Commit Graph

518 Commits

Author SHA1 Message Date
GregoryTravis
67b5745806
Add range info to Out_Of_Range error (#9586) 2024-04-04 12:25:45 +00:00
Hubert Plociniczak
a83c75f8ec
Add missing API for Jackson_Object (#9591)
`Jackson_Object` supported parsing but not creating JSON from text. With this change, `Jackson_Object` is on par with `JS_Object` API and replaces the latter.

The most visible differences come from more detailed parsing exception's messages. Had to add some special cases for corner cases like `NaN` or infinity.

Closes #9473.
2024-04-02 20:45:35 +00:00
GregoryTravis
ff562cb454
Conversions from Decimal to Integer and Float. (#9462)
* wip

* wip

* Remove unnecessary conversion

* changelog

* also integer

* cleanup

* to_float inf

* review

* Using multiple simple if conditions

* review

* logging in conversions

* cleanup

* revert

* pending test

* truncation

* Use TypeOfNode instead of TypesLibrary in InvokeConversionNode

* combine cases

* rename typesLib

* String is not Enso type, return Text

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2024-03-30 10:42:56 -04:00
Pavel Marek
0b94493258
Equality does not swallow errors (#9560)
`42 == (Error.throw "foo")` now correctly returns an `Error` rather than False

# Important Notes
The error was in the wrong usage of the `org.enso.interpreter.dsl.AcceptsError` DSL annotation.
2024-03-29 04:11:16 +00:00
GregoryTravis
f80e3f3415
Decimal remainder, pow, div (#9566) 2024-03-28 20:20:29 +00:00
GregoryTravis
b74fb5c5c9
Set.to_text should not assume contents are Text (#9548)
`Set.to_text` fails if the contents are anything other than `Text`.
2024-03-27 19:20:31 +00:00
James Dunkerley
e262801daa
Restructure Standard.Table. (#9559)
Move the types from `Standard.Table.Data` to `Standard.Table`.

Exceptions:
- `Standard.Table.Data.Report_Unmatched` => `Standard.Table.Constants`.
- `Standard.Table.Data.Join_Kind_Cross` => `Standard.Table.Internal.Join_Kind_Cross`.
Also removed constructor as an atom type.
- `Standard.Table.Extensions.Table_Ref` => `Standard.Table.Internal.Table_Ref`.
- `Standard.Table.Data.Type.Value_Type_Helpers` => `Standard.Table.Internal.Value_Type_Helpers`.
- `Standard.Table.Data.Type.Enso_Types` => `Standard.Table.Internal.Value_Type_Helpers`.
- `Standard.Table.Data.Type.Storage` => `Standard.Table.Internal.Storage`.

Changed all `Standard.Table` imports inside project to be project.
Favoured importing from `Standard.Table.Main` in `Standard.Database`.
Also fixed some linting in Enso_File.
2024-03-27 17:10:43 +00:00
Radosław Waśko
f2d6079ac4
Fix missing AWS region in S3 operations (#9546)
- Closes #9284
- Now our tests run without the default `AWS_` config, thus ensuring that the tested setups work in a clean environment.
- After all, more complicated logic was needed for buckets access - apparently the AWS SDK only allows for some operations on buckets to happen if the client is connected to the correct region. Thus detection of bucket regions had to be implemented.
- Added `AWS_Region` widget based on autoscoping.
- Fixed `AWS_Credential.profile_names` crashing if no AWS config was found. Now it returns no profiles if not found. Added a regression test.
2024-03-27 12:00:15 +00:00
Radosław Waśko
af5354b869
Data Link for reading Enso_File (#9525)
- Closes #9282
2024-03-27 04:17:07 +00:00
Radosław Waśko
431a055608
Use DELETE /assets/<id> for deleting files and secrets (#9551)
- Closes #9288
2024-03-26 21:48:17 +00:00
James Dunkerley
7098c5e3e6
Apply ICONs and other tweaks. (#9552)
Lots of ICONs...
2024-03-26 19:44:06 +00:00
Radosław Waśko
6665c22eb9
Make data-links behave more like 'symlinks' (#9485)
- Closes #9324
2024-03-22 17:01:54 +00:00
James Dunkerley
d2d5b3ba8d
Some fixes working towards a sales demo (#9501)
- Fixed the `write` format dropdown.
- Added `Text_Input` widget to path in various places.
- ICONs and Widgets in various places.
- Secret name dropdown.
2024-03-20 16:44:50 +00:00
AdRiley
933608e26a
Add new aliases (#9500) 2024-03-20 16:15:00 +00:00
Radosław Waśko
6e5b4d93a3
Implement refreshing the Cloud token in Enso libraries (#9390)
- Closes #9300
- Now the Enso libraries are themselves capable of refreshing the access token, thus there is no more problems if the token expires during a long running workflow.
- Adds `get_optional_field` sibling to `get_required_field` for more unified parsing of JSON responses from the Cloud.
- Adds `expected_type` that checks the type of extracted fields. This way, if the response is malformed we get a nice Enso Cloud error telling us what is wrong with the payload instead of a `Type_Error` later down the line.
- Fixes `Test.expect_panic_with` to actually catch only panics. Before it used to also handle dataflow errors - but these have `.should_fail_with` instead. We should distinguish these scenarios.
2024-03-19 19:26:34 +00:00
GregoryTravis
9a9eff1aa6
Decimal type: constuctors, comparisons, and arithmetic (#9272) 2024-03-15 21:13:41 +00:00
Sergei Garin
6bf37b3d17
Update ~/.enso/credentials file (#9287)
* Pass additional metadata to ~/.enso/credentials

* support old and new credentials format

* fix some issues after the review

* use domain instead of iss field

* small fixes

* rename Uri -> Url

* rename cognito -> Cognito

* remove unneeded file

---------

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2024-03-12 12:41:47 +03:00
AdRiley
cd4a6d456c
Add xml.write tests and .write to XML_Element (#9332)
* Add xml.write tests and write to XML_Element

* Code Review changes

* Fix tests
2024-03-11 16:26:06 +00:00
Radosław Waśko
e98306f170
Excel DataLink (#9346)
- Adds the Excel format as one of the formats supported when creating a data link.
- The data link can choose to read the file as a workbook, or read a sheet or range from it as a table, like `Excel_Format`.
- Also updated Delimited format dialog to allow customizing the quote style.
2024-03-11 16:12:12 +00:00
Radosław Waśko
2e35189d83
Better error message in Data.read when an argument to format is missing (#9337) 2024-03-11 14:50:32 +00:00
AdRiley
8e1bb16afd
Remove File from File./ (#9330)
Closes #8905
2024-03-11 10:12:09 +00:00
Hubert Plociniczak
f80dd9f593
Demonstrate usage of foreign arrow function (#9150)
Including arrow language in the distribution by default. Added a basic example for creating an Arrow array.
Making sure that memory layout agrees with Arrow specification (padding, continuous allocation of memory chunks).
Related to #9118.

This should unblock work on allowing serialization/deserialization to/from Parquet but I'd like to delay it to a follow up ticket as it is going to be a significant amount of specialized work.
2024-03-08 15:20:55 +00:00
Radosław Waśko
a3bf5a0be5
Split Excel_Format into 3 constructors (#9308)
Splitting the `Excel_Format` into 3 constructors.
2024-03-08 14:26:30 +00:00
James Dunkerley
c7d693dfc8
Move Standard.Database.Data to Standard.Database. (#9321)
Moves the types out of `Data`.
2024-03-07 14:43:38 +00:00
James Dunkerley
1e33723ce9
ICONs and linting... (#9319)
- Apply all the icons from @Cassandra-Clark.
- Fix various linting issues.
- Remove circular dependency between Table and Database.
2024-03-07 12:26:32 +00:00
AdRiley
3ebf1340e8
Add write to xml document (#9299)
* First commit

* Add xml.write

* Add comment

* Changelog.md

* Code review changes

* Code review changes

* Update import
2024-03-06 17:13:28 +00:00
James Dunkerley
7f6d10a9e0
Some minor improvements suggested from Ned's use (#9249)
- Adjusted `AWS_Credential` to have a `Default` and removed support for `Nothing` from functions.
- Renamed `Response_Body.to_file` to `Response_Body.write`.
- Add `write` to `Response`.
- Add `Table.get_value` and `DB_Table.get_value` allowing getting a single value from a table.
- Added `Data.download` allowing downloading from a URL to a file.
- Added text widget as input widget for `Data` methods.

![image](https://github.com/enso-org/enso/assets/4699705/fcfc8b1e-1197-4106-b8a7-43b1435327c0)
2024-03-04 16:33:31 +00:00
Jaroslav Tulach
5676618bad
Autoscoped constructors (#9190)
Fixes #8645 by recognizing `~` prefix to constructor names.
2024-03-04 11:41:02 +00:00
Radosław Waśko
39af372bcd
Allow resolving enso:// URIs in Data.read and other places (#9225)
- Implements the core parts of #9048
- Currently the path resolution is done by resolving each segment, one by one - requiring as many API calls as there are segments in the path.
- This should be replaced in a followup PR, once https://github.com/enso-org/cloud-v2/issues/899 is implemented.
2024-03-02 16:04:30 +00:00
James Dunkerley
964fdfd7ea
Align XML_Document and XML_Element APIs more. (#9233)
- Added `to_table` extensions on some core types.
- Added ICON to `Any.to`.
- Added ICON to `Column.info`, `Table.info`, `DB_Column.info` and `DB_Table.info`.
- Added defaults to `Table.cross_tab` and `DB_Table.cross_tab`.
- Added `name`, `get`, `at`, `inner_xml` and `outer_xml` to `XML_Document`.
- Added constants into left hand side of simple expressions.
- Added widget to `get` and `at` on `XML_Document` and `XML_Element`. (Some bug in annotation code with Dmitry)
- Altered `get` and `at` to not allow XPath and just get direct child/attribute values.
- Added `get_xpath` to `XML_Document`.
- Renamed `get_elements_by_tag_name` to `get_descendants_by_tag_name` and added new `get_children_by_tag_name`.
- Added `child_names` and `attribute_names` to `XML_Document` and `XML_Element`.
2024-03-01 17:42:44 +00:00
Radosław Waśko
4316709379
Implementing reading Data Links (#9215)
- Close #9123
2024-03-01 15:33:21 +00:00
Radosław Waśko
b33802a1d1
Start using pre-signed URLs for Enso Files (#9227)
Since the cloud finally gives us the pre-signed URLs, we can start using that.
2024-03-01 01:01:21 +00:00
GregoryTravis
54675b1e4d
Implement Table.replace for the database backend (#8986) 2024-02-29 18:36:42 +00:00
AdRiley
259ad09957
Update dry run warning messages (#9183) 2024-02-27 10:15:03 +00:00
AdRiley
0d635626aa
Added unique, deduplicate aliases for distinct (#9152)
Added unique, deduplicate aliases for distinct
2024-02-23 11:12:21 +00:00
Radosław Waśko
d817df94a5
Move Enso_Cloud from Data to top-level in Standard.Base (#9140)
- 4 weeks ago we discussed with @jdunkerley we may want to move this module.
- `Data` is a submodule mostly for various data-structures etc., the cloud stuff did not fit that well in there.
2024-02-22 14:42:35 +00:00
Pavel Marek
21e1284086
Enso tests can be run with filter from cmdline (#9065)
Simplify the `Test.Suite.run_with_filter` to accept a single filter parameter that searches for all the groups and specs that matches that filter. This filter can be a simple text provided from the command line.

# Important Notes
- Pending groups are now printed at the end of the run
- `Test.Suite.run_with_filter` is simplified to accept a single filter parameter that is either `Text` or `Nothing`. See the docs.
- Passing a filter from the command line is therefore straightforward, it is treated as a regex.
- For convenience, I have left all the `main` methods in all the test sources. I have just refactored them to accept the `filter` argument from the command line.
- For example, to run only a single spec from `Vector_Spec.enso`, invoke `enso --run test/Base_Tests/src/Data/Vector_Spec.enso "should allow vector creation with a programmatic constructor"`
- **Majority of the PR is a regex replace** of `^main =` for `main filter=Nothing =` and of `suite.run_with_filter` for `suite.run_with_filter filter`.
- **Fixed some internal engine bugs:**
- `AtomWithHole` allows to specify only one hole - https://github.com/enso-org/enso/pull/9065/files#diff-0f7bb7e85cf86a965de133aa7e6b5958ceb889bd1921c01e00d3a9ceb19626ef
- NaN keys in hash maps are handled in polyglot maps as well - c5257f6c2b78f893214ff67300893b593ea05e21..db4b3c0e9828ee79208d52e02586b24bb845b0d6
2024-02-22 12:31:44 +00:00
James Dunkerley
fa6fccb99e
Refactoring Table.set for GUI2 and other GUI fixes (#9128)
- Added `regex` function to `Standard.Base` as a way to easily and cleanly make regular expressions.
- Added `expr` and `Expression.Value` to distinguish expressions from text values.
- Fixed issues with `Table.join` widget so dropdown exists.
- Needed fully qualified name.
- Added default empty text values for right column to provided text input boxes.
- Deprecate `Table.filter_by_expression` and allow `Table.filter` to take an `Expression`.
- Added `Simple_Expression` and deprecated `Column_Operation`. Changes the order so takes a column then a calculation.
- Rename `column` to `value` and `new_name` to `as` on `Table.set`.
- Rename `name_column` to `names` on `Table.cross_tab`.
- Removed `Column_Ref.Expression` in favour of using `Expression.Value`.
2024-02-21 23:51:41 +00:00
Radosław Waśko
167a91dab1
Updating Enso Cloud integration - cheaper Enso_Secret.create, introducing Enso_File.creation_time and last_modified_time (#9085)
- Closes #9047
2024-02-19 18:53:20 +00:00
Jaroslav Tulach
f1d4e546d5
Upgrading to GraalVM 21.0.2 (#8883)
Upgrades to latest GraalVM 21.0.2
2024-02-19 12:08:59 +00:00
Radosław Waśko
cf71a05c3b
Refactor common context handling pattern (#9087) 2024-02-19 11:59:42 +00:00
Radosław Waśko
642d5a691e
Implement copy_to and move_to for S3_File (#9054)
- Closes #8833
- Tests for copying between S3 and `Enso_File` will only be added once we implement Enso_File writing.
2024-02-16 10:42:28 +00:00
James Dunkerley
08584b0423
Improving experience with format and parse. (#9045)
- Add format dropdown to `Number.format`.
![image](https://github.com/enso-org/enso/assets/4699705/8aa74910-c6ad-4480-a7f2-04dacd9686e8)

- Support case insensitive month names and abbreviations in dates.
https://github.com/enso-org/enso/assets/4699705/4dbd8755-e1c2-4207-a8a1-65b427ca4fab

- Improve locale dropdown for `parse_date` and `parse_date_time`.
![image](https://github.com/enso-org/enso/assets/4699705/5d605a2d-1248-46be-bc74-34a4afecf609)

- Added dropdown to `Table.parse` and amended so now doesn't accept `Nothing` (using empty string instead).
![image](https://github.com/enso-org/enso/assets/4699705/340dd093-77db-4685-a34b-45ce09e3c3b3)

- Added dropdown to `Table.format` and amended so now doesn't accept `Nothing` (using empty string instead).

- Altered `Column.parse` to not accept `Nothing` and added drop down for format.
- Altered `Column.format` to not accept `Nothing` and added drop down for format conditional on type.
- Improved the locale date/time format drop to have the suggested formats too.
2024-02-14 16:30:37 +00:00
Radosław Waśko
d45f0fe4df
Check type of self in static dispatch (#8867)
- Fixes #8805
- also btw. fixes #8706
2024-02-14 15:50:13 +00:00
James Dunkerley
8c197f325b
Update the aggregate API to take a separate group_by (#9027)
Separate out the `Group_By` from the column definition in `aggregate`.
![image](https://github.com/enso-org/enso/assets/4699705/6b4f03bc-1c4a-4582-b38a-ba528ae94167)

Supports the old API with a warning attached about deprecation:
![image](https://github.com/enso-org/enso/assets/4699705/0cc42ff7-6047-41a5-bb99-c717d06d0d93)

Widgets have been updated with `Group_By` removed from the dropdown.
2024-02-13 10:23:59 +00:00
Radosław Waśko
eb59b475f6
Write support for S3 (#8921)
- Closes #8809
2024-02-12 19:04:13 +00:00
James Dunkerley
0c3e8f1e50
Update ICONs as per Cass's work (#9023)
Apply the Date and Time based ICON updates.
2024-02-12 18:15:28 +00:00
James Dunkerley
f257952680
Changing Widgets to work better with GUI2 (#9017)
- Rename `Column.replace` to `Column.text_replace` (consistency with Table and allow for adding `Column.replace`).
- Changed Regex insert to be `.to_regex` rather than `Regex.compile`.
https://github.com/enso-org/enso/assets/4699705/3694e876-5b46-49f9-8e8c-352fa0684777

- Adjusted `Table.rename_columns` to insert a `Pair.new` for each rename.
![image](https://github.com/enso-org/enso/assets/4699705/8240f302-498c-46d6-9b77-b1a0f5bb585e)

- Added a `<Regular Expression>` option to the `rename_columns`.
https://github.com/enso-org/enso/assets/4699705/d3c695ad-93f2-419d-b1c1-1f607ca825bd

- Added constant values to `Table.fill_nothing` and made it value type dependent on `Column.fill_nothing`.
https://github.com/enso-org/enso/assets/4699705/26e2e975-e972-46f0-ae1a-6b09dcd39fe6
![image](https://github.com/enso-org/enso/assets/4699705/51393d09-c06b-4eb1-9471-1856430fbfdc)
![image](https://github.com/enso-org/enso/assets/4699705/064fa42b-6187-4816-b3f0-0b7f0f00f6a9)

- Removed `use_regex` option from `Sort_Column.By_Name` (in favour of Regex type).

- Improved widgets for `order_by`. Drop down now on the column name and allow using the Index method with specified numeric range.
![image](https://github.com/enso-org/enso/assets/4699705/549b9c4c-5753-4518-801c-ecd06db5c2e7)
2024-02-12 15:12:19 +00:00
AdRiley
48a61d7b78
Change empty_error type to array. Add tests (#9019)
* Change error type tp array. Add tests

* Fix tests

* Put this back as it was
2024-02-12 09:15:10 +00:00
AdRiley
1dc8c1cb24
Merge empty_error (#9000)
We don't need 3 copies of empty_error. This PR makes them one.
2024-02-09 14:18:14 +00:00
James Dunkerley
93f2a44633
Add some ICON tags and a few new Icons. (#8996)
- Apply @Cassandra-Clark's ICON tags to the library code.

- Fix bug with Table viz not showing row count in dropdown list. **Doesn't fix defect with dropdown not changing data.**
![image](https://github.com/enso-org/enso/assets/4699705/dfbdba91-5f4f-4c89-8ae8-f30156047162)

- Added missing icons.
![image](https://github.com/enso-org/enso/assets/4699705/145191cd-d31a-45f1-8bb2-fcc3cf0f1207)
![image](https://github.com/enso-org/enso/assets/4699705/2a8e0530-2470-4ec8-b836-57bed8520c28)
2024-02-08 09:56:57 +00:00
James Dunkerley
0c39f8ec04
Allow Filter_Condition to be inverted. (#8861)
- Various linting fixes (doc comments and type annotations etc.).
- Add an action to determine if a `Filter_Condition` is keep or remove.

https://github.com/enso-org/enso/assets/4699705/69ba2bd3-8893-4237-acc4-eb01f534a209

- Remove `Not_In`, `Not_Contains` and `Not_Like` from `Filter_Condition`.

- Ability to use an `Expression` as a `Column_Ref`.

https://github.com/enso-org/enso/assets/4699705/16a2e030-f8f9-4f59-beca-2646f56fcb90
2024-02-07 14:36:14 +00:00
James Dunkerley
eeaddbc434
Add parser for line by line processing (#8719)
- Linting fixes and groups.
- Add `File.from that:Text` and use `File` conversions instead of taking both `File` and `Text` and calling `File.new`.
- Align Unix Epoc with the UTC timezone and add converting from long value to `Date_Time` using it.
- Add simple first logging API allowing writing to log messages from Enso.
- Fix minor style issue where a test type had a empty constructor.
- Added a `long` based array builder.
- Added `File_By_Line` to read a file line by line.
- Added "fast" JSON parser based off Jackson.
- Altered range `to_vector` to be a proxy Vector.
- Added `at` and `get` to `Database.Column`.
- Added `get` to `Table.Column`.
- Added ability to expand `Vector`, `Array` `Range`, `Date_Range` to columns.
- Altered so `expand_to_column` default column name will be the same as the input column (i.e. no `Value` suffix).
- Added ability to expand `Map`, `JS_Object` and `Jackson_Object` to rows with two columns coming out (and extra key column).
-  Fixed bug where couldn't use integer index to expand to rows.
2024-02-01 07:29:50 +00:00
Jaroslav Tulach
9a37357247
Binary operator resolution based on that value (#8779) 2024-01-27 08:38:47 +01:00
Radosław Waśko
ca4f98c78e
Adding tests and missing methods for Enso_File. (#8815)
- Closes #8808 - adds tests for various scenarios.
- Implements `size` using HEAD.
- Updates existing functions to changes in Cloud API.
- Adds stubs for `*_time` methods, `parent`, `path`.
- [x] TODO: resolve the `Enso_File.current_working_directory` from an environment variable.
- ~~TODO: recursive directory deletion?~~ left for later

# Important Notes
- Currently, the Cloud API does not offer an easy way to extract metadata for a file, in particular to get the parent folder from the file `id`.
- We should be able to get the parent, and stuff like creation/modified time.
- We need a way to resolve paths to asset ids, for `path` to work as well as `current_working_directory`.
- What is the environment variable that will be used to feed the `current_working_directory` property?
2024-01-26 19:04:42 +00:00
GregoryTravis
5eb3f3bd1d
Implement relational NULL semantics for Nothing for in-memory Column operations (#8816)
Updates in-memory table column operations to treat Nothing as a relational NULL.
This PR does not include changes to Table.join.
2024-01-24 17:02:45 +00:00
AdRiley
23d6fcdd9c
Make max of Random methods inclusive rather than exclusive (#8768)
* Make max inclusive rather than exclusive

* Update names to min max. Remove end_inclusive

* Indicies should remain exclusive

* Make the tests pass
2024-01-24 16:58:33 +00:00
Radosław Waśko
edfcfde11c
Tests and improvements for secrets in cloud subdirectories (#8791)
- Closes #8723
- Adds some missing features that were needed to make this work:
- `Enso_File.create_directory` and `Enso_File.delete`, and basic tests for it
- Changes how `Enso_Secret.list` is obtained - using a different Cloud endpoint allows us to implement the desired logic, the default endpoint was giving us _all_ secrets which was not what we wanted here.
- Implements `Enso_Secret.update` and tests for it

# Important Notes
Notes describing any problems with the current Cloud API:
https://docs.google.com/document/d/1x8RUt3KkwyhlxGux7XUGfOdtFSAZV3fI9lSSqQ3XsXk/edit

Apparently, everything that was needed to make this feature work has already been implemented, although a few features needed workarounds on Enso side to work properly.
2024-01-24 10:17:22 +00:00
Radosław Waśko
14be36c401
Allow secrets in Header.authorization_* (#8761)
- Closes #8739
2024-01-18 12:49:47 +00:00
James Dunkerley
d55c9c99b4
First few changes from the Churn workflow (#8782)
- `up_to` should have the `step` as an optional argument.
- `Date_Range` conversion can do a clever auto rename, so if a period use the name of it.
- Add `to Table` for `Range`, `Date_Range`.
2024-01-17 21:57:24 +00:00
AdRiley
ac0d4c9f5f
Make Random.Seed private. Remove unused TEXT_ONLY (#8783)
Random.Seed doesn't work in the GUI and the TEXT_ONLY tag doesn't do anything so it was incorrectly showing up in the component browser.

This MR makes Random.Seed private to hide it from the GUI and completely removes the TEXT_ONLY tag which is unused and unimplemented.
2024-01-17 17:15:51 +00:00
AdRiley
ef7b11fb67
Add new aliases for cross_join, info and drop (#8773)
Adds new aliases
append -> cross_join
metadata -> info
skip -> drop
2024-01-16 16:08:54 +00:00
GregoryTravis
f2cb1f097e
Support on_problems=Problem_Behavior.Report_Warning and Map_Error wrapping in Vector.map (#8595)
Implements `Warnings.get_all wrap_errors=True` which wraps warnings attached to values inside vectors with `Map_Error`, which includes the position of the value within the vector. See [the documentation](https://github.com/enso-org/enso/blob/develop/docs/semantics/wrapped-errors.md) for more details.

`get_all wrap_errors=True` does not change the warnings that are attached to values -- it wraps them before returning them to the caller, but does not change the original warnings attached to the values.

Wrapped warnings only appear attached to the vector itself. The values inside the vector do not have their warnings wrapped.

Warning propagation is not changed at all; `Warnings.get_all` (with default `wrap_errors=False`) behaves as before. `get_all wrap_errors=True` is meant to be used primarily by the IDE, although it can be used anywhere this wrapping is desired.
2024-01-16 09:36:22 +00:00
Radosław Waśko
5b70ff25f7
Remove set_user_info from URI (#8738)
I have added this in #8591, but I have realised it may not be a good idea to have it, so I am removing that particular change.
2024-01-15 17:35:17 +00:00
Radosław Waśko
f34abeda0c
Add tests for Enso_Secrets, update to new cloud API (#8736)
- Closes #8556
2024-01-15 16:12:08 +00:00
Pavel Marek
6ae35abc46
Fix Runtime.assert (#8742) 2024-01-12 18:47:40 +01:00
Pavel Marek
428e83de36
Remove org.bouncycastle dependency (#8664)
Remove `org.bouncycastle` dependency from `org.enso.runtime`.
2024-01-04 17:16:41 +01:00
GregoryTravis
e6ae366917
Ignore User-Agent in HTTP tests (#8630)
`User-Agent` depends on the precise JDK version, so we ignore it when checking HTTP responses.
Clsoes #8629.
2024-01-04 15:53:17 +00:00
AdRiley
bf8dd1888c
Give file read its own helper widget for delimiters. (#8627)
Give file read its own helper widget for delimiters. Remove newline add none. The file read delimiter is similar but different to the split one and so should have its own set of options.
2024-01-04 11:59:42 +00:00
Radosław Waśko
a1207e029d
Unify File_Format_Metadata with File_For_Read (#8628)
- Closes #8555
- Refactors the file format detection logic, compacting lots of repetitive logic for HTTP handling into helper functions.
- Some updates to CODEOWNERS.
2024-01-04 03:57:05 +00:00
Jaroslav Tulach
07d58f2c02
DataflowError.withoutTrace shall not store a trace (#8608) 2023-12-24 11:07:32 +01:00
Radosław Waśko
b3de42eb23
Handle Nothing values in Filter_Condition.to_predicate (#8600)
- Fixes #8549
- Ensures that a `Type_Error` is thrown instead of a `No_Such_Method` error on type mismatches.
- I think this is more readable.
2023-12-21 19:17:55 +00:00
Radosław Waśko
d41d48e8a0
Merge URI_With_Query into URI, extend API of URI (#8591)
- Closes #8544
- Adds `reset_query_arguments` and `/` operators allowing to transform a URI.
- Adding tests for handling of various edge cases.
2023-12-21 18:39:26 +00:00
Radosław Waśko
dfdb547616
Better context info in Type_Error raised from return type checks (#8566)
- Followup to #8502 that adds better error messages
2023-12-20 18:22:47 +00:00
Radosław Waśko
d56b800c11
Remove the Apache dependency from std-base (#8571)
- After [suggestion](https://github.com/enso-org/enso/pull/8497#discussion_r1429543815) from @JaroslavTulach I have tried reimplementing the URL encoding using just `URLEncode` builtin util. I will see if this does not complicate other followup improvements, but most likely all should work so we should be able to get rid of the unnecessary bloat.
2023-12-20 18:01:08 +00:00
James Dunkerley
2e9bd86854
Small linting fixes. (#8592) 2023-12-20 17:25:43 +00:00
Pavel Marek
4cb2439890
Submodules can be private (#8581) 2023-12-19 19:13:44 +01:00
Radosław Waśko
724f8d2a56
Add tests for Enso Cloud auth + simple API mock for Enso_User (#8511)
- Closes #8354
- Extends `simple-httpbin` with a simple mock of the Cloud API (currently it checks the token and serves the `/users` endpoint).
- Renames `simple-httpbin` to `http-test-helper`.
2023-12-19 17:41:09 +00:00
Radosław Waśko
d4714af826
Add a few new Filter_Conditions (#8539)
- Closes #8045
2023-12-16 15:12:23 +00:00
Radosław Waśko
940b8f7d51
Improving tests and edge cases for URI and HTTP (#8497)
- Closes #8352
- ~~Proposed fix for #8493~~
- The temporary fix is deemed not viable. I will try to figure out a workaround and leave fixing #8493 to the engine team.
2023-12-15 17:58:45 +00:00
Radosław Waśko
b5c995a7bf
Reworking Excel support to allow for reading of big files (#8403)
- Closes #8111 by making sure that all Excel workbooks are read using a backing file (which should be more memory efficient).
- If the workbook is being opened from an input stream, that stream is materialized to a `Temporary_File`.
- Adds tests fetching Table formats from HTTP.
- Extends `simple-httpbin` with ability to serve files for our tests.
- Ensures that the `Infer` option on `Excel` format also works with streams, if content-type metadata is available (e.g. from HTTP headers).
- Implements a `Temporary_File` facility that can be used to create a temporary file that is deleted once all references to the `Temporary_File` instance are GCed.
2023-12-15 00:02:15 +00:00
GregoryTravis
1c815a3d45
Better Error Trapping in map (#8307)
* tests

* wip

* wip

* additional warnings

* wip

* wip

* cleanup

* nested wrapping

* multiple nestings

* wraps_error uses looks_for, test for should_fail_with

* wip

* stack trace line fix

* use catch_primitive internally

* fix warning mapping, dtf spec

* just one wrapper checker, vector spec

* missing ctor, back to non-primitive catch

* back to c_p

* put old map back

* wip

* unnest tests

* Array.map on_problems

* wip

* Revert "wip"

This reverts commit c30d171457.

* better test names

* warning logging

* wip

* wip

* move logic into ALH

* doc

* constant

* My_Error.Error

* nested

* doc

* map_primtiive in warning mapper

* composition

* ref spec

* Remove warnings prior to matching on the value

If an expression has warnings and is matched we:
1) extract the warnings
2) execute the branch of a pattern that matches the value
3) attach extracted warnings to the result

This caused warnings to reappear when doing the custom warnings
manipulation.
This is also consistent with how `CaseNode`'s `doWarning` specialization
is defined.

* fix 1

* do not auto unwrap in test error checkers

* nested error matcher

* in problems too

* dtf

* v

* statistics

* wip

* Table_Spec, map_with_index_primitive

* Column_Operations_Spec

* disable warning wrapping and Report_Warning

* unimpl test

* Warnings_Spec

* DCS

* ACG JP

* zip_primitive

* join_helpers

* Lookup_Helpers

* Table

* Data_Formatter

* Value_Type_Helpers

* revert check types changes

* table_helpers

* table tests

* remove st

* do not remove warnings from value

* vec docs, tests for zip, mwi, flat_map

* docs, fixes

* remove nested_error_matcher

* cleanup

* benchmark

* one error

* alter

* add bench to main

* review

* review

* review

* tail call

* changelog

* tail call was not a tail call

* ws

* bad import

* Added missing import

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Array.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* review, ref example

* lazy benchmark data

* extra paren

* check outside of catch

* review

* vector too

* actually lazy

* disambiguate Map_Error

* finish rename

* move to extensions

* combine Additional_Warnings error

* rename to map_no_wrap

* do not catch and rethrow

* review

* wip

* remove _primitives entirely

* remove unused should_fail_with function options

* remove expected_warning as function in Problems

---------

Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2023-12-13 09:38:09 -05:00
somebody1234
3419c77bc6
Fix typo in Match.enso (#8515)
Fix issue causing visualizations to break:
![image](https://github.com/enso-org/enso/assets/4046547/d63239e2-3fde-424a-ba6c-3e163c6ce983)

# Important Notes
None
2023-12-12 17:59:44 +00:00
James Dunkerley
f60836d9e1
Apply ICONs (#8360)
- Amend a couple of missed groups.
- Add the first pass of some ICONs.

The linter tool has been updated to support rewriting the ICON as well.
2023-11-22 15:24:16 +00:00
James Dunkerley
347b5a7cf5
Linting and Groups update (#8357)
- Fix issues from the linter.
- Rename the constructors for `Blank_Selector`.
- Update various GROUP tags.
2023-11-21 18:12:27 +00:00
James Dunkerley
ecaca12df1
Integrating Enso Cloud with the libraries (part 1...) (#8006)
- Add a `File_For_Read` type. Used for `File_Format` to read files.
- Added `Enso_User` representing the current user in `Enso_Cloud`.
- *Will be later able to list known users.*
- Added `Enso_Secret` representing a value defined in `Enso_Cloud`.
- Value not used within Enso only accessed within polyglot Java.
- Integrated into `Username_And_Password` and can be used within JDBC connections.
- Integrated into HTTP Headers so a secret can be used as a value.
- New `URI_With_Query` with the same API as `URI`. Supporting secrets in the value.
- *Will be integrated with AWS credentials.*
- Added `Enso_File` representing a file or a folder in the cloud.
- Support the same API as `File` (like the `S3_File`).
- *Will support `enso://` URI style access.*
2023-11-20 23:21:14 +00:00
Jaroslav Tulach
1138dfe147
Specify expression to get more advanced results on_return callback (#8331) 2023-11-20 18:47:11 +01:00
Pavel Marek
5a7ad6bfe4
Upgrade enso to GraalVM for jdk 21 (#7991)
Upgrade to GraalVM JDK 21.
```
> java -version
openjdk version "21" 2023-09-19
OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15)
OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing)
```

With SDKMan, download with `sdk install java 21-graalce`.

# Important Notes
- After this PR, one can theoretically run enso with any JRE with version at least 21.
- Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution.
- `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded.
- sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368
- Which means that we can have `module-info.java` files only for Java-only projects.
- Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar.
- `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent.
- Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory.
- The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module.
- The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725)
- [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 18:02:36 +00:00
Radosław Waśko
2ce1567384
Limit max_rows that are downloaded in Table.read by default, and warn if more rows are available (#8159)
- Sets the default limit for `Table.read` in Database to be max 1000 rows.
- The limit for in-memory compatible API still defaults to `Nothing`.
- Adds a warning if there are more rows than limit.
- Enables a few unrelated asserts.
2023-11-06 16:41:47 +00:00
GregoryTravis
1480f50207
Overhaul the random number and item generation code (#8127)
Rewrite most of Random.enso.
2023-10-31 15:25:37 +00:00
Kaz Wesley
2edd2bd7ff
Ensure all spans have document offsets (#8039)
- Validate spans during existing lexer and parser unit tests, and in `enso_parser_debug`.
- Fix lost span info causing failures of updated tests.

# Important Notes
- [x] Output of `parse_all_enso_files.sh` is unchanged since before #7881 (modulo libs changes since then).
- When the parser encounters an input with the first line indented, it now creates a sub-block for lines at than indent level, and emits a syntax error (every indented block must have a parent).
- When the parser encounters a number with a base but no digits (e.g. `0x`), it now emits a `Number` with `None` in the digits field rather than a 0-length digits token.
2023-10-19 12:36:42 +00:00
James Dunkerley
ad95dc9815
Fixes for Table viz (#8102)
- Fixes issue with `to_display_text` on `Date_Time`. Now a reversible format.
- Adjusted the auto width code on table so it works.
- Manually sized columns should remain fixed size.
- Enable Excel range style selection in table.
- Removed page support as not implemented yet.

# Important Notes
Will need to apply to Vue based viz at some point as well.
2023-10-19 08:22:23 +00:00
GregoryTravis
7383db0e04
Restructuring XML into Table form (#8083)
# Important Notes
Adds `.to Table` support, as well as XML support for `expand_column`.
2023-10-19 07:02:48 +00:00
Radosław Waśko
08b717eb54
Refactor Table problem handling to a more robust and hopefully cleaner approach (#7879)
Closes #7514
2023-10-16 15:09:08 +00:00
James Dunkerley
b7d7910a88
Use US locale for Date/Time parsing by default. (#8053)
Fixes issue with parsing long format month names.
2023-10-13 17:47:14 +00:00
Jaroslav Tulach
31624b5314
Helper method to turn iterator to a Vector (#8035) 2023-10-12 17:50:51 +02:00
James Dunkerley
0dcfc3e9bf
Minor improvements from last couple of Book Clubs (#8034)
- Added some ALIASes.
- Added `sheet` to `Excel_Workbook` to give familiar API to read sheet.
- Added conversion from range to vector allowing easy use with Zip.
- Add `Map.from_keys_and_values.
2023-10-12 14:29:59 +00:00
Jaroslav Tulach
7ae0971d11
Runtime checks of signatures with polyglot java classes (#8016) 2023-10-11 15:05:36 +02:00
Jaroslav Tulach
a234e82ee9
Instrumenter to observe behavior of nodes with UUID (#7833)
Exposing instrumentation capabilities to Enso. Fixes #7683.
2023-10-10 02:36:59 +00:00
Pavel Marek
4db61210c0
Exporting non-existing symbols fails with compiler error (#7960)
Adds a new compiler analysis pass that ensures that all the symbols exported via `export ...` or `from ... export ...` statements exist. If not, generates an IR Error.

# Important Notes
We already have such a compiler pass for the version with imports in [ImportSymbolAnalysis.scala](https://github.com/enso-org/enso/blob/develop/engine/runtime/src/main/scala/org/enso/compiler/pass/analyse/ImportSymbolAnalysis.scala)
2023-10-09 09:48:43 +00:00
GregoryTravis
a90af9f844
Cache filtered children in XML (#7998)
We currently scan the entire child list for each call to children, num_children, and get_child_element.

Instead, we should cache the filtered child list on first access.
2023-10-06 19:44:01 +00:00
GregoryTravis
9ba7be20af
Basic XML support (#7947)
This PR includes
* Reading XML from a file, stream, or string
* Reading XML via Data.fetch
* Accessing the root element, element children, and attributes
* Accessing tag text contents
* Get tags by name
* Inner / Outer XML string
2023-10-06 17:52:19 +00:00
Jaroslav Tulach
6a59fa5e93
Meta.Type.find a type by FQN (#7885) 2023-10-02 17:02:00 +02:00
James Dunkerley
fb50eb7595
Using conversions in a few places (#7859)
- Shuffle a few `from`s into correct places:
- `Day_Of_Week.from` removing `Day_Of_Week_From` module.
- Adding short cut for `http` and `https` in `Data.read` so it calls onto `Data.fetch` giving a single entry point.
- Moved `URI` extensions from `Standard.Base.Data` module into `Standard.Base.Network.Extensions`.
- Added `post` extension for `URI`.
- Added `contains_key` to `JS_Object`.
- Restored `into` in `JS_Object`:
- Follows old logic populating a constructor.
- Will use conversion from `JS_Object` if present.
- Added automatic deserialization of `Date`, `Time_Of_Day` and `Date_Time` from JSON.
- Uses conversion from `JS_Object`.
- Added conversion from `Text` to a `HTTP_Method` and type checking where `HTTP_Method` used in public APIs.
- Added support for `Date`, `Time_Of_Day` and `Date_Time` in `Table.from_objects`.
- Added `expand_column` to `Table` to expand `JS_Object` to values.
- Add type checking for `Table` in `right` arguments (allowing `Column`s to be used).
- Use type checking in `Table.set` to allow for conversion to a `Column`.
- Remove some unused imports.
- Fix for bug in S3 edge case.
2023-10-02 14:54:22 +00:00
Cassandra-Clark
d7258abbf5
Add Text.substring to allow for an easy short hand of Text.take (start.up_to end) (#7913)
* Add Text.substring function and get_position helper function

For #7876 adds a Text.substring function which supports negative indexes and returns a part of a string from 0-based index 'start' and continuing for 'length'

* added substring and simplified get function

For #7876 adds a Text.substring function which supports negative indexes and returns a part of a string from 0-based index 'start' and continuing for 'length'.

Also simplified get function as it looped unnecessarily.

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Text/Extensions.enso

punctuation corrections

Co-authored-by: GregoryTravis <greg.m.travis@gmail.com>

* Update Text_Spec.enso

Added test for start index larger than string

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Text/Extensions.enso

updated Arguments: section to use consistent style

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Text/Extensions.enso

updated Index_Out_Of_Bounds error to reference cached length

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* Removed Slice label and added changelog entry

* Re-added slice tag to substring

Per conversation with James, added slice back to substring

* Update CHANGELOG.md add link

---------

Co-authored-by: GregoryTravis <greg.m.travis@gmail.com>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2023-09-29 10:57:57 -06:00
Pavel Marek
9f15b90caa
Implement Enso-specific assert (#7883)
Implement Enso-specific assert - `Runtime.assert` that works like asserts in any other runtime.

# Important Notes
- Enso-specific assertions are enabled when JVM assertions are enabled, or when `ENSO_ENABLE_ASSERTIONS` env var is not empty (See 72cd8361cb/engine/runtime/src/main/java/org/enso/interpreter/runtime/EnsoContext.java (L139))
2023-09-29 14:46:58 +00:00
Jaroslav Tulach
ffa036411d
Make sure Integer can be treated as Float (#7909) 2023-09-28 11:56:54 +02:00
Radosław Waśko
8d926166ea
Follow up improvements to Date_Time_Formatter (#7875)
- Closes #7872
- Also closes #7866
2023-09-28 09:38:00 +00:00
Radosław Waśko
c690559ec4
Implement auto_value_type operation (#7908)
Closes #6113
2023-09-27 15:45:34 +00:00
GregoryTravis
b03712390c
Improve HTTP tests (#7847)
* simple-httpbin encodes response using the Content-encoding header value
* Return sent body verbatim
2023-09-27 14:02:32 +00:00
Radosław Waśko
12c4f2981d
More robust Date/Time format patterns parsing (#7826)
- Closes #7461 by introducing a `Date_Time_Formatter` type and making parsing date time formats more robust and safer.
- The default ('simple') set of patterns is slightly simplified and made case insensitive (except for `M/m` and `H/h`) to avoid the `YYYY` vs `yyyy` issues and make it less error prone.
- The `YYYY` now has the same meaning as `yyyy` in simple mode. The old meaning (week-based year) is moved to a _separate mode_, triggered by `Date_Time_Formatter.from_iso_week_date_pattern`.
- Full Java syntax, as well as custom-built Java `DateTimeFormatter` can also be used by `Date_Time_Formatter.from_java`.
- Text-based constants (e.g. `ISO_ZONED_DATE_TIME`) have now become methods on `Date_Time_Formatter`, e.g. `Date_Time_Formatter.iso_zoned_date_time`).
2023-09-22 10:12:18 +00:00
Jaroslav Tulach
5150c14afd
Avoid duplicated conversion & search target type scope (#7849) 2023-09-20 17:23:09 +02:00
James Dunkerley
74d1d0861c
S3 Read Access, Input Stream based reading (#7776)
- Added a `FileSystemSPI` allowing protocol resolution to a target type.
- Separated `Input_Stream` and `Output_Stream` from `File` to allow use in other spaces.
- `File_Format` types `read_web` changed to be `read_stream` working with `InputStream`.
- Added directory listing to `Auto_Detect` allowing for `Data.read` to list a folder.
- Adjusted HTTP to return an `InputStream` not a `byte[]`:
- `Response_Body` adjusted to wrap an `InputStream`.
- Added ability to materialize to either and in-memory vector (<4KB) or a temporary file.
- `Data.fetch` will materialize if not a recognized mime-type.
- Added `HTTP_Error` to handle IO exceptions from the stream.
- `Excel_Format` now supports mime-type and reading a stream.
- `Excel_Workbook` can now get a `Excel_Section` using `read_section`.
- Added S3 APIs:
- `parse_uri`: splits an S3 URI into bucket and key.
- `list_objects`: list the items in a S3 bucket with specified prefix.
- `read_bucket`: list prefixes and keys with a delimiter in a S3 bucket with specified prefix.
- `head`: either head_bucket (tests existance) or head_object API (reads object meta data).
- `get_object`: gets an object from S3 returning as a `Response_Body`.
- Added `S3_File` type acting like a `File`:
- No support for writing in this PR.
- **ToDo:** recursive listing, glob filtering, exists, size.
- Fixed a few invalid type signature line.
- Moved `create` methods for `Postgres_Connection` and `SQLite_Connection` into type instead of module.
- Renamed `Column_Fetcher.Builder` to `Column_Fetcher_Builder`.
- Fixed bug with `select_into` in Dry Run mode creating permanent tables.

**ToDo:** Unit tests.
2023-09-20 15:09:11 +00:00
GregoryTravis
b0c1f3b00e
New Data.post for sending a payload to a Web API (#7700) 2023-09-19 11:26:29 +00:00
Hubert Plociniczak
1ee3d8f4f0
Rename Decimal to Float (#7807)
Implements #6889.
2023-09-14 15:01:30 +00:00
James Dunkerley
f0ae9bf9c5
Fixes issue writing to a dry run Excel File (#7763)
- Adds `size` to `File`.
- If file is empty, then create a new Excel file.
- Fixes dry run Excel write issue.
2023-09-08 08:52:00 +00:00
Radosław Waśko
7d424bf8a2
Implement Table.delete_rows. (#7709)
- Closes #7238
- Aligns `update_database_table` to a more consistent and clearer API - `update_rows`.
- Adds a `truncate_table` helper function, to pair up with `drop_table`. Both are `PRIVATE` for now.
- Adds tests for NULLs in keys in `update_rows` and `delete_rows`.
- The behaviour is sometimes unexpected, so instead these fail with `Null_Values_In_Key_Columns`.
- Adds a workaround for https://github.com/oracle/graal/issues/7359
- Adds a workaround for a related bug where a stack frame has no name (its `rootNode.getName() == null`).
- I could not track down this bug to provide a neat repro.
2023-09-07 11:07:53 +00:00
Jaroslav Tulach
1437a671e1
Introducing generic Any.to type conversion method (#7704) 2023-09-01 08:05:48 +02:00
Jaroslav Tulach
6461e20870
Special support for Python Date/Time/Zone interop (#7617) 2023-08-25 10:27:16 +02:00
GregoryTravis
ddf18f212b
Handle writing to a relative file (#7638)
Fixes bug in writing to a non-absolute file (with backup).
2023-08-24 21:01:37 +00:00
Jaroslav Tulach
20e18d22df
More descriptive function information (#7629)
Fixes #7359 by printing more information about the function including partially applied arguments and over-saturated arguments.
2023-08-24 18:04:08 +00:00
James Dunkerley
7d83b3d7b4
Add GROUP to functions (#7622)
- Update list of groups to agreed list.
- Lower case `ALIAS` names to be consistent with function names.
- Add `GROUP` to methods.
- All constructors and functions have doc comments.
- Correct a few typos (e.g. `PRVIATE`).
- Mark some more things as `PRIVATE`.
- Use `ToDo:` and `Note:` consistently.
- Order tags in doc comment.

# Important Notes
We don't have all the doc comments on types and will want to add them in future,
2023-08-23 13:20:38 +00:00
GregoryTravis
c9d7c5cb2b
Convert in-memory Column.round to Java (#7521) 2023-08-16 14:45:23 +00:00
Jaroslav Tulach
aa0413e5a2
Use only Type instances as keys for State (#7585) 2023-08-16 15:54:17 +02:00
James Dunkerley
296c95d414
Fix for empty column on replace and out of memory catching for join and tab (#7593)
- Added a Panic.catch to catch heap memory error in joins and cross_tab.
- Adjusted column replace so type is correct.
2023-08-15 17:06:51 +00:00
Jaroslav Tulach
7a272ec152
Encapsulating array-like data and operations into a single package (#7544) 2023-08-15 13:00:47 +02:00
GregoryTravis
d3436fae70
Implement Number.round as a builtin (#7460) 2023-08-14 15:43:39 +00:00
GregoryTravis
037a687401
Expose Unicode normalization methods on Texts (#7425)
Exposes Text_Utils.normalize().
2023-08-03 18:07:00 +00:00
GregoryTravis
628a51d8e2
Convert Number.round to Java (#7360) 2023-07-26 12:03:09 +00:00
James Dunkerley
7345f0fd9a
Speed up statistics (#7390)
- Allow `parse_to_columns` to take a `Regex` object.
- Add `pattern` to the `Regex` object.
- Add `column_names` to the `Row` object.
- Improve statistics performance.
- Add benchmarks for stats.

| Benchmark | Reference | New | Improvement |
| --- | --- | --- | --- |
| Max (by reduce) | 16.4ms | 16.3ms | - |
| Max (stats) | 703ms | 224ms | 68% |
| Sum (by reduce) | 38ms | 38ms | - |
| Sum (stats) | 753ms | 420ms | 44% |
| Variance (stats) | 745ms | 553s | 26% |

Also tried using a Ref approach for stats but as slower (7e13c45224).
2023-07-26 10:01:18 +00:00
Radosław Waśko
4b5a2e2176
Fixing operations on Mixed types (#7368)
- Fixes #7231
- Cleans up vectorized operations to distinguish unary and binary operations.
- Introduces MixedStorage which may pretend to be a more specialized storage on demand.
- Ensures that operations request a more specialized storage on right-hand side to ensure compatibility with reported inferred storage type.
- Ensures that a dataflow error returned by an Enso callback in Java is propagated as a polyglot exception and can be caught back in Enso
- Tests for comparison of Mixed storages with each other and other types
- Started using `Set` for `Filter_Condition.Is_In` for better performance.
- ~~Migrated `Column.map` and `Column.zip` to use the Java-to-Enso callbacks.~~
- This does not forward warnings. IMO we should not be losing them. We can switch and add a ticket to fix the warnings, but that would be a regression (current implementation handles them correctly). Instead, we should first gain some ability to work with warnings in polyglot. I created a ticket to get this figured out #7371
- ~~Trying to avoid conversions when calling Enso functions from Java.~~
- Needs extra care as dataflow errors may not be handled right then. So only works for simple functions that should not error.
- Not sure how much it really helps. [Benchmarks](https://github.com/enso-org/enso/pull/7270#issuecomment-1635618393) suggested it could improve the performance quite significantly, but the practical solution is not exactly the same as the one measured, so we may have to measure and tune it to get the best results.
- Created #7378 to track this.
2023-07-25 23:25:17 +00:00
GregoryTravis
1f6fcf189b
Implement replace on the Database Column (#7275)
Implements `replace` for database text columns, for text, regex, and column patterns.
2023-07-25 18:09:50 +00:00
James Dunkerley
2dc565b366
Fix failing test (#7394)
Fix a failing test.
2023-07-25 14:06:11 +00:00
Adam Obuchowicz
1d2371f986
Groups in DocTags (#7337)
Fixes #7336 in a quick way.

Next to the old way of defining groups, the library can just add `GROUP` tag to some entities, and it will be added to the group specified in tag's description.

The group name may be qualified (with project name, like `Standard.Base.Input/Output`) or just name - in the latter case, IDE will assume a group defined in the same library as the entity.

Also moved some entities from "export" list in package.yaml to GROUP tag to give an example. I didn't move all of those, as I assume the library team will reorganize those groups anyway.

### Important Notes

@jdunkerley @radeusgd @GregoryTravis When you will start specifying groups in tags, remember that:
* The groups still belongs to a concrete project; if some entity outside a project wants to be added to its group, the "qualified" name should be specified. See `Table.new` example in this PR.
* If the group name does not reflect any group in package.yaml **the tag is ignored**.
* A single entity may be only in a single group. If it's specified in both package.yaml and in tag, the tag takes precedence.

---------

Co-authored-by: Ilya Bogdanov <fumlead@gmail.com>
2023-07-24 15:54:16 +02:00
James Dunkerley
88f32d9b2a
Various small tickets... (#7367)
- Added `Text.length` into Text class so CB lists the built in.
- Added `File.starts_with` and tests for the built in method.
- Add `to_js_object` and `to_display_text` to `Regex`.
![image](https://github.com/enso-org/enso/assets/4699705/3b197c94-9c49-4bc5-a2cc-ce53b917942e)
- Add `to_js_object` and `to_display_text` to `Match`.
![image](https://github.com/enso-org/enso/assets/4699705/962ec4f2-324d-4f10-8ec0-932b093c6729)
- Remove the `bit_shift_l` alias from the built-ins.
- Add test and Enso wrapper for `Text.is_normalized`.
2023-07-23 09:04:11 +00:00
Pavel Marek
7264d81f2a
Builtin methods can support array-like arguments (#7235)
This PR modifies the builtin method processor such that it forbids arrays of non-primitive and non-guest objects in builtin methods. And provides a proper implementation for the builtin methods in `EnsoFile`.

- Remove last `to_array` calls from `File.enso`
2023-07-17 09:17:39 +02:00
James Dunkerley
aaa235fbad
Add drop down for replace, remove Column_Selector (#7295)
- Add dropdowns for `replace` functions.
- Retire `Column_Selector` type.
- Add `select_blank_columns` and `remove_blank_columns` functions to table types.
- Allow Regex to be used to pick columns.
2023-07-14 17:30:52 +00:00
Radosław Waśko
866283c0a8
Improve error message on Filter_Condition missing arguments in Table.filter (#7290)
In #7148 I improved the error message when a `Filter_Condition` constructor without arguments is provided to `Vector.filter` and its friends. This PR applies the same check to the `Table.filter`.

This is useful, because when we select a Filter_Condition from a widget, initially it does not have all its arguments applied. This used to lead to confusing errors being reported to the user, now, a much clearer error is shown:

![image](https://github.com/enso-org/enso/assets/1436948/19140a7b-d6fc-4292-81d3-dc6d61135cb9)
2023-07-14 08:00:13 +00:00
Radosław Waśko
620cc361ce
Add date_diff, date_add and date_part to scalar Enso date-time values. (#7273)
Followup of #7221, adding `date_diff`, `date_add` and `date_part` to scalar Enso date-time values.
2023-07-13 15:17:21 +00:00
Radosław Waśko
ca68dd94da
Adding new Date/Time operations (-, date_add, date_diff, date_part) (#7221)
- Adds `Column.date_diff` for computing date/time difference as integer multiply of some unit.
- Adds `Column.date_add` for shifting date/time by a unit.
- Adds `Column.date_part` for extracting various parts of the date/time value as integer.
- Adds widgets for the 3 methods above whose content depends on the column value type.
- Adds shorthands: `Column.hour`, `Column.minute` and `Column.second` to extract these date parts.
- Extends `Time_Period` with support for milli-, micro- and nano- seconds; and adapts functions taking `Time_Period` to support these wherever possible.
2023-07-13 12:56:54 +00:00
James Dunkerley
0adab6c68c
Round on a column was always adding a warning (#7246)
- Only warn if outside allowed range.
- Added `is_infinite` to In-Memory column.
- Allow integer value type for `is_nan` and `is_infinite`.
2023-07-10 17:35:23 +00:00
GregoryTravis
bd26e95fd6
Add Table.replace; Change Text.replace to take a Text|Pattern, and remove the use_regex param. (#7223) 2023-07-06 16:13:11 +00:00
James Dunkerley
7749286c69
Tidy up the imports using script (#7220)
Ordering the imports to test a script.
2023-07-06 14:22:50 +00:00
GregoryTravis
6eb46afb40
Do not rename column on fill_nothing and add version to the Table allowing filling multiple (include fill_empty as well). (#7166)
Updated Column.fill_nothing and .fill_empty, and added the same to Table. (Both in-memory and db.)
2023-07-05 17:20:23 +00:00
GregoryTravis
966f8b773a
Combine Regex and Pattern (#7172)
Merge Pattern into Regex.
2023-07-05 13:51:53 +00:00
James Dunkerley
4fbe7e3830
Remove Array.new and Array.copy and move Vector functions to builtins. (#7147)
- Removed Array methods: `new`, `copy` and `new_[1234]`.
- New builtins for `Vector.insert`, `Vector.remove` and `Vector.flatten`.
- Replaced `Vector_Builder` use of `Array.copy` to a `Vector.Builder` approach.
2023-07-03 12:41:41 +00:00
GregoryTravis
550d146493
Add round, ceil, floor, truncate to the In-Database Column type (#6988) 2023-06-30 16:47:40 +00:00
Radosław Waśko
6eac095579
Add support for Filter_Condition in any, all, find, partition and index_of (#7148)
Closes #6628
2023-06-30 16:06:01 +00:00
Pavel Marek
60ef7e0ba3
Referencing self from static method raises compiler error (#7131)
Add a new `NoStaticInSelf` compiler pass that ensures that `self` keyword is used only from instance methods.
2023-06-28 19:04:32 +02:00
James Dunkerley
56688ec1e7
Minor fixes. (#7122)
Mostly stuff to tidy up the static methods in the CB.

- Remove default pattern from `parse_to_table` (caused IDE to freeze).
- Rename any `_` arguments to what they are.
- Merge `Date.now` into `Date.today`
- Merge the Interval constructors into a single constructor.
- Hide various methods.
2023-06-27 18:18:15 +00:00