Commit Graph

187 Commits

Author SHA1 Message Date
Radosław Waśko
39af372bcd
Allow resolving enso:// URIs in Data.read and other places (#9225)
- Implements the core parts of #9048
- Currently the path resolution is done by resolving each segment, one by one - requiring as many API calls as there are segments in the path.
- This should be replaced in a followup PR, once https://github.com/enso-org/cloud-v2/issues/899 is implemented.
2024-03-02 16:04:30 +00:00
James Dunkerley
964fdfd7ea
Align XML_Document and XML_Element APIs more. (#9233)
- Added `to_table` extensions on some core types.
- Added ICON to `Any.to`.
- Added ICON to `Column.info`, `Table.info`, `DB_Column.info` and `DB_Table.info`.
- Added defaults to `Table.cross_tab` and `DB_Table.cross_tab`.
- Added `name`, `get`, `at`, `inner_xml` and `outer_xml` to `XML_Document`.
- Added constants into left hand side of simple expressions.
- Added widget to `get` and `at` on `XML_Document` and `XML_Element`. (Some bug in annotation code with Dmitry)
- Altered `get` and `at` to not allow XPath and just get direct child/attribute values.
- Added `get_xpath` to `XML_Document`.
- Renamed `get_elements_by_tag_name` to `get_descendants_by_tag_name` and added new `get_children_by_tag_name`.
- Added `child_names` and `attribute_names` to `XML_Document` and `XML_Element`.
2024-03-01 17:42:44 +00:00
Radosław Waśko
4316709379
Implementing reading Data Links (#9215)
- Close #9123
2024-03-01 15:33:21 +00:00
GregoryTravis
54675b1e4d
Implement Table.replace for the database backend (#8986) 2024-02-29 18:36:42 +00:00
Radosław Waśko
386132cdab
Fix matching JS strings (#9203)
- Fixes #9202
2024-02-29 15:07:29 +00:00
James Dunkerley
8f2b9da664
IsNa to IsNothing, missing to Nothing in Table code. (#9154)
Starting to use Nothing everywhere...
2024-02-26 10:52:07 +00:00
James Dunkerley
0e2a91cfe1
Remove countMissing from Storage and replace with a new CountNothing operation. (#9137)
Removing another small piece of logic from the storages to it's own operation.
2024-02-22 19:32:46 +00:00
Radosław Waśko
d817df94a5
Move Enso_Cloud from Data to top-level in Standard.Base (#9140)
- 4 weeks ago we discussed with @jdunkerley we may want to move this module.
- `Data` is a submodule mostly for various data-structures etc., the cloud stuff did not fit that well in there.
2024-02-22 14:42:35 +00:00
Pavel Marek
21e1284086
Enso tests can be run with filter from cmdline (#9065)
Simplify the `Test.Suite.run_with_filter` to accept a single filter parameter that searches for all the groups and specs that matches that filter. This filter can be a simple text provided from the command line.

# Important Notes
- Pending groups are now printed at the end of the run
- `Test.Suite.run_with_filter` is simplified to accept a single filter parameter that is either `Text` or `Nothing`. See the docs.
- Passing a filter from the command line is therefore straightforward, it is treated as a regex.
- For convenience, I have left all the `main` methods in all the test sources. I have just refactored them to accept the `filter` argument from the command line.
- For example, to run only a single spec from `Vector_Spec.enso`, invoke `enso --run test/Base_Tests/src/Data/Vector_Spec.enso "should allow vector creation with a programmatic constructor"`
- **Majority of the PR is a regex replace** of `^main =` for `main filter=Nothing =` and of `suite.run_with_filter` for `suite.run_with_filter filter`.
- **Fixed some internal engine bugs:**
- `AtomWithHole` allows to specify only one hole - https://github.com/enso-org/enso/pull/9065/files#diff-0f7bb7e85cf86a965de133aa7e6b5958ceb889bd1921c01e00d3a9ceb19626ef
- NaN keys in hash maps are handled in polyglot maps as well - c5257f6c2b78f893214ff67300893b593ea05e21..db4b3c0e9828ee79208d52e02586b24bb845b0d6
2024-02-22 12:31:44 +00:00
Jaroslav Tulach
2a42388905
Including missing Polyglot_Spec in all the tests (#9126) 2024-02-21 16:16:00 +01:00
Radosław Waśko
167a91dab1
Updating Enso Cloud integration - cheaper Enso_Secret.create, introducing Enso_File.creation_time and last_modified_time (#9085)
- Closes #9047
2024-02-19 18:53:20 +00:00
James Dunkerley
ee66b9fb1d
Refactoring the Unary operations so uncoupled from Storage. (#9090)
In order to allow clever masking, slicing, filtering and arrow backing stores...

- Adding ColumnStorage interface with the base API a storage will need.
- Refactored each of the unary operations to a new `UnaryOperation` interface which makes them responsible for deciding if they can be executed.
2024-02-19 17:11:52 +00:00
Jaroslav Tulach
a664dd9d56
Equality with conversions (#9070) 2024-02-19 17:18:56 +01:00
Radosław Waśko
cf71a05c3b
Refactor common context handling pattern (#9087) 2024-02-19 11:59:42 +00:00
James Dunkerley
f2d2f73e89
Starting to refactor Storage and Operations (#9076)
Cleaning up some of the structures in Storage before working on UnaryOperations.

- Removed some legacy code: `countMask`, `Index` and `DefaultIndex`.
- Renamed `mask` to `applyFilter` on `Column` and `Storage`.
- Renamed `Table.mask` to `Table.filter`.
2024-02-15 18:21:07 +00:00
James Dunkerley
08584b0423
Improving experience with format and parse. (#9045)
- Add format dropdown to `Number.format`.
![image](https://github.com/enso-org/enso/assets/4699705/8aa74910-c6ad-4480-a7f2-04dacd9686e8)

- Support case insensitive month names and abbreviations in dates.
https://github.com/enso-org/enso/assets/4699705/4dbd8755-e1c2-4207-a8a1-65b427ca4fab

- Improve locale dropdown for `parse_date` and `parse_date_time`.
![image](https://github.com/enso-org/enso/assets/4699705/5d605a2d-1248-46be-bc74-34a4afecf609)

- Added dropdown to `Table.parse` and amended so now doesn't accept `Nothing` (using empty string instead).
![image](https://github.com/enso-org/enso/assets/4699705/340dd093-77db-4685-a34b-45ce09e3c3b3)

- Added dropdown to `Table.format` and amended so now doesn't accept `Nothing` (using empty string instead).

- Altered `Column.parse` to not accept `Nothing` and added drop down for format.
- Altered `Column.format` to not accept `Nothing` and added drop down for format conditional on type.
- Improved the locale date/time format drop to have the suggested formats too.
2024-02-14 16:30:37 +00:00
Radosław Waśko
d45f0fe4df
Check type of self in static dispatch (#8867)
- Fixes #8805
- also btw. fixes #8706
2024-02-14 15:50:13 +00:00
GregoryTravis
3bdd1a0dce
Re-fetch Warnings in Warning.getWarnings() to correctly wrap them (#9011) 2024-02-13 15:12:32 +00:00
Radosław Waśko
eb59b475f6
Write support for S3 (#8921)
- Closes #8809
2024-02-12 19:04:13 +00:00
AdRiley
48a61d7b78
Change empty_error type to array. Add tests (#9019)
* Change error type tp array. Add tests

* Fix tests

* Put this back as it was
2024-02-12 09:15:10 +00:00
AdRiley
1dc8c1cb24
Merge empty_error (#9000)
We don't need 3 copies of empty_error. This PR makes them one.
2024-02-09 14:18:14 +00:00
Pavel Marek
f3f0697d56
Merge Test_New into Test (#8991)
Merges the temporary `Test_New` library into `Test`. This is the last PR in the series of PRs that refactor all the stdlib tests to the builder API.
2024-02-08 11:25:13 +00:00
James Dunkerley
0c39f8ec04
Allow Filter_Condition to be inverted. (#8861)
- Various linting fixes (doc comments and type annotations etc.).
- Add an action to determine if a `Filter_Condition` is keep or remove.

https://github.com/enso-org/enso/assets/4699705/69ba2bd3-8893-4237-acc4-eb01f534a209

- Remove `Not_In`, `Not_Contains` and `Not_Like` from `Filter_Condition`.

- Ability to use an `Expression` as a `Column_Ref`.

https://github.com/enso-org/enso/assets/4699705/16a2e030-f8f9-4f59-beca-2646f56fcb90
2024-02-07 14:36:14 +00:00
Pavel Marek
83fffd9c05
Refactor stdlib tests to the builder API (#8968)
Follow-up of #8890

Refactor the rest of the tests to the builder API (`Test_New`):
- `Image_Tests`
- `Geo_Tests`
- `Google_Api_Test`
- `Examples_Test`
- `AWS_Tests`
- `Meta_Test_Suite_Tests`
- `Visualization_Tests`

# Important Notes
- Unrelated: Fix NPE in `File.new "/" . name`
2024-02-07 13:22:17 +00:00
Jaroslav Tulach
2028d6b2c1
Better runEngineDistribution --run test/Base_Tests "filter" (#8940) 2024-02-04 11:39:36 +01:00
James Dunkerley
eeaddbc434
Add parser for line by line processing (#8719)
- Linting fixes and groups.
- Add `File.from that:Text` and use `File` conversions instead of taking both `File` and `Text` and calling `File.new`.
- Align Unix Epoc with the UTC timezone and add converting from long value to `Date_Time` using it.
- Add simple first logging API allowing writing to log messages from Enso.
- Fix minor style issue where a test type had a empty constructor.
- Added a `long` based array builder.
- Added `File_By_Line` to read a file line by line.
- Added "fast" JSON parser based off Jackson.
- Altered range `to_vector` to be a proxy Vector.
- Added `at` and `get` to `Database.Column`.
- Added `get` to `Table.Column`.
- Added ability to expand `Vector`, `Array` `Range`, `Date_Range` to columns.
- Altered so `expand_to_column` default column name will be the same as the input column (i.e. no `Value` suffix).
- Added ability to expand `Map`, `JS_Object` and `Jackson_Object` to rows with two columns coming out (and extra key column).
-  Fixed bug where couldn't use integer index to expand to rows.
2024-02-01 07:29:50 +00:00
Pavel Marek
bb8ff8f89e
Refactor Base_Tests/src/Data to Test_New (#8890)
Refactor `Base_Tests` to `Test_New` testing framework. Mostly automatic text replacements.

# Important Notes
List of changes that were not done automatically (not via automatic text replacement):
- Fix indexes in Instrumentor_Spec - f590c4a398
- If group or spec is pending, its block is not evaluated - 8d797f1a4a
- Spec_Result is not private - 5767535af2

Tests marked as *pending*:
- #8913
- #8910
2024-01-31 14:25:16 +00:00
Jaroslav Tulach
9a37357247
Binary operator resolution based on that value (#8779) 2024-01-27 08:38:47 +01:00
Radosław Waśko
ca4f98c78e
Adding tests and missing methods for Enso_File. (#8815)
- Closes #8808 - adds tests for various scenarios.
- Implements `size` using HEAD.
- Updates existing functions to changes in Cloud API.
- Adds stubs for `*_time` methods, `parent`, `path`.
- [x] TODO: resolve the `Enso_File.current_working_directory` from an environment variable.
- ~~TODO: recursive directory deletion?~~ left for later

# Important Notes
- Currently, the Cloud API does not offer an easy way to extract metadata for a file, in particular to get the parent folder from the file `id`.
- We should be able to get the parent, and stuff like creation/modified time.
- We need a way to resolve paths to asset ids, for `path` to work as well as `current_working_directory`.
- What is the environment variable that will be used to feed the `current_working_directory` property?
2024-01-26 19:04:42 +00:00
Pavel Marek
d0fdeca6df
Refactor Table_Tests to the builder API (#8622)
Refactor `test/Table_Test` to the builder API. The builder API is in a new library called `Test_New` that is alongside the old `Test` library. There will be follow-up PRs that will migrate the rest of the tests. Meanwhile, let's keep these two libraries, and merge them after the last PR.

# Important Notes
- For a brief introduction into the new API, see **Prototype 1** section in https://github.com/enso-org/enso/pull/8622#issuecomment-1889706168
- When executing all the tests, the behavior should be the same as with the old library. With the only exception that if `ENSO_TEST_ANSI_COLORS` env var is set, the output is more colorful than it used to be.
2024-01-26 12:08:24 +00:00
GregoryTravis
5eb3f3bd1d
Implement relational NULL semantics for Nothing for in-memory Column operations (#8816)
Updates in-memory table column operations to treat Nothing as a relational NULL.
This PR does not include changes to Table.join.
2024-01-24 17:02:45 +00:00
AdRiley
23d6fcdd9c
Make max of Random methods inclusive rather than exclusive (#8768)
* Make max inclusive rather than exclusive

* Update names to min max. Remove end_inclusive

* Indicies should remain exclusive

* Make the tests pass
2024-01-24 16:58:33 +00:00
Radosław Waśko
edfcfde11c
Tests and improvements for secrets in cloud subdirectories (#8791)
- Closes #8723
- Adds some missing features that were needed to make this work:
- `Enso_File.create_directory` and `Enso_File.delete`, and basic tests for it
- Changes how `Enso_Secret.list` is obtained - using a different Cloud endpoint allows us to implement the desired logic, the default endpoint was giving us _all_ secrets which was not what we wanted here.
- Implements `Enso_Secret.update` and tests for it

# Important Notes
Notes describing any problems with the current Cloud API:
https://docs.google.com/document/d/1x8RUt3KkwyhlxGux7XUGfOdtFSAZV3fI9lSSqQ3XsXk/edit

Apparently, everything that was needed to make this feature work has already been implemented, although a few features needed workarounds on Enso side to work properly.
2024-01-24 10:17:22 +00:00
Radosław Waśko
368e4867b4
Allow secrets in AWS_Credential (#8774)
- Closes #8722
2024-01-19 19:00:56 +00:00
Radosław Waśko
14be36c401
Allow secrets in Header.authorization_* (#8761)
- Closes #8739
2024-01-18 12:49:47 +00:00
James Dunkerley
d55c9c99b4
First few changes from the Churn workflow (#8782)
- `up_to` should have the `step` as an optional argument.
- `Date_Range` conversion can do a clever auto rename, so if a period use the name of it.
- Add `to Table` for `Range`, `Date_Range`.
2024-01-17 21:57:24 +00:00
Radosław Waśko
71427c3319
Rename Tests to Base_Tests to be more descriptive (#8781) 2024-01-17 16:19:19 +01:00