Commit Graph

640 Commits

Author SHA1 Message Date
AdRiley
2fdb2fca62
Added Table.running (#9382)
This is a first naïve implementation of Table.running that only supports count. Adding it as a scaffold for the rest of the functionality and to give us a place to agree on the API. (Which I changed slightly from the design)

![image](https://github.com/enso-org/enso/assets/1720119/a62a83ed-f864-4295-98ea-1007f62381b1)

# Important Notes
Only supports Statistic.Count. Other functionality to follow.
2024-03-13 13:28:33 +00:00
Sergei Garin
6bf37b3d17
Update ~/.enso/credentials file (#9287)
* Pass additional metadata to ~/.enso/credentials

* support old and new credentials format

* fix some issues after the review

* use domain instead of iss field

* small fixes

* rename Uri -> Url

* rename cognito -> Cognito

* remove unneeded file

---------

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2024-03-12 12:41:47 +03:00
AdRiley
cd4a6d456c
Add xml.write tests and .write to XML_Element (#9332)
* Add xml.write tests and write to XML_Element

* Code Review changes

* Fix tests
2024-03-11 16:26:06 +00:00
Radosław Waśko
e98306f170
Excel DataLink (#9346)
- Adds the Excel format as one of the formats supported when creating a data link.
- The data link can choose to read the file as a workbook, or read a sheet or range from it as a table, like `Excel_Format`.
- Also updated Delimited format dialog to allow customizing the quote style.
2024-03-11 16:12:12 +00:00
Radosław Waśko
2e35189d83
Better error message in Data.read when an argument to format is missing (#9337) 2024-03-11 14:50:32 +00:00
AdRiley
8e1bb16afd
Remove File from File./ (#9330)
Closes #8905
2024-03-11 10:12:09 +00:00
James Dunkerley
9f9cf58d28
Add selecting by type to the table (#9334)
- Adds `select_by_type` and `remove_by_type` to tables.
2024-03-08 18:40:53 +00:00
Hubert Plociniczak
f80dd9f593
Demonstrate usage of foreign arrow function (#9150)
Including arrow language in the distribution by default. Added a basic example for creating an Arrow array.
Making sure that memory layout agrees with Arrow specification (padding, continuous allocation of memory chunks).
Related to #9118.

This should unblock work on allowing serialization/deserialization to/from Parquet but I'd like to delay it to a follow up ticket as it is going to be a significant amount of specialized work.
2024-03-08 15:20:55 +00:00
Radosław Waśko
a3bf5a0be5
Split Excel_Format into 3 constructors (#9308)
Splitting the `Excel_Format` into 3 constructors.
2024-03-08 14:26:30 +00:00
James Dunkerley
c7d693dfc8
Move Standard.Database.Data to Standard.Database. (#9321)
Moves the types out of `Data`.
2024-03-07 14:43:38 +00:00
James Dunkerley
1e33723ce9
ICONs and linting... (#9319)
- Apply all the icons from @Cassandra-Clark.
- Fix various linting issues.
- Remove circular dependency between Table and Database.
2024-03-07 12:26:32 +00:00
AdRiley
3ebf1340e8
Add write to xml document (#9299)
* First commit

* Add xml.write

* Add comment

* Changelog.md

* Code review changes

* Code review changes

* Update import
2024-03-06 17:13:28 +00:00
Radosław Waśko
e37862b09d
Implement a Data Link for Postgres (#9269)
- Closes #9124
2024-03-06 11:57:12 +00:00
James Dunkerley
7f6d10a9e0
Some minor improvements suggested from Ned's use (#9249)
- Adjusted `AWS_Credential` to have a `Default` and removed support for `Nothing` from functions.
- Renamed `Response_Body.to_file` to `Response_Body.write`.
- Add `write` to `Response`.
- Add `Table.get_value` and `DB_Table.get_value` allowing getting a single value from a table.
- Added `Data.download` allowing downloading from a URL to a file.
- Added text widget as input widget for `Data` methods.

![image](https://github.com/enso-org/enso/assets/4699705/fcfc8b1e-1197-4106-b8a7-43b1435327c0)
2024-03-04 16:33:31 +00:00
AdRiley
8b889f0977
Make Table.To_Xml return a XML_Document (#9263)
As part of the XML improvements it makes more sense for Table.To_Xml to return a XML_Document.
2024-03-04 15:19:20 +00:00
Jaroslav Tulach
5676618bad
Autoscoped constructors (#9190)
Fixes #8645 by recognizing `~` prefix to constructor names.
2024-03-04 11:41:02 +00:00
Radosław Waśko
39af372bcd
Allow resolving enso:// URIs in Data.read and other places (#9225)
- Implements the core parts of #9048
- Currently the path resolution is done by resolving each segment, one by one - requiring as many API calls as there are segments in the path.
- This should be replaced in a followup PR, once https://github.com/enso-org/cloud-v2/issues/899 is implemented.
2024-03-02 16:04:30 +00:00
James Dunkerley
964fdfd7ea
Align XML_Document and XML_Element APIs more. (#9233)
- Added `to_table` extensions on some core types.
- Added ICON to `Any.to`.
- Added ICON to `Column.info`, `Table.info`, `DB_Column.info` and `DB_Table.info`.
- Added defaults to `Table.cross_tab` and `DB_Table.cross_tab`.
- Added `name`, `get`, `at`, `inner_xml` and `outer_xml` to `XML_Document`.
- Added constants into left hand side of simple expressions.
- Added widget to `get` and `at` on `XML_Document` and `XML_Element`. (Some bug in annotation code with Dmitry)
- Altered `get` and `at` to not allow XPath and just get direct child/attribute values.
- Added `get_xpath` to `XML_Document`.
- Renamed `get_elements_by_tag_name` to `get_descendants_by_tag_name` and added new `get_children_by_tag_name`.
- Added `child_names` and `attribute_names` to `XML_Document` and `XML_Element`.
2024-03-01 17:42:44 +00:00
Radosław Waśko
4316709379
Implementing reading Data Links (#9215)
- Close #9123
2024-03-01 15:33:21 +00:00
Radosław Waśko
b33802a1d1
Start using pre-signed URLs for Enso Files (#9227)
Since the cloud finally gives us the pre-signed URLs, we can start using that.
2024-03-01 01:01:21 +00:00
GregoryTravis
54675b1e4d
Implement Table.replace for the database backend (#8986) 2024-02-29 18:36:42 +00:00
Radosław Waśko
47c64167ef
Clearer warnings in license review (#9134)
- Closes #9120
- Reorders CI steps to do the license check last (to avoid it preventing tests from running which are more important than the license check)
- Tries to reword the warnings to be clearer
- Adds some CSS to the report to more clearly indicate which elements can be clicked.
2024-02-27 16:32:08 +00:00
AdRiley
259ad09957
Update dry run warning messages (#9183) 2024-02-27 10:15:03 +00:00
James Dunkerley
8f2b9da664
IsNa to IsNothing, missing to Nothing in Table code. (#9154)
Starting to use Nothing everywhere...
2024-02-26 10:52:07 +00:00
AdRiley
4420a403a9
Add ability to choose dimensions and metrics to google analytics report (#9156)
* Metrics and Dimensions from Vector

* Add dropdowns

* Small cleanup

* Updated list from Cass
2024-02-24 21:31:54 +00:00
AdRiley
0d635626aa
Added unique, deduplicate aliases for distinct (#9152)
Added unique, deduplicate aliases for distinct
2024-02-23 11:12:21 +00:00
James Dunkerley
288a00a01f
Fix issue with date columns being fetched from Postgres. (#9143)
Quick bug fix that was caused by us not handling `java.sql.Date` and mapping to `java.time.LocalDate`.
2024-02-22 20:39:42 +00:00
James Dunkerley
0e2a91cfe1
Remove countMissing from Storage and replace with a new CountNothing operation. (#9137)
Removing another small piece of logic from the storages to it's own operation.
2024-02-22 19:32:46 +00:00
Radosław Waśko
d817df94a5
Move Enso_Cloud from Data to top-level in Standard.Base (#9140)
- 4 weeks ago we discussed with @jdunkerley we may want to move this module.
- `Data` is a submodule mostly for various data-structures etc., the cloud stuff did not fit that well in there.
2024-02-22 14:42:35 +00:00
Pavel Marek
21e1284086
Enso tests can be run with filter from cmdline (#9065)
Simplify the `Test.Suite.run_with_filter` to accept a single filter parameter that searches for all the groups and specs that matches that filter. This filter can be a simple text provided from the command line.

# Important Notes
- Pending groups are now printed at the end of the run
- `Test.Suite.run_with_filter` is simplified to accept a single filter parameter that is either `Text` or `Nothing`. See the docs.
- Passing a filter from the command line is therefore straightforward, it is treated as a regex.
- For convenience, I have left all the `main` methods in all the test sources. I have just refactored them to accept the `filter` argument from the command line.
- For example, to run only a single spec from `Vector_Spec.enso`, invoke `enso --run test/Base_Tests/src/Data/Vector_Spec.enso "should allow vector creation with a programmatic constructor"`
- **Majority of the PR is a regex replace** of `^main =` for `main filter=Nothing =` and of `suite.run_with_filter` for `suite.run_with_filter filter`.
- **Fixed some internal engine bugs:**
- `AtomWithHole` allows to specify only one hole - https://github.com/enso-org/enso/pull/9065/files#diff-0f7bb7e85cf86a965de133aa7e6b5958ceb889bd1921c01e00d3a9ceb19626ef
- NaN keys in hash maps are handled in polyglot maps as well - c5257f6c2b78f893214ff67300893b593ea05e21..db4b3c0e9828ee79208d52e02586b24bb845b0d6
2024-02-22 12:31:44 +00:00
James Dunkerley
e50ded8b24
Rename new_name to as on Aggregate_Column. (#9135)
Small tweak renaming a property.
![image](https://github.com/enso-org/enso/assets/4699705/21f4ca29-2219-412b-a650-c900360f9dea)
2024-02-22 10:34:00 +00:00
Radosław Waśko
d845e709a1
Refactor S3 path handling (#9092)
- Closes #9021
2024-02-22 09:15:34 +00:00
James Dunkerley
fa6fccb99e
Refactoring Table.set for GUI2 and other GUI fixes (#9128)
- Added `regex` function to `Standard.Base` as a way to easily and cleanly make regular expressions.
- Added `expr` and `Expression.Value` to distinguish expressions from text values.
- Fixed issues with `Table.join` widget so dropdown exists.
- Needed fully qualified name.
- Added default empty text values for right column to provided text input boxes.
- Deprecate `Table.filter_by_expression` and allow `Table.filter` to take an `Expression`.
- Added `Simple_Expression` and deprecated `Column_Operation`. Changes the order so takes a column then a calculation.
- Rename `column` to `value` and `new_name` to `as` on `Table.set`.
- Rename `name_column` to `names` on `Table.cross_tab`.
- Removed `Column_Ref.Expression` in favour of using `Expression.Value`.
2024-02-21 23:51:41 +00:00
AdRiley
4cbdc9dfb6
Remove table aliases (#9098)
Now Table and DB_Table have different names we can remove a lot of aliases. This closes #8981
2024-02-20 01:03:57 +00:00
Radosław Waśko
167a91dab1
Updating Enso Cloud integration - cheaper Enso_Secret.create, introducing Enso_File.creation_time and last_modified_time (#9085)
- Closes #9047
2024-02-19 18:53:20 +00:00
AdRiley
27ec236c19
rename database Table to DB_Table (#9067)
Rename database.Table to database.DB_Table to avoid name collisions as part of https://github.com/enso-org/enso/issues/8981
2024-02-19 17:31:58 +00:00
James Dunkerley
ee66b9fb1d
Refactoring the Unary operations so uncoupled from Storage. (#9090)
In order to allow clever masking, slicing, filtering and arrow backing stores...

- Adding ColumnStorage interface with the base API a storage will need.
- Refactored each of the unary operations to a new `UnaryOperation` interface which makes them responsible for deciding if they can be executed.
2024-02-19 17:11:52 +00:00
Jaroslav Tulach
a664dd9d56
Equality with conversions (#9070) 2024-02-19 17:18:56 +01:00
Jaroslav Tulach
f1d4e546d5
Upgrading to GraalVM 21.0.2 (#8883)
Upgrades to latest GraalVM 21.0.2
2024-02-19 12:08:59 +00:00
Radosław Waśko
cf71a05c3b
Refactor common context handling pattern (#9087) 2024-02-19 11:59:42 +00:00
Radosław Waśko
642d5a691e
Implement copy_to and move_to for S3_File (#9054)
- Closes #8833
- Tests for copying between S3 and `Enso_File` will only be added once we implement Enso_File writing.
2024-02-16 10:42:28 +00:00
James Dunkerley
f2d2f73e89
Starting to refactor Storage and Operations (#9076)
Cleaning up some of the structures in Storage before working on UnaryOperations.

- Removed some legacy code: `countMask`, `Index` and `DefaultIndex`.
- Renamed `mask` to `applyFilter` on `Column` and `Storage`.
- Renamed `Table.mask` to `Table.filter`.
2024-02-15 18:21:07 +00:00
AdRiley
f4955815b2
Remove Column Aliases (#9056) 2024-02-14 19:06:29 +00:00
James Dunkerley
08584b0423
Improving experience with format and parse. (#9045)
- Add format dropdown to `Number.format`.
![image](https://github.com/enso-org/enso/assets/4699705/8aa74910-c6ad-4480-a7f2-04dacd9686e8)

- Support case insensitive month names and abbreviations in dates.
https://github.com/enso-org/enso/assets/4699705/4dbd8755-e1c2-4207-a8a1-65b427ca4fab

- Improve locale dropdown for `parse_date` and `parse_date_time`.
![image](https://github.com/enso-org/enso/assets/4699705/5d605a2d-1248-46be-bc74-34a4afecf609)

- Added dropdown to `Table.parse` and amended so now doesn't accept `Nothing` (using empty string instead).
![image](https://github.com/enso-org/enso/assets/4699705/340dd093-77db-4685-a34b-45ce09e3c3b3)

- Added dropdown to `Table.format` and amended so now doesn't accept `Nothing` (using empty string instead).

- Altered `Column.parse` to not accept `Nothing` and added drop down for format.
- Altered `Column.format` to not accept `Nothing` and added drop down for format conditional on type.
- Improved the locale date/time format drop to have the suggested formats too.
2024-02-14 16:30:37 +00:00
Radosław Waśko
d45f0fe4df
Check type of self in static dispatch (#8867)
- Fixes #8805
- also btw. fixes #8706
2024-02-14 15:50:13 +00:00
AdRiley
5c7947ce60
Rename database column to DB_Column (#9046)
Rename database column to DB_Column to avoid name collisions as part of https://github.com/enso-org/enso/issues/8981
2024-02-14 09:11:52 +00:00
Cassandra-Clark
88c1cc14f7
8830 add google analytics authentication and reporting to enso (#8907)
Updates Google_Api version for authentication and adds Google Analytics reporting api and run_google_report method.

This is an initial method for proof of concept, with further design changes to follow.

# Important Notes
Updates google-api-client to v 2.2.0 from 1.35.2
Adds google-analytics-data v 0.44.0
2024-02-13 16:23:48 +00:00
AdRiley
c4701a9bd3
Rename Builder to SQL_Builder (#9032)
* Rename Builder to SQL_Builder

* Missed one
2024-02-13 11:04:52 +00:00
James Dunkerley
8c197f325b
Update the aggregate API to take a separate group_by (#9027)
Separate out the `Group_By` from the column definition in `aggregate`.
![image](https://github.com/enso-org/enso/assets/4699705/6b4f03bc-1c4a-4582-b38a-ba528ae94167)

Supports the old API with a warning attached about deprecation:
![image](https://github.com/enso-org/enso/assets/4699705/0cc42ff7-6047-41a5-bb99-c717d06d0d93)

Widgets have been updated with `Group_By` removed from the dropdown.
2024-02-13 10:23:59 +00:00
Radosław Waśko
eb59b475f6
Write support for S3 (#8921)
- Closes #8809
2024-02-12 19:04:13 +00:00