Commit Graph

964 Commits

Author SHA1 Message Date
Jaroslav Tulach
fe2cf49568
Run whole test/Base_Tests in native image runner (#10296) 2024-06-21 06:03:53 +02:00
Hubert Plociniczak
114b3a5c5e
Cleaning up YAML parsing in preparation for circe-yaml removal (#10309)
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.

As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.

Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.

In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
2024-06-20 13:07:54 +00:00
James Dunkerley
5042592d24
Some formatting issues and a few tweaks (#10298)
- Linting fixes.
- Adjust doc comments with `<` or `>` in plain text areas to use `&lt;` and `&gt;`.
- Refactor Statistics module and add auto-scoping.
- Add auto-scoping to `Text.to_case`.
- Fix type issue with `Table.get_value`.
- Private constructor for `Excel_Workbook` and move `xls_format` to public method.
- Add `fields` widget to `to_table`.
- Add `simple_expr` to make `Table.set` clearer.

![image](https://github.com/enso-org/enso/assets/4699705/3e21e800-142c-4006-a6c2-dd6196b76c9a)

![image](https://github.com/enso-org/enso/assets/4699705/d40dcbfd-a35e-4849-9e1a-f4a418d562dd)
2024-06-20 10:44:36 +00:00
Radosław Waśko
a8358512ad
Small fixes to Cloud Integration (#10303)
- Includes HTTP method in error message
- Does not do special handling for `403` status code - this was wrong and led to `Unauthorized` error when the real cause was lack of permssions in the Cloud. The errors should be more understandable now.
- Adds `projectSessionId` to audit log metadata.
- Fixes a test (`Secrets_Spec`) that did not have unique names and would fail if cleanup of previous runs failed (or if ran in parallel).
2024-06-18 09:41:33 +00:00
marthasharkey
bb16db9c79
Remove legacy matrix and legacy objects from table visualization and remove the index column (#10260)
- remove legacy matrix and object types from vue component and any further code relating to those
- remove the index and index header being sent in the json for tables
- add has_index_col flag for json hat previously sent 'indicies_header' and 'indicies' so that index/# column is still rendered where required
2024-06-17 09:17:20 +00:00
Radosław Waśko
3a4784c226
Initial separation of Snowflake_Dialect from Postgres_Dialect (#10266)
- Part of #9486
- Building on top of initial work by @jdunkerley and finishing it
- Reverted the changes to the Postgres_Dialect from last Snowflake work and split the Snowflake_Dialect into a separate module.
- Moved from `rounding_decimal_places_not_allowed_for_floats` to `supports_float_round_decimal_places` (as too confusing).
- Added Snowflake_Dialect type.
- Extracted `Snowflake_Spec` into separate `Snowflake_Tests`
- It imports the common tests from `Table_Tests`.
- Some initial adaptations to make the snowflake dialect not-crash.
- Adding `Internals_Access` proxy to allow external implementations to access our internal data structures without directly exposing them to users. Users should not use these.
- Adding profiling of SQL to check performance.
2024-06-13 16:12:20 +00:00
AdRiley
fadb81abe6
Fix Text_Cleanse tests (#10263)
I hadn't connected the Text_Cleanse tests up properly and as a result they weren't running or working. This fixes that.
2024-06-13 08:25:32 +00:00
Hubert Plociniczak
d6d370925a
Removing dependencies to speed up startup (#10249)
Reducing the number of dependencies. Explicit `cats` are almost gone (present in `cli`). `enumeration` is completely gone.  `cats` is also still included implicitly via `io.circe` but that's a different kind of beast.
Also, really removed `jackson` from dependencies by fixing the dependency on `http-test-helper`.

# Important Notes
In a number of places importing all cats implicits could be simply replaced with a single or two method calls. Not to mention that this will reduce compilation times due to reduced implicit search space.

One example of how the changes affect performance (not only startup):

Before:
![Screenshot from 2024-06-11 12-05-24](https://github.com/enso-org/enso/assets/292128/a1a772a9-635d-4a16-a543-e2fd2124a22c)
Now:
![Screenshot from 2024-06-11 14-27-47](https://github.com/enso-org/enso/assets/292128/b17c7fcc-9a6d-48b9-8200-60708354ee03)
(frequently executed)

![Screenshot from 2024-06-12 12-46-34](https://github.com/enso-org/enso/assets/292128/31bc4dfd-4edc-45c9-9c5d-13e3472089b9)
Also appears to be gone.

This PR is by no means finished. The purge will continue in follow up PRs.
2024-06-12 18:15:36 +00:00
marthasharkey
55af1b9ffd
Add icon to indicate value type in table (#10225)
closes #10018

Sends the value type within the json for table visualisation
Header uses a html template to show it's value type to the right of the title
Displays the value type also in a tooltip, this displays the types "display_text"

<img width="392" alt="image" src="https://github.com/enso-org/enso/assets/170310417/0828e6a2-b30f-4ac7-9a8f-46b4a9cfac91">


tooltip:
<img width="498" alt="image" src="https://github.com/enso-org/enso/assets/170310417/f9964f90-9337-42d3-a0ef-3c58f6d74621">
2024-06-12 09:06:55 +00:00
AdRiley
783276ca34
Make test framework list out failed tests to use in filter (#10241)
* List out failed tests

* Code review changes
2024-06-11 21:15:28 +01:00
Radosław Waśko
c4b0ca8f69
Ensure type ascriptions are correctly transformed to facilitate checking of more complex return types (#10229)
- Fixes #9980
- Adds some tests to ensure types like `|` or `&` (in addition to `!` from the ticket) correctly work in return type check.
- Fixes a weird behaviour where we used to avoid processing type related IR transformations inside of type ascriptions.
- Adds parentheses to type representations if they are more complex: `A | B & C` is unclear as it can either mean `A | (B & C)` or `(A | B) & C` which have different meanings. If we now have an operation with such nesting, the sub expressions are wrapped in parentheses to disambiguate.
2024-06-11 17:11:03 +00:00
Hubert Plociniczak
4da5e611c9
Replace Jackson serde (#10035)
JSON serialization setup between Language Server and Runtime is a major contributor to startup time. This PR experiments with an alternative implementation that remedies the problem.
The new serializer uses [jsoniter-scala](https://github.com/plokhotnyuk/jsoniter-scala) which by some accounts claims to be really fast. In our case, more importantly, we pay negligible cost of startup setup compared to Jackson which was horribly slow.

# Important Notes
Before:
![Screenshot from 2024-06-06 15-35-18](https://github.com/enso-org/enso/assets/292128/56103b82-777e-459f-966e-abdef25c2430)
After:
![Screenshot from 2024-06-06 15-35-02](https://github.com/enso-org/enso/assets/292128/00a36647-dfae-4dc8-a1b7-bf69069ef109)

Yes. About 0.8sec.
2024-06-11 15:03:12 +00:00
Pavel Marek
5fa29c51b5
Fix Meta.enso_project (#10192)
Fixes `Standard.Base.Meta.Enso_Project.enso_project` to return a project descriptor for the *main* project, i.e., the one configured as a *root* for the engine.

# Important Notes
`enso_project` builtin no longer iterates the stack frames to infer the project descriptor. It derives it from the default package repository.
2024-06-11 08:31:40 +00:00
Radosław Waśko
41d02e95ef
Implement Windows-1252 fallback logic for Encoding.Default (#10190)
- Closes #10148
- [x] Tests for `Restartable_Input_Stream`, `peek_bytes` and `skip_n_bytes`.
- [x] Report `Managed_Resource` stack overflow bug: #10211
- [x] Followup possible optimization: #10220
- [x] Test use-case from blog.
2024-06-10 10:49:26 +00:00
James Dunkerley
d938c96c55
Adding type annotations and enabling auto-scoping (#10173)
- Renamed `Missing_Required_Argument` to `Missing_Argument`, and added `throw` method.
- Add default widget to `Case_Sensitivity.Insensitive locale`.
- Switch to auto scoping for `parse_type_selector`.
- Add type annotation to various simple typed arguments in `Table` and `DB_Table`.
- Altered `Filter_Condition` to have `Missing_Argument` for all non-defaulted arguments.
- Added resolution of `Column_Ref` passed as auto-scoped to `Table_Ref`.
- Altered `Simple_Calculation` to have `Missing_Argument` for all non-defaulted arguments.
- Altered `Join_Condition` to have `Missing_Argument` for all non-defaulted arguments.
- Altered `Sort_Column` to have `Missing_Argument` for all non-defaulted arguments.
- Altered `Aggregate_Column` to have `Missing_Argument` for all non-defaulted arguments.

**rename_columns:**
![image](https://github.com/enso-org/enso/assets/4699705/08aaba0f-687a-450c-9781-8eadc062bd50)

**aggregate:**
![image](https://github.com/enso-org/enso/assets/4699705/c29e7944-1a1c-4020-9fe0-528d874b8049)

**join:**
![image](https://github.com/enso-org/enso/assets/4699705/50038166-e56d-48c5-9eeb-bd46fa415e46)

**set:**
![image](https://github.com/enso-org/enso/assets/4699705/bee2462a-dafb-4bd4-b102-ec73edb4fb93)
2024-06-10 07:52:32 +00:00
GregoryTravis
4aa3d52b60
Implement conversions for Decimal column (#10206)
* treat scale nothing as unspecifed

* cast to decimal

* float int biginteger

* conversion failure ints

* loss of decimal precision

* precision loss for mixed column to float

* mixed columns

* loss of precision on inexact float conversion

* cleanup, reuse

* changelog

* review

* no fits bd

* no warning on 0.1 conversion

* fmt
2024-06-07 15:37:32 -04:00
AdRiley
fe6eafd06e
Add By_Type option to more components (#10183)
* reorder_columns

* format

* cast

* auto_value_types

* replace

* fill_nothing

* fill_empty

* text_replace

* text_cleanse

* Add By_Type

* Fix cast

* Fix more tests

* Fix the table helper

* Bug fix

* Remove check
2024-06-07 16:26:23 +01:00
Hubert Plociniczak
1bc14252df
Extract mutable builder from ModuleScope (#9914)
Refactored mutable parts of `ModuleScope` into builder to make it easier to reduce unnecessary locks.

# Important Notes
Elements of ModuleScope (types, imports etc) are used while _building_ of it may still be in progress. In order to make static typing happy, every `ModuleScope.Builder` can be exposed as (unmodifiable) `ModuleScope`.
2024-06-05 16:57:08 +00:00
Radosław Waśko
698f4b5640
Remove workaround for #8937 after it has been fixed (#10149)
- Since #8937 was fixed by #10104 the workaround is no longer needed
2024-06-05 14:23:12 +00:00
Jaroslav Tulach
8332118ff4
What's the name of my method? (#10164)
While working on #10056 I realized the names of method and closure nodes are incomprehensible to anyone. This PR replaces the infamous `<anonymous>` with a name hinting where the method actually is.

# Important Notes
I assume this change will be visible not only in IGV, but also in _stacktraces_ and we may need to adjust few tests.
2024-06-05 07:49:53 +00:00
GregoryTravis
5fad3558a6
BigDecimalBuilder and arithmetic operations. (#9950)
* hack

* make a column

* add

* no scale=0 on BD type

* a test

* wip

* 3 arithmetic ops

* /

* wip

* BigDecimalPowerOp

* wip

* mod test

* NumericBinaryOpReturningBigDecimal

* with scalar

* misc arithmetic tests

* fix integralBigDecimalToInteger

* mixed columns

* bigdecimal pow via double

* cleanup

* j2e on get

* arithmetic exception

* mod 0

* cleanup

* fmt

* changelog

* check type first

* merge

* mc error message

* add BD case to Builder.java

* fmt

* changelog

* add BD case to StorageConverter.java

* fmt

* fix test
2024-06-04 13:59:31 -04:00
AdRiley
7c35781a14
Add select/remove column by type (#10159)
* stash

* Working first pass

* Tests

* remove columns

* more tests

* More tests

* DB_Table

* In DB

* Deprecate

* Update documentation

* More docs

* Remove strict

* Remove tests

* Fix widgets

* Add doc

* Spaces

* Fix In-DB
2024-06-04 16:23:46 +01:00
marthasharkey
24d209abf9
Add Columns_To_Add type to move away from Integer | Nothing (#10152)
Adds new type "Coulmns_To_Add" that replaces the Integer | Nothing type
And adds new widget to split_to_columns and tokenise_to_columns
<img width="397" alt="image" src="https://github.com/enso-org/enso/assets/170310417/1b155682-0992-4cc0-8964-62b389ee7072">

Closes #10041
2024-06-04 15:06:08 +00:00
Radosław Waśko
7cf80f3196
Handle UTF BOM when decoding text (#10130)
- Improve BOM handling: detect and skip the BOM character, Default encoding that detects encoding based on BOM if present, warnings if unexpected BOM is encountered.
- Closes #9849
- Windows-1252 fallback will be done as a separate PR as it has additional complexity. Tracked in ticket #10148.
2024-06-04 13:22:19 +00:00
GregoryTravis
1c2955655c
Add no_warning_propagation flag to Vector.build and Builder.new (#10079)
* no_warning_propagation flag

* doc

* benchmarks

* change to propagate_warnings, test

* fix java benchmark

* add benchmark to main
2024-06-04 09:06:09 -04:00
James Dunkerley
f1a1901058
Bug fix for read by line. (#10161)
Needs to pass -1 not Nothing to JSON.
2024-06-03 21:33:45 +00:00
Jaroslav Tulach
9632f04e9b
Directly allocate warnings asVectorEnsoObjects (#10143) 2024-06-01 06:23:49 +02:00
AdRiley
af4177e61f
Make table default visualisation (#10128)
Makes table the default vis type.

Before:
![image](https://github.com/enso-org/enso/assets/1720119/8a5d113a-ac06-4dd8-afdb-151cdbc149ab)

After:
![image](https://github.com/enso-org/enso/assets/1720119/d7c33cf7-355d-4517-8967-1963ff506f97)

Closes #10093
2024-05-31 15:02:02 +00:00
James Dunkerley
ffdfc7195e
Linter and small widget change (#10136)
Linter issues to allow doc site generation.
Small change in order for aggregate column.
2024-05-31 12:30:38 +00:00
AdRiley
06327f8fde
Add statistic product (#10122)
Add Statistic.Product

![image](https://github.com/enso-org/enso/assets/1720119/f7fc7bb5-9efe-4dbe-9150-cd9e5101c553)
2024-05-31 09:29:52 +00:00
Radosław Waśko
233f28235a
Small fixes to Postgres integration (#10105)
- Better message when saving datalink in disabled Output context:
![image](https://github.com/enso-org/enso/assets/1436948/540d615b-79ff-4811-8262-a0475a7b6923)
Before it was:
![image](https://github.com/enso-org/enso/assets/1436948/51198bf1-1e50-41bc-a56b-f829bc32d09a)

- Hack to get Postgres widget to display connection options:
![image](https://github.com/enso-org/enso/assets/1436948/39f3db39-1163-4815-b59f-c629d812e2ab)
Before the `Postgres` constructor was created without any parameters and it was not showing any parameters for modification.
2024-05-28 14:34:44 +00:00
Dmitry Bushev
858e646328
Start Ydoc with the language server (#9862)
- related #7954

Changelog:
- update: Ydoc starts with the language server on the `localhost:1234` by default. The hostname and ports can be configured by setting environment variables `LANGUAGE_SERVER_YDOC_HOSTNAME` and `LANGUAGE_SERVER_YDOC_PORT`
- update: by default `npm dev run` uses the node Ydoc server. You can control it with `POLYGLOT_YDOC_SERVER` env variable. For example,
```
env POLYGLOT_YDOC_SERVER='true' npm --workspace=enso-gui2 run dev
```
To connect to the Ydoc server running on the 1234 port (the one started with the language server)
⠀
```
env POLYGLOT_YDOC_SERVER='ws://127.0.0.1:1235' npm --workspace=enso-gui2 run dev
```
To connect to the provided URL. Can be useful for debugging when you start a separate Ydoc process.
- update: run `npm install` before the engine build. It is required to create the Ydoc JS bundle.
2024-05-28 13:51:42 +00:00
AdRiley
28bd5c522b
Add named_pattern and make it usable in Text_Replace (#10040)
Creates a new type Named_Expression and allows it to be used in text_replace

![image](https://github.com/enso-org/enso/assets/1720119/673a62a1-1ce5-4e1f-8289-0fa10e87b9de)

![image](https://github.com/enso-org/enso/assets/1720119/2f555e9b-d8c5-41e3-8000-d959d5818666)

![image](https://github.com/enso-org/enso/assets/1720119/d2ae2b0f-5bc4-4e9e-a391-402b58ee72d5)

![image](https://github.com/enso-org/enso/assets/1720119/3bf0547f-c2c7-4987-a45e-d922be2b0bae)
2024-05-27 13:01:26 +00:00
James Dunkerley
ab4b1f0f35
Add day_of_week and day_of_year to Column and DB_Column (#10081)
- Adds support for getting the weekday as an integer (1 Monday - 7 Sunday - ISO standard).
- Add support for getting the day of year.
2024-05-27 11:29:25 +00:00
James Dunkerley
200da1ad50
Remove old GUI1 ICONs and mapping function (#10086)
* Remove old GUI1 ICONs and mapping:

```
  dataframe_clean: 'table_clean',
  dataframe_map_row: 'map_row',
  dataframe_map_column: 'column_add',
  dataframes_join: 'join2-1',
  dataframes_union: 'union',
  sigma: 'transform4',
  io: 'in_out',
  date_and_time: 'time',
  spatial: 'location',
  predictive: 'predict',
  machine_learning: 'robot',
  split_text: 'split',
```

* Linter.

* Linter (2).

* Sort out some Returns:

* Sort out some ? Example
2024-05-27 11:17:21 +01:00
James Dunkerley
d8059fd22c
Some tweaks following Steve's testings (#10042)
- Add ranged number widget to `at` and `get`.
- Add defaults to `at` and `get` picking the first item.
- PRIVATE on various Excel_Workbook methods. It still works like a connection but not shown in CB.
2024-05-27 09:04:29 +00:00
AdRiley
adfe2dbab4
Add expression alias (#10067)
We call formulas expressions so having expression as well as formula makes sense as an alias.
2024-05-24 14:13:00 +00:00
AdRiley
084be279c5
Fix input_number icons (#10066)
icon is called input_number not number_input

![image](https://github.com/enso-org/enso/assets/1720119/c68790d3-05ad-4d17-8423-df3634fff29a)
2024-05-24 11:30:06 +00:00
Adam Obuchowicz
8ac0ee41c6
Allow picking non-existing files in write component (#10047)
Fixes #9984

Added a flag to `File_Browse` widget configuration specifying if we require an existing file. Also, the FileBrowserWidget will match against all "Writable_File", also displaying Save dialog.
2024-05-24 09:21:32 +00:00
Radosław Waśko
9d75f79ff9
Cloud Integration updates: renames in file metadata structure, remove path resolver workaround, partial fix for datalink resolver issue (#10050)
- Supersedes #9966 as I wanted to test these changes in one go.
- Fixes #10037 caused by lack of CI check and my oversight (forgot to run full tests after a minor change).
- Fixes a regression after [file metadata fields were renamed](c09d856ac8 (diff-9f59b6a0ee3155efecdc70c1ea0c90ab5cde00b5623d84363118b1793f941c46R2037)).
- Fixes handling of creating new datalinks and using them after cache was cleared (e.g. workflow restart).
- This was caused by troubles with path resolver.
- The fix addresses the most common issue and adds a test for it (test flushes the caches to ensure path resolver is used instead of the cached value).
- Some related issues were discovered on the cloud side, tracked by https://github.com/enso-org/cloud-v2/issues/1252
2024-05-23 16:05:48 +00:00
Jaroslav Tulach
16c1b74218
Enso Library Feature to execute (a bit of) Base_Tests (#9997) 2024-05-23 08:20:19 +02:00
Radosław Waśko
1e0649fda1
Improvements to Table.union (#9968)
- Closes #9952
2024-05-22 09:38:10 +00:00
marthasharkey
517299bb09
Add new aliases for take and drop methods (#10022)
Adds the aliases 'keep' and 'remove' to take/drop methods
2024-05-22 08:02:48 +00:00
James Dunkerley
2677e556e8
Align headers in doc comments... (#10025)
- Always use `Arguments:` for parameters or fields.
- Always use `> Example` for examples.
- Align sections behind `!` or `?` headers.
- Whitespace fixes.
2024-05-21 19:44:58 +00:00
AdRiley
3b59b40657
Update test framework to default to only show failures (#9986)
* Update test framework to default to only show failures

* Fix
2024-05-21 11:23:23 +03:00
James Dunkerley
31dd5944d7
Some fixes to make linter happy (#10004)
- A few import orderings.
- Missing doc comments on Text_Cleanse.
- Indent correction.
2024-05-20 16:15:11 +00:00
AdRiley
c7476c10f3
Add data cleanse component (#9879)
* Signature

* More API work

* Red

* Still red

* Green

* Red

* Green

* Red

* Green

* Refactor

* Refactor

* Refactor

* Red

* Green

* Red

* Green

* Red

* Green

* More tests

* non-ascii

* Ordering tests

* Remove tabs

* Numbers and Letters

* Changelog.md

* Add documentation

* Tests for non-text columns

* Add punctuation

* Add symbols

* Fix

* Refactor

* Refactor

* Move to base

* Fix

* Start of in-db tests. Not working

* DB versions

* Update widget

* Fix widgets

* Move tests to base

* Code review changes
2024-05-20 09:33:18 +01:00
James Dunkerley
3b29de313d
Update ICONs, linting fixes and add select_columns ALIASes. (#9985)
- Linting fixes.
- Add ICONs to all functions missing them.
- Add ALIASes for select columns.
2024-05-17 19:24:17 +00:00
GregoryTravis
4d49b00375
Combine builders for Vector.build and Vector.new_builder (#9922)
We have decided to keep the old new_builder, and to combine the new and old builders into one builder.
2024-05-17 16:18:47 +00:00
AdRiley
5c9ee94812
Make parse_fast_json private (#9978)
Micro MR to make deprecated method private
2024-05-17 15:47:11 +00:00
Radosław Waśko
38ad7b0afa
Creating datalinks from code (#9957)
- Closes #9673
- Adds the ability to save an existing Postgres connection as a datalink into the Enso Cloud, automatically promoting plain-text passwords into a Secret.
- Fixes dataflow error propagation in `JS_Object.from_pairs`.
2024-05-16 13:29:41 +00:00
Radosław Waśko
a2481036e6
Prepare for the Path resolver change to GET (#9958)
- Prepares for the Cloud API change, together with a fallback for the old API to avoid problems during migration.
- This PR should be merged before the https://github.com/enso-org/cloud-v2/pull/1236 PR is _deployed_.
2024-05-15 13:00:32 +00:00
James Dunkerley
b2aeb9fc84
Moving away from Integer | Nothing to Rows_To_Read for limiting number of rows. (#9925)
- Added a new `Rows_To_Read` type with conversions from `Nothing` and integers.
- Updated `read` on `Table`, `Column`, `DB_Table` and `DB_Column`.
- Updated `Delimited_Format.Delimited` to use `Rows_To_Read` for `row_limit`.
- Updated `Excel_Format.Sheet` and `Excel_Format.Range` to use `Rows_To_Read` for `row_limit`.
- Updated `Excel_Workbook.read` to use `Rows_To_Read`.
- Updated `Connection.read` (in all connection types) to use `Rows_To_Read`.

![image](https://github.com/enso-org/enso/assets/4699705/553c027f-f4c3-4855-9f51-2c4bcaec48a0)

![image](https://github.com/enso-org/enso/assets/4699705/a06c3912-77e0-4c10-abb8-73aed667458d)
2024-05-14 16:31:26 +00:00
James Dunkerley
2d6735edc7
Allow a Table to be used to rename_columns. (#9940)
Following an example from Steve, adjusted `rename_column` to allow a `Table` to define the mapping.

- A single Text column table gives a list of new names.
- A two Text column table gives a map from old name to new name.
- If a `DB_Table` is passed errors and tells the user to materialize the table.

![image](https://github.com/enso-org/enso/assets/4699705/3ed98330-1fce-465e-bf96-4a86e0872dd3)
2024-05-14 15:57:10 +00:00
Radosław Waśko
5f0a16c87c
Audit Logs for Postgres connections opened through a data link (#9873)
- Closes #9599
- Implemented API for sending audit logs to the cloud on a background thread.
- If the Postgres connection is opened through a datalink, its internal JDBC connection is replaced by a wrapper that reports executed queries to the audit log.
- Also introduces `EnsoMeta` - a helper Java class that can be used in our helper libraries to access Enso types.
- I have replaced the common pattern scattered throughout the codebase with calls to this 'library' to avoid repetitive code.
- Refactored `Table.display` to share code between in-memory and DB - it was needed as the function stopped working for `DB_Table` after adding making the `Table` constructor `private`.
- Clearer error when reading a SQLite database from a remote file (tells the user to download it first).
- Follow up - correlate asset id of the data link:
#9869
- Follow up - include project name (once bug is fixed):
#9875
- Some problems/improvements of the audit log:
- The audit log system is not yet ready for high throughput of logs
#9870
- The logs may be lost if `System.exit` is used
#9871
2024-05-11 08:54:33 +00:00
James Dunkerley
d97754da17
Error messages for rename_columns and Vector.duplicates (#9917)
- Improve error message for `rename_columns`.
- Add `length` to `Set` and `Map`.
- Add `duplicates` to `Vector` (and `Array`).

![image](https://github.com/enso-org/enso/assets/4699705/623df253-52e8-4bdc-a69c-ac8dc3ca594e)
2024-05-10 17:43:50 +00:00
GregoryTravis
720d32cbe3
Convert most remaining uses of Vector.new_builder to Vector.build (#9891) 2024-05-09 15:52:53 +00:00
Radosław Waśko
1d8fc1bf6e
Update endpoint name after it was updated in the Cloud (#9912)
Micro PR updating our API integration to the Cloud-side changes.
2024-05-09 15:17:19 +00:00
AdRiley
e25ec96aaa
Add table running variance skew sd and kurtosis (#9854)
Adds support for Variance, Skew, Standard Deviation and Kurtosis to Table.Running.
2024-05-09 08:45:29 +00:00
James Dunkerley
ce360f9002
Fix the group colours for consistent colouring. (#9893)
Fix the group colour to a fixed set.

![image](https://github.com/enso-org/enso/assets/4699705/476046ce-0ce3-4f98-9a75-a51f2e55f4e6)
2024-05-08 16:52:42 +00:00
James Dunkerley
4e758e5c9d
Undo change to autoscoping. (#9881)
Can't use auto-scoping for filter yet.
2024-05-08 07:52:12 +00:00
GregoryTravis
a2c36def26
Add Vector.build_multiple and handle dataflow error and warning propagation in build and build_multiple (#9766) 2024-05-07 19:30:21 +00:00
James Dunkerley
c0fd6eed2d
Small tweaks from Ned, Mark, Cass and Steve's feedback. (#9859)
- Rename `.info` to `.column_info`.
- Rename `.select_by_type` to `.select_columns_by_type`. (to be merged into `select_columns` functionality)
- Rename `.remove_by_type` to `.remove_columns_by_type`. (to be merged into `remove_columns` functionality)
- Adding "field" `ALIAS`es.
- Making `Table.Value` private (and hence hiding `java_table`).
- Making `Column.Value` private (and hence hiding `java_column`, added method to `Storage` to allow tests to work).
- Making `DB_Table.Value` private (and hence hiding `connection`, `context` and `internal_columns`).
- Making `DB_Column.Value` private (and hence hiding `connection`, `sql_type_reference`, `expression` and `context`).
- Add widgets for everywhere `Filter_Condition` used (fix missing entry for lower, can't use auto-scoping).
![image](https://github.com/enso-org/enso/assets/4699705/bad7d5c8-cd01-4384-a1c8-dbcd7a5ad92b)
- Fix for widgets in `Simple_Calculation`: `date_add` and `format`.
![image](https://github.com/enso-org/enso/assets/4699705/14eeef2e-5069-4a88-8b68-eb675497addf)
![image](https://github.com/enso-org/enso/assets/4699705/75d4e097-f171-4bf6-a27e-f9d396c44a92)
- Fix for nested `Row`, `Column` and `DB_Column` rendering in Table Viz.
![image](https://github.com/enso-org/enso/assets/4699705/8c50c519-7c34-4228-bab7-be938a6ce1bf)
![image](https://github.com/enso-org/enso/assets/4699705/c83ad301-f31e-451d-a8be-b022e1456b4e)

# Important Notes
- Can't use `.` to access a private constructor field when mapping, have to use a lambda.
- We should agree convention for private backing fields which are wrapped by a public getter (went with `internal_` for this pass).
2024-05-06 13:49:12 +00:00
Dmitry Bushev
a44bb2b1b1
Language Server API for AI (#9679)
close #9656

Changelog:
- add: `ai/completion_v2` method
- add: `Visualization.AI.print` method for converting the expression to text format
- update: The default system prompt was updated to tell AI to use the `Visualization.AI.print` method for printing.

# Important Notes
The project [New_Project_1.zip](https://github.com/enso-org/enso/files/15152993/New_Project_1.zip) contains the following main file:
```py
from Standard.Base import all
from Standard.Table import all
from Standard.Database import all
from Standard.AWS import all
import Standard.Visualization
import Standard.Visualization.Warnings
from Standard.Base.Errors.Common import Dry_Run_Operation

type Student
Value id region

main =
operator70395 = 226
operator47321 = 'east'
operator76980 = Student.Value operator70395 operator47321
operator31302 = operator47321.words True
operator91574 = 1
operator34358 = operator47321.take (Index_Sub_Range.By_Index [0, operator91574])



#### METADATA ####
[[{"index":{"value":0},"size":{"value":4}},"4cf8de7f-2014-4dfd-9ceb-0164fe26c8bf"],[{"index":{"value":0},"size":{"value":29}},"389fe7a5-e59b-440a-8801-e0bd86716094"],[{"index":{"value":0},"size":{"value":558}},"a8a81ce3-199e-479e-98dd-5f919425842d"],[{"index":{"value":5},"size":{"value":8}},"c6f0de99-3bd3-459d-a69b-5fc774dd1e3f"],[{"index":{"value":5},"size":{"value":13}},"05827411-7354-478b-99d8-8370dadd560b"],[{"index":{"value":13},"size":{"value":1}},"41de94cc-ccd2-4ccf-b9db-cf686443efa0"],[{"index":{"value":14},"size":{"value":4}},"8abd24ea-b1cb-4c10-ad16-5a190f7ed0c7"],[{"index":{"value":19},"size":{"value":6}},"5c57a62d-ed82-4195-8b78-2ed660556dd0"],[{"index":{"value":26},"size":{"value":3}},"23544334-1117-4dbc-ac7f-bb3f84575264"],[{"index":{"value":30},"size":{"value":4}},"6058ba89-dc1b-45e1-b8f3-d1a4d27975f7"],[{"index":{"value":30},"size":{"value":30}},"bdd4b4cc-9c1e-4554-b0fb-3f73dcd5a590"],[{"index":{"value":35},"size":{"value":8}},"7e7bb9b4-8e9b-4700-9dcd-83c353800da5"],[{"index":{"value":35},"size":{"value":14}},"524bafb6-d3ca-4ee5-b8de-84b5c47ca87b"],[{"index":{"value":43},"size":{"value":1}},"b68b3929-8b63-49ad-977c-b04e058f8ee4"],[{"index":{"value":44},"size":{"value":5}},"7173f374-5f53-462b-ab1b-f7b1845d729b"],[{"index":{"value":50},"size":{"value":6}},"911d381e-1bdb-4e9c-826d-a00495c8fd23"],[{"index":{"value":57},"size":{"value":3}},"2f8fa1ee-9d13-4a73-99b2-bc77d1656387"],[{"index":{"value":61},"size":{"value":4}},"11abd658-4888-476c-afcd-592edaaa53c1"],[{"index":{"value":61},"size":{"value":33}},"9f6ae949-410b-48e6-bb75-84d4ba8a1989"],[{"index":{"value":66},"size":{"value":8}},"f4726e40-dd5b-49a0-aa2a-6cea6df88d5d"],[{"index":{"value":66},"size":{"value":17}},"539b4dcf-4487-49b6-9c90-6ae58d51c2db"],[{"index":{"value":74},"size":{"value":1}},"a34a41a0-5dcf-4c47-a36c-a4f427c87c03"],[{"index":{"value":75},"size":{"value":8}},"8fdf4ee7-2e74-4cac-b9b9-4230183f91c5"],[{"index":{"value":84},"size":{"value":6}},"7ff7efdc-fed6-4cb6-bed6-392d110ff991"],[{"index":{"value":91},"size":{"value":3}},"eacff407-dc6d-4f17-a28b-329f7a0582e1"],[{"index":{"value":95},"size":{"value":4}},"f2feac9f-ea02-4070-879a-f1d493473993"],[{"index":{"value":95},"size":{"value":28}},"f9b37b67-fb54-4e47-8281-e922edb4c5d3"],[{"index":{"value":100},"size":{"value":8}},"6cf398f8-d0c9-42fa-a6e6-e740de11562f"],[{"index":{"value":100},"size":{"value":12}},"28e4d0e3-7da4-4ee2-94c7-d9a7c692f8de"],[{"index":{"value":108},"size":{"value":1}},"955e8788-816c-4eb7-b5ec-020b4863536a"],[{"index":{"value":109},"size":{"value":3}},"bdc65807-0438-4296-8b1f-1fab3a885c17"],[{"index":{"value":113},"size":{"value":6}},"fe3dd265-1855-4d66-8a2d-51153b21e699"],[{"index":{"value":120},"size":{"value":3}},"3b7a14ac-3224-4b9a-b474-f6136a658f8b"],[{"index":{"value":124},"size":{"value":6}},"7f7f18a4-06ef-46e0-bddd-b9fe60878855"],[{"index":{"value":124},"size":{"value":29}},"d49e0fe3-a638-4d7e-ba58-12db7abad20f"],[{"index":{"value":131},"size":{"value":8}},"c9411433-5e73-4927-9175-15d3b1ccb0ef"],[{"index":{"value":131},"size":{"value":22}},"b53ad817-ea4c-491e-aaf8-49967e22aff6"],[{"index":{"value":139},"size":{"value":1}},"837a4d65-8af6-4860-ae0e-ad9172af6ad2"],[{"index":{"value":140},"size":{"value":13}},"3b662906-1887-4dd6-a0e2-a222ace135ce"],[{"index":{"value":154},"size":{"value":6}},"548ca873-544f-48c9-b7ab-6aa1e7f03d72"],[{"index":{"value":154},"size":{"value":38}},"7c45872c-22b4-4f37-bcd7-b3791408f737"],[{"index":{"value":161},"size":{"value":8}},"21152f0d-4222-48e4-9376-a2a28f8f6be6"],[{"index":{"value":161},"size":{"value":22}},"1cb71d49-a06d-4dd9-bc1b-54603b416601"],[{"index":{"value":161},"size":{"value":31}},"faa46cf2-3e90-44ed-9d9e-77b7f874b338"],[{"index":{"value":169},"size":{"value":1}},"6d5fdcd5-0ddf-4c83-aafe-a446ac5ce461"],[{"index":{"value":170},"size":{"value":13}},"830d73c2-1898-41a9-8046-22d86f159ef0"],[{"index":{"value":183},"size":{"value":1}},"ab897380-c596-4182-a2c5-b7b23cf59893"],[{"index":{"value":184},"size":{"value":8}},"29f35cf2-d79d-44d0-96d0-79b11589726c"],[{"index":{"value":193},"size":{"value":4}},"45bf8220-52f2-49f4-bbcf-9566d1f1b1c4"],[{"index":{"value":193},"size":{"value":57}},"478005ac-f0ca-497a-b298-554bd64946a9"],[{"index":{"value":198},"size":{"value":8}},"c11511ce-252c-4b84-9f61-dfa5544d4916"],[{"index":{"value":198},"size":{"value":13}},"dbae65a1-bdca-44b0-a497-7934d1e42616"],[{"index":{"value":198},"size":{"value":20}},"355392ad-8c81-4304-a6d0-2bbf3690da0c"],[{"index":{"value":198},"size":{"value":27}},"0720ccb5-ebd3-4c29-8113-0aa1f251e873"],[{"index":{"value":206},"size":{"value":1}},"f5a8d310-1f58-456e-84a3-78490f05ff13"],[{"index":{"value":207},"size":{"value":4}},"10046e78-5164-407c-adef-2ab11ce77b4c"],[{"index":{"value":211},"size":{"value":1}},"ed23da0c-9193-4eea-ab9c-fee90ba8f924"],[{"index":{"value":212},"size":{"value":6}},"7a09de10-35e2-45e0-9eb8-f79fa054b733"],[{"index":{"value":218},"size":{"value":1}},"b85a0981-9d82-4204-a03a-cd3a4720c72d"],[{"index":{"value":219},"size":{"value":6}},"1b04c19d-cdf0-4661-8f4e-d980cf12c996"],[{"index":{"value":226},"size":{"value":6}},"87d57f2c-e009-48a7-98bb-0e1612905564"],[{"index":{"value":233},"size":{"value":17}},"60fd6a1c-7ada-4546-ae20-e1f8b432b4fd"],[{"index":{"value":252},"size":{"value":4}},"9558affa-b913-4576-8ee3-6af22985e0f6"],[{"index":{"value":252},"size":{"value":32}},"8479091c-9232-49d6-b375-32f89693c389"],[{"index":{"value":257},"size":{"value":7}},"3d1f13f5-e22a-4cb6-9e75-9c717e016922"],[{"index":{"value":264},"size":{"value":1}},"aa429f07-973a-4c69-8e3f-74aa4780f85a"],[{"index":{"value":269},"size":{"value":5}},"5b304cf6-5ec4-4b23-ad0d-20e38b41cdcd"],[{"index":{"value":269},"size":{"value":15}},"e3315ef2-d0ec-43b1-b8f6-5b436054aaee"],[{"index":{"value":275},"size":{"value":2}},"bc7205c4-68bf-461e-a1ba-31400b9337ff"],[{"index":{"value":278},"size":{"value":6}},"78870c05-9117-4e24-bf77-a72e7aef9c18"],[{"index":{"value":286},"size":{"value":4}},"517c743f-c775-4031-8f87-222d0f0a365f"],[{"index":{"value":286},"size":{"value":271}},"d0d107bc-4997-41d6-9c20-32295590eaac"],[{"index":{"value":291},"size":{"value":1}},"91174930-f18b-4322-84cc-deb88637d012"],[{"index":{"value":292},"size":{"value":265}},"f09a4372-3231-4f2e-99f4-84aa751f9b60"],[{"index":{"value":297},"size":{"value":13}},"613df8c9-0a40-4c94-886f-9668c2c360c2"],[{"index":{"value":297},"size":{"value":19}},"5b88a4d4-6840-4a25-a862-8b5d0b75303d"],[{"index":{"value":311},"size":{"value":1}},"6bfe1dea-09df-4358-9613-74d8326bc680"],[{"index":{"value":313},"size":{"value":3}},"e3b7fac7-0c08-4f20-8a0c-a58f1b118097"],[{"index":{"value":321},"size":{"value":13}},"37d007fe-9cb6-4f2e-a8b7-30842eac60b2"],[{"index":{"value":321},"size":{"value":22}},"4233204a-543d-437a-a9fc-9f79853a9540"],[{"index":{"value":335},"size":{"value":1}},"6c01783b-27ba-413c-84a4-6a94a37d714a"],[{"index":{"value":337},"size":{"value":1}},"34bc865a-58b5-4c49-bf85-a8cfedaf9b54"],[{"index":{"value":337},"size":{"value":6}},"bb556514-570d-4a87-8b2a-6e7f198a975f"],[{"index":{"value":338},"size":{"value":4}},"87c2cedc-898e-43c3-be3d-24e15689c373"],[{"index":{"value":342},"size":{"value":1}},"13387ff5-ec5e-453a-86f2-deae721b5549"],[{"index":{"value":348},"size":{"value":13}},"6280a86b-5469-43f3-94d5-1e1726028a54"],[{"index":{"value":348},"size":{"value":57}},"a17c5e55-5e99-4e4f-961d-9b4072ecd6e4"],[{"index":{"value":362},"size":{"value":1}},"c0720382-9ce4-451a-a04b-f248143d2528"],[{"index":{"value":364},"size":{"value":7}},"9cd51399-d9d2-4898-9bcb-be2f540999ff"],[{"index":{"value":364},"size":{"value":13}},"b4980bfc-13cc-429f-bc43-9edfc07d2406"],[{"index":{"value":364},"size":{"value":27}},"1a3f6a8c-ab27-43dd-a494-bf413334fd9a"],[{"index":{"value":364},"size":{"value":41}},"71a97e88-9b19-4ec9-b3c8-ffd2940c4cb8"],[{"index":{"value":371},"size":{"value":1}},"99f38c61-3ea7-4b02-be71-f293fbecb7d7"],[{"index":{"value":372},"size":{"value":5}},"08820818-ddb9-408a-b913-e9382e5b6dc7"],[{"index":{"value":378},"size":{"value":13}},"2cf06b4e-af9f-492e-ac13-5723a896a508"],[{"index":{"value":392},"size":{"value":13}},"6c887279-5998-45ce-81ab-c37fdeb03144"],[{"index":{"value":410},"size":{"value":13}},"6c126d22-c018-46cc-bdae-cf8b89104173"],[{"index":{"value":410},"size":{"value":40}},"a65c1fa8-9fce-4c30-bef7-d85ff1b428eb"],[{"index":{"value":424},"size":{"value":1}},"208c049e-e221-49ce-99d7-96142f2d1b1c"],[{"index":{"value":426},"size":{"value":13}},"0adf4ddd-1402-4b96-bfb0-1917cc275063"],[{"index":{"value":426},"size":{"value":19}},"f2158ac6-2dd5-482d-9f3d-0bf4812d6d6e"],[{"index":{"value":426},"size":{"value":24}},"c7bbc3e7-1377-429e-b60a-0c2d2cd4ea74"],[{"index":{"value":439},"size":{"value":1}},"1a17a7ee-31d7-4885-84ce-c902bff4ea4a"],[{"index":{"value":440},"size":{"value":5}},"47016cac-40a0-45f6-8057-2b81edb052a5"],[{"index":{"value":446},"size":{"value":4}},"ce4fa4cd-5064-4752-9dbc-5fcf4e92138a"],[{"index":{"value":455},"size":{"value":13}},"3300001d-13ea-45f4-acbe-959813bcf85b"],[{"index":{"value":455},"size":{"value":17}},"b8f5900b-2370-4b21-8a0c-06d3bbb45068"],[{"index":{"value":469},"size":{"value":1}},"19ea3187-83d6-47d1-994c-64cf0ce9af3b"],[{"index":{"value":471},"size":{"value":1}},"9457c3f8-c878-4f01-a5c7-39336f163a28"],[{"index":{"value":477},"size":{"value":13}},"a5139627-05a8-4838-b171-02666c62c348"],[{"index":{"value":477},"size":{"value":80}},"8b5c864c-055d-4d11-a470-2679328fc131"],[{"index":{"value":491},"size":{"value":1}},"eb01b0af-06cf-4d75-81ad-d0bf7664b2d7"],[{"index":{"value":493},"size":{"value":13}},"6c042eb7-e895-473d-b456-6e0eec23e958"],[{"index":{"value":493},"size":{"value":18}},"f40730ff-ac6c-45db-8970-534f3becda2a"],[{"index":{"value":493},"size":{"value":64}},"b2df8ba8-683c-45c2-9932-8e1c970c799a"],[{"index":{"value":506},"size":{"value":1}},"2289bf31-74dd-431c-97f1-3ac179715453"],[{"index":{"value":507},"size":{"value":4}},"6db4523a-0bcd-4ba5-b6c2-7720db57d93c"],[{"index":{"value":512},"size":{"value":1}},"b6faea17-664d-470a-9641-999b6a2be5bd"],[{"index":{"value":512},"size":{"value":45}},"98aea277-bc44-4ceb-85a6-24ee453b7f72"],[{"index":{"value":513},"size":{"value":15}},"c0ae1c05-026d-4712-a571-979ce260b310"],[{"index":{"value":513},"size":{"value":24}},"502e8e4d-5105-4abe-8dba-4d7293cbe80b"],[{"index":{"value":513},"size":{"value":43}},"71cee748-378b-4817-be0e-b87bc1d3d847"],[{"index":{"value":528},"size":{"value":1}},"fcaad3ac-e080-4170-9d63-b2c01295447d"],[{"index":{"value":529},"size":{"value":8}},"a053e05a-4862-4407-abe8-2746784bdfc3"],[{"index":{"value":538},"size":{"value":1}},"9adae78d-7667-4865-83e9-b64464b13696"],[{"index":{"value":538},"size":{"value":18}},"92f153e3-da05-439e-bb4f-596ec8cfa9c8"],[{"index":{"value":539},"size":{"value":1}},"dddedc5e-415d-4f5d-ab1b-19468aa079a9"],[{"index":{"value":540},"size":{"value":1}},"4adce2e5-1e60-4a63-9ff3-5781d967b704"],[{"index":{"value":542},"size":{"value":13}},"ef06ed1e-246f-4106-a4c0-1e648b8ad5e7"],[{"index":{"value":555},"size":{"value":1}},"7bd1cc5f-20b1-4dd2-8a34-ccc40f5f091a"],[{"index":{"value":556},"size":{"value":1}},"720adbd6-24b0-49ac-b123-61cc037a9636"]]
{"ide":{"node":{"e3b7fac7-0c08-4f20-8a0c-a58f1b118097":{"position":{"vector":[-224,13]}},"bb556514-570d-4a87-8b2a-6e7f198a975f":{"position":{"vector":[115,19]}},"71a97e88-9b19-4ec9-b3c8-ffd2940c4cb8":{"position":{"vector":[-224,-51]}},"c7bbc3e7-1377-429e-b60a-0c2d2cd4ea74":{"position":{"vector":[360,19]},"visualization":{"show":false,"fullscreen":false,"width":200}},"f2158ac6-2dd5-482d-9f3d-0bf4812d6d6e":{"position":{"vector":[297,-45]},"visualization":{"show":false,"fullscreen":false,"width":96}},"9457c3f8-c878-4f01-a5c7-39336f163a28":{"position":{"vector":[640,40]},"visualization":{"show":false,"fullscreen":false,"width":200}},"b2df8ba8-683c-45c2-9932-8e1c970c799a":{"position":{"vector":[219,-80]},"visualization":{"show":true,"fullscreen":false,"width":200}},"f40730ff-ac6c-45db-8970-534f3becda2a":{"position":{"vector":[219,-48]}}},"import":{}}}
```

To test the functionality, I asked AI to show me the result of the `operator70395` variable:

1. Init protocol connection
```json
{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}
```

2. Create execution context
```json
{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}
```

3. Push the main method
```json
{"jsonrpc":"2.0","id":0,"method":"executionContext/push","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","stackItem":{"methodPointer":{"module":"local.New_Project_1.Main","definedOnType":"local.New_Project_1.Main","name":"main"},"positionalArgumentsExpressions":[],"type":"ExplicitCall"}}}
```

4. Ask AI for the variable contents
```json
{"jsonrpc":"2.0","id":1,"method":"ai/completion_v2","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","expressionId":"f09a4372-3231-4f2e-99f4-84aa751f9b60","prompt":"There is 'operator70395' variable defined in the program. What is the result of the variable 'operator70395'?"}}
```

I got the following responses:
```json
{"jsonrpc":"2.0","method":"ai/completionProgress","params":{"code":"Visualization.AI.print(operator70395)","reason":"To provide the result of 'operator70395', I need to know its current value.","visualizationId":"edfb00a3-6ce5-41e1-bb8f-ab191809114e"}}
```
```json
{"jsonrpc":"2.0","id":1,"result":{"Success":{"fn":"def get_operator70395_result():\n    return operator70395","fnCall":"get_operator70395_result()"}}}
```
2024-05-02 16:55:06 +00:00
AdRiley
5350b2d00d
Refactor add row number (#9822)
* Refactor add row number

* Refactor

* Green

* Green

* Remove dead code

* Cleanup

* Deduplicate check
2024-05-02 12:29:54 +01:00
James Dunkerley
d2e6ff260e
Restructure SQLite_Details. (#9832)
```
type SQLite_Details
SQLite location:File|In_Memory

type In_Memory
```
to
```
type SQLite
From_File location:File

In_Memory
```

# Important Notes
Splits the In-Memory entry for Database Connect but still works nicely.

![image](https://github.com/enso-org/enso/assets/4699705/ec798ce0-9f41-4903-a2fd-722a9e37743c)

![image](https://github.com/enso-org/enso/assets/4699705/f233b055-893e-4c56-a23d-562e982560f6)
2024-05-01 22:15:41 +00:00
James Dunkerley
4d6d6f239c
Handle URL encoding automatically in query string. (#9823)
A small fix to automatically encode the query string.
Attaches a warning if needed.

![image](https://github.com/enso-org/enso/assets/4699705/032bdb59-6896-46c0-b970-f5a542cc6adf)

![image](https://github.com/enso-org/enso/assets/4699705/6b2075b9-3c98-4de2-8a34-c860ecd65d0c)
2024-04-30 22:03:46 +00:00
Radosław Waśko
f87c75a3da
Prepare for Cloud Environment variable name change (#9821)
Changing `ENSO_PROJECT_DIRECTORY_PATH` to `ENSO_CLOUD_PROJECT_DIRECTORY_PATH`.
2024-04-30 16:02:12 +00:00
AdRiley
d1bf4cb771
Add Ignored_Nothing_Values (#9770)
Add a `IgnoredNothing` warning for Table.Running

![image](https://github.com/enso-org/enso/assets/1720119/1941d278-2c33-43fe-a175-8bcc65bae51a)

![image](https://github.com/enso-org/enso/assets/1720119/b5f6b235-d939-4868-9490-de0f226ea1a2)

![image](https://github.com/enso-org/enso/assets/1720119/a1d617a6-a684-4cc1-be13-c4907d2e6876)
2024-04-30 13:30:40 +00:00
Pavel Marek
660c5e7a9d
Atom constructors can be private (#9692)
Closes #8836.

Atom constructors can be declared as private (project-private). project-private constructors can be called only from the same project. See the encapsulation.md docs for more info.

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
2024-04-29 14:43:18 +02:00
Radosław Waśko
16e1999040
Fix copy_to/move_to edge case when source and target are the same (#9798)
- Adds tests for an edge case where the source and target of copy/move operation are the same.
- And fixes the behaviour to be as expected.
2024-04-29 12:29:07 +00:00
Radosław Waśko
8e03e3be9c
Print pending groups in-line and include their reason, also add progress bar to the runner (#9796)
- Closes #9534 by printing the pending groups with pending reason
- Re-introduces original ordering of tests
- Adds a progress bar to the test suite runner in 'interactive' mode (if ANSI colors are enabled, progress bar will also be)
2024-04-26 16:05:05 +00:00
Jaroslav Tulach
c5bf2384e4
Removing Truffle API dependency from runtime-compiler project (#9785)
Fixes #8888 by removing dependency on Truffle API and `polyglot-api` from `runtime-compiler` project.
2024-04-26 13:14:22 +00:00
James Dunkerley
c5179309fd
Improve Table Viz handling of Numbers (#9793)
- Added support for NaN, Infinity and -Infinity in the Table viz.
- Simplified the logic so all the formatting in a single place in the Table viz.
- Sorted Number formatting for BigInt and < 10000.

![image](https://github.com/enso-org/enso/assets/4699705/1486eb7e-df87-4516-90d0-75fda1e9743d)

![image](https://github.com/enso-org/enso/assets/4699705/e55c0e6b-bd3c-4dcf-a65f-41f8516568a6)
2024-04-26 07:52:40 +00:00
Radosław Waśko
df4a7efcdd
Add AWS SSO JARs to the Standard.AWS library (#9782)
As reported by our users, when using the AWS SSO, our code was failing with:
```
Execution finished with an error: To use Sso related properties in the 'xyz' profile, the 'sso' service module must be on the class path.
```

This PR adds the missing JARs to fix that.

Additionally it improves the license review tool UX a bit (parts of #9122):
- sorting the report by amount of problems, so that dependencies with unresolved problems appear at the top,
- semi-automatic helper button to rename package configurations after a version bump,
- button to remove stale entries from config (files or copyrights that disappeared after update),
- button to add custom copyright notice text straight from the report UI,
- button to set a file as the license for the project (creating the `custom-license` file automatically)
- ability to filter processed projects - e.g. `openLegalReviewReport AWS` will only run on the AWS subproject - saving time processing unchanged dependencies,
- updated the license search heuristic, fixing a problem with duplicates:
- if we had dependencies `netty-http` and `netty-http2`, because of a prefix-check logic, the notices for `netty-http` would also appear again for `netty-http2`, which is not valid. I have improved the heuristic to avoid these false positives and removed them from the current report.
- WIP: button to mark a license type as reviewed (not finished in this PR).
2024-04-25 18:44:51 +00:00
Radosław Waśko
0c989c6938
Writing to Data Links (#9750)
- Closes #9292
- and also closes #9522
2024-04-25 17:55:49 +00:00
GregoryTravis
b150b091b0
Convert Vector.new_builder to Vector.build in Standard.Base (#9754)
* Vector.build

* new_builder deprecated

* convert one

* convert vector_spec

* limit param

* convert vector

* docs

* changelog

* wip

* rename limit to capacity

* wip

* review

* append comments

* review

* wip

* wip

* fix tests

* docs

* wip

* wip

* fix filter warning propagation

* doc weird dependencies

* fix tests

* cleanup

* check builder result
2024-04-25 10:38:29 -04:00
Jaroslav Tulach
0d495ffd97
Make conversion of double to BigDecimal exact (#9740)
Resolves #9607 by computing `Number.hash` by converting given number to `Float` first and then computing the hash. Also the conversion from `Float.to Decimal` is exact - done via `new BigDecimal(double)`. There is `Decimal.new` that handles the user-friendly conversion. However as a result `Decimal.from 2.1 != Decimal.new 2.1` - that's the only way to ensure consistency between hash code and conversions.
2024-04-25 11:22:50 +00:00
James Dunkerley
fb9cf38914
Excel_Workbook.read_many (#9759)
- Some minor linting fixes.
- Adjust `headers` parameter so a dedicated type.
![image](https://github.com/enso-org/enso/assets/4699705/989f464d-df95-410e-a03b-36661f1c4a37)
- Fix bug with `read` on an `Excel_Workbook` so error handled more gracefully and not panicking to UI.
![image](https://github.com/enso-org/enso/assets/4699705/23b4575f-daad-4719-a5cc-30d064bd7f7a)
- Fix bug when writing to a file with an `Excel_Format` with an invalid extension which was causing a panic.
![image](https://github.com/enso-org/enso/assets/4699705/dc0e055c-c1b6-482f-b129-eb69f6554d72)
- Add `read_many` to `Excel_Workbook` allowing reading more than one sheet at a time.
2024-04-24 13:16:44 +00:00
Jaroslav Tulach
ff62c1e25f
Access non-self arguments when widget annotations are computed (#9410) 2024-04-24 03:58:32 +02:00
AdRiley
6f3e649e7a
Make errors propagate out of each (#9742)
Make errors propagate out of each
2024-04-23 17:14:54 +00:00
AdRiley
4a97bfa31f
Add table running functionality for Sum, Mean, Min, Max. (#9577)
* Add Table.Running

* Code Review fixes

* Code Review changes

* Change null handling
2024-04-23 09:45:43 +01:00
GregoryTravis
bc61d686c1
Integer.to_decimal and Float.to_decimal (#9716) 2024-04-23 08:06:31 +00:00
Hubert Plociniczak
58009b7c04
In-memory suggestions (#9751)
This change replaces an sqllite-backed suggestions' repo with a simple, in-memory, one.
As `completion` functionality has been implemented completely in GUI, there is no need to support it in backend, which simplifies a lot of functionality.

Closes #9650 and #9471.

# Important Notes
Loading suggestions and sending them to GUI on startup is almost instantaneous. Previously it would take ~10s just for `Standard.Base`.
2024-04-22 11:02:17 +00:00
GregoryTravis
681df82697
Implement Vector.build: implementation and some tests (#9725)
* Vector.build

* new_builder deprecated

* convert one

* convert vector_spec

* limit param

* convert vector

* docs

* changelog

* rename limit to capacity

* review

* append comments

* review
2024-04-18 17:52:48 -04:00
James Dunkerley
16126a0c71
Working through various issues from OKR / blog use. (#9739)
- Fixed `list_directory` so that it works with defaults and uses the `Folder_Browse`.
![image](https://github.com/enso-org/enso/assets/4699705/cf8eb187-a1fe-4eb7-a017-e7bb40e30e12)
- Adjusted types of `order_by` in `Aggregate_Column.First` and `Aggregate_Column.Last` so supports single `Sort_Column`.
- Adjusted types of `order_by` in `Aggregate_Column.First` and `Aggregate_Column.Last` so an empty Vector is valid (treated as Nothing).
- Adjusted `Count_Distinct` so if an empty set of columns then equivalent to `Count`.
- Use the multi-select widget for `columns` in `Count_Distinct`.
![image](https://github.com/enso-org/enso/assets/4699705/86e8f3c1-9cbe-4fb3-b6fd-43b937346082)
- Sort the widget for `order_by` in `First` / `Last`.
![image](https://github.com/enso-org/enso/assets/4699705/d624544c-6f4d-42ef-bb97-79b68aa60569)
- Bug fix for widgets with both tagValues and dynamic data. (Thanks @farmaazon).
- Use the multi-select for various things.
2024-04-18 21:50:20 +00:00
GregoryTravis
86ecd3e027
Add Decimal.floor, .ceil, and .trunc (#9694) 2024-04-17 18:42:38 +00:00
Radosław Waśko
34f3331561
Implement recursive argument for File.delete (#9719)
- Closes #8993
2024-04-17 16:53:45 +00:00
Radosław Waśko
0310eb46ab
Highlight stack trace elements from the current project in Test suite reports (#9724)
If ANSI coloring is enabled, stack trace entries corresponding to the currently running project will be highlighted in bold:
![image](https://github.com/enso-org/enso/assets/1436948/b02da409-b67c-421c-a6d0-07a6cbbaf5f8)
2024-04-17 13:22:29 +00:00
Radosław Waśko
41a39eb019
Fix Json.parse "" edge case and add a test for it (#9722)
- Fixes #9720
2024-04-17 08:48:48 +00:00
Hubert Plociniczak
d6f7afc624
Open Java modules for Snowflake setup (#9664)
# Important Notes
Some workarounds are being considered but we need this change sooner than later.
2024-04-16 21:01:32 +00:00
Radosław Waśko
fda41cbfd1
Writing Cloud files (#9686)
- Closes #9291
2024-04-16 14:01:03 +00:00
Pavel Marek
0d915d0344
Update to GraalVM 24.0.0 (#9647)
Update the GraalVM-related Maven packages from **23.1.2** to **24.0.0**.

# Important Notes
- Reverted workarounds mentioned in https://github.com/enso-org/enso/issues/9477

The language home search functionality in Truffle changed. It used to find the home for Enso from the class loader of `EnsoLanguage`:
https://github.com/oracle/graal/blob/graal-23.1.2/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/LanguageCache.java#L362

In 24.0.0, the language home is no longer found via class loader, but via system properties:
https://github.com/oracle/graal/blob/graal-24.0.0/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/LanguageCache.java#L353

I provided a "fix" for this by an explicit specification of the `org.graalvm.language.enso.home` system property in our launchers.
2024-04-12 17:01:49 +00:00
Radosław Waśko
bdda1830b7
Integrate Cloud path resolver (#9662)
- Closes #9363
- Cleans up the Cloud mock as it got a bit messy. It still implements the bare minimum to be able to test basic secret and auth handling logic 'offline' (added very simple path resolution, only handling the minimum set of cases for the tests to work).
- Adds first implementation of caching Cloud replies.
- Currently only caching the `Enso_User.current`. This is a simple one to cache because we do not expect it to ever change, so it can be safely cached for a long period of time (I chose 2h to make it still refresh from time to time while not being noticeable).
- We may try using this for caching other values in future PRs.
2024-04-12 13:03:09 +00:00
Pavel Marek
4dc7992ab5
Export works without import (#9683) 2024-04-12 14:23:34 +02:00
Michał Wawrzyniec Urbańczyk
cdf031f61f
[CI] Provide latest FlatBuffers to the Engine build. (#9654)
This PR bumps the FlatBuffers version used by the backend to `24.3.25` (the latest version as of now).

Since the newer FlatBuffers releases come with prebuilt binaries for all platforms we target, we can simplify the build process by simply downloading the required `flatc` binary from the official FlatBuffers GitHub release page. This allows us to remove the dependency on `conda`, which was the only reliable way to get the outdated `flatc`.

The `conda` setup has been removed from the CI steps and the relevant code has been removed from the build script.

The FlatBuffers version is no longer hard-coded in the Rust build script, it is inferred from the `build.sbt` definition (similar to GraalVM).

# Important Notes
This does not affect the GUI binary protocol implementation.

While I initially wanted to update it, it turned out farly non-trivial.

As there are multiple issues with the generated TS code, it was significantly refactored by hand and it is impossible to automatically update it. Work to address this problem is left as [a future task](https://github.com/enso-org/enso/issues/9658).

As the Flatbuffers binary protocol is guaranteed to be compatible between versions (unlike the generated sources), there should be no adverse effects from bumping `flatc` only on the backend side.
2024-04-12 10:10:44 +00:00
GregoryTravis
e3afa5561d
Add Decimal.round (#9672) 2024-04-11 15:47:50 +00:00
GregoryTravis
bf90e2e8a8
Attach warning when converting from Decimal to Integer (#9660) 2024-04-11 15:44:58 +00:00
GregoryTravis
676e989f7f
Add Decimal.min and .max (#9663)
* min and max

* changelog

* wip

* scale approach mostly works

* Revert "scale approach mostly works"

This reverts commit 88e6073f7a.

* review

* review

* return value type
2024-04-11 09:58:10 -04:00
GregoryTravis
9ac85b93b7
Decimal.parse and .format (#9637)
* to_text + tests

* locale note

* format

* parse

* parse examples

* wip

* type

* changelog
2024-04-11 09:29:59 -04:00