Commit Graph

1082 Commits

Author SHA1 Message Date
GregoryTravis
e836373d9b
Mixed Decimal/Float operations throw error or attach warning (#10725) 2024-08-14 16:45:28 +00:00
Radosław Waśko
fc3ac6ced1
Various fixes to visualizations (#10745)
- Closes #10716 by un-marking the `Snowflake_Connection` as `private`.
- Does the same to `SQLServer_Connection`.
- Makes sure that `Snowflake_Connection` has correct visualization and enables links for it.
- Adds a fallback in `to_default_visualization_data` and in Table's `prepare_visualization` so that if `to_js_object` of a given type is malformed, we still get some kind of visualization + debug info about the underlying error.
2024-08-13 15:20:55 +00:00
AdRiley
9fc6af12c5
Clean up Text option from read and query (#10790)
* Revert

* from

* Fix

* fix

* Last few details

* Fix

* Fix test

* Fix

* Cleanup

* Fix error message

* fix

* Fix

* Improve error message
2024-08-13 13:25:21 +03:00
Radosław Waśko
b1958f8aa3
Adding vectorized implementations to some Column operations (#10795)
- Part of #6256 - implements operations that could have been vectorized without changes to the overall infrastructure
2024-08-13 08:53:39 +00:00
marthasharkey
0d5d01a294
Wip/mk/drive drilldown from enso (#10724)
This moves some of the drilldown logic to the enso code and utilises the new 'get_rows' method for Table
2024-08-12 12:51:51 +00:00
AdRiley
3581fd8e21
Make sqlserver temp tables work (#10781)
* Temp table support

* Fix

* Add query

* Clean up

* Clean up

* Fix

* Code review changes

* Added comment

* Code review changes

* Add execute
2024-08-09 13:06:16 +03:00
Pavel Marek
c2c6712b77
WithWarnings uses EnsoHashMap to speed things up (#10555)
Majority of warnings handling is now done via newly introduced nodes. Moreover, the underlying representation of warnings storage in `WithWarnings` was changed from `Warning[]` to `EnsoHashMap`.

# Important Notes
- Remove `ArrayRope`.
2024-08-07 15:29:57 +00:00
James Dunkerley
b8c036c476
Initial Tableau Reading Support (#10733)
- Adds `Hyper_File` allowing reading a Tableau hyper file.
- Can read the schema and table list.
- Can read the structure of a table.
- Can read data into an Enso Table.
2024-08-07 09:23:05 +00:00
AdRiley
0f688d0a25
Add newlines option to text cleanse/replace (#10761)
* Auto-commit work in progress before clean build on 2024-08-06 11:32:46

* Fixed Regex and additional test

* changelog

* .

* Make non-capturing
2024-08-06 18:59:54 +03:00
Pavel Marek
f0de43a970
Add org.enso.compiler.dumpIr system prop (#10740)
Working on compiler IR is a daunting task. I have therefore added a new system property `enso.compiler.dumpIr` that will help with that. It dumps the encountered IRs to `ir-dumps` directory in the [GraphViz](www.graphviz.org) format. More info in updated docs.

Note that all the functionality to dump IRs to `dot` files was already implemented. This PR just adds the command line option and updates docs.

# Important Notes
- `--dump-graphs` cmd line option is removed as per [Jaroslav's request](https://github.com/enso-org/enso/pull/10740#pullrequestreview-2216676140).
- To dump graphs, use `-Dgraal.Dump=Truffle:2` system property passed via `JAVA_OPTS` env var.

If you run `env JAVA_OPTS='-Denso.compiler.dumpIr=true' enso --run tmp.enso` where `tmp.enso` is, e.g.:
```
from Standard.Base import all
main = 42
```
You will then have something like:
```
$ ls ir-dumps
Standard.Base.Data.Filter_Condition.dot     Standard.Base.Data.Time.dot              Standard.Base.System.Advanced.dot       Standard.Base.Warning.dot
Standard.Base.Data.Locale.dot               Standard.Base.Enso_Cloud.Enso_File.dot   Standard.Base.System.File.Advanced.dot  tmp.dot
Standard.Base.Data.Numeric.dot              Standard.Base.Errors.dot                 Standard.Base.System.File.dot
Standard.Base.Data.Numeric.Internal.dot     Standard.Base.Network.HTTP.Internal.dot  Standard.Base.System.File.Generic.dot
Standard.Base.Data.Text.Regex.Internal.dot  Standard.Base.Runtime.dot                Standard.Base.System.Internal.dot
```
You can then visualize any of these with `dot -Tsvg -O ir-dumps/tmp.dot`.

An example how that could look like is
![image.svg](https://github.com/user-attachments/assets/26ab8415-72cf-46da-bc63-f475e9fa628e)
2024-08-06 12:00:27 +00:00
Radosław Waśko
3fd14642d9
Fix upload/delete transactions in Snowflake backend (#10738)
Fixes #10609 by rewriting all our upload-related operations to rely on `DDL_Transaction` - an abstraction that handles 'transactionality' of `CREATE TABLE` statements dependent on if a given backend allows DDLs within transactions or not (if not it emulates transactionality by creating the tables outside of transaction and then dropping them on rollback).
2024-08-06 08:14:44 +00:00
Kaz Wesley
aafdef1aeb
Improve parser contextualization (#10734) 2024-08-05 15:46:58 +00:00
Jaroslav Tulach
e94974a0a8
Internal tests run with --disable-private-check (#10743) 2024-08-05 15:22:22 +02:00
Radosław Waśko
6ad3faf03b
Refactor Upload_Table to be more readable: split into separate smaller submodules (#10713)
- First step of #10609 - before I actually modify it, I decided I need to refactor the `Upload_Table` logic as it was quite convoluted. Doing this as a separate PR for easier review. A big 600+ line file was replaced by several smaller ones grouped by topics.
- Practically no changes apart from moving stuff into separate modules.
- One small change - added `Missing_Argument` to `SQL_Query` as I noticed that lack of defaults was giving rise to confusing errors when working with `query` in the GUI.

Before:
![image](https://github.com/user-attachments/assets/b586caec-f25c-406e-be5a-d402f10feb86)
After:
![image](https://github.com/user-attachments/assets/6b1d4206-05b1-4587-b3e1-43ca95ea7c2e)
![image](https://github.com/user-attachments/assets/58c098bd-db0c-4ee2-823c-bf5c9e758ce4)
2024-07-31 09:54:17 +00:00
Radosław Waśko
9b2f611402
Relative paths are relative to current project locally and in Cloud (#10660)
- Close #10622
- Changes `project-manager` and `ensoup` launcher to run the engine/language-server with working directory set to the directory containing currently running project.
- If the working directory is _not_ "the directory containing currently running project", a warning is written to logs. This can happen if the raw `/bin/enso` engine runner is used in a different directory.
- In the Cloud, the `File.new` interprets relative paths as cloud paths relative to the Cloud directory containing the current project. Absolute paths are unaffected.
2024-07-31 09:43:17 +00:00
AdRiley
0c552489e3
Add Initial SQL Server support (#10624)
* Squash all commits to resolve merge conflicts

* Fix merge problems

* Merge fix

* Fix port

* Fix warning

* cargo fmt

* legal review

* Small fixes

* Update instructions

* Code review feedback

* Cleanup

* typo

* Fix

* Remove leftover snowflake code

* Remove comment

* Add underscore

* Type cleanup

* Code review fix

* Cleanup

* Add datetime roundtrip test

* add comment

* drop

* Refactor

* Refactor

* Fix merge

* Fix

* Fix

* fix

* Add comment
2024-07-30 11:13:08 +01:00
GregoryTravis
f0e9616d39
Implement Runtime.Context.Dataflow_Stack_Trace for dataflow errors thrown from Enso (#9625) 2024-07-26 19:20:41 +00:00
James Dunkerley
74acc1de24
Tweaks from client Demo (#10685)
- Adjusted Filter_Condition removing keep/drop from basic filters.
- Fix Is_In to have selector.
- Fix for Date simple expressions.
- Add get_row to Table and DB_Table.
2024-07-26 18:44:36 +00:00
Radosław Waśko
46e8bab429
Fix aggregate after sort in DB (#10677)
* add test for https://github.com/enso-org/enso/issues/10321

* wrap in subquery if ordering was present to fix the bug

* adding one more test just to be sure

* fix import
2024-07-26 08:43:19 +01:00
GregoryTravis
f31c084f43
Implement in-memory and database mixed decimal column comparisons (#10614) 2024-07-25 21:27:19 +00:00
Radosław Waśko
ba8ae4502c
Fix date_diff in Snowflake (#10672)
- Closes #10438
- The results are again aligned across backends.
2024-07-25 16:16:33 +00:00
James Dunkerley
7e0870267e
Fix Simple Expression issue where column arguments weren't resolved. (#10658)
Closes #10657

Was missing the `table.resolve` for arguments.
2024-07-24 16:39:10 +00:00
Jaroslav Tulach
c6da3a93ba
Treat WithWarnings via dedicated @Specialization (#10650) 2024-07-24 15:31:27 +02:00
Kaz Wesley
e5b85bf16e
Space-precedence does not apply to value-level operators (#10597)
In a sequence of value-level operators, whitespace does not affect relative precedence. Functional operators still follow the space-precedence rules.

The "functional" operators are: `>> << |> |>> <| <<| : .`, application, and any operator containing `<-` or `->`. All other operators are considered value-level operators.

Asymmetric whitespace can still be used to form *operator sections* of value-level operators, e.g. `+2 * 3` is still equivalent to `x -> (x+2) * 3`.

Precedence of application is unchanged, so `f x+y` is still equivalent to `f (x + y)` and `f x+y * z` is still equivalent to `(f (x + y)) * z`.

Any attempt to use spacing to override value-level operator precedence will be caught by the new enso linter. Mixed spacing (for clarity) in value-operator expressions is allowed, as long as it is consistent with the precedences of the operators.

Closes #10366.

# Important Notes
Precedence warnings:
- The parser emits a warning if the whitespace in an expression is inconsistent with its effective precedence.
- A new enso linter can be run with `./run libraries lint`. It parses all `.enso` files in `distribution/lib` and `test`, and reports any errors or warnings. It can also be run on individual files: `cargo run --release --bin check_syntax -- file1 file2...` (the result may be easier to read than the `./run` output).
- The linter is also run as part of `./run lint`, so it is checked in CI.

Additional language change:
- The exponentiation operator (`^`) now has higher precedence than the multiplication class (`*`, `/`, `%`). This change did not affect any current enso files.

Library changes:
- The libraries have been updated. The new warnings were used to identify all affected code; the changes themselves have not been programmatically verified (in many cases their equivalence relies on the commutativity of string concatenation).
2024-07-24 10:55:44 +00:00
Radosław Waśko
ba56f8e89b
Snowflake Dialect - pt. 7 (#10612)
- Closes #9486
- All tests are succeeding or marked pending
- Created follow up tickets for things that still need to be addressed, including:
- Fixing upload / table update #10609
- Fixing `Count_Distinct` on Boolean columns #10611
- Running the tests on CI is not part of this PR - to be addressed separately
2024-07-23 06:58:11 +00:00
Radosław Waśko
7fd8701690
Snowflake Dialect pt. 6 - Union, Distinct and other improvements (#10576)
- Part of #9486
- Fixes `Table.union`, `merge` and `distinct` tests
- Replaces `distinct_on` in `Context` that was actually a Postgres specific addition leaking into the base with a more abstract `Context_Extension` mechanism.
- This allows us to implement the Snowflake-specific `DISTINCT` using `QUALIFY`.
2024-07-19 16:04:00 +00:00
GregoryTravis
2e0fa89928
Enable and fix mixed-comparison Decimal tests (#10583)
* from_string to from_text

* dec

* int/dec tests

* tests for f/d conversion warnings

* test warnings on mixed comparisons

* return inline types

* review

* disable mixed comparison test

* chaqngelog

* wip

* wip

* wip

* enable tests

* fix tests

* impl

* Revert "impl"

This reverts commit baeea8989f.
2024-07-19 11:55:46 -04:00
Jaroslav Tulach
c0f5759e07
Speeding up Ordering_Comparator (#10593) 2024-07-19 04:08:53 +02:00
James Dunkerley
15810a4675
Linting fixes. (#10589)
Generating Doc data so fixed linting.
2024-07-18 12:42:34 +00:00
James Dunkerley
771402acf6
Widget Improvements (#10575)
- New centralised `make_any_selector` for creating dropdown for various types.
- Updated `Filter_Condition.default_widget` to be auto-scoped.
![image](https://github.com/user-attachments/assets/2786d512-6716-4d4b-b119-15054611c11c)
- Nested structure for Simple_Expressions.
![image](https://github.com/user-attachments/assets/09eda332-0888-41e7-8462-fc8566959da5)
2024-07-17 19:53:17 +00:00
GregoryTravis
0268cbb55b
Various Decimal usability tweaks (#10517) 2024-07-16 18:37:25 +00:00
AdRiley
cd1d61710f
Rename run workflow button to write (#10568)
Request from Product team to not have a run button. Renaming it to write
2024-07-16 18:04:31 +00:00
James Dunkerley
2442ebc52e
Restore Encoding.Default. (#10567)
Following the fix of Input Stream, restore the encoding parts.

No significant performance impact on reading the client test data.
2024-07-16 16:49:46 +00:00
Radosław Waśko
a30b0c60eb
Snowflake Dialect pt. 5 (#10528)
- Related to #9486
- Batching of expression tests
- Fixing arithmetic by simplifying `%` and `/` operations
- Trying to share some more tables, sometimes improving performance sometimes not really
- Adding sorting and other fixes to tests to make them pass: Missing_Values_Spec, Filter_Spec, Map_Spec
- Fixing warnings related to materialization of Decimal->Integer, thus fixing Join_Spec.
2024-07-16 09:38:57 +00:00
James Dunkerley
cc1ac87c99
Linting, XML to_table and fix JSON viz for XML (#10554)
- Linting fixes.
- `XML_Document` and `XML_Element` have `to_table` method.
- Add `to_default_visualization_data` to `XML_Document` and `XML_Element`.
- Add support to Table viz to render.

![image](https://github.com/user-attachments/assets/f01a3508-443e-48db-ad4f-605094a04c2b)

![image](https://github.com/user-attachments/assets/c7573b68-7549-494f-9c59-ea240178c0eb)
2024-07-15 18:33:37 +00:00
Radosław Waśko
632355f85b
Snowflake Dialect pt. 4 - reading a column of small integers as Integer type, other type mapping tests (#10518)
- Related to #9486
- Ensures that even though an integer column in Snowflake is represented by `Decimal` type, if the values are small enough, they are materialized as `Integer`.
- If the values are larger, they are still read in as `Decimal`.
- Adds tests for some other `Decimal` edge cases (various precisions and scales), and for `Float`.
2024-07-11 20:14:46 +00:00
Pavel Marek
0f9852aab2
Replace all from ... export all with explicit exports (#10369)
Replace all exports with explicit exports.
2024-07-11 19:34:25 +02:00
Jaroslav Tulach
077b86f98c
Speed up EnsoInputStream again (#10515)
Fix #10503 by creating a benchmark and then speeding it up by making sure usage of `InteropLibrary` reminds in partially evaluated code and isn't hidden behind `@TruffleBoundary`.
2024-07-11 10:08:26 +00:00
Jaroslav Tulach
220b40a1cd
Enforce conversion method return type & introduce Comparable.new (#10468) 2024-07-11 06:58:51 +02:00
Adam Obuchowicz
60c1a0e1f6
Record control tweaks (#10509)
Fixes #10388

[Screencast from 2024-07-10 13-55-46.webm](https://github.com/enso-org/enso/assets/3919101/4ad7c039-84f6-4e42-aad9-5e287ccd88bb)

# Important Notes
Also, removed unnecessary  setExecutionEnvironment call on startup
2024-07-10 20:19:23 +00:00
Radosław Waśko
48c17845a7
Fixing Database tests and Snowflake Dialect - part 3 out of ... (#10458)
- Related to #9486
- Fixes types in literal tables that are used throughout the tests
- Tries to makes testing faster by disabling some edge cases, trying batching some queries, re-using the main connection and trying to re-use tables more
- Implements date/time type mapping and operations for Snowflake
- Updates type mapping to correctly reflect what Snowflake does
- Disables warnings for Integer->Decimal coercion as that's too annoying and implicitly understood in Snowflake
- Allows to select a Decimal column with `..By_Type ..Integer` (only in Snowflake backend) because the Decimal column there is its 'de-facto' Integer column replacement.
2024-07-10 13:21:30 +00:00
Jaroslav Tulach
ee0175c364
Support re-running failed tests with runEngineDistribution (#10483) 2024-07-10 07:05:29 +02:00
James Dunkerley
8da06309e9
Date Time Pickers, Temporarily Disable Encoding.default (#10493)
- Widgets for Date_Time, Time_Of_Day and Time_Zone.
- Disable Encoding.default for now as big performance impact on CSVs.

![image](https://github.com/enso-org/enso/assets/4699705/c1b936f0-3ab4-490c-8fe5-2310ef1ed079)

![image](https://github.com/enso-org/enso/assets/4699705/d5e29ec4-cc52-41e5-a532-17cd6dff34b9)

![image](https://github.com/enso-org/enso/assets/4699705/61455519-ea63-4275-9c7a-603714ff9f85)

![image](https://github.com/enso-org/enso/assets/4699705/48ccd3ad-5e15-49f9-87cd-4710ca559843)
2024-07-09 21:04:08 +00:00
GregoryTravis
71a6e2162e
Fix return type for Postgres Decimal division (#10479) 2024-07-09 15:22:14 +00:00
Radosław Waśko
a3dc50fe1e
Replace presigned S3 URL with lambda request (#10456)
- Closes #10419
2024-07-09 09:36:10 +00:00
James Dunkerley
4b3e4ae15e
Rename Map to Dictionary and Set to Hashset. (#10474)
- Rename `Map` to `Dictionary`.
- Rename `Set` to `Hashset`.
- Add a deprecated place holder for the static method of `Map`.
2024-07-09 09:12:23 +00:00
James Dunkerley
4c0647ea29
Stop publishing First/Last as constructors and use auto-scoping for take and drop. (#10467)
- Removes `First` and `Last` from the `Standard.Base` exports.
- Enable auto-scoping for all `Index_Sub_Range` and `Text_Sub_Range`.
- Update all use of those methods to use auto-scoping.
2024-07-08 10:26:30 +00:00
James Dunkerley
018d4c312f
Stop publishing Postgres constructor, update Postgres_Details.Postgres to Postgres.Server. (#10466)
![image](https://github.com/enso-org/enso/assets/4699705/6d0d4167-e97b-4765-8079-650ad091ce60)

- Rename `Postgres_Details` to `Postgres`.
- Rename `Postgres` constructor to `Server`.
- Update SPI.
- Linting issues (indent, missing doc comment)
2024-07-08 07:58:08 +00:00
Jaroslav Tulach
515d8238bb
Support for --jvm option in Enso runner (#10374)
Addresses one of two concerns of #5298 - adds support for `--jvm` argument to allow us to switch from _native image_ built Enso binary (as developed by #10126) to regular JVM based Enso execution. This change _doesn't affect production builds_. The _native executable_ continues to be only built by `engine-runner/buildNativeImage` which is tested on CI, but not in the production jobs.
2024-07-06 07:02:20 +00:00
James Dunkerley
d65371096b
Deserialise large integer and decimals from JSON automatically. (#10463)
- Sort large integer and decimal JSON deserialization.
- Change type to be Integer instead of BigInt for large integers.
- Add tests.
- Update Table viz.
- Preserve white space in JSON viz.
![image](https://github.com/enso-org/enso/assets/4699705/48c83616-c0ed-4cb4-862a-34cd4fff09aa)

![image](https://github.com/enso-org/enso/assets/4699705/5bae9ccd-1d0f-4b70-aea5-d4cebc3d9df8)
2024-07-05 21:56:08 +00:00
James Dunkerley
c2c4b95116
Final step removing the Problem_Behavior publishing. (#10461)
- Remove publishing the constructors.
- Fix any missed use in libs.
- Alter tests to generally use auto-scoped calls.
- `on_incomparable` to `on_problems`.
2024-07-05 18:41:36 +00:00
James Dunkerley
2124cd022a
Batch Two of Problem_Behavior (#10454)
Part 2 of removing the public constructors.
Completes review of the libraries.

Tests left to do then can remove the export.
2024-07-05 13:40:29 +00:00
AdRiley
fc93b4d121
Refactor database dialect types (#10437)
* Auto-commit work in progress before clean build on 2024-07-03 14:17:22

* Refactor

* Revert

* revert

* Code Review feedback

* Green

* 2 Red

* Green

* Renames

* Code review changes

* Code review changes
2024-07-05 13:08:25 +01:00
James Dunkerley
82678aa7cc
Starting to clean up exporting Problem_Behavior constructor refactor. (#10449)
Use type check everywhere in Standard.Base for Problem_Behavior.
2024-07-05 10:05:51 +00:00
Hubert Plociniczak
c54c3b7e9d
Eliminate circe-yaml dependency (#10326)
* Eliminating circe-yaml

This change adds our very-own YAML parser on top of SnakeYAML. Compared
to Circe parser on top of SnakeYAML. The advantage? In some not-so-distant
future we might actually get rid of circe and the related performance
issues.

The logic is similar to what circe does i.e. analyzing SnakeYAML to
build our own structure.
This change is not complete, as there are still some tests failing, but
most common Configs are already parseable.
We _could_ auto-generate some of the code but still some of the logic
would have to be tweaked by hand; the current logic has a number of
special cases, as I found out the hard way.

* wip: more tests passing

* Fix remaining tests in ConfigSpec

* Fixing YAML decoder for editions

Dropping circe as a decoder for Editions revealed some problems. Turns
out the current implementation had even more special cases to deal with.

* nit

* Allow for empty exports

* Mostly complete encodin part

Replaced almost all `toYAML` locations with SnakeYAML equivalent.
The encoding has to use Java collections for which there exists a
built-in support. If we were to use Scala collections we would have to
deal with tagging, at the very least.

* Remove the last remaining Circe's YAML parser

* Bug fix + further loop optimization

* removal of some dependencies

* Remove circe-yaml

Added a custom SnakeYAML Node updater to mimick the JSON -> YAML -> JSON
conversion needed for updating fields. The algorithm recursively follows
the key-path and inserts the desired Node. This is not a performance
oriented code on purpose.

* Fix compilation issues

`circe-core` was marked as `provided` but no one eventually included it
in the final jar, hence `NoClassFoundException`.

* fix licensing

* Removing obsolete circe definitions

* fmt

* nits

* s/SnakeYamlDecoder/YamlDecoder

* fmt

* Partial revert, PM needs JSON decoders/encoders

* style

* incremental compilation gone wrong
2024-07-05 09:32:45 +02:00
James Dunkerley
0661f17d1c
Tune Text.trim, fix for Text.split (#10445)
- Rename `Location.Start` to `Location.Left`.
- Rename `Location.End` to `Location.Right`.
- Use auto-scoping for `Location`.
- Tune widgets for `Text.trim`.
- Correct signature of `Text.split`.
- Adjist `generateLocallyUniqueIdent` to not fail on bad signature.
2024-07-04 22:24:56 +00:00
James Dunkerley
9a2aed92f1
Separating list from read function and other small tweaks. (#10434)
- Rename `Faker.string_value` to `Faker.text_value`.
- Remove `Regex.pattern_string` as duplicate of `Regex.pattern`.
- Sort the Date picker.
- Rename `Data.list_directory` to `Data.list`.
- Remove support for reading a directory.

![image](https://github.com/enso-org/enso/assets/4699705/b42bb3c9-e63b-49f2-8cdc-4666cb6d968e)

![image](https://github.com/enso-org/enso/assets/4699705/97f49891-5ae5-4f0a-9a41-6200888fcd86)
2024-07-03 22:00:53 +00:00
marthasharkey
ee39fd7f53
Visualise excelwork book sheets, db tables/schemas and add node adding gets to query them (#10362)
changes to current table (and db table) on index click:
- node added on double click
- tooltip added
- cursor changes to pointer
- styled as a link

![7473-table-w-links](https://github.com/enso-org/enso/assets/170310417/5e60c177-3f83-4db7-be86-fa8a9d493204)


Row table types:
Added on click to show value
![7473-row-links](https://github.com/enso-org/enso/assets/170310417/82f878ea-420d-4308-99bf-c77a6340a8c3)



Excel Workbook connections
- show sheet names in column
- sheet names are clickable that add a 'read' node for the corresponding sheet
- Shown in table with header title: Sheets

![7473-excel-links-sheets](https://github.com/enso-org/enso/assets/170310417/748f524e-5cca-4c20-b458-132af9a57ec1)


SQLite Connection:
- shows available schemas in column
- schemas are clickable that add a 'query' node for the corresponding schema/table
![7473-sql-lite-links](https://github.com/enso-org/enso/assets/170310417/21a8006f-c462-4128-874d-05380d3bab00)


Postgres Connection:
- shows available tables in a column
- tables are clickable and add a 'query' node for the corresponding table
![7473-postgres-links](https://github.com/enso-org/enso/assets/170310417/66b134ff-80fd-4995-b445-505919e25cfa)

JS_Object
- style keys as links
![7473-json-links](https://github.com/enso-org/enso/assets/170310417/9ec17c85-c1f4-42ab-b40a-06d6bc29d54f)
2024-07-03 13:58:07 +00:00
James Dunkerley
4bb82fae4a
Fix issues with Date.new Widgets and Some Viz Fixes. (#10430)
- Do not include nothing values for optional parameters when serializing widgets.
- Serialize Nothing as `{"type": "Nothing"}` for the default JSON viz.
- Add JS Object support for decimal.
- Fix `to_json` for large integers

![image](https://github.com/enso-org/enso/assets/4699705/88a404a3-537d-4be7-9a47-c896d2f8b378)

![image](https://github.com/enso-org/enso/assets/4699705/ce09955e-b377-431a-aca3-28f442c772a6)

![image](https://github.com/enso-org/enso/assets/4699705/d0c99a0b-1964-4874-8d87-41979a17a359)

![image](https://github.com/enso-org/enso/assets/4699705/32ade951-0921-4837-9c35-9b60488f863f)

![image](https://github.com/enso-org/enso/assets/4699705/f0fba27a-dfcc-4f17-a883-deb23757f74e)
2024-07-03 06:41:40 +00:00
GregoryTravis
48fb999eb3
Implement Decimal support for Postgres backend (#10216)
* treat scale nothing as unspecifed

* cast to decimal

* float int biginteger

* conversion failure ints

* loss of decimal precision

* precision loss for mixed column to float

* mixed columns

* loss of precision on inexact float conversion

* cleanup, reuse

* changelog

* review

* no fits bd

* no warning on 0.1 conversion

* fmt

* big_decimal_fetcher

* default fetcher and statement setting

* round-trip d

* fix warning

* expr +10

* double builder retype to bigdecimal

* Use BD fetcher for underspecified postgres numeric column, not inferred builder, and do not use biginteger builder for integral bigdecimal values

* fix tests

* fix test

* cast_op_type

* no-ops for other dialects

* Types

* sum + avg

* avg + sum test

* fix test

* update agg type inference test

* wip

* is_int8, stddev

* more doc, overflow check

* fmt

* finish round-trip test

* wip
2024-07-02 15:01:55 -04:00
James Dunkerley
69b5f719e8
Adjust Colours. (#10417)
- Change the lightness of the component colours to match Figma.
- Remove the background image from tailwind.css.
- Alter the code editor background to be white.
- Make the selection colour much brighter in the dashboard.

![image](https://github.com/enso-org/enso/assets/4699705/4ae8f03a-bdeb-48cf-86eb-aa058a2a1b01)

![image](https://github.com/enso-org/enso/assets/4699705/eb5d4cf8-d275-4c0d-b876-4a29ce7768bf)

![image](https://github.com/enso-org/enso/assets/4699705/0df64403-75e5-4e44-a818-58cedbbcd3c9)

# Important Notes
Removed the `is-hidden-file` package from the project manager shim. It seems to be unreliable in Win11 and as just for local testing made just do dot file logic.
2024-07-01 20:01:18 +00:00
Radosław Waśko
db4f7ab3b5
Fixing Database tests and Snowflake Dialect - part 2 out of ... (#10319)
- Part of #9486
- Fixing our tests to not rely on deterministic ordering of created Tables in Database backends
- Before, SQLite and Postgres used to mostly return rows in the order they were inserted in, but Snowflake does not.
- Fixing various parts of Snowflake dialect.
2024-06-27 14:54:00 +00:00
James Dunkerley
d92078471b
Rename order_by to sort for Table and DB_Table. (#10372)
- Rename `order_by` to `sort` for `Table` and `DB_Table`.
- Added deprecated placeholder.
- Fixed a couple of minor deprecated mistakes.

![image](https://github.com/enso-org/enso/assets/4699705/96c32fa7-33e5-400a-9d3a-ebf330886911)
2024-06-26 17:46:09 +00:00
AdRiley
6bc7d4f786
Fix signatures for distinct and duplicates (#10365)
Fixes #10364
2024-06-25 12:59:27 +00:00
Jaroslav Tulach
0cde0e7f96
More detailed No_Such_Method error message for function like objects (#10328) 2024-06-25 11:36:56 +02:00
James Dunkerley
e6c8ec7ab5
Changes from session with Ned (#10349)
- Removed `second_row` and `second_column` from the `Table` and `DB_Table`.
- Added `first_value` and `last_value` to the `Table` and `DB_Table`.
- Fixed bug where negative index access wasn't allowed on `Column`.
- Added error if negative index access used on `DB_Column`. Tells user they have to materialize.
- Fix argument order for `Table.text_cleanse` and a couple of typo corrections.
- Rename `auto_value_type` to `auto_cast` on table and columns.
2024-06-24 12:47:14 +00:00
AdRiley
c324c78e23
Add duplicates component (#10323)
* Update existing behaviou to match new

* Add signatures

* Red test

* First test green

* sbt javafmtAll

* In-Memory working

* Not implemeted for In-Db

* Docs

* Disable tests for in-db

* Changelog

* Code review changes

* Fix

* Fix

* Fixc tests
2024-06-24 13:29:03 +03:00
James Dunkerley
791dba6729
Autoscoping for File_Format and minor tweaks. (#10348)
- Add `rename` ALIAS to `Table.use_first_row_as_names`.
- Add a shorthand to `Faker.string_value` to allow quick creation of fake values such as National Insurance numbers.
- Add `match` ALIAS for `Text.find` and `Text.find_all`.
- Auto scoping for `File_Format`. Demonstrates how to use it in a dynamic context.
- `SQLite_Format.For_File` renamed to `SQLite_Format.SQLite` (though kept for backwards compatibility.
- Fixed bug in `SQLite_Format` which was calling a non-existent constructor.

![image](https://github.com/enso-org/enso/assets/4699705/4506d27c-c1ff-4ad6-9276-53c2ae00de17)

![image](https://github.com/enso-org/enso/assets/4699705/9043ffb0-6740-42ba-91f8-ab0df555f20f)

![image](https://github.com/enso-org/enso/assets/4699705/03122fac-bdbb-4bcf-ac96-9491da41a8b9)

![image](https://github.com/enso-org/enso/assets/4699705/79122aac-a74a-435d-9849-ac3421f7d080)

![image](https://github.com/enso-org/enso/assets/4699705/54544de8-9aea-4dc6-bb4d-a7d8233c6814)
2024-06-24 08:28:54 +00:00
Jaroslav Tulach
fe2cf49568
Run whole test/Base_Tests in native image runner (#10296) 2024-06-21 06:03:53 +02:00
Hubert Plociniczak
114b3a5c5e
Cleaning up YAML parsing in preparation for circe-yaml removal (#10309)
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.

As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.

Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.

In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
2024-06-20 13:07:54 +00:00
James Dunkerley
5042592d24
Some formatting issues and a few tweaks (#10298)
- Linting fixes.
- Adjust doc comments with `<` or `>` in plain text areas to use `&lt;` and `&gt;`.
- Refactor Statistics module and add auto-scoping.
- Add auto-scoping to `Text.to_case`.
- Fix type issue with `Table.get_value`.
- Private constructor for `Excel_Workbook` and move `xls_format` to public method.
- Add `fields` widget to `to_table`.
- Add `simple_expr` to make `Table.set` clearer.

![image](https://github.com/enso-org/enso/assets/4699705/3e21e800-142c-4006-a6c2-dd6196b76c9a)

![image](https://github.com/enso-org/enso/assets/4699705/d40dcbfd-a35e-4849-9e1a-f4a418d562dd)
2024-06-20 10:44:36 +00:00
Radosław Waśko
a8358512ad
Small fixes to Cloud Integration (#10303)
- Includes HTTP method in error message
- Does not do special handling for `403` status code - this was wrong and led to `Unauthorized` error when the real cause was lack of permssions in the Cloud. The errors should be more understandable now.
- Adds `projectSessionId` to audit log metadata.
- Fixes a test (`Secrets_Spec`) that did not have unique names and would fail if cleanup of previous runs failed (or if ran in parallel).
2024-06-18 09:41:33 +00:00
marthasharkey
bb16db9c79
Remove legacy matrix and legacy objects from table visualization and remove the index column (#10260)
- remove legacy matrix and object types from vue component and any further code relating to those
- remove the index and index header being sent in the json for tables
- add has_index_col flag for json hat previously sent 'indicies_header' and 'indicies' so that index/# column is still rendered where required
2024-06-17 09:17:20 +00:00
Radosław Waśko
3a4784c226
Initial separation of Snowflake_Dialect from Postgres_Dialect (#10266)
- Part of #9486
- Building on top of initial work by @jdunkerley and finishing it
- Reverted the changes to the Postgres_Dialect from last Snowflake work and split the Snowflake_Dialect into a separate module.
- Moved from `rounding_decimal_places_not_allowed_for_floats` to `supports_float_round_decimal_places` (as too confusing).
- Added Snowflake_Dialect type.
- Extracted `Snowflake_Spec` into separate `Snowflake_Tests`
- It imports the common tests from `Table_Tests`.
- Some initial adaptations to make the snowflake dialect not-crash.
- Adding `Internals_Access` proxy to allow external implementations to access our internal data structures without directly exposing them to users. Users should not use these.
- Adding profiling of SQL to check performance.
2024-06-13 16:12:20 +00:00
AdRiley
fadb81abe6
Fix Text_Cleanse tests (#10263)
I hadn't connected the Text_Cleanse tests up properly and as a result they weren't running or working. This fixes that.
2024-06-13 08:25:32 +00:00
Hubert Plociniczak
d6d370925a
Removing dependencies to speed up startup (#10249)
Reducing the number of dependencies. Explicit `cats` are almost gone (present in `cli`). `enumeration` is completely gone.  `cats` is also still included implicitly via `io.circe` but that's a different kind of beast.
Also, really removed `jackson` from dependencies by fixing the dependency on `http-test-helper`.

# Important Notes
In a number of places importing all cats implicits could be simply replaced with a single or two method calls. Not to mention that this will reduce compilation times due to reduced implicit search space.

One example of how the changes affect performance (not only startup):

Before:
![Screenshot from 2024-06-11 12-05-24](https://github.com/enso-org/enso/assets/292128/a1a772a9-635d-4a16-a543-e2fd2124a22c)
Now:
![Screenshot from 2024-06-11 14-27-47](https://github.com/enso-org/enso/assets/292128/b17c7fcc-9a6d-48b9-8200-60708354ee03)
(frequently executed)

![Screenshot from 2024-06-12 12-46-34](https://github.com/enso-org/enso/assets/292128/31bc4dfd-4edc-45c9-9c5d-13e3472089b9)
Also appears to be gone.

This PR is by no means finished. The purge will continue in follow up PRs.
2024-06-12 18:15:36 +00:00
marthasharkey
55af1b9ffd
Add icon to indicate value type in table (#10225)
closes #10018

Sends the value type within the json for table visualisation
Header uses a html template to show it's value type to the right of the title
Displays the value type also in a tooltip, this displays the types "display_text"

<img width="392" alt="image" src="https://github.com/enso-org/enso/assets/170310417/0828e6a2-b30f-4ac7-9a8f-46b4a9cfac91">


tooltip:
<img width="498" alt="image" src="https://github.com/enso-org/enso/assets/170310417/f9964f90-9337-42d3-a0ef-3c58f6d74621">
2024-06-12 09:06:55 +00:00
AdRiley
783276ca34
Make test framework list out failed tests to use in filter (#10241)
* List out failed tests

* Code review changes
2024-06-11 21:15:28 +01:00
Radosław Waśko
c4b0ca8f69
Ensure type ascriptions are correctly transformed to facilitate checking of more complex return types (#10229)
- Fixes #9980
- Adds some tests to ensure types like `|` or `&` (in addition to `!` from the ticket) correctly work in return type check.
- Fixes a weird behaviour where we used to avoid processing type related IR transformations inside of type ascriptions.
- Adds parentheses to type representations if they are more complex: `A | B & C` is unclear as it can either mean `A | (B & C)` or `(A | B) & C` which have different meanings. If we now have an operation with such nesting, the sub expressions are wrapped in parentheses to disambiguate.
2024-06-11 17:11:03 +00:00
Hubert Plociniczak
4da5e611c9
Replace Jackson serde (#10035)
JSON serialization setup between Language Server and Runtime is a major contributor to startup time. This PR experiments with an alternative implementation that remedies the problem.
The new serializer uses [jsoniter-scala](https://github.com/plokhotnyuk/jsoniter-scala) which by some accounts claims to be really fast. In our case, more importantly, we pay negligible cost of startup setup compared to Jackson which was horribly slow.

# Important Notes
Before:
![Screenshot from 2024-06-06 15-35-18](https://github.com/enso-org/enso/assets/292128/56103b82-777e-459f-966e-abdef25c2430)
After:
![Screenshot from 2024-06-06 15-35-02](https://github.com/enso-org/enso/assets/292128/00a36647-dfae-4dc8-a1b7-bf69069ef109)

Yes. About 0.8sec.
2024-06-11 15:03:12 +00:00
Pavel Marek
5fa29c51b5
Fix Meta.enso_project (#10192)
Fixes `Standard.Base.Meta.Enso_Project.enso_project` to return a project descriptor for the *main* project, i.e., the one configured as a *root* for the engine.

# Important Notes
`enso_project` builtin no longer iterates the stack frames to infer the project descriptor. It derives it from the default package repository.
2024-06-11 08:31:40 +00:00
Radosław Waśko
41d02e95ef
Implement Windows-1252 fallback logic for Encoding.Default (#10190)
- Closes #10148
- [x] Tests for `Restartable_Input_Stream`, `peek_bytes` and `skip_n_bytes`.
- [x] Report `Managed_Resource` stack overflow bug: #10211
- [x] Followup possible optimization: #10220
- [x] Test use-case from blog.
2024-06-10 10:49:26 +00:00
James Dunkerley
d938c96c55
Adding type annotations and enabling auto-scoping (#10173)
- Renamed `Missing_Required_Argument` to `Missing_Argument`, and added `throw` method.
- Add default widget to `Case_Sensitivity.Insensitive locale`.
- Switch to auto scoping for `parse_type_selector`.
- Add type annotation to various simple typed arguments in `Table` and `DB_Table`.
- Altered `Filter_Condition` to have `Missing_Argument` for all non-defaulted arguments.
- Added resolution of `Column_Ref` passed as auto-scoped to `Table_Ref`.
- Altered `Simple_Calculation` to have `Missing_Argument` for all non-defaulted arguments.
- Altered `Join_Condition` to have `Missing_Argument` for all non-defaulted arguments.
- Altered `Sort_Column` to have `Missing_Argument` for all non-defaulted arguments.
- Altered `Aggregate_Column` to have `Missing_Argument` for all non-defaulted arguments.

**rename_columns:**
![image](https://github.com/enso-org/enso/assets/4699705/08aaba0f-687a-450c-9781-8eadc062bd50)

**aggregate:**
![image](https://github.com/enso-org/enso/assets/4699705/c29e7944-1a1c-4020-9fe0-528d874b8049)

**join:**
![image](https://github.com/enso-org/enso/assets/4699705/50038166-e56d-48c5-9eeb-bd46fa415e46)

**set:**
![image](https://github.com/enso-org/enso/assets/4699705/bee2462a-dafb-4bd4-b102-ec73edb4fb93)
2024-06-10 07:52:32 +00:00
GregoryTravis
4aa3d52b60
Implement conversions for Decimal column (#10206)
* treat scale nothing as unspecifed

* cast to decimal

* float int biginteger

* conversion failure ints

* loss of decimal precision

* precision loss for mixed column to float

* mixed columns

* loss of precision on inexact float conversion

* cleanup, reuse

* changelog

* review

* no fits bd

* no warning on 0.1 conversion

* fmt
2024-06-07 15:37:32 -04:00
AdRiley
fe6eafd06e
Add By_Type option to more components (#10183)
* reorder_columns

* format

* cast

* auto_value_types

* replace

* fill_nothing

* fill_empty

* text_replace

* text_cleanse

* Add By_Type

* Fix cast

* Fix more tests

* Fix the table helper

* Bug fix

* Remove check
2024-06-07 16:26:23 +01:00
Hubert Plociniczak
1bc14252df
Extract mutable builder from ModuleScope (#9914)
Refactored mutable parts of `ModuleScope` into builder to make it easier to reduce unnecessary locks.

# Important Notes
Elements of ModuleScope (types, imports etc) are used while _building_ of it may still be in progress. In order to make static typing happy, every `ModuleScope.Builder` can be exposed as (unmodifiable) `ModuleScope`.
2024-06-05 16:57:08 +00:00
Radosław Waśko
698f4b5640
Remove workaround for #8937 after it has been fixed (#10149)
- Since #8937 was fixed by #10104 the workaround is no longer needed
2024-06-05 14:23:12 +00:00
Jaroslav Tulach
8332118ff4
What's the name of my method? (#10164)
While working on #10056 I realized the names of method and closure nodes are incomprehensible to anyone. This PR replaces the infamous `<anonymous>` with a name hinting where the method actually is.

# Important Notes
I assume this change will be visible not only in IGV, but also in _stacktraces_ and we may need to adjust few tests.
2024-06-05 07:49:53 +00:00
GregoryTravis
5fad3558a6
BigDecimalBuilder and arithmetic operations. (#9950)
* hack

* make a column

* add

* no scale=0 on BD type

* a test

* wip

* 3 arithmetic ops

* /

* wip

* BigDecimalPowerOp

* wip

* mod test

* NumericBinaryOpReturningBigDecimal

* with scalar

* misc arithmetic tests

* fix integralBigDecimalToInteger

* mixed columns

* bigdecimal pow via double

* cleanup

* j2e on get

* arithmetic exception

* mod 0

* cleanup

* fmt

* changelog

* check type first

* merge

* mc error message

* add BD case to Builder.java

* fmt

* changelog

* add BD case to StorageConverter.java

* fmt

* fix test
2024-06-04 13:59:31 -04:00
AdRiley
7c35781a14
Add select/remove column by type (#10159)
* stash

* Working first pass

* Tests

* remove columns

* more tests

* More tests

* DB_Table

* In DB

* Deprecate

* Update documentation

* More docs

* Remove strict

* Remove tests

* Fix widgets

* Add doc

* Spaces

* Fix In-DB
2024-06-04 16:23:46 +01:00
marthasharkey
24d209abf9
Add Columns_To_Add type to move away from Integer | Nothing (#10152)
Adds new type "Coulmns_To_Add" that replaces the Integer | Nothing type
And adds new widget to split_to_columns and tokenise_to_columns
<img width="397" alt="image" src="https://github.com/enso-org/enso/assets/170310417/1b155682-0992-4cc0-8964-62b389ee7072">

Closes #10041
2024-06-04 15:06:08 +00:00
Radosław Waśko
7cf80f3196
Handle UTF BOM when decoding text (#10130)
- Improve BOM handling: detect and skip the BOM character, Default encoding that detects encoding based on BOM if present, warnings if unexpected BOM is encountered.
- Closes #9849
- Windows-1252 fallback will be done as a separate PR as it has additional complexity. Tracked in ticket #10148.
2024-06-04 13:22:19 +00:00
GregoryTravis
1c2955655c
Add no_warning_propagation flag to Vector.build and Builder.new (#10079)
* no_warning_propagation flag

* doc

* benchmarks

* change to propagate_warnings, test

* fix java benchmark

* add benchmark to main
2024-06-04 09:06:09 -04:00
James Dunkerley
f1a1901058
Bug fix for read by line. (#10161)
Needs to pass -1 not Nothing to JSON.
2024-06-03 21:33:45 +00:00
Jaroslav Tulach
9632f04e9b
Directly allocate warnings asVectorEnsoObjects (#10143) 2024-06-01 06:23:49 +02:00
AdRiley
af4177e61f
Make table default visualisation (#10128)
Makes table the default vis type.

Before:
![image](https://github.com/enso-org/enso/assets/1720119/8a5d113a-ac06-4dd8-afdb-151cdbc149ab)

After:
![image](https://github.com/enso-org/enso/assets/1720119/d7c33cf7-355d-4517-8967-1963ff506f97)

Closes #10093
2024-05-31 15:02:02 +00:00
James Dunkerley
ffdfc7195e
Linter and small widget change (#10136)
Linter issues to allow doc site generation.
Small change in order for aggregate column.
2024-05-31 12:30:38 +00:00
AdRiley
06327f8fde
Add statistic product (#10122)
Add Statistic.Product

![image](https://github.com/enso-org/enso/assets/1720119/f7fc7bb5-9efe-4dbe-9150-cd9e5101c553)
2024-05-31 09:29:52 +00:00
Radosław Waśko
233f28235a
Small fixes to Postgres integration (#10105)
- Better message when saving datalink in disabled Output context:
![image](https://github.com/enso-org/enso/assets/1436948/540d615b-79ff-4811-8262-a0475a7b6923)
Before it was:
![image](https://github.com/enso-org/enso/assets/1436948/51198bf1-1e50-41bc-a56b-f829bc32d09a)

- Hack to get Postgres widget to display connection options:
![image](https://github.com/enso-org/enso/assets/1436948/39f3db39-1163-4815-b59f-c629d812e2ab)
Before the `Postgres` constructor was created without any parameters and it was not showing any parameters for modification.
2024-05-28 14:34:44 +00:00
Dmitry Bushev
858e646328
Start Ydoc with the language server (#9862)
- related #7954

Changelog:
- update: Ydoc starts with the language server on the `localhost:1234` by default. The hostname and ports can be configured by setting environment variables `LANGUAGE_SERVER_YDOC_HOSTNAME` and `LANGUAGE_SERVER_YDOC_PORT`
- update: by default `npm dev run` uses the node Ydoc server. You can control it with `POLYGLOT_YDOC_SERVER` env variable. For example,
```
env POLYGLOT_YDOC_SERVER='true' npm --workspace=enso-gui2 run dev
```
To connect to the Ydoc server running on the 1234 port (the one started with the language server)
⠀
```
env POLYGLOT_YDOC_SERVER='ws://127.0.0.1:1235' npm --workspace=enso-gui2 run dev
```
To connect to the provided URL. Can be useful for debugging when you start a separate Ydoc process.
- update: run `npm install` before the engine build. It is required to create the Ydoc JS bundle.
2024-05-28 13:51:42 +00:00
AdRiley
28bd5c522b
Add named_pattern and make it usable in Text_Replace (#10040)
Creates a new type Named_Expression and allows it to be used in text_replace

![image](https://github.com/enso-org/enso/assets/1720119/673a62a1-1ce5-4e1f-8289-0fa10e87b9de)

![image](https://github.com/enso-org/enso/assets/1720119/2f555e9b-d8c5-41e3-8000-d959d5818666)

![image](https://github.com/enso-org/enso/assets/1720119/d2ae2b0f-5bc4-4e9e-a391-402b58ee72d5)

![image](https://github.com/enso-org/enso/assets/1720119/3bf0547f-c2c7-4987-a45e-d922be2b0bae)
2024-05-27 13:01:26 +00:00
James Dunkerley
ab4b1f0f35
Add day_of_week and day_of_year to Column and DB_Column (#10081)
- Adds support for getting the weekday as an integer (1 Monday - 7 Sunday - ISO standard).
- Add support for getting the day of year.
2024-05-27 11:29:25 +00:00
James Dunkerley
200da1ad50
Remove old GUI1 ICONs and mapping function (#10086)
* Remove old GUI1 ICONs and mapping:

```
  dataframe_clean: 'table_clean',
  dataframe_map_row: 'map_row',
  dataframe_map_column: 'column_add',
  dataframes_join: 'join2-1',
  dataframes_union: 'union',
  sigma: 'transform4',
  io: 'in_out',
  date_and_time: 'time',
  spatial: 'location',
  predictive: 'predict',
  machine_learning: 'robot',
  split_text: 'split',
```

* Linter.

* Linter (2).

* Sort out some Returns:

* Sort out some ? Example
2024-05-27 11:17:21 +01:00
James Dunkerley
d8059fd22c
Some tweaks following Steve's testings (#10042)
- Add ranged number widget to `at` and `get`.
- Add defaults to `at` and `get` picking the first item.
- PRIVATE on various Excel_Workbook methods. It still works like a connection but not shown in CB.
2024-05-27 09:04:29 +00:00
AdRiley
adfe2dbab4
Add expression alias (#10067)
We call formulas expressions so having expression as well as formula makes sense as an alias.
2024-05-24 14:13:00 +00:00
AdRiley
084be279c5
Fix input_number icons (#10066)
icon is called input_number not number_input

![image](https://github.com/enso-org/enso/assets/1720119/c68790d3-05ad-4d17-8423-df3634fff29a)
2024-05-24 11:30:06 +00:00
Adam Obuchowicz
8ac0ee41c6
Allow picking non-existing files in write component (#10047)
Fixes #9984

Added a flag to `File_Browse` widget configuration specifying if we require an existing file. Also, the FileBrowserWidget will match against all "Writable_File", also displaying Save dialog.
2024-05-24 09:21:32 +00:00
Radosław Waśko
9d75f79ff9
Cloud Integration updates: renames in file metadata structure, remove path resolver workaround, partial fix for datalink resolver issue (#10050)
- Supersedes #9966 as I wanted to test these changes in one go.
- Fixes #10037 caused by lack of CI check and my oversight (forgot to run full tests after a minor change).
- Fixes a regression after [file metadata fields were renamed](c09d856ac8 (diff-9f59b6a0ee3155efecdc70c1ea0c90ab5cde00b5623d84363118b1793f941c46R2037)).
- Fixes handling of creating new datalinks and using them after cache was cleared (e.g. workflow restart).
- This was caused by troubles with path resolver.
- The fix addresses the most common issue and adds a test for it (test flushes the caches to ensure path resolver is used instead of the cached value).
- Some related issues were discovered on the cloud side, tracked by https://github.com/enso-org/cloud-v2/issues/1252
2024-05-23 16:05:48 +00:00
Jaroslav Tulach
16c1b74218
Enso Library Feature to execute (a bit of) Base_Tests (#9997) 2024-05-23 08:20:19 +02:00
Radosław Waśko
1e0649fda1
Improvements to Table.union (#9968)
- Closes #9952
2024-05-22 09:38:10 +00:00
marthasharkey
517299bb09
Add new aliases for take and drop methods (#10022)
Adds the aliases 'keep' and 'remove' to take/drop methods
2024-05-22 08:02:48 +00:00
James Dunkerley
2677e556e8
Align headers in doc comments... (#10025)
- Always use `Arguments:` for parameters or fields.
- Always use `> Example` for examples.
- Align sections behind `!` or `?` headers.
- Whitespace fixes.
2024-05-21 19:44:58 +00:00
AdRiley
3b59b40657
Update test framework to default to only show failures (#9986)
* Update test framework to default to only show failures

* Fix
2024-05-21 11:23:23 +03:00
James Dunkerley
31dd5944d7
Some fixes to make linter happy (#10004)
- A few import orderings.
- Missing doc comments on Text_Cleanse.
- Indent correction.
2024-05-20 16:15:11 +00:00
AdRiley
c7476c10f3
Add data cleanse component (#9879)
* Signature

* More API work

* Red

* Still red

* Green

* Red

* Green

* Red

* Green

* Refactor

* Refactor

* Refactor

* Red

* Green

* Red

* Green

* Red

* Green

* More tests

* non-ascii

* Ordering tests

* Remove tabs

* Numbers and Letters

* Changelog.md

* Add documentation

* Tests for non-text columns

* Add punctuation

* Add symbols

* Fix

* Refactor

* Refactor

* Move to base

* Fix

* Start of in-db tests. Not working

* DB versions

* Update widget

* Fix widgets

* Move tests to base

* Code review changes
2024-05-20 09:33:18 +01:00
James Dunkerley
3b29de313d
Update ICONs, linting fixes and add select_columns ALIASes. (#9985)
- Linting fixes.
- Add ICONs to all functions missing them.
- Add ALIASes for select columns.
2024-05-17 19:24:17 +00:00
GregoryTravis
4d49b00375
Combine builders for Vector.build and Vector.new_builder (#9922)
We have decided to keep the old new_builder, and to combine the new and old builders into one builder.
2024-05-17 16:18:47 +00:00
AdRiley
5c9ee94812
Make parse_fast_json private (#9978)
Micro MR to make deprecated method private
2024-05-17 15:47:11 +00:00
Radosław Waśko
38ad7b0afa
Creating datalinks from code (#9957)
- Closes #9673
- Adds the ability to save an existing Postgres connection as a datalink into the Enso Cloud, automatically promoting plain-text passwords into a Secret.
- Fixes dataflow error propagation in `JS_Object.from_pairs`.
2024-05-16 13:29:41 +00:00
Radosław Waśko
a2481036e6
Prepare for the Path resolver change to GET (#9958)
- Prepares for the Cloud API change, together with a fallback for the old API to avoid problems during migration.
- This PR should be merged before the https://github.com/enso-org/cloud-v2/pull/1236 PR is _deployed_.
2024-05-15 13:00:32 +00:00
James Dunkerley
b2aeb9fc84
Moving away from Integer | Nothing to Rows_To_Read for limiting number of rows. (#9925)
- Added a new `Rows_To_Read` type with conversions from `Nothing` and integers.
- Updated `read` on `Table`, `Column`, `DB_Table` and `DB_Column`.
- Updated `Delimited_Format.Delimited` to use `Rows_To_Read` for `row_limit`.
- Updated `Excel_Format.Sheet` and `Excel_Format.Range` to use `Rows_To_Read` for `row_limit`.
- Updated `Excel_Workbook.read` to use `Rows_To_Read`.
- Updated `Connection.read` (in all connection types) to use `Rows_To_Read`.

![image](https://github.com/enso-org/enso/assets/4699705/553c027f-f4c3-4855-9f51-2c4bcaec48a0)

![image](https://github.com/enso-org/enso/assets/4699705/a06c3912-77e0-4c10-abb8-73aed667458d)
2024-05-14 16:31:26 +00:00
James Dunkerley
2d6735edc7
Allow a Table to be used to rename_columns. (#9940)
Following an example from Steve, adjusted `rename_column` to allow a `Table` to define the mapping.

- A single Text column table gives a list of new names.
- A two Text column table gives a map from old name to new name.
- If a `DB_Table` is passed errors and tells the user to materialize the table.

![image](https://github.com/enso-org/enso/assets/4699705/3ed98330-1fce-465e-bf96-4a86e0872dd3)
2024-05-14 15:57:10 +00:00
Radosław Waśko
5f0a16c87c
Audit Logs for Postgres connections opened through a data link (#9873)
- Closes #9599
- Implemented API for sending audit logs to the cloud on a background thread.
- If the Postgres connection is opened through a datalink, its internal JDBC connection is replaced by a wrapper that reports executed queries to the audit log.
- Also introduces `EnsoMeta` - a helper Java class that can be used in our helper libraries to access Enso types.
- I have replaced the common pattern scattered throughout the codebase with calls to this 'library' to avoid repetitive code.
- Refactored `Table.display` to share code between in-memory and DB - it was needed as the function stopped working for `DB_Table` after adding making the `Table` constructor `private`.
- Clearer error when reading a SQLite database from a remote file (tells the user to download it first).
- Follow up - correlate asset id of the data link:
#9869
- Follow up - include project name (once bug is fixed):
#9875
- Some problems/improvements of the audit log:
- The audit log system is not yet ready for high throughput of logs
#9870
- The logs may be lost if `System.exit` is used
#9871
2024-05-11 08:54:33 +00:00
James Dunkerley
d97754da17
Error messages for rename_columns and Vector.duplicates (#9917)
- Improve error message for `rename_columns`.
- Add `length` to `Set` and `Map`.
- Add `duplicates` to `Vector` (and `Array`).

![image](https://github.com/enso-org/enso/assets/4699705/623df253-52e8-4bdc-a69c-ac8dc3ca594e)
2024-05-10 17:43:50 +00:00
GregoryTravis
720d32cbe3
Convert most remaining uses of Vector.new_builder to Vector.build (#9891) 2024-05-09 15:52:53 +00:00
Radosław Waśko
1d8fc1bf6e
Update endpoint name after it was updated in the Cloud (#9912)
Micro PR updating our API integration to the Cloud-side changes.
2024-05-09 15:17:19 +00:00
AdRiley
e25ec96aaa
Add table running variance skew sd and kurtosis (#9854)
Adds support for Variance, Skew, Standard Deviation and Kurtosis to Table.Running.
2024-05-09 08:45:29 +00:00
James Dunkerley
ce360f9002
Fix the group colours for consistent colouring. (#9893)
Fix the group colour to a fixed set.

![image](https://github.com/enso-org/enso/assets/4699705/476046ce-0ce3-4f98-9a75-a51f2e55f4e6)
2024-05-08 16:52:42 +00:00
James Dunkerley
4e758e5c9d
Undo change to autoscoping. (#9881)
Can't use auto-scoping for filter yet.
2024-05-08 07:52:12 +00:00
GregoryTravis
a2c36def26
Add Vector.build_multiple and handle dataflow error and warning propagation in build and build_multiple (#9766) 2024-05-07 19:30:21 +00:00
James Dunkerley
c0fd6eed2d
Small tweaks from Ned, Mark, Cass and Steve's feedback. (#9859)
- Rename `.info` to `.column_info`.
- Rename `.select_by_type` to `.select_columns_by_type`. (to be merged into `select_columns` functionality)
- Rename `.remove_by_type` to `.remove_columns_by_type`. (to be merged into `remove_columns` functionality)
- Adding "field" `ALIAS`es.
- Making `Table.Value` private (and hence hiding `java_table`).
- Making `Column.Value` private (and hence hiding `java_column`, added method to `Storage` to allow tests to work).
- Making `DB_Table.Value` private (and hence hiding `connection`, `context` and `internal_columns`).
- Making `DB_Column.Value` private (and hence hiding `connection`, `sql_type_reference`, `expression` and `context`).
- Add widgets for everywhere `Filter_Condition` used (fix missing entry for lower, can't use auto-scoping).
![image](https://github.com/enso-org/enso/assets/4699705/bad7d5c8-cd01-4384-a1c8-dbcd7a5ad92b)
- Fix for widgets in `Simple_Calculation`: `date_add` and `format`.
![image](https://github.com/enso-org/enso/assets/4699705/14eeef2e-5069-4a88-8b68-eb675497addf)
![image](https://github.com/enso-org/enso/assets/4699705/75d4e097-f171-4bf6-a27e-f9d396c44a92)
- Fix for nested `Row`, `Column` and `DB_Column` rendering in Table Viz.
![image](https://github.com/enso-org/enso/assets/4699705/8c50c519-7c34-4228-bab7-be938a6ce1bf)
![image](https://github.com/enso-org/enso/assets/4699705/c83ad301-f31e-451d-a8be-b022e1456b4e)

# Important Notes
- Can't use `.` to access a private constructor field when mapping, have to use a lambda.
- We should agree convention for private backing fields which are wrapped by a public getter (went with `internal_` for this pass).
2024-05-06 13:49:12 +00:00
Dmitry Bushev
a44bb2b1b1
Language Server API for AI (#9679)
close #9656

Changelog:
- add: `ai/completion_v2` method
- add: `Visualization.AI.print` method for converting the expression to text format
- update: The default system prompt was updated to tell AI to use the `Visualization.AI.print` method for printing.

# Important Notes
The project [New_Project_1.zip](https://github.com/enso-org/enso/files/15152993/New_Project_1.zip) contains the following main file:
```py
from Standard.Base import all
from Standard.Table import all
from Standard.Database import all
from Standard.AWS import all
import Standard.Visualization
import Standard.Visualization.Warnings
from Standard.Base.Errors.Common import Dry_Run_Operation

type Student
Value id region

main =
operator70395 = 226
operator47321 = 'east'
operator76980 = Student.Value operator70395 operator47321
operator31302 = operator47321.words True
operator91574 = 1
operator34358 = operator47321.take (Index_Sub_Range.By_Index [0, operator91574])



#### METADATA ####
[[{"index":{"value":0},"size":{"value":4}},"4cf8de7f-2014-4dfd-9ceb-0164fe26c8bf"],[{"index":{"value":0},"size":{"value":29}},"389fe7a5-e59b-440a-8801-e0bd86716094"],[{"index":{"value":0},"size":{"value":558}},"a8a81ce3-199e-479e-98dd-5f919425842d"],[{"index":{"value":5},"size":{"value":8}},"c6f0de99-3bd3-459d-a69b-5fc774dd1e3f"],[{"index":{"value":5},"size":{"value":13}},"05827411-7354-478b-99d8-8370dadd560b"],[{"index":{"value":13},"size":{"value":1}},"41de94cc-ccd2-4ccf-b9db-cf686443efa0"],[{"index":{"value":14},"size":{"value":4}},"8abd24ea-b1cb-4c10-ad16-5a190f7ed0c7"],[{"index":{"value":19},"size":{"value":6}},"5c57a62d-ed82-4195-8b78-2ed660556dd0"],[{"index":{"value":26},"size":{"value":3}},"23544334-1117-4dbc-ac7f-bb3f84575264"],[{"index":{"value":30},"size":{"value":4}},"6058ba89-dc1b-45e1-b8f3-d1a4d27975f7"],[{"index":{"value":30},"size":{"value":30}},"bdd4b4cc-9c1e-4554-b0fb-3f73dcd5a590"],[{"index":{"value":35},"size":{"value":8}},"7e7bb9b4-8e9b-4700-9dcd-83c353800da5"],[{"index":{"value":35},"size":{"value":14}},"524bafb6-d3ca-4ee5-b8de-84b5c47ca87b"],[{"index":{"value":43},"size":{"value":1}},"b68b3929-8b63-49ad-977c-b04e058f8ee4"],[{"index":{"value":44},"size":{"value":5}},"7173f374-5f53-462b-ab1b-f7b1845d729b"],[{"index":{"value":50},"size":{"value":6}},"911d381e-1bdb-4e9c-826d-a00495c8fd23"],[{"index":{"value":57},"size":{"value":3}},"2f8fa1ee-9d13-4a73-99b2-bc77d1656387"],[{"index":{"value":61},"size":{"value":4}},"11abd658-4888-476c-afcd-592edaaa53c1"],[{"index":{"value":61},"size":{"value":33}},"9f6ae949-410b-48e6-bb75-84d4ba8a1989"],[{"index":{"value":66},"size":{"value":8}},"f4726e40-dd5b-49a0-aa2a-6cea6df88d5d"],[{"index":{"value":66},"size":{"value":17}},"539b4dcf-4487-49b6-9c90-6ae58d51c2db"],[{"index":{"value":74},"size":{"value":1}},"a34a41a0-5dcf-4c47-a36c-a4f427c87c03"],[{"index":{"value":75},"size":{"value":8}},"8fdf4ee7-2e74-4cac-b9b9-4230183f91c5"],[{"index":{"value":84},"size":{"value":6}},"7ff7efdc-fed6-4cb6-bed6-392d110ff991"],[{"index":{"value":91},"size":{"value":3}},"eacff407-dc6d-4f17-a28b-329f7a0582e1"],[{"index":{"value":95},"size":{"value":4}},"f2feac9f-ea02-4070-879a-f1d493473993"],[{"index":{"value":95},"size":{"value":28}},"f9b37b67-fb54-4e47-8281-e922edb4c5d3"],[{"index":{"value":100},"size":{"value":8}},"6cf398f8-d0c9-42fa-a6e6-e740de11562f"],[{"index":{"value":100},"size":{"value":12}},"28e4d0e3-7da4-4ee2-94c7-d9a7c692f8de"],[{"index":{"value":108},"size":{"value":1}},"955e8788-816c-4eb7-b5ec-020b4863536a"],[{"index":{"value":109},"size":{"value":3}},"bdc65807-0438-4296-8b1f-1fab3a885c17"],[{"index":{"value":113},"size":{"value":6}},"fe3dd265-1855-4d66-8a2d-51153b21e699"],[{"index":{"value":120},"size":{"value":3}},"3b7a14ac-3224-4b9a-b474-f6136a658f8b"],[{"index":{"value":124},"size":{"value":6}},"7f7f18a4-06ef-46e0-bddd-b9fe60878855"],[{"index":{"value":124},"size":{"value":29}},"d49e0fe3-a638-4d7e-ba58-12db7abad20f"],[{"index":{"value":131},"size":{"value":8}},"c9411433-5e73-4927-9175-15d3b1ccb0ef"],[{"index":{"value":131},"size":{"value":22}},"b53ad817-ea4c-491e-aaf8-49967e22aff6"],[{"index":{"value":139},"size":{"value":1}},"837a4d65-8af6-4860-ae0e-ad9172af6ad2"],[{"index":{"value":140},"size":{"value":13}},"3b662906-1887-4dd6-a0e2-a222ace135ce"],[{"index":{"value":154},"size":{"value":6}},"548ca873-544f-48c9-b7ab-6aa1e7f03d72"],[{"index":{"value":154},"size":{"value":38}},"7c45872c-22b4-4f37-bcd7-b3791408f737"],[{"index":{"value":161},"size":{"value":8}},"21152f0d-4222-48e4-9376-a2a28f8f6be6"],[{"index":{"value":161},"size":{"value":22}},"1cb71d49-a06d-4dd9-bc1b-54603b416601"],[{"index":{"value":161},"size":{"value":31}},"faa46cf2-3e90-44ed-9d9e-77b7f874b338"],[{"index":{"value":169},"size":{"value":1}},"6d5fdcd5-0ddf-4c83-aafe-a446ac5ce461"],[{"index":{"value":170},"size":{"value":13}},"830d73c2-1898-41a9-8046-22d86f159ef0"],[{"index":{"value":183},"size":{"value":1}},"ab897380-c596-4182-a2c5-b7b23cf59893"],[{"index":{"value":184},"size":{"value":8}},"29f35cf2-d79d-44d0-96d0-79b11589726c"],[{"index":{"value":193},"size":{"value":4}},"45bf8220-52f2-49f4-bbcf-9566d1f1b1c4"],[{"index":{"value":193},"size":{"value":57}},"478005ac-f0ca-497a-b298-554bd64946a9"],[{"index":{"value":198},"size":{"value":8}},"c11511ce-252c-4b84-9f61-dfa5544d4916"],[{"index":{"value":198},"size":{"value":13}},"dbae65a1-bdca-44b0-a497-7934d1e42616"],[{"index":{"value":198},"size":{"value":20}},"355392ad-8c81-4304-a6d0-2bbf3690da0c"],[{"index":{"value":198},"size":{"value":27}},"0720ccb5-ebd3-4c29-8113-0aa1f251e873"],[{"index":{"value":206},"size":{"value":1}},"f5a8d310-1f58-456e-84a3-78490f05ff13"],[{"index":{"value":207},"size":{"value":4}},"10046e78-5164-407c-adef-2ab11ce77b4c"],[{"index":{"value":211},"size":{"value":1}},"ed23da0c-9193-4eea-ab9c-fee90ba8f924"],[{"index":{"value":212},"size":{"value":6}},"7a09de10-35e2-45e0-9eb8-f79fa054b733"],[{"index":{"value":218},"size":{"value":1}},"b85a0981-9d82-4204-a03a-cd3a4720c72d"],[{"index":{"value":219},"size":{"value":6}},"1b04c19d-cdf0-4661-8f4e-d980cf12c996"],[{"index":{"value":226},"size":{"value":6}},"87d57f2c-e009-48a7-98bb-0e1612905564"],[{"index":{"value":233},"size":{"value":17}},"60fd6a1c-7ada-4546-ae20-e1f8b432b4fd"],[{"index":{"value":252},"size":{"value":4}},"9558affa-b913-4576-8ee3-6af22985e0f6"],[{"index":{"value":252},"size":{"value":32}},"8479091c-9232-49d6-b375-32f89693c389"],[{"index":{"value":257},"size":{"value":7}},"3d1f13f5-e22a-4cb6-9e75-9c717e016922"],[{"index":{"value":264},"size":{"value":1}},"aa429f07-973a-4c69-8e3f-74aa4780f85a"],[{"index":{"value":269},"size":{"value":5}},"5b304cf6-5ec4-4b23-ad0d-20e38b41cdcd"],[{"index":{"value":269},"size":{"value":15}},"e3315ef2-d0ec-43b1-b8f6-5b436054aaee"],[{"index":{"value":275},"size":{"value":2}},"bc7205c4-68bf-461e-a1ba-31400b9337ff"],[{"index":{"value":278},"size":{"value":6}},"78870c05-9117-4e24-bf77-a72e7aef9c18"],[{"index":{"value":286},"size":{"value":4}},"517c743f-c775-4031-8f87-222d0f0a365f"],[{"index":{"value":286},"size":{"value":271}},"d0d107bc-4997-41d6-9c20-32295590eaac"],[{"index":{"value":291},"size":{"value":1}},"91174930-f18b-4322-84cc-deb88637d012"],[{"index":{"value":292},"size":{"value":265}},"f09a4372-3231-4f2e-99f4-84aa751f9b60"],[{"index":{"value":297},"size":{"value":13}},"613df8c9-0a40-4c94-886f-9668c2c360c2"],[{"index":{"value":297},"size":{"value":19}},"5b88a4d4-6840-4a25-a862-8b5d0b75303d"],[{"index":{"value":311},"size":{"value":1}},"6bfe1dea-09df-4358-9613-74d8326bc680"],[{"index":{"value":313},"size":{"value":3}},"e3b7fac7-0c08-4f20-8a0c-a58f1b118097"],[{"index":{"value":321},"size":{"value":13}},"37d007fe-9cb6-4f2e-a8b7-30842eac60b2"],[{"index":{"value":321},"size":{"value":22}},"4233204a-543d-437a-a9fc-9f79853a9540"],[{"index":{"value":335},"size":{"value":1}},"6c01783b-27ba-413c-84a4-6a94a37d714a"],[{"index":{"value":337},"size":{"value":1}},"34bc865a-58b5-4c49-bf85-a8cfedaf9b54"],[{"index":{"value":337},"size":{"value":6}},"bb556514-570d-4a87-8b2a-6e7f198a975f"],[{"index":{"value":338},"size":{"value":4}},"87c2cedc-898e-43c3-be3d-24e15689c373"],[{"index":{"value":342},"size":{"value":1}},"13387ff5-ec5e-453a-86f2-deae721b5549"],[{"index":{"value":348},"size":{"value":13}},"6280a86b-5469-43f3-94d5-1e1726028a54"],[{"index":{"value":348},"size":{"value":57}},"a17c5e55-5e99-4e4f-961d-9b4072ecd6e4"],[{"index":{"value":362},"size":{"value":1}},"c0720382-9ce4-451a-a04b-f248143d2528"],[{"index":{"value":364},"size":{"value":7}},"9cd51399-d9d2-4898-9bcb-be2f540999ff"],[{"index":{"value":364},"size":{"value":13}},"b4980bfc-13cc-429f-bc43-9edfc07d2406"],[{"index":{"value":364},"size":{"value":27}},"1a3f6a8c-ab27-43dd-a494-bf413334fd9a"],[{"index":{"value":364},"size":{"value":41}},"71a97e88-9b19-4ec9-b3c8-ffd2940c4cb8"],[{"index":{"value":371},"size":{"value":1}},"99f38c61-3ea7-4b02-be71-f293fbecb7d7"],[{"index":{"value":372},"size":{"value":5}},"08820818-ddb9-408a-b913-e9382e5b6dc7"],[{"index":{"value":378},"size":{"value":13}},"2cf06b4e-af9f-492e-ac13-5723a896a508"],[{"index":{"value":392},"size":{"value":13}},"6c887279-5998-45ce-81ab-c37fdeb03144"],[{"index":{"value":410},"size":{"value":13}},"6c126d22-c018-46cc-bdae-cf8b89104173"],[{"index":{"value":410},"size":{"value":40}},"a65c1fa8-9fce-4c30-bef7-d85ff1b428eb"],[{"index":{"value":424},"size":{"value":1}},"208c049e-e221-49ce-99d7-96142f2d1b1c"],[{"index":{"value":426},"size":{"value":13}},"0adf4ddd-1402-4b96-bfb0-1917cc275063"],[{"index":{"value":426},"size":{"value":19}},"f2158ac6-2dd5-482d-9f3d-0bf4812d6d6e"],[{"index":{"value":426},"size":{"value":24}},"c7bbc3e7-1377-429e-b60a-0c2d2cd4ea74"],[{"index":{"value":439},"size":{"value":1}},"1a17a7ee-31d7-4885-84ce-c902bff4ea4a"],[{"index":{"value":440},"size":{"value":5}},"47016cac-40a0-45f6-8057-2b81edb052a5"],[{"index":{"value":446},"size":{"value":4}},"ce4fa4cd-5064-4752-9dbc-5fcf4e92138a"],[{"index":{"value":455},"size":{"value":13}},"3300001d-13ea-45f4-acbe-959813bcf85b"],[{"index":{"value":455},"size":{"value":17}},"b8f5900b-2370-4b21-8a0c-06d3bbb45068"],[{"index":{"value":469},"size":{"value":1}},"19ea3187-83d6-47d1-994c-64cf0ce9af3b"],[{"index":{"value":471},"size":{"value":1}},"9457c3f8-c878-4f01-a5c7-39336f163a28"],[{"index":{"value":477},"size":{"value":13}},"a5139627-05a8-4838-b171-02666c62c348"],[{"index":{"value":477},"size":{"value":80}},"8b5c864c-055d-4d11-a470-2679328fc131"],[{"index":{"value":491},"size":{"value":1}},"eb01b0af-06cf-4d75-81ad-d0bf7664b2d7"],[{"index":{"value":493},"size":{"value":13}},"6c042eb7-e895-473d-b456-6e0eec23e958"],[{"index":{"value":493},"size":{"value":18}},"f40730ff-ac6c-45db-8970-534f3becda2a"],[{"index":{"value":493},"size":{"value":64}},"b2df8ba8-683c-45c2-9932-8e1c970c799a"],[{"index":{"value":506},"size":{"value":1}},"2289bf31-74dd-431c-97f1-3ac179715453"],[{"index":{"value":507},"size":{"value":4}},"6db4523a-0bcd-4ba5-b6c2-7720db57d93c"],[{"index":{"value":512},"size":{"value":1}},"b6faea17-664d-470a-9641-999b6a2be5bd"],[{"index":{"value":512},"size":{"value":45}},"98aea277-bc44-4ceb-85a6-24ee453b7f72"],[{"index":{"value":513},"size":{"value":15}},"c0ae1c05-026d-4712-a571-979ce260b310"],[{"index":{"value":513},"size":{"value":24}},"502e8e4d-5105-4abe-8dba-4d7293cbe80b"],[{"index":{"value":513},"size":{"value":43}},"71cee748-378b-4817-be0e-b87bc1d3d847"],[{"index":{"value":528},"size":{"value":1}},"fcaad3ac-e080-4170-9d63-b2c01295447d"],[{"index":{"value":529},"size":{"value":8}},"a053e05a-4862-4407-abe8-2746784bdfc3"],[{"index":{"value":538},"size":{"value":1}},"9adae78d-7667-4865-83e9-b64464b13696"],[{"index":{"value":538},"size":{"value":18}},"92f153e3-da05-439e-bb4f-596ec8cfa9c8"],[{"index":{"value":539},"size":{"value":1}},"dddedc5e-415d-4f5d-ab1b-19468aa079a9"],[{"index":{"value":540},"size":{"value":1}},"4adce2e5-1e60-4a63-9ff3-5781d967b704"],[{"index":{"value":542},"size":{"value":13}},"ef06ed1e-246f-4106-a4c0-1e648b8ad5e7"],[{"index":{"value":555},"size":{"value":1}},"7bd1cc5f-20b1-4dd2-8a34-ccc40f5f091a"],[{"index":{"value":556},"size":{"value":1}},"720adbd6-24b0-49ac-b123-61cc037a9636"]]
{"ide":{"node":{"e3b7fac7-0c08-4f20-8a0c-a58f1b118097":{"position":{"vector":[-224,13]}},"bb556514-570d-4a87-8b2a-6e7f198a975f":{"position":{"vector":[115,19]}},"71a97e88-9b19-4ec9-b3c8-ffd2940c4cb8":{"position":{"vector":[-224,-51]}},"c7bbc3e7-1377-429e-b60a-0c2d2cd4ea74":{"position":{"vector":[360,19]},"visualization":{"show":false,"fullscreen":false,"width":200}},"f2158ac6-2dd5-482d-9f3d-0bf4812d6d6e":{"position":{"vector":[297,-45]},"visualization":{"show":false,"fullscreen":false,"width":96}},"9457c3f8-c878-4f01-a5c7-39336f163a28":{"position":{"vector":[640,40]},"visualization":{"show":false,"fullscreen":false,"width":200}},"b2df8ba8-683c-45c2-9932-8e1c970c799a":{"position":{"vector":[219,-80]},"visualization":{"show":true,"fullscreen":false,"width":200}},"f40730ff-ac6c-45db-8970-534f3becda2a":{"position":{"vector":[219,-48]}}},"import":{}}}
```

To test the functionality, I asked AI to show me the result of the `operator70395` variable:

1. Init protocol connection
```json
{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}
```

2. Create execution context
```json
{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}
```

3. Push the main method
```json
{"jsonrpc":"2.0","id":0,"method":"executionContext/push","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","stackItem":{"methodPointer":{"module":"local.New_Project_1.Main","definedOnType":"local.New_Project_1.Main","name":"main"},"positionalArgumentsExpressions":[],"type":"ExplicitCall"}}}
```

4. Ask AI for the variable contents
```json
{"jsonrpc":"2.0","id":1,"method":"ai/completion_v2","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","expressionId":"f09a4372-3231-4f2e-99f4-84aa751f9b60","prompt":"There is 'operator70395' variable defined in the program. What is the result of the variable 'operator70395'?"}}
```

I got the following responses:
```json
{"jsonrpc":"2.0","method":"ai/completionProgress","params":{"code":"Visualization.AI.print(operator70395)","reason":"To provide the result of 'operator70395', I need to know its current value.","visualizationId":"edfb00a3-6ce5-41e1-bb8f-ab191809114e"}}
```
```json
{"jsonrpc":"2.0","id":1,"result":{"Success":{"fn":"def get_operator70395_result():\n    return operator70395","fnCall":"get_operator70395_result()"}}}
```
2024-05-02 16:55:06 +00:00
AdRiley
5350b2d00d
Refactor add row number (#9822)
* Refactor add row number

* Refactor

* Green

* Green

* Remove dead code

* Cleanup

* Deduplicate check
2024-05-02 12:29:54 +01:00
James Dunkerley
d2e6ff260e
Restructure SQLite_Details. (#9832)
```
type SQLite_Details
SQLite location:File|In_Memory

type In_Memory
```
to
```
type SQLite
From_File location:File

In_Memory
```

# Important Notes
Splits the In-Memory entry for Database Connect but still works nicely.

![image](https://github.com/enso-org/enso/assets/4699705/ec798ce0-9f41-4903-a2fd-722a9e37743c)

![image](https://github.com/enso-org/enso/assets/4699705/f233b055-893e-4c56-a23d-562e982560f6)
2024-05-01 22:15:41 +00:00
James Dunkerley
4d6d6f239c
Handle URL encoding automatically in query string. (#9823)
A small fix to automatically encode the query string.
Attaches a warning if needed.

![image](https://github.com/enso-org/enso/assets/4699705/032bdb59-6896-46c0-b970-f5a542cc6adf)

![image](https://github.com/enso-org/enso/assets/4699705/6b2075b9-3c98-4de2-8a34-c860ecd65d0c)
2024-04-30 22:03:46 +00:00
Radosław Waśko
f87c75a3da
Prepare for Cloud Environment variable name change (#9821)
Changing `ENSO_PROJECT_DIRECTORY_PATH` to `ENSO_CLOUD_PROJECT_DIRECTORY_PATH`.
2024-04-30 16:02:12 +00:00
AdRiley
d1bf4cb771
Add Ignored_Nothing_Values (#9770)
Add a `IgnoredNothing` warning for Table.Running

![image](https://github.com/enso-org/enso/assets/1720119/1941d278-2c33-43fe-a175-8bcc65bae51a)

![image](https://github.com/enso-org/enso/assets/1720119/b5f6b235-d939-4868-9490-de0f226ea1a2)

![image](https://github.com/enso-org/enso/assets/1720119/a1d617a6-a684-4cc1-be13-c4907d2e6876)
2024-04-30 13:30:40 +00:00
Pavel Marek
660c5e7a9d
Atom constructors can be private (#9692)
Closes #8836.

Atom constructors can be declared as private (project-private). project-private constructors can be called only from the same project. See the encapsulation.md docs for more info.

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
2024-04-29 14:43:18 +02:00
Radosław Waśko
16e1999040
Fix copy_to/move_to edge case when source and target are the same (#9798)
- Adds tests for an edge case where the source and target of copy/move operation are the same.
- And fixes the behaviour to be as expected.
2024-04-29 12:29:07 +00:00
Radosław Waśko
8e03e3be9c
Print pending groups in-line and include their reason, also add progress bar to the runner (#9796)
- Closes #9534 by printing the pending groups with pending reason
- Re-introduces original ordering of tests
- Adds a progress bar to the test suite runner in 'interactive' mode (if ANSI colors are enabled, progress bar will also be)
2024-04-26 16:05:05 +00:00
Jaroslav Tulach
c5bf2384e4
Removing Truffle API dependency from runtime-compiler project (#9785)
Fixes #8888 by removing dependency on Truffle API and `polyglot-api` from `runtime-compiler` project.
2024-04-26 13:14:22 +00:00
James Dunkerley
c5179309fd
Improve Table Viz handling of Numbers (#9793)
- Added support for NaN, Infinity and -Infinity in the Table viz.
- Simplified the logic so all the formatting in a single place in the Table viz.
- Sorted Number formatting for BigInt and < 10000.

![image](https://github.com/enso-org/enso/assets/4699705/1486eb7e-df87-4516-90d0-75fda1e9743d)

![image](https://github.com/enso-org/enso/assets/4699705/e55c0e6b-bd3c-4dcf-a65f-41f8516568a6)
2024-04-26 07:52:40 +00:00
Radosław Waśko
df4a7efcdd
Add AWS SSO JARs to the Standard.AWS library (#9782)
As reported by our users, when using the AWS SSO, our code was failing with:
```
Execution finished with an error: To use Sso related properties in the 'xyz' profile, the 'sso' service module must be on the class path.
```

This PR adds the missing JARs to fix that.

Additionally it improves the license review tool UX a bit (parts of #9122):
- sorting the report by amount of problems, so that dependencies with unresolved problems appear at the top,
- semi-automatic helper button to rename package configurations after a version bump,
- button to remove stale entries from config (files or copyrights that disappeared after update),
- button to add custom copyright notice text straight from the report UI,
- button to set a file as the license for the project (creating the `custom-license` file automatically)
- ability to filter processed projects - e.g. `openLegalReviewReport AWS` will only run on the AWS subproject - saving time processing unchanged dependencies,
- updated the license search heuristic, fixing a problem with duplicates:
- if we had dependencies `netty-http` and `netty-http2`, because of a prefix-check logic, the notices for `netty-http` would also appear again for `netty-http2`, which is not valid. I have improved the heuristic to avoid these false positives and removed them from the current report.
- WIP: button to mark a license type as reviewed (not finished in this PR).
2024-04-25 18:44:51 +00:00
Radosław Waśko
0c989c6938
Writing to Data Links (#9750)
- Closes #9292
- and also closes #9522
2024-04-25 17:55:49 +00:00
GregoryTravis
b150b091b0
Convert Vector.new_builder to Vector.build in Standard.Base (#9754)
* Vector.build

* new_builder deprecated

* convert one

* convert vector_spec

* limit param

* convert vector

* docs

* changelog

* wip

* rename limit to capacity

* wip

* review

* append comments

* review

* wip

* wip

* fix tests

* docs

* wip

* wip

* fix filter warning propagation

* doc weird dependencies

* fix tests

* cleanup

* check builder result
2024-04-25 10:38:29 -04:00
Jaroslav Tulach
0d495ffd97
Make conversion of double to BigDecimal exact (#9740)
Resolves #9607 by computing `Number.hash` by converting given number to `Float` first and then computing the hash. Also the conversion from `Float.to Decimal` is exact - done via `new BigDecimal(double)`. There is `Decimal.new` that handles the user-friendly conversion. However as a result `Decimal.from 2.1 != Decimal.new 2.1` - that's the only way to ensure consistency between hash code and conversions.
2024-04-25 11:22:50 +00:00
James Dunkerley
fb9cf38914
Excel_Workbook.read_many (#9759)
- Some minor linting fixes.
- Adjust `headers` parameter so a dedicated type.
![image](https://github.com/enso-org/enso/assets/4699705/989f464d-df95-410e-a03b-36661f1c4a37)
- Fix bug with `read` on an `Excel_Workbook` so error handled more gracefully and not panicking to UI.
![image](https://github.com/enso-org/enso/assets/4699705/23b4575f-daad-4719-a5cc-30d064bd7f7a)
- Fix bug when writing to a file with an `Excel_Format` with an invalid extension which was causing a panic.
![image](https://github.com/enso-org/enso/assets/4699705/dc0e055c-c1b6-482f-b129-eb69f6554d72)
- Add `read_many` to `Excel_Workbook` allowing reading more than one sheet at a time.
2024-04-24 13:16:44 +00:00
Jaroslav Tulach
ff62c1e25f
Access non-self arguments when widget annotations are computed (#9410) 2024-04-24 03:58:32 +02:00
AdRiley
6f3e649e7a
Make errors propagate out of each (#9742)
Make errors propagate out of each
2024-04-23 17:14:54 +00:00
AdRiley
4a97bfa31f
Add table running functionality for Sum, Mean, Min, Max. (#9577)
* Add Table.Running

* Code Review fixes

* Code Review changes

* Change null handling
2024-04-23 09:45:43 +01:00