Commit Graph

171 Commits

Author SHA1 Message Date
Radosław Waśko
dc55b4e9ce
Support for Table output in Data.read_many (#11546)
- Closes #11311
2024-11-29 12:06:30 +00:00
James Dunkerley
7363377331
Linting and Widgets (#11551)
- Fix any issues identified by the doc writer.
- Alter all `default_widget` functions: all now take display and all use the type check signature.
- Review widgets on AWS APIs and make sure display is correct.
2024-11-14 12:01:04 +00:00
Pavel Marek
701bba6504
Convert Array_Like_Helpers.map to a builtin to reduce stack size (#11363)
The ultimate goal is to reduce the method calls necessary for `Vector.map`.

# Important Notes
- I managed to reduce the number of Java stack frames needed for each `Vector.map` call from **150** to **22** (See https://github.com/enso-org/enso/pull/11363#issuecomment-2432996902)
- Introduced `Stack_Size_Spec` regression test that will ensure that Java stack frames needed for `Vector.map` method call does not exceed **40**.
2024-11-06 11:14:48 +00:00
Radosław Waśko
2619399799
Enso_File integration update: Multi-part upload and presigned URL for download (#11440)
- Closes #11330
- Closes #11331
2024-10-31 13:11:42 +00:00
Radosław Waśko
244effde0c
Enable audit logs for Snowflake backend (#11306)
- Closes #11292
- Tries to fix #11300
2024-10-14 14:30:42 +00:00
Radosław Waśko
3458fe4fe1
Accessing and modifying description and labels of Enso Cloud assets (#11255)
- Closes #11227
- Additionally, it should fix #11278 by ensuring that every scheduled message goes to the desired endpoint, by splitting each batch by endpoint.
2024-10-10 12:11:10 +00:00
Gregory Michael Travis
cce50fab3a
Extend the range of int/float arguments to outside the range of Java long, in ceil, floor, and truncate (#11135) 2024-10-07 11:36:17 -04:00
Radosław Waśko
0302670092
Reporting Enso test failures as annotations on the PR (#10821) 2024-10-02 16:01:28 +00:00
James Dunkerley
e6ea3b2607
Default Project, Templates and Docs (#10980)
- Linting updates.
- Add an `Examples.welcome` and adjust the start up project to use it.
- Merge all of Cass's work into the source code.
- Make example render in mono space font.
2024-09-05 14:36:08 +00:00
Jaroslav Tulach
b6f0675563
IO.print without new line (#10858) 2024-08-24 07:00:16 +02:00
James Dunkerley
422fa8c16b
Adding support for creating Atoms in expressions (#10820)
- Enables the `..` autoscoping style for creating Atoms in expressions.
- Add type checking to methods in columns.
- Auto wrap returns from method in expressions into a column as needed.
- Remove `Time_Period.Day` to remove confusion..
2024-08-15 15:52:30 +00:00
Radosław Waśko
b1958f8aa3
Adding vectorized implementations to some Column operations (#10795)
- Part of #6256 - implements operations that could have been vectorized without changes to the overall infrastructure
2024-08-13 08:53:39 +00:00
Kaz Wesley
e5b85bf16e
Space-precedence does not apply to value-level operators (#10597)
In a sequence of value-level operators, whitespace does not affect relative precedence. Functional operators still follow the space-precedence rules.

The "functional" operators are: `>> << |> |>> <| <<| : .`, application, and any operator containing `<-` or `->`. All other operators are considered value-level operators.

Asymmetric whitespace can still be used to form *operator sections* of value-level operators, e.g. `+2 * 3` is still equivalent to `x -> (x+2) * 3`.

Precedence of application is unchanged, so `f x+y` is still equivalent to `f (x + y)` and `f x+y * z` is still equivalent to `(f (x + y)) * z`.

Any attempt to use spacing to override value-level operator precedence will be caught by the new enso linter. Mixed spacing (for clarity) in value-operator expressions is allowed, as long as it is consistent with the precedences of the operators.

Closes #10366.

# Important Notes
Precedence warnings:
- The parser emits a warning if the whitespace in an expression is inconsistent with its effective precedence.
- A new enso linter can be run with `./run libraries lint`. It parses all `.enso` files in `distribution/lib` and `test`, and reports any errors or warnings. It can also be run on individual files: `cargo run --release --bin check_syntax -- file1 file2...` (the result may be easier to read than the `./run` output).
- The linter is also run as part of `./run lint`, so it is checked in CI.

Additional language change:
- The exponentiation operator (`^`) now has higher precedence than the multiplication class (`*`, `/`, `%`). This change did not affect any current enso files.

Library changes:
- The libraries have been updated. The new warnings were used to identify all affected code; the changes themselves have not been programmatically verified (in many cases their equivalence relies on the commutativity of string concatenation).
2024-07-24 10:55:44 +00:00
Radosław Waśko
7fd8701690
Snowflake Dialect pt. 6 - Union, Distinct and other improvements (#10576)
- Part of #9486
- Fixes `Table.union`, `merge` and `distinct` tests
- Replaces `distinct_on` in `Context` that was actually a Postgres specific addition leaking into the base with a more abstract `Context_Extension` mechanism.
- This allows us to implement the Snowflake-specific `DISTINCT` using `QUALIFY`.
2024-07-19 16:04:00 +00:00
James Dunkerley
15810a4675
Linting fixes. (#10589)
Generating Doc data so fixed linting.
2024-07-18 12:42:34 +00:00
Pavel Marek
0f9852aab2
Replace all from ... export all with explicit exports (#10369)
Replace all exports with explicit exports.
2024-07-11 19:34:25 +02:00
Jaroslav Tulach
220b40a1cd
Enforce conversion method return type & introduce Comparable.new (#10468) 2024-07-11 06:58:51 +02:00
Radosław Waśko
48c17845a7
Fixing Database tests and Snowflake Dialect - part 3 out of ... (#10458)
- Related to #9486
- Fixes types in literal tables that are used throughout the tests
- Tries to makes testing faster by disabling some edge cases, trying batching some queries, re-using the main connection and trying to re-use tables more
- Implements date/time type mapping and operations for Snowflake
- Updates type mapping to correctly reflect what Snowflake does
- Disables warnings for Integer->Decimal coercion as that's too annoying and implicitly understood in Snowflake
- Allows to select a Decimal column with `..By_Type ..Integer` (only in Snowflake backend) because the Decimal column there is its 'de-facto' Integer column replacement.
2024-07-10 13:21:30 +00:00
Jaroslav Tulach
ee0175c364
Support re-running failed tests with runEngineDistribution (#10483) 2024-07-10 07:05:29 +02:00
James Dunkerley
4b3e4ae15e
Rename Map to Dictionary and Set to Hashset. (#10474)
- Rename `Map` to `Dictionary`.
- Rename `Set` to `Hashset`.
- Add a deprecated place holder for the static method of `Map`.
2024-07-09 09:12:23 +00:00
James Dunkerley
0661f17d1c
Tune Text.trim, fix for Text.split (#10445)
- Rename `Location.Start` to `Location.Left`.
- Rename `Location.End` to `Location.Right`.
- Use auto-scoping for `Location`.
- Tune widgets for `Text.trim`.
- Correct signature of `Text.split`.
- Adjist `generateLocallyUniqueIdent` to not fail on bad signature.
2024-07-04 22:24:56 +00:00
James Dunkerley
9a2aed92f1
Separating list from read function and other small tweaks. (#10434)
- Rename `Faker.string_value` to `Faker.text_value`.
- Remove `Regex.pattern_string` as duplicate of `Regex.pattern`.
- Sort the Date picker.
- Rename `Data.list_directory` to `Data.list`.
- Remove support for reading a directory.

![image](https://github.com/enso-org/enso/assets/4699705/b42bb3c9-e63b-49f2-8cdc-4666cb6d968e)

![image](https://github.com/enso-org/enso/assets/4699705/97f49891-5ae5-4f0a-9a41-6200888fcd86)
2024-07-03 22:00:53 +00:00
Radosław Waśko
db4f7ab3b5
Fixing Database tests and Snowflake Dialect - part 2 out of ... (#10319)
- Part of #9486
- Fixing our tests to not rely on deterministic ordering of created Tables in Database backends
- Before, SQLite and Postgres used to mostly return rows in the order they were inserted in, but Snowflake does not.
- Fixing various parts of Snowflake dialect.
2024-06-27 14:54:00 +00:00
James Dunkerley
d92078471b
Rename order_by to sort for Table and DB_Table. (#10372)
- Rename `order_by` to `sort` for `Table` and `DB_Table`.
- Added deprecated placeholder.
- Fixed a couple of minor deprecated mistakes.

![image](https://github.com/enso-org/enso/assets/4699705/96c32fa7-33e5-400a-9d3a-ebf330886911)
2024-06-26 17:46:09 +00:00
James Dunkerley
791dba6729
Autoscoping for File_Format and minor tweaks. (#10348)
- Add `rename` ALIAS to `Table.use_first_row_as_names`.
- Add a shorthand to `Faker.string_value` to allow quick creation of fake values such as National Insurance numbers.
- Add `match` ALIAS for `Text.find` and `Text.find_all`.
- Auto scoping for `File_Format`. Demonstrates how to use it in a dynamic context.
- `SQLite_Format.For_File` renamed to `SQLite_Format.SQLite` (though kept for backwards compatibility.
- Fixed bug in `SQLite_Format` which was calling a non-existent constructor.

![image](https://github.com/enso-org/enso/assets/4699705/4506d27c-c1ff-4ad6-9276-53c2ae00de17)

![image](https://github.com/enso-org/enso/assets/4699705/9043ffb0-6740-42ba-91f8-ab0df555f20f)

![image](https://github.com/enso-org/enso/assets/4699705/03122fac-bdbb-4bcf-ac96-9491da41a8b9)

![image](https://github.com/enso-org/enso/assets/4699705/79122aac-a74a-435d-9849-ac3421f7d080)

![image](https://github.com/enso-org/enso/assets/4699705/54544de8-9aea-4dc6-bb4d-a7d8233c6814)
2024-06-24 08:28:54 +00:00
Jaroslav Tulach
fe2cf49568
Run whole test/Base_Tests in native image runner (#10296) 2024-06-21 06:03:53 +02:00
Radosław Waśko
3a4784c226
Initial separation of Snowflake_Dialect from Postgres_Dialect (#10266)
- Part of #9486
- Building on top of initial work by @jdunkerley and finishing it
- Reverted the changes to the Postgres_Dialect from last Snowflake work and split the Snowflake_Dialect into a separate module.
- Moved from `rounding_decimal_places_not_allowed_for_floats` to `supports_float_round_decimal_places` (as too confusing).
- Added Snowflake_Dialect type.
- Extracted `Snowflake_Spec` into separate `Snowflake_Tests`
- It imports the common tests from `Table_Tests`.
- Some initial adaptations to make the snowflake dialect not-crash.
- Adding `Internals_Access` proxy to allow external implementations to access our internal data structures without directly exposing them to users. Users should not use these.
- Adding profiling of SQL to check performance.
2024-06-13 16:12:20 +00:00
AdRiley
783276ca34
Make test framework list out failed tests to use in filter (#10241)
* List out failed tests

* Code review changes
2024-06-11 21:15:28 +01:00
James Dunkerley
200da1ad50
Remove old GUI1 ICONs and mapping function (#10086)
* Remove old GUI1 ICONs and mapping:

```
  dataframe_clean: 'table_clean',
  dataframe_map_row: 'map_row',
  dataframe_map_column: 'column_add',
  dataframes_join: 'join2-1',
  dataframes_union: 'union',
  sigma: 'transform4',
  io: 'in_out',
  date_and_time: 'time',
  spatial: 'location',
  predictive: 'predict',
  machine_learning: 'robot',
  split_text: 'split',
```

* Linter.

* Linter (2).

* Sort out some Returns:

* Sort out some ? Example
2024-05-27 11:17:21 +01:00
AdRiley
084be279c5
Fix input_number icons (#10066)
icon is called input_number not number_input

![image](https://github.com/enso-org/enso/assets/1720119/c68790d3-05ad-4d17-8423-df3634fff29a)
2024-05-24 11:30:06 +00:00
James Dunkerley
2677e556e8
Align headers in doc comments... (#10025)
- Always use `Arguments:` for parameters or fields.
- Always use `> Example` for examples.
- Align sections behind `!` or `?` headers.
- Whitespace fixes.
2024-05-21 19:44:58 +00:00
AdRiley
3b59b40657
Update test framework to default to only show failures (#9986)
* Update test framework to default to only show failures

* Fix
2024-05-21 11:23:23 +03:00
James Dunkerley
3b29de313d
Update ICONs, linting fixes and add select_columns ALIASes. (#9985)
- Linting fixes.
- Add ICONs to all functions missing them.
- Add ALIASes for select columns.
2024-05-17 19:24:17 +00:00
GregoryTravis
4d49b00375
Combine builders for Vector.build and Vector.new_builder (#9922)
We have decided to keep the old new_builder, and to combine the new and old builders into one builder.
2024-05-17 16:18:47 +00:00
GregoryTravis
720d32cbe3
Convert most remaining uses of Vector.new_builder to Vector.build (#9891) 2024-05-09 15:52:53 +00:00
GregoryTravis
a2c36def26
Add Vector.build_multiple and handle dataflow error and warning propagation in build and build_multiple (#9766) 2024-05-07 19:30:21 +00:00
James Dunkerley
d2e6ff260e
Restructure SQLite_Details. (#9832)
```
type SQLite_Details
SQLite location:File|In_Memory

type In_Memory
```
to
```
type SQLite
From_File location:File

In_Memory
```

# Important Notes
Splits the In-Memory entry for Database Connect but still works nicely.

![image](https://github.com/enso-org/enso/assets/4699705/ec798ce0-9f41-4903-a2fd-722a9e37743c)

![image](https://github.com/enso-org/enso/assets/4699705/f233b055-893e-4c56-a23d-562e982560f6)
2024-05-01 22:15:41 +00:00
Radosław Waśko
8e03e3be9c
Print pending groups in-line and include their reason, also add progress bar to the runner (#9796)
- Closes #9534 by printing the pending groups with pending reason
- Re-introduces original ordering of tests
- Adds a progress bar to the test suite runner in 'interactive' mode (if ANSI colors are enabled, progress bar will also be)
2024-04-26 16:05:05 +00:00
Radosław Waśko
0c989c6938
Writing to Data Links (#9750)
- Closes #9292
- and also closes #9522
2024-04-25 17:55:49 +00:00
Jaroslav Tulach
0d495ffd97
Make conversion of double to BigDecimal exact (#9740)
Resolves #9607 by computing `Number.hash` by converting given number to `Float` first and then computing the hash. Also the conversion from `Float.to Decimal` is exact - done via `new BigDecimal(double)`. There is `Decimal.new` that handles the user-friendly conversion. However as a result `Decimal.from 2.1 != Decimal.new 2.1` - that's the only way to ensure consistency between hash code and conversions.
2024-04-25 11:22:50 +00:00
Radosław Waśko
0310eb46ab
Highlight stack trace elements from the current project in Test suite reports (#9724)
If ANSI coloring is enabled, stack trace entries corresponding to the currently running project will be highlighted in bold:
![image](https://github.com/enso-org/enso/assets/1436948/b02da409-b67c-421c-a6d0-07a6cbbaf5f8)
2024-04-17 13:22:29 +00:00
Radosław Waśko
5650c7aed2
Refactoring Enso_File to be path based (#9581)
- Closes #9289
- Ensures that we can refer through `Enso_File` to files that do not _yet_ exist - preparing us for implementing the Write functionalities for `Enso_File` (#9291).
2024-04-09 11:15:29 +00:00
Radosław Waśko
354ee94a2f
Make HTTP tests more robust by adding retries to the tests (#9652)
- As asked for by @hubertp who was encountering flaky test failures on CI in the Http_Spec and related ones, I'm adding retry logic to make such cases much less likely.
- I've made the test server randomly fail 50% of tests and with the retry logic the tests are still passing, so I think that should be much more robust, in practice the failure rate is much much less (I imagine <1% as most of the time these tests were working and we do a ton of requests in a single CI run).
- I move the `with_retries` method to now be `Test.with_retries` which can be used anywhere in our tests for the retry logic.
- It sleeps for 0.1s between retries. Not all kinds of tests need it, this was mostly for propagation delays in the Cloud in our tests. I was thinking if the delay should be configurable, but I think the 0.1s delay is not problematic and if our tests are sometimes failing due to high machine load, the delay could also help.
- This _does not_ add retry logic to raw HTTP operations or `Data.fetch`. We may add that later, but that needs some further design. In such case we may remove some retries from tests if they become unnecessary.
2024-04-09 10:07:22 +00:00
GregoryTravis
973d2c6aea
Decimal arithmetic benchmarks (#9582) 2024-04-04 16:03:35 +00:00
James Dunkerley
7098c5e3e6
Apply ICONs and other tweaks. (#9552)
Lots of ICONs...
2024-03-26 19:44:06 +00:00
Radosław Waśko
6e5b4d93a3
Implement refreshing the Cloud token in Enso libraries (#9390)
- Closes #9300
- Now the Enso libraries are themselves capable of refreshing the access token, thus there is no more problems if the token expires during a long running workflow.
- Adds `get_optional_field` sibling to `get_required_field` for more unified parsing of JSON responses from the Cloud.
- Adds `expected_type` that checks the type of extracted fields. This way, if the response is malformed we get a nice Enso Cloud error telling us what is wrong with the payload instead of a `Type_Error` later down the line.
- Fixes `Test.expect_panic_with` to actually catch only panics. Before it used to also handle dataflow errors - but these have `.should_fail_with` instead. We should distinguish these scenarios.
2024-03-19 19:26:34 +00:00
James Dunkerley
1e33723ce9
ICONs and linting... (#9319)
- Apply all the icons from @Cassandra-Clark.
- Fix various linting issues.
- Remove circular dependency between Table and Database.
2024-03-07 12:26:32 +00:00
Pavel Marek
21e1284086
Enso tests can be run with filter from cmdline (#9065)
Simplify the `Test.Suite.run_with_filter` to accept a single filter parameter that searches for all the groups and specs that matches that filter. This filter can be a simple text provided from the command line.

# Important Notes
- Pending groups are now printed at the end of the run
- `Test.Suite.run_with_filter` is simplified to accept a single filter parameter that is either `Text` or `Nothing`. See the docs.
- Passing a filter from the command line is therefore straightforward, it is treated as a regex.
- For convenience, I have left all the `main` methods in all the test sources. I have just refactored them to accept the `filter` argument from the command line.
- For example, to run only a single spec from `Vector_Spec.enso`, invoke `enso --run test/Base_Tests/src/Data/Vector_Spec.enso "should allow vector creation with a programmatic constructor"`
- **Majority of the PR is a regex replace** of `^main =` for `main filter=Nothing =` and of `suite.run_with_filter` for `suite.run_with_filter filter`.
- **Fixed some internal engine bugs:**
- `AtomWithHole` allows to specify only one hole - https://github.com/enso-org/enso/pull/9065/files#diff-0f7bb7e85cf86a965de133aa7e6b5958ceb889bd1921c01e00d3a9ceb19626ef
- NaN keys in hash maps are handled in polyglot maps as well - c5257f6c2b78f893214ff67300893b593ea05e21..db4b3c0e9828ee79208d52e02586b24bb845b0d6
2024-02-22 12:31:44 +00:00
Jaroslav Tulach
a664dd9d56
Equality with conversions (#9070) 2024-02-19 17:18:56 +01:00
James Dunkerley
f257952680
Changing Widgets to work better with GUI2 (#9017)
- Rename `Column.replace` to `Column.text_replace` (consistency with Table and allow for adding `Column.replace`).
- Changed Regex insert to be `.to_regex` rather than `Regex.compile`.
https://github.com/enso-org/enso/assets/4699705/3694e876-5b46-49f9-8e8c-352fa0684777

- Adjusted `Table.rename_columns` to insert a `Pair.new` for each rename.
![image](https://github.com/enso-org/enso/assets/4699705/8240f302-498c-46d6-9b77-b1a0f5bb585e)

- Added a `<Regular Expression>` option to the `rename_columns`.
https://github.com/enso-org/enso/assets/4699705/d3c695ad-93f2-419d-b1c1-1f607ca825bd

- Added constant values to `Table.fill_nothing` and made it value type dependent on `Column.fill_nothing`.
https://github.com/enso-org/enso/assets/4699705/26e2e975-e972-46f0-ae1a-6b09dcd39fe6
![image](https://github.com/enso-org/enso/assets/4699705/51393d09-c06b-4eb1-9471-1856430fbfdc)
![image](https://github.com/enso-org/enso/assets/4699705/064fa42b-6187-4816-b3f0-0b7f0f00f6a9)

- Removed `use_regex` option from `Sort_Column.By_Name` (in favour of Regex type).

- Improved widgets for `order_by`. Drop down now on the column name and allow using the Index method with specified numeric range.
![image](https://github.com/enso-org/enso/assets/4699705/549b9c4c-5753-4518-801c-ecd06db5c2e7)
2024-02-12 15:12:19 +00:00