Commit Graph

165 Commits

Author SHA1 Message Date
Gregory Michael Travis
cce50fab3a
Extend the range of int/float arguments to outside the range of Java long, in ceil, floor, and truncate (#11135) 2024-10-07 11:36:17 -04:00
Radosław Waśko
0302670092
Reporting Enso test failures as annotations on the PR (#10821) 2024-10-02 16:01:28 +00:00
James Dunkerley
e6ea3b2607
Default Project, Templates and Docs (#10980)
- Linting updates.
- Add an `Examples.welcome` and adjust the start up project to use it.
- Merge all of Cass's work into the source code.
- Make example render in mono space font.
2024-09-05 14:36:08 +00:00
Jaroslav Tulach
b6f0675563
IO.print without new line (#10858) 2024-08-24 07:00:16 +02:00
James Dunkerley
422fa8c16b
Adding support for creating Atoms in expressions (#10820)
- Enables the `..` autoscoping style for creating Atoms in expressions.
- Add type checking to methods in columns.
- Auto wrap returns from method in expressions into a column as needed.
- Remove `Time_Period.Day` to remove confusion..
2024-08-15 15:52:30 +00:00
Radosław Waśko
b1958f8aa3
Adding vectorized implementations to some Column operations (#10795)
- Part of #6256 - implements operations that could have been vectorized without changes to the overall infrastructure
2024-08-13 08:53:39 +00:00
Kaz Wesley
e5b85bf16e
Space-precedence does not apply to value-level operators (#10597)
In a sequence of value-level operators, whitespace does not affect relative precedence. Functional operators still follow the space-precedence rules.

The "functional" operators are: `>> << |> |>> <| <<| : .`, application, and any operator containing `<-` or `->`. All other operators are considered value-level operators.

Asymmetric whitespace can still be used to form *operator sections* of value-level operators, e.g. `+2 * 3` is still equivalent to `x -> (x+2) * 3`.

Precedence of application is unchanged, so `f x+y` is still equivalent to `f (x + y)` and `f x+y * z` is still equivalent to `(f (x + y)) * z`.

Any attempt to use spacing to override value-level operator precedence will be caught by the new enso linter. Mixed spacing (for clarity) in value-operator expressions is allowed, as long as it is consistent with the precedences of the operators.

Closes #10366.

# Important Notes
Precedence warnings:
- The parser emits a warning if the whitespace in an expression is inconsistent with its effective precedence.
- A new enso linter can be run with `./run libraries lint`. It parses all `.enso` files in `distribution/lib` and `test`, and reports any errors or warnings. It can also be run on individual files: `cargo run --release --bin check_syntax -- file1 file2...` (the result may be easier to read than the `./run` output).
- The linter is also run as part of `./run lint`, so it is checked in CI.

Additional language change:
- The exponentiation operator (`^`) now has higher precedence than the multiplication class (`*`, `/`, `%`). This change did not affect any current enso files.

Library changes:
- The libraries have been updated. The new warnings were used to identify all affected code; the changes themselves have not been programmatically verified (in many cases their equivalence relies on the commutativity of string concatenation).
2024-07-24 10:55:44 +00:00
Radosław Waśko
7fd8701690
Snowflake Dialect pt. 6 - Union, Distinct and other improvements (#10576)
- Part of #9486
- Fixes `Table.union`, `merge` and `distinct` tests
- Replaces `distinct_on` in `Context` that was actually a Postgres specific addition leaking into the base with a more abstract `Context_Extension` mechanism.
- This allows us to implement the Snowflake-specific `DISTINCT` using `QUALIFY`.
2024-07-19 16:04:00 +00:00
James Dunkerley
15810a4675
Linting fixes. (#10589)
Generating Doc data so fixed linting.
2024-07-18 12:42:34 +00:00
Pavel Marek
0f9852aab2
Replace all from ... export all with explicit exports (#10369)
Replace all exports with explicit exports.
2024-07-11 19:34:25 +02:00
Jaroslav Tulach
220b40a1cd
Enforce conversion method return type & introduce Comparable.new (#10468) 2024-07-11 06:58:51 +02:00
Radosław Waśko
48c17845a7
Fixing Database tests and Snowflake Dialect - part 3 out of ... (#10458)
- Related to #9486
- Fixes types in literal tables that are used throughout the tests
- Tries to makes testing faster by disabling some edge cases, trying batching some queries, re-using the main connection and trying to re-use tables more
- Implements date/time type mapping and operations for Snowflake
- Updates type mapping to correctly reflect what Snowflake does
- Disables warnings for Integer->Decimal coercion as that's too annoying and implicitly understood in Snowflake
- Allows to select a Decimal column with `..By_Type ..Integer` (only in Snowflake backend) because the Decimal column there is its 'de-facto' Integer column replacement.
2024-07-10 13:21:30 +00:00
Jaroslav Tulach
ee0175c364
Support re-running failed tests with runEngineDistribution (#10483) 2024-07-10 07:05:29 +02:00
James Dunkerley
4b3e4ae15e
Rename Map to Dictionary and Set to Hashset. (#10474)
- Rename `Map` to `Dictionary`.
- Rename `Set` to `Hashset`.
- Add a deprecated place holder for the static method of `Map`.
2024-07-09 09:12:23 +00:00
James Dunkerley
0661f17d1c
Tune Text.trim, fix for Text.split (#10445)
- Rename `Location.Start` to `Location.Left`.
- Rename `Location.End` to `Location.Right`.
- Use auto-scoping for `Location`.
- Tune widgets for `Text.trim`.
- Correct signature of `Text.split`.
- Adjist `generateLocallyUniqueIdent` to not fail on bad signature.
2024-07-04 22:24:56 +00:00
James Dunkerley
9a2aed92f1
Separating list from read function and other small tweaks. (#10434)
- Rename `Faker.string_value` to `Faker.text_value`.
- Remove `Regex.pattern_string` as duplicate of `Regex.pattern`.
- Sort the Date picker.
- Rename `Data.list_directory` to `Data.list`.
- Remove support for reading a directory.

![image](https://github.com/enso-org/enso/assets/4699705/b42bb3c9-e63b-49f2-8cdc-4666cb6d968e)

![image](https://github.com/enso-org/enso/assets/4699705/97f49891-5ae5-4f0a-9a41-6200888fcd86)
2024-07-03 22:00:53 +00:00
Radosław Waśko
db4f7ab3b5
Fixing Database tests and Snowflake Dialect - part 2 out of ... (#10319)
- Part of #9486
- Fixing our tests to not rely on deterministic ordering of created Tables in Database backends
- Before, SQLite and Postgres used to mostly return rows in the order they were inserted in, but Snowflake does not.
- Fixing various parts of Snowflake dialect.
2024-06-27 14:54:00 +00:00
James Dunkerley
d92078471b
Rename order_by to sort for Table and DB_Table. (#10372)
- Rename `order_by` to `sort` for `Table` and `DB_Table`.
- Added deprecated placeholder.
- Fixed a couple of minor deprecated mistakes.

![image](https://github.com/enso-org/enso/assets/4699705/96c32fa7-33e5-400a-9d3a-ebf330886911)
2024-06-26 17:46:09 +00:00
James Dunkerley
791dba6729
Autoscoping for File_Format and minor tweaks. (#10348)
- Add `rename` ALIAS to `Table.use_first_row_as_names`.
- Add a shorthand to `Faker.string_value` to allow quick creation of fake values such as National Insurance numbers.
- Add `match` ALIAS for `Text.find` and `Text.find_all`.
- Auto scoping for `File_Format`. Demonstrates how to use it in a dynamic context.
- `SQLite_Format.For_File` renamed to `SQLite_Format.SQLite` (though kept for backwards compatibility.
- Fixed bug in `SQLite_Format` which was calling a non-existent constructor.

![image](https://github.com/enso-org/enso/assets/4699705/4506d27c-c1ff-4ad6-9276-53c2ae00de17)

![image](https://github.com/enso-org/enso/assets/4699705/9043ffb0-6740-42ba-91f8-ab0df555f20f)

![image](https://github.com/enso-org/enso/assets/4699705/03122fac-bdbb-4bcf-ac96-9491da41a8b9)

![image](https://github.com/enso-org/enso/assets/4699705/79122aac-a74a-435d-9849-ac3421f7d080)

![image](https://github.com/enso-org/enso/assets/4699705/54544de8-9aea-4dc6-bb4d-a7d8233c6814)
2024-06-24 08:28:54 +00:00
Jaroslav Tulach
fe2cf49568
Run whole test/Base_Tests in native image runner (#10296) 2024-06-21 06:03:53 +02:00
Radosław Waśko
3a4784c226
Initial separation of Snowflake_Dialect from Postgres_Dialect (#10266)
- Part of #9486
- Building on top of initial work by @jdunkerley and finishing it
- Reverted the changes to the Postgres_Dialect from last Snowflake work and split the Snowflake_Dialect into a separate module.
- Moved from `rounding_decimal_places_not_allowed_for_floats` to `supports_float_round_decimal_places` (as too confusing).
- Added Snowflake_Dialect type.
- Extracted `Snowflake_Spec` into separate `Snowflake_Tests`
- It imports the common tests from `Table_Tests`.
- Some initial adaptations to make the snowflake dialect not-crash.
- Adding `Internals_Access` proxy to allow external implementations to access our internal data structures without directly exposing them to users. Users should not use these.
- Adding profiling of SQL to check performance.
2024-06-13 16:12:20 +00:00
AdRiley
783276ca34
Make test framework list out failed tests to use in filter (#10241)
* List out failed tests

* Code review changes
2024-06-11 21:15:28 +01:00
James Dunkerley
200da1ad50
Remove old GUI1 ICONs and mapping function (#10086)
* Remove old GUI1 ICONs and mapping:

```
  dataframe_clean: 'table_clean',
  dataframe_map_row: 'map_row',
  dataframe_map_column: 'column_add',
  dataframes_join: 'join2-1',
  dataframes_union: 'union',
  sigma: 'transform4',
  io: 'in_out',
  date_and_time: 'time',
  spatial: 'location',
  predictive: 'predict',
  machine_learning: 'robot',
  split_text: 'split',
```

* Linter.

* Linter (2).

* Sort out some Returns:

* Sort out some ? Example
2024-05-27 11:17:21 +01:00
AdRiley
084be279c5
Fix input_number icons (#10066)
icon is called input_number not number_input

![image](https://github.com/enso-org/enso/assets/1720119/c68790d3-05ad-4d17-8423-df3634fff29a)
2024-05-24 11:30:06 +00:00
James Dunkerley
2677e556e8
Align headers in doc comments... (#10025)
- Always use `Arguments:` for parameters or fields.
- Always use `> Example` for examples.
- Align sections behind `!` or `?` headers.
- Whitespace fixes.
2024-05-21 19:44:58 +00:00
AdRiley
3b59b40657
Update test framework to default to only show failures (#9986)
* Update test framework to default to only show failures

* Fix
2024-05-21 11:23:23 +03:00
James Dunkerley
3b29de313d
Update ICONs, linting fixes and add select_columns ALIASes. (#9985)
- Linting fixes.
- Add ICONs to all functions missing them.
- Add ALIASes for select columns.
2024-05-17 19:24:17 +00:00
GregoryTravis
4d49b00375
Combine builders for Vector.build and Vector.new_builder (#9922)
We have decided to keep the old new_builder, and to combine the new and old builders into one builder.
2024-05-17 16:18:47 +00:00
GregoryTravis
720d32cbe3
Convert most remaining uses of Vector.new_builder to Vector.build (#9891) 2024-05-09 15:52:53 +00:00
GregoryTravis
a2c36def26
Add Vector.build_multiple and handle dataflow error and warning propagation in build and build_multiple (#9766) 2024-05-07 19:30:21 +00:00
James Dunkerley
d2e6ff260e
Restructure SQLite_Details. (#9832)
```
type SQLite_Details
SQLite location:File|In_Memory

type In_Memory
```
to
```
type SQLite
From_File location:File

In_Memory
```

# Important Notes
Splits the In-Memory entry for Database Connect but still works nicely.

![image](https://github.com/enso-org/enso/assets/4699705/ec798ce0-9f41-4903-a2fd-722a9e37743c)

![image](https://github.com/enso-org/enso/assets/4699705/f233b055-893e-4c56-a23d-562e982560f6)
2024-05-01 22:15:41 +00:00
Radosław Waśko
8e03e3be9c
Print pending groups in-line and include their reason, also add progress bar to the runner (#9796)
- Closes #9534 by printing the pending groups with pending reason
- Re-introduces original ordering of tests
- Adds a progress bar to the test suite runner in 'interactive' mode (if ANSI colors are enabled, progress bar will also be)
2024-04-26 16:05:05 +00:00
Radosław Waśko
0c989c6938
Writing to Data Links (#9750)
- Closes #9292
- and also closes #9522
2024-04-25 17:55:49 +00:00
Jaroslav Tulach
0d495ffd97
Make conversion of double to BigDecimal exact (#9740)
Resolves #9607 by computing `Number.hash` by converting given number to `Float` first and then computing the hash. Also the conversion from `Float.to Decimal` is exact - done via `new BigDecimal(double)`. There is `Decimal.new` that handles the user-friendly conversion. However as a result `Decimal.from 2.1 != Decimal.new 2.1` - that's the only way to ensure consistency between hash code and conversions.
2024-04-25 11:22:50 +00:00
Radosław Waśko
0310eb46ab
Highlight stack trace elements from the current project in Test suite reports (#9724)
If ANSI coloring is enabled, stack trace entries corresponding to the currently running project will be highlighted in bold:
![image](https://github.com/enso-org/enso/assets/1436948/b02da409-b67c-421c-a6d0-07a6cbbaf5f8)
2024-04-17 13:22:29 +00:00
Radosław Waśko
5650c7aed2
Refactoring Enso_File to be path based (#9581)
- Closes #9289
- Ensures that we can refer through `Enso_File` to files that do not _yet_ exist - preparing us for implementing the Write functionalities for `Enso_File` (#9291).
2024-04-09 11:15:29 +00:00
Radosław Waśko
354ee94a2f
Make HTTP tests more robust by adding retries to the tests (#9652)
- As asked for by @hubertp who was encountering flaky test failures on CI in the Http_Spec and related ones, I'm adding retry logic to make such cases much less likely.
- I've made the test server randomly fail 50% of tests and with the retry logic the tests are still passing, so I think that should be much more robust, in practice the failure rate is much much less (I imagine <1% as most of the time these tests were working and we do a ton of requests in a single CI run).
- I move the `with_retries` method to now be `Test.with_retries` which can be used anywhere in our tests for the retry logic.
- It sleeps for 0.1s between retries. Not all kinds of tests need it, this was mostly for propagation delays in the Cloud in our tests. I was thinking if the delay should be configurable, but I think the 0.1s delay is not problematic and if our tests are sometimes failing due to high machine load, the delay could also help.
- This _does not_ add retry logic to raw HTTP operations or `Data.fetch`. We may add that later, but that needs some further design. In such case we may remove some retries from tests if they become unnecessary.
2024-04-09 10:07:22 +00:00
GregoryTravis
973d2c6aea
Decimal arithmetic benchmarks (#9582) 2024-04-04 16:03:35 +00:00
James Dunkerley
7098c5e3e6
Apply ICONs and other tweaks. (#9552)
Lots of ICONs...
2024-03-26 19:44:06 +00:00
Radosław Waśko
6e5b4d93a3
Implement refreshing the Cloud token in Enso libraries (#9390)
- Closes #9300
- Now the Enso libraries are themselves capable of refreshing the access token, thus there is no more problems if the token expires during a long running workflow.
- Adds `get_optional_field` sibling to `get_required_field` for more unified parsing of JSON responses from the Cloud.
- Adds `expected_type` that checks the type of extracted fields. This way, if the response is malformed we get a nice Enso Cloud error telling us what is wrong with the payload instead of a `Type_Error` later down the line.
- Fixes `Test.expect_panic_with` to actually catch only panics. Before it used to also handle dataflow errors - but these have `.should_fail_with` instead. We should distinguish these scenarios.
2024-03-19 19:26:34 +00:00
James Dunkerley
1e33723ce9
ICONs and linting... (#9319)
- Apply all the icons from @Cassandra-Clark.
- Fix various linting issues.
- Remove circular dependency between Table and Database.
2024-03-07 12:26:32 +00:00
Pavel Marek
21e1284086
Enso tests can be run with filter from cmdline (#9065)
Simplify the `Test.Suite.run_with_filter` to accept a single filter parameter that searches for all the groups and specs that matches that filter. This filter can be a simple text provided from the command line.

# Important Notes
- Pending groups are now printed at the end of the run
- `Test.Suite.run_with_filter` is simplified to accept a single filter parameter that is either `Text` or `Nothing`. See the docs.
- Passing a filter from the command line is therefore straightforward, it is treated as a regex.
- For convenience, I have left all the `main` methods in all the test sources. I have just refactored them to accept the `filter` argument from the command line.
- For example, to run only a single spec from `Vector_Spec.enso`, invoke `enso --run test/Base_Tests/src/Data/Vector_Spec.enso "should allow vector creation with a programmatic constructor"`
- **Majority of the PR is a regex replace** of `^main =` for `main filter=Nothing =` and of `suite.run_with_filter` for `suite.run_with_filter filter`.
- **Fixed some internal engine bugs:**
- `AtomWithHole` allows to specify only one hole - https://github.com/enso-org/enso/pull/9065/files#diff-0f7bb7e85cf86a965de133aa7e6b5958ceb889bd1921c01e00d3a9ceb19626ef
- NaN keys in hash maps are handled in polyglot maps as well - c5257f6c2b78f893214ff67300893b593ea05e21..db4b3c0e9828ee79208d52e02586b24bb845b0d6
2024-02-22 12:31:44 +00:00
Jaroslav Tulach
a664dd9d56
Equality with conversions (#9070) 2024-02-19 17:18:56 +01:00
James Dunkerley
f257952680
Changing Widgets to work better with GUI2 (#9017)
- Rename `Column.replace` to `Column.text_replace` (consistency with Table and allow for adding `Column.replace`).
- Changed Regex insert to be `.to_regex` rather than `Regex.compile`.
https://github.com/enso-org/enso/assets/4699705/3694e876-5b46-49f9-8e8c-352fa0684777

- Adjusted `Table.rename_columns` to insert a `Pair.new` for each rename.
![image](https://github.com/enso-org/enso/assets/4699705/8240f302-498c-46d6-9b77-b1a0f5bb585e)

- Added a `<Regular Expression>` option to the `rename_columns`.
https://github.com/enso-org/enso/assets/4699705/d3c695ad-93f2-419d-b1c1-1f607ca825bd

- Added constant values to `Table.fill_nothing` and made it value type dependent on `Column.fill_nothing`.
https://github.com/enso-org/enso/assets/4699705/26e2e975-e972-46f0-ae1a-6b09dcd39fe6
![image](https://github.com/enso-org/enso/assets/4699705/51393d09-c06b-4eb1-9471-1856430fbfdc)
![image](https://github.com/enso-org/enso/assets/4699705/064fa42b-6187-4816-b3f0-0b7f0f00f6a9)

- Removed `use_regex` option from `Sort_Column.By_Name` (in favour of Regex type).

- Improved widgets for `order_by`. Drop down now on the column name and allow using the Index method with specified numeric range.
![image](https://github.com/enso-org/enso/assets/4699705/549b9c4c-5753-4518-801c-ecd06db5c2e7)
2024-02-12 15:12:19 +00:00
Pavel Marek
f3f0697d56
Merge Test_New into Test (#8991)
Merges the temporary `Test_New` library into `Test`. This is the last PR in the series of PRs that refactor all the stdlib tests to the builder API.
2024-02-08 11:25:13 +00:00
Radosław Waśko
583345d8f2
Allow Test asserts to be run outside of the test suite, still raising sensible Panics (#8778)
- Fixes #5962 by defaulting to no Clue if run outside of tests, ensuring that the assertions still throw a sensible panic.
2024-01-16 17:17:20 +00:00
GregoryTravis
f2cb1f097e
Support on_problems=Problem_Behavior.Report_Warning and Map_Error wrapping in Vector.map (#8595)
Implements `Warnings.get_all wrap_errors=True` which wraps warnings attached to values inside vectors with `Map_Error`, which includes the position of the value within the vector. See [the documentation](https://github.com/enso-org/enso/blob/develop/docs/semantics/wrapped-errors.md) for more details.

`get_all wrap_errors=True` does not change the warnings that are attached to values -- it wraps them before returning them to the caller, but does not change the original warnings attached to the values.

Wrapped warnings only appear attached to the vector itself. The values inside the vector do not have their warnings wrapped.

Warning propagation is not changed at all; `Warnings.get_all` (with default `wrap_errors=False`) behaves as before. `get_all wrap_errors=True` is meant to be used primarily by the IDE, although it can be used anywhere this wrapping is desired.
2024-01-16 09:36:22 +00:00
Radosław Waśko
d41d48e8a0
Merge URI_With_Query into URI, extend API of URI (#8591)
- Closes #8544
- Adds `reset_query_arguments` and `/` operators allowing to transform a URI.
- Adding tests for handling of various edge cases.
2023-12-21 18:39:26 +00:00
Radosław Waśko
724f8d2a56
Add tests for Enso Cloud auth + simple API mock for Enso_User (#8511)
- Closes #8354
- Extends `simple-httpbin` with a simple mock of the Cloud API (currently it checks the token and serves the `/users` endpoint).
- Renames `simple-httpbin` to `http-test-helper`.
2023-12-19 17:41:09 +00:00
GregoryTravis
1c815a3d45
Better Error Trapping in map (#8307)
* tests

* wip

* wip

* additional warnings

* wip

* wip

* cleanup

* nested wrapping

* multiple nestings

* wraps_error uses looks_for, test for should_fail_with

* wip

* stack trace line fix

* use catch_primitive internally

* fix warning mapping, dtf spec

* just one wrapper checker, vector spec

* missing ctor, back to non-primitive catch

* back to c_p

* put old map back

* wip

* unnest tests

* Array.map on_problems

* wip

* Revert "wip"

This reverts commit c30d171457.

* better test names

* warning logging

* wip

* wip

* move logic into ALH

* doc

* constant

* My_Error.Error

* nested

* doc

* map_primtiive in warning mapper

* composition

* ref spec

* Remove warnings prior to matching on the value

If an expression has warnings and is matched we:
1) extract the warnings
2) execute the branch of a pattern that matches the value
3) attach extracted warnings to the result

This caused warnings to reappear when doing the custom warnings
manipulation.
This is also consistent with how `CaseNode`'s `doWarning` specialization
is defined.

* fix 1

* do not auto unwrap in test error checkers

* nested error matcher

* in problems too

* dtf

* v

* statistics

* wip

* Table_Spec, map_with_index_primitive

* Column_Operations_Spec

* disable warning wrapping and Report_Warning

* unimpl test

* Warnings_Spec

* DCS

* ACG JP

* zip_primitive

* join_helpers

* Lookup_Helpers

* Table

* Data_Formatter

* Value_Type_Helpers

* revert check types changes

* table_helpers

* table tests

* remove st

* do not remove warnings from value

* vec docs, tests for zip, mwi, flat_map

* docs, fixes

* remove nested_error_matcher

* cleanup

* benchmark

* one error

* alter

* add bench to main

* review

* review

* review

* tail call

* changelog

* tail call was not a tail call

* ws

* bad import

* Added missing import

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Array.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* review, ref example

* lazy benchmark data

* extra paren

* check outside of catch

* review

* vector too

* actually lazy

* disambiguate Map_Error

* finish rename

* move to extensions

* combine Additional_Warnings error

* rename to map_no_wrap

* do not catch and rethrow

* review

* wip

* remove _primitives entirely

* remove unused should_fail_with function options

* remove expected_warning as function in Problems

---------

Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2023-12-13 09:38:09 -05:00