Commit Graph

494 Commits

Author SHA1 Message Date
Hubert Plociniczak
ae3f9025e3
Invoke instance methods for Any overrides (#6441)
This change modifies method dispatch for methods that override Any's definitions. When an overrided method is invoked statically we call Any's method to stay consistent.
This change primarily addresses the plethora of problems related to `to_text` invocations. It does not attempt to completely modify method dispatch logic.

Closes #6300.
2023-04-28 07:18:37 +00:00
Hubert Plociniczak
c6790f1e9c
Report only unique warnings (#6372)
This change makes sure that reported warnings are unique, based on the value of internal clock tick and ignoring differences in reassignments.

Before:
![Screenshot from 2023-04-20 15-42-55](https://user-images.githubusercontent.com/292128/233415710-925c1045-37c7-49f5-9bc3-bfbfd30270a3.png)
After:
![Screenshot from 2023-04-20 15-27-27](https://user-images.githubusercontent.com/292128/233415807-8cb67bc2-ac37-4db7-924e-ae7619074b5b.png)

On the positive side, no further changes, like in LS, have to be done.


Closes #6257.
2023-04-28 07:16:00 +00:00
Radosław Waśko
462016a428
Create database table from memory (#6429) 2023-04-27 20:06:17 +00:00
James Dunkerley
0c7c3bdeaf
Fix for the massive number of warnings when renaming with invalid names. (#6450)
* Rename makeUnique overloads to avoid issue when Nothing is passed.
Suspend warnings when building the output table to avoid mass warning duplication.

* Add test for mixed invalid names.
Adjust so a single warning attached.

* PR comments.
2023-04-27 14:51:59 +01:00
Dmitry Bushev
9a0f739cb0
Visualization preprocessor should not catch errors (#6423)
related #6323

Fixes the following scenario:
- preprocessor function catches `ThreadInterruptedException` and returns it as a JSON
```json
{"error":"org.enso.interpreter.runtime.control.ThreadInterruptedException"}
```
- engine thinks that the visualization was computed successfully and just returns it to the user (without doing any fallback logic like retrying the computation)
- IDE displays blank visualization
2023-04-27 12:42:34 +00:00
James Dunkerley
0e51131809
Table Visualization and display text changes. (#6382)
- Missing tests from number parsing.
- Fix type signature on some warning methods.
- Fix warnings on `Standard.Database.Data.Table.parse_values`.
- Added test for `Nothing` and empty string on `use_first_row_as_names`.
- New API for `Number.format` taking a simple format string and `Locale`.
- Add ellipsis to truncated `Text.to_display_text`.
- Adjusted built-in `to_display_text` for numbers to not include type (but also to display BigInteger as value).
- Remove `Noise.Generator` interface type.
- Json: Added `to_display_text` to `JS_Object`.
- Time: Added `to_display_text` for `Date`, `Time_Of_Day`, `Date_Time`, `Duration` and `Period`.
- Text: Added `to_display_text` to `Locale`, `Case_Sensitivity`, `Encoding`, `Text_Sub_Range`, `Span`, `Utf_16_Span`.
- System: Added `to_display_text` to `File`, `File_Permissions`, `Process_Result` and `Exit_Code`.
- Network: Added `to_display_text` to `URI`, `HTTP_Status_Code` and `Header`.
- Added `to_display_text` to `Maybe`, `Regression`, `Pair`, `Range`, `Filter_Condition`.
- Added support for `to_js_object` and `to_display_text` to `Random_Number_Generator`.
- Verified all error types have `to_display_text`.
- Removed `BigInt`, `Date`, `Date_Time` and `Time_Of_Day` JS based rendering as using `to_display_text` now.
- Added support for rendering nested structures in the table viz.
2023-04-26 18:15:48 +00:00
GregoryTravis
255edda146
Add parse extensions to Text type. #6330 (#6404)
Add type-specific parse stubs to Text, e.g.:
Text.parse_json self -> Json =
Text.parse_url self -> Url =
Text.parse_number self -> Number =
2023-04-26 15:37:09 +00:00
GregoryTravis
dcfbf841b3
Replace Table should_equal with should_equal_verbose (#6405)
Table.should_equal_verbose shows the contents of the tables on failure; let's make this the standard comparison.
2023-04-26 14:01:49 +00:00
GregoryTravis
afd804d529
5127 Add Table.parse_to_columns to parse a single column to a set of columns. (#6383)
Implement Table.parse_to_columns
2023-04-24 15:21:38 +00:00
Radosław Waśko
f3873f9768
Infer SQLite types locally (#6381)
Closes #6208
2023-04-24 10:55:12 +00:00
Radosław Waśko
a43d524336
Add typechecks to Aggregate and Cross Tab (#6380)
Follow up of #6298 as it grew too much. Adds the needed typechecks to aggregate operations. Ensures that the DB operations report `Floating_Point_Equality` warning consistently with in-memory.
2023-04-24 08:55:54 +00:00
GregoryTravis
22f820feb7
Add Table.parse_text_to_table to convert Text to a Table. (#6294) 2023-04-21 17:43:19 +00:00
Radosław Waśko
8db2ad51a1
Adding typechecks to Column Operations (#6298)
Closes #6106
2023-04-21 12:20:12 +00:00
James Dunkerley
0350762386
Add replace, trim to Column. Better number parsing. (#6253)
- Add `replace` with same syntax as on `Text` to an in-memory `Column`.
- Add `trim` with same syntax as on `Text` to an in-memory `Column`.
- Add `trim` to in-database `Column`.
- Added `is_supported` to dialects and exposed the dialect consistently on the `Connection`.
- Add `write_table` support to `JSON_File` allowing `Table.write` to write JSON.
- Updated the parsing for integers and decimals:
- Support for currency symbols.
- Support for brackets for negative numbers.
- Automatic detection of decimal points and thousand separators.
- Tighter rules for scientific and thousand separated numbers.
- Remove `replace_text` from `Table`.
- Remove `write_json` from `Table`.
2023-04-20 16:04:59 +00:00
Pavel Marek
4076a64f33
NaN can be used as a key in Map (#6301)
`Number.nan` can be used as a key in `Map`. This PR basically implements the support for [JavaScript's Same Value Zero Equality](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Equality_comparisons_and_sameness#same-value-zero_equality) so that `Number.nan` can be used as a key in `Map`.

# Important Notes
- For NaN, it holds that `Meta.is_same_object Number.nan Number.nan`, and `Number.nan != Number.nan` - inspired by JS spec.
- `Meta.is_same_object x y` implies `Any.== x y`, except for `Number.nan`.
2023-04-20 09:25:36 +00:00
Hubert Plociniczak
6d3151f32d
UnresolvedSymbol is now accepted by Vector.sort (#6334)
`Vector.sort` does some custom method dispatch logic which always expected a function as `by` and `on` arguments. At the same time, `UnresolvedSymbol` is treated like a (to be resolved) `Function` and under normal circumstances there would be no difference between `_.foo` and `.foo` provided as arguments.

Rather than adding an additional phase that does some form of eta-expansion, to accomodate for this custom dispatch, this change only fixes the problem locally. We accept `Function` and `UnresolvedSymbol` and perform the resolution on the fly. Ideally, we would have a specialization on the latter but again, it would be dependent on the contents of the `Vector` so unclear if that is better.

Closes #6276,

# Important Notes
There was a suggestion to somehow modify our codegen to accomodate for this scenario but I went against it. In fact a lot of name literals have `isMethod` flag and that information is used in the passes but it should not control how (late) codegen is done. If we were to make this more generic, I would suggest maybe to add separate eta-expansion pass. But it could affect other things and could be potentially a significant change with limited potential initially, so potential future work item.
2023-04-20 07:58:58 +00:00
Hubert Plociniczak
f288198b4b
Change order of arguments when controlling context (#6357)
https://github.com/orgs/enso-org/discussions/6344 requested to change the order of arguments when controlling context permissions.

# Important Notes
The change brings it closer to the design doc but IMHO also a bit cumbersome to use (see changed tests) - applications involving default arguments don't play well when the last argument is not the default 🤷 .`
2023-04-19 14:00:27 +00:00
Hubert Plociniczak
4c1e73c864
Don't propagate warnings on suspended arguments (#6345)
* Don't propagate warnings on suspended arguments

In the current implementation, application of arguments with warnings
first extracts warnings, does the application and appends the warnings
to the result.
This process was however too eager if the suspended argument was a
literal (we don't know if it will be executed after all).

The change modifies method processor to take into account the
`@Suspend` annotation and not gather warnings before the application
takes place.

* PR review
2023-04-19 11:55:23 +02:00
Hubert Plociniczak
f720bd2516
Type of UnresolvedSymbol is Function (#6284)
`Meta.is_a` and `Meta.type_of` now recognize UnresolvedSymbol.
Closes #6277.
2023-04-17 13:44:15 +00:00
Jaroslav Tulach
413661b366
Splitting Gigantic EqualsNode into few Smaller Ones (#6280)
Splitting gigantic `EqualsNode` into few smaller ones
2023-04-17 11:29:54 +02:00
Pavel Marek
b42e910280
sort handles incomparable values (#5998)
* Update type ascriptions in some operators in Any

* Add @GenerateUncached to AnyToTextNode.

Will be used in another node with @GenerateUncached.

* Add tests for "sort handles incomparable types"

* Vector.sort handles incomparable types

* Implement sort handling for different comparators

* Comparison operators in Any do not throw Type_Error

* Fix some issues in Ordering_Spec

* Remove the remaining comparison operator overrides for numbers.

* Consolidate all sorting functionality into a single builtin node.

* Fix warnings attachment in sort

* PrimitiveValuesComparator handles other types than primitives

* Fix byFunc calling

* on function can be called from the builtin

* Fix build of native image

* Update changelog

* Add VectorSortTest

* Builtin method should not throw DataflowError.

If yes, the message is discarded (a bug?)

* TypeOfNode may not return only Type

* UnresolvedSymbol is not supported as `on` argument to Vector.sort_builtin

* Fix docs

* Fix bigint spec in LessThanNode

* Small fixes

* Small fixes

* Nothings and Nans are sorted at the end of default comparator group.

But not at the whole end of the resulting vector.

* Fix checking of `by` parameter - now accepts functions with default arguments.

* Fix changelog formatting

* Fix imports in DebuggingEnsoTest

* Remove Array.sort_builtin

* Add comparison operators to micro-distribution

* Remove Array.sort_builtin

* Replace Incomparable_Values by Type_Error in some tests

* Add on_incomparable argument to Vector.sort_builtin

* Fix after merge - Array.sort delegates to Vector.sort

* Add more tests for problem_behavior on Vector.sort

* SortVectorNode throws only Incomparable_Values.

* Delete Collections helper class

* Add test for expected failure for custom incomparable values

* Cosmetics.

* Fix test expecting different comparators warning

* isNothing is checked via interop

* Remove TruffleLogger from SortVectorNode

* Small review refactorings

* Revert "Remove the remaining comparison operator overrides for numbers."

This reverts commit 0df66b1080.

* Improve bench_download.py tool's `--compare` functionality.

- Output table is sorted by benchmark labels.
- Do not fail when there are different benchmark labels in both runs.

* Wrap potential interop values with `HostValueToEnsoNode`

* Use alter function in Vector_Spec

* Update docs

* Invalid comparison throws Incomparable_Values rather than Type_Error

* Number comparison builtin methods return Nothing in case of incomparables
2023-04-16 16:40:12 +02:00
Hubert Plociniczak
e163353d16
Deprecate Error's to_display_text builtin methods (#6275)
The primary motivation for this change was
https://github.com/enso-org/enso/issues/6248, which requested the possibility of defining `to_display_text` methods of common errors via regular method definitions. Until now one could only define them via builtins.

To be able to support that, polyglot invocation had to report `to_display_text` in the list of (invokable) members, which it didn't. Until now, it only considered fields of constructors and builtin methods. That is now fixed as indicated by the change in `Atom`.

Closes #6248.

# Important Notes
Once most of builtins have been translated to regular Enso code, it became apparent how the usage of `.` at the end of the message is not consistent and inflexible. The pure message should never follow with a dot or it makes it impossible to pretty print consistently for the purpose of error reporting. Otherwise we regularly end up with errors ending with `..` or worse. So I went medieval on the reasons for failures and removed all the dots.
The overall result is mostly the same except now we are much more consistent.

Finally, there was a bit of a good reason for using builtins as it simplified our testing.
Take for example `No_Such_Method.Error`. If we do not import `Errors.Common` module we only rely on builtin error types. The type obviously has the constructor but it **does not have** `to_display_text` in scope; the latter is no longer a builtin method but a regular method. This is not really a problem for users who will always import stdlib but our tests often don't. Hence the number of changes and sometimes lack of human-readable errors there.
2023-04-14 19:14:49 +00:00
GregoryTravis
4dcf5faddd
Add split and tokenize to the Table. (#6233)
Implement split and tokenize for tables.
2023-04-14 16:03:02 +00:00
Jaroslav Tulach
a74933d10f
Speed cascade of if statements up (#6255)
Fixes #5709. We have a test and a generic fix that improves inlining of every builtin. Everything seems to be faster.
2023-04-14 13:27:23 +00:00
James Dunkerley
4e92d76665
Add Warning methods onto Any to make working with them easier. (#6176)
Adds `has_warnings`, `remove_warnings` and `throw_on_warning` extension methods.
2023-04-12 15:08:01 +00:00
Jaroslav Tulach
5b3cf6f503
Consolidate Vector and Array methods (#6218)
Fixes #5011 by making sure the same methods that are on `Vector` are also available on `Array`.
2023-04-11 16:20:11 +00:00
Radosław Waśko
0f4f8a0542
Full-joins in SQLite (#6215)
Closes #5254

In #6189 the SQLite version was bumped to a newer release which has builtin support for Full and Right joins, so no workaround is no longer needed.
2023-04-06 16:49:14 +00:00
Hubert Plociniczak
3cce3b3caf
Replace IOContexts with execution env and contexts (#6171)
As per design, IOContexts controlled via type signatures are going away. They are replaced by explicit `Context.if_enabled` runtime checks that will be added to particular method implementations.

`production`/`development` `IOPermissions` are replaced with `live` and `design` execution enviornment. Currently, the `live` env has a hardcoded list of allowed contexts i.e. `Input` and `Output`.

# Important Notes
As per design PR-55. Closes #6129. Closes #6131.
2023-04-06 15:47:40 +00:00
Radosław Waśko
f5db35af07
Adjust {Table|Column}.parse to use Value_Type (#6213)
Closes #5660
2023-04-06 10:58:55 +00:00
Radosław Waśko
83b10a2088
Implement Table.union for Database backend (#6204)
Closes #5235
2023-04-06 08:40:34 +00:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Jaroslav Tulach
741b394b0d
Suspended atom fields are evaluated only once (#6151)
Implements #6134.

# Important Notes
One can define lazy atom fields as:
```haskell
type Lazy
Value ~x ~y
```
the evaluation of the `x` and `y` fields is then delayed until they are needed. The evaluation happens once. Then the computed value is kept in the atom for further use.
2023-04-05 23:46:08 +00:00
Jaroslav Tulach
4805193428
Text.to_display_text is (shortened) identity (#6174)
Fixes #5971.
2023-04-05 19:53:07 +00:00
James Dunkerley
a05cee2571
Add PRIVATE so function hidden from Component Browser and other tidying... (#6207)
Review the set of public functions. Mark many as PRIVATE so not in CB.
Some other small tidying.
2023-04-05 17:54:06 +00:00
GregoryTravis
d9bc5246ba
Remove old (Java) Regex library and replace with new (Truffle) library. (#6195)
Remove old (Java) Regex library and replace with new (Truffle) library.
2023-04-04 19:58:26 +00:00
GregoryTravis
fb77f42fd5
Update Text.split to take a Vector Text parameter (#6156)
Allows you to pass a vector of delimiters to `split`.
2023-04-04 14:44:47 +00:00
Jaroslav Tulach
519df66d39
Always try to resolve conversion for Any type (#6184)
Fixes #5898 by removing `Catch.panic` and speeding the `sieve.enso` benchmark from 1058 ms to 514 ms. Should there be no dedicated conversion, let's use one defined on `Any` type - e.g. defining a conversion `from(Any)` makes such a conversion is always available.
2023-04-04 09:24:35 +00:00
GregoryTravis
6766389cd7
#5123 Text.tokenize (#6150)
Implement Text.tokenize.
2023-04-03 20:13:49 +00:00
James Dunkerley
f26bcf6ab6
Small issues from working with Ned (#6160)
- `Process.run` now returns a `Process_Result` allowing the easy capture of stdout and stderr.
- Joining a column with a column name does not warn if adding just the prefix.
- Stop the table viz from changing case and adding spaces to the headers.
2023-04-03 13:01:42 +00:00
Radosław Waśko
6ddcb553e5
Date/time support for Postgres. Year/month/day operations on Columns. (#6153)
Closes #6115
2023-03-31 18:37:04 +00:00
Radosław Waśko
6f86115498
Proper implementation of Value Types in Table (#6073)
This is the first part of the #5158 umbrella task. It closes #5158, follow-up tasks are listed as a comment in the issue.

- Updates all prototype methods dealing with `Value_Type` with a proper implementation.
- Adds a more precise mapping from in-memory storage to `Value_Type`.
- Adds a dialect-dependent mapping between `SQL_Type` and `Value_Type`.
- Removes obsolete methods and constants on `SQL_Type` that were not portable.
- Ensures that in the Database backend, operation results are computed based on what the Database is meaning to return (by asking the Database about expected types of each operation).
- But also ensures that the result types are sane.
- While SQLite does not officially support a BOOLEAN affinity, we add a set of type overrides to our operations to ensure that Boolean operations will return Boolean values and will not be changed to integers as SQLite would suggest.
- Some methods in SQLite fallback to a NUMERIC affinity unnecessarily, so stuff like `max(text, text)` will keep the `text` type instead of falling back to numeric as SQLite would suggest.
- Adds ability to use custom fetch / builder logic for various types, so that we can support vendor specific types (for example, Postgres dates).

# Important Notes
- There are some TODOs left in the code. I'm still aligning follow-up tasks - once done I will try to add references to relevant tasks in them.
2023-03-31 16:16:18 +00:00
GregoryTravis
c8f5a91d6c
Implement Regular Expression split and update Text.split to the new API (#6116)
Re-implement split on top of Truffle regex.
2023-03-30 13:05:30 +00:00
GregoryTravis
6b9cbeacb2
Implement Regular Expression replace and update Text.replace to the new API (#5959)
Re-implement replace on top of Truffle regex.
2023-03-28 06:13:12 +00:00
James Dunkerley
58f2c7643f
Use new Enso Hash Codes and Comparable (#6060)
Enables `distinct`, `aggregate` and `cross_tab` to use the Enso hashing and equality operations.
Also, I rewired the way the ObjectComparators are obtained in polyglot code to be more consistent.

Add Comparator for `Day_Of_Week`, `Header`, `SQL_Type`, `Image` and `Matrix`.
Also, removed the custom `==` from these types as needed. (Closes #5626)
2023-03-24 15:02:25 +00:00
James Dunkerley
dd009fd1af
Tidy up the public module level statics (#6032)
Tidies up a lot of PUBLIC module statics - marked some as PRIVATE, made some methods of types.
2023-03-22 18:02:37 +00:00
James Dunkerley
546cb0c4ab
Deal with warnings attached to value when making widgets. (#5994)
- Fixes InvokeCallableNode to support warnings.
- Strips warnings from annotations in `get_widget_json`.
- Remove `get_full_annotations_json`.
- Fix warnings on Dialect.
2023-03-22 10:50:19 +00:00
Hubert Plociniczak
8c6fd60aaf
Detect conflicts between exported types and FQNs (#5986)
Exporting types named the same as the module where they are defined in `Main` modules of library components may lead to accidental name conflicts. This became apparent when trying to access `Problem_Behavior` module via a fully qualified name and the compiler rejected it. This is due to the fact that `Main` module exported `Error` type defined in `Standard.Base.Error` module, thus making it impossible to access any other submodules of `Standard.Base.Error` via a fully qualified name.

This change adds a warning to FullyQualifiedNames pass that detects any such future problems.
While only `Error` module was affected, it was widely used in the stdlib, hence the number of changes.

Closes #5902.

# Important Notes
I left out the potential conflict in micro-distribution, thus ensuring we actually detect and report the warning.
2023-03-21 21:09:41 +00:00
Jaroslav Tulach
b46be10f63
Introducing Meta.Type (#5956)
Fixing #5768 and #5765 and co. Introducing `Meta.Type` and giving it the desired methods.

# Important Notes
`Type` is no longer a `Meta.Atom`, but it has a dedicated `Meta.Type` representation.
2023-03-17 20:08:18 +00:00
GregoryTravis
e9600f5eac
Truffle regex: finish group accessors (#5883) 2023-03-15 19:15:09 +00:00
James Dunkerley
919e9474c5
Fix Location, a type signature issue, clean up warnings. (#5953)
Makes Location a normal type.
2023-03-15 18:19:28 +00:00
Jaroslav Tulach
888307f03f
Meta.get_constructor_fields works for Type (#5869)
Fixes #5805 by returning `[]` as list of fields of `Type`.

# Important Notes
`Type` is recognized as `Meta.is_atom` since #3671. However `Type` isn't an `Atom` internally. We have to provide special handling for it where needed.
2023-03-13 09:58:11 +00:00
Radosław Waśko
952beba8d1
Fix cross_tab column naming edge cases, add fill_empty (#5863)
Closes #5151 and adds some additional tests for `cross_tab` that verify duplicated and invalid names.

I decided that for empty or `Nothing` names, instead of replacing them with `Column` and implicitly losing connection with the value that was in the column, we should just error on such values.

To make handling of these easier, `fill_empty` was added allowing to easily replace the empty values with something else.

Also, `{is,fill}_missing` was renamed to `{is,fill}_nothing` to align with `Filter_Condition.Is_Nothing`.
2023-03-11 11:58:54 +00:00
Pavel Marek
5f7a4a5a39
Merge ordered and unordered comparators (#5845)
Merge _ordered_ and _unordered_ comparators into a single one.

# Important Notes
Comparator is now required to have only `compare` method:
```
type Comparator
comapre : T -> T -> (Ordering|Nothing)
hash : T -> Integer
```
2023-03-11 05:43:22 +00:00
James Dunkerley
7887fb8d40
Regex.Compile using Truffle Regex, update find, match and match_all (#5785) 2023-03-10 21:49:50 +00:00
Radosław Waśko
91ef8acf35
Review generated Column names (#5850)
Closes #5583 and closes #5157
2023-03-10 19:07:58 +00:00
Radosław Waśko
62e57f5557
Test some Mismatched Quote edge cases in Delimited reader (#5810)
Follow-up to #5113 - I add some more edge case tests as we discussed with @jdunkerley

When debugging some quoting issues, I also realised the current `Mismatched_Quote` error provided not enough information. So I amended it to at least include some context indicating which was the 'offending' cell.
2023-03-10 15:47:57 +00:00
Jaroslav Tulach
8bbdd1af5b
Meta.is_a consistent with case-type-of check (#5853)
Removing special handling of `AtomConstructor` in `Meta.is_a` check.

# Important Notes
A lot of tests are about to fail. Many of them indirectly call `Meta.is_a` with a constructor rather than type.
2023-03-10 07:41:04 +00:00
Dmitry Bushev
9397a6ec2f
Pre compute suggestion db during build time (#5698)
Close #5068

Cache suggestions during the `buildEngineDistribution` command, and read them from the disk when the library is loaded. Initial graph coloring takes ~20 seconds vs ~25 seconds on the develop branch.

[peek-develop-branch.webm](https://user-images.githubusercontent.com/357683/223504462-e7d48262-4f5e-4724-b2b0-2cb97fc05140.webm)
[peek-suggestions-branch.webm](https://user-images.githubusercontent.com/357683/223504464-0fe86c04-8c4b-443c-ba96-6c5e2fb1e396.webm)
2023-03-08 12:37:48 +00:00
James Dunkerley
299bfd6b7d
Fixes from the Demo on 2nd March (#5823)
- Fix issue with Geo Map viz.
- Handle invalid format strings better in `Data_Formatter`.
- New constants for the ISO format strings (and a special ENSO_ZONED_DATE_TIME)
- Consistent Date Time format for parsing in all places.
- Avoid throwing exception in datetime parsing.
- Support for milliseconds (well nanoseconds) in Date_Time and Time_Of_Day.
- `Column.map` stays within Enso.
- Allow `Aggregate_Column.Group_By` in `cross_tab` group_by parameter.
2023-03-07 20:58:00 +00:00
Jaroslav Tulach
e3f0888c5f
Coerce values obtained from polyglot calls (#5832)
Coerce values obtained from polyglot calls to fix #5177.

# Important Notes
Adds `IntHolder` class into the `test/Tests` project to simulate access to a class with integer field.
2023-03-07 18:21:30 +00:00
Pavel Marek
b6e2319fcc
Comparators support partial ordering (#5778) 2023-03-07 04:16:38 +00:00
Radosław Waśko
da760aa27d
Review Text/Table.write problem behavior (#5816)
Closes #5114

Added tests for various problems scenarios when writing files.

And ensured that those tests are passing by fixing a few edge cases.
2023-03-07 02:25:13 +00:00
Radosław Waśko
2d29456ed1
Review File/Data read and read_text warnings (#5799)
Closes #5113

Fixes a bug where read-only files would be overwritten if File.write was used in backup mode, and added tests to avoid such regression. To implement it, introduced a `is_writable` property on `File`.
2023-03-06 03:43:38 +00:00
Jaroslav Tulach
f64edd806d
Switch to Truffle PE mode before calling into Enso (#5783)
Enter partial evaluation mode via `CallTarget.call` before invoking `InteropLibrary`. Fixes #5782.
2023-03-03 14:39:37 +00:00
James Dunkerley
01fc34c18a
Improving Expression Support for In Database (#5790)
- Adjust Excel Workbook write behaviour.
- Support Nothing / Null constants.
- Deduce the type of arithmetic operations and `iif`.
- Allow Date_Time constants, treating as local timezone.
- Removed the `to_column_name` and `ensure_sane_name` code.
2023-03-03 12:03:05 +00:00
Radosław Waśko
b764b0b7b7
Improve error handling of Connection.query (#5693)
Closes #5252
2023-02-24 17:15:10 +00:00
Radosław Waśko
793eafc866
Improve Table.parse_values API (#5692)
Closes #5111
2023-02-24 13:35:01 +00:00
James Dunkerley
652b8d5db3
Update rename_columns to new API design, add first_row, second_row and last_row functions to the table. (#5719)
- Updates the `rename_columns` API.
- Add `first_row`, `second_row` and `last_row` to the Table types.
- New option for reading only last row of ResultSet.
2023-02-23 19:42:45 +00:00
Radosław Waśko
ed6d3d0f97
Review Text encoding/decoding problem handling (#5701)
Closes #5112
2023-02-23 12:53:44 +00:00
GregoryTravis
3a09ee88f6
Wip/gmt/match find only text (#5721)
Rename is_match + match to match + find (respectively), and remove all non-regexp functionality.

Regexp flags and Match_Mode are also no longer supported by these methods.
2023-02-23 09:47:10 +00:00
Jaroslav Tulach
97bcd87dc6
File.parent can return Nothing (#5699)
`File.parent` can be `Nothing`
2023-02-20 19:13:56 +00:00
GregoryTravis
afb853a03f
Update Text.locate, Text.locate_all, Text.index_of and Text.last_index_of #5119 (#5679)
Remove regex support from .locate and .locate_all; regex functionality is moved to .match and .match_all where appropriate. This is in preparation for simplifying regex support across the board.

Also change Matching_Mode types to a single type with two variants.

Note: the matcher parameter to .locate and .locate_all has been replaced by a case_sensitivity parameter, of type Case_Sensitivity, which differs in that it also has a Default option. Default is treated as Sensitive.
2023-02-17 19:41:55 +00:00
Radosław Waśko
4dcf802831
Ensure that warnings are preserved on Nothing values passing back to Enso through polyglot boundary (#5677)
Fixes #5672

# Important Notes
- Added a subproject `enso-test-java-helpers` which allows the in-Enso tests to add Java helpers for testing.
2023-02-17 13:38:26 +00:00
Radosław Waśko
3027c6f3a2
Ensure entries containing newlines are quoted when writing Delimited Files (#5652)
Fixes #5638
2023-02-17 00:57:48 +00:00
James Dunkerley
1bc27501e6
Remove Column type from Aggregate_Column, simplify Column_Selector, some new File_Formats (#5646)
- Updated `Widget.Vector_Editor` ready for use by IDE team.
- Added `get` to `Row` to make API more aligned.
- Added `first_column`, `second_column` and `last_column` to `Table` APIs.
- Adjusted `Column_Selector` and associated methods to have simpler API.
- Removed `Column` from `Aggregate_Column` constructors.
- Added new `Excel_Workbook` type and added to `Excel_Section`.
- Added new `SQLiteFormatSPI` and `SQLite_Format`.
- Added new `IamgeFormatSPI` and `Image_Format`.
2023-02-16 15:15:49 +00:00
Pavel Marek
3e8467c204
Update GraalVM to 22.3.1 (#5602)
Updates the engine to GraalVM 22.3.1 version, which contains fixes for:
- Chrome inspector issues - 38eb3b5932

# Important Notes
- Update to GraalVM 22.3.1
- Remove host object wrapping workaround
2023-02-14 15:51:17 +00:00
Michael Mauderer
edfa845b34
Implement Backend API for Lazy Table Visualization. (#4087)
Adds relevant code to the standard library that can be used to access table visualization data in a lazy visualization.
2023-02-14 14:05:53 +00:00
Radosław Waśko
a02eab451e
Implement basic warnings for column arithmetic, review warnings on expressions and filter (#5605)
Closes #5109

# Important Notes
- Currently the tests pass for the in-memory parts of Common_Table_Operations, but still some stuff not working on DB backends - in progress.
2023-02-14 09:33:04 +00:00
GregoryTravis
99b9728995
Improve Text.toDisplayString (#5045) (#5598) 2023-02-13 18:38:15 +00:00
Pavel Marek
1f8511dab2
Add Comparator conversion for all types (#4067)
Add `Comparator` type class emulation for all types. Migrate all the types in stdlib to this new `Comparator` API. The main documentation is in `Ordering.enso`.

Fixes these pivotals:
- https://www.pivotaltracker.com/story/show/183945328
- https://www.pivotaltracker.com/story/show/183958734
- https://www.pivotaltracker.com/story/show/184380208

# Important Notes
- The new Comparator API forces users to specify both `equals` and `hash` methods on their custom comparators.
- All the `compare_to` overrides were replaced by definition of a custom _ordered_ comparator.
- All the call sites of `x.compare_to y` method were replaced with `Ordering.compare x y`.
- `Ordering.compare` is essentially a shortcut for `Comparable.from x . compare x y`.
- The default comparator for `Any` is `Default_Unordered_Comparator`, which just forwards to the builtin `EqualsNode` and `HashCodeNode` nodes.
- For `x`, one can get its hash with `Comparable.from x . hash x`.
- This makes `hash` as _hidden_ as possible. There are no other public methods to get a hash code of an object.
- Comparing `x` and `y` can be done either by `Ordering.compare x y` or `Comparable.from x . compare x y` instead of `x.compare_to y`.
2023-02-10 09:22:11 +00:00
James Dunkerley
1c821e22cf
Some fixed form the Anagrams experiment. (#5592)
- Fixes the display of Date, Time_Of_Day and Date_Time so doesn't wrap.
- Adjust serialization of large integer values for JS and display within table.
- Workaround for issue with using `.lines` in the Table (new bug filed).
- Disabled warning on no specified `separator` on `Concatenate`.

Does not include fix for aggregation on integer values outside of `long` range.
2023-02-08 22:17:00 +00:00
Radosław Waśko
4f90946d1e
Rework Invalid Aggregations (#5579)
Closes #5108
2023-02-08 18:39:09 +00:00
Michael Mauderer
d3e46e9957
Ensure Table Visualisation is the Default Visualization for Tables (#4120)
Adds a mechanism for the GUI to choose an appropriate default visualization for a newly created node.

https://user-images.githubusercontent.com/1428930/216641714-2410d2c9-44d2-45f0-856d-1f995421c94b.mp4

# Important Notes
This is a workaround for [missing functionality](https://www.pivotaltracker.com/story/show/184390437) that would take much longer to implement.
2023-02-07 23:23:24 +00:00
Radosław Waśko
3c72ab08c4
Review Missing_Input_Column and Column_Index_Out_Of_Range warnings (#4118)
Implements https://www.pivotaltracker.com/story/show/184226383
2023-02-06 19:52:25 +00:00
James Dunkerley
0790ce494f
New set function, parse a column (#4097)
- New `set` function design - takes a `Column` and works with that more easily and supports control of `Set_Mode`.
- New simple `parse` API on `Column`.
- Separated expression support for `filter` to new `filter_by_expression` on `Table`.
- New `compute` function allowing creation of a column from an expression.
- Added case sensitivity argument to `Column` based on `starts_with`, `ends_with` and `contains`.
- Added case sensitivity argument to `Filter_Condition` for `Starts_With`, `Ends_With`, `Contains` and `Not_Contains`.
- Fixed the issue in JS Table visualisation where JavaScript date was incorrectly set.
- Some dynamic dropdown expressions - experimenting with ways to use them.
- Fixed issue with `.pretty` that wasn't escaping `\`.
- Changed default Postgres DB to `postgres`.
- Fixed SQLite support for starts_with, ends_with and contains to be consistent (using GLOB not LIKE).
2023-01-31 20:48:16 +00:00
Radosław Waśko
c965ad3455
Review Table.order_by (#4104) 2023-01-31 18:29:02 +00:00
Hubert Plociniczak
d3b350f460
Custom type conversion to double from large long (#4099)
When a large long would be passed to a host call expecting a double, it would crash with a
```
Cannot convert '<some long>'(language: Java, type: java.lang.Long) to Java type 'double': Invalid or lossy primitive coercion
```

That is unlikely to be expected by users. It also came up in the Statistics examples during Sum. One could workaround it by forcing the conversion manually with `.to_decimal` but it is not a permanent solution.

Instead this change adds a custom type mapping from Long to Double that will do it behind the scenes with no user interaction. The mapping kicks in only for really large longs.

# Important Notes
Note that the _safe_ range is hardcoded in Truffle and it is not accessible in enso packages. Therefore a simple c&p for that max safe long value was necessary.
2023-01-31 15:13:00 +00:00
Hubert Plociniczak
be91b1e8dd
Create static wrappers for builtin types (#4077)
https://github.com/enso-org/enso/pull/3764 introduced static wrappers for instance methods. Except it had a limitation to only be allowed for types with at least a single constructor.
That excluded builtin types as well which, by default, don't have them. This limitation is problematic for Array/Vector consolidation and makes builtin types somehow second-citizens.

This change lifts the limitation for builtin types only. Note that we do want to share the implementation of the generated builtin methods. At the same time due to the additional argument we have to adjust the starting index of the arguments.
This change avoids messing with the existing dispatch logic, to avoid unnecessary complexity.

As a result it is now possible to call builtin types' instance methods, statically:
```
arr = Array.new_1 42
Array.length arr
```
That would previously lead to missing method exception in runtime.

# Important Notes
The only exception is `Nothing`. Primarily because it requires `Nothing` to have a proper eigentype (`Nothing.type`) which would messed up a lot of existing logic for no obvious benefit (no more calling of `foo=Nothing` in parameters being one example).
2023-01-30 19:54:51 +00:00
Radosław Waśko
b9dbfd036f
First steps of the Problem Handling refactor to the new design (#4086)
Implements:
- https://www.pivotaltracker.com/story/show/184226137
- https://www.pivotaltracker.com/story/show/184226434
- https://www.pivotaltracker.com/story/show/184226462
2023-01-30 16:48:06 +00:00
Jaroslav Tulach
ca2f108513
Enable asserts in the tests (#4074)
Enso unit tests were running without `-ea` check enabled and as such various invariant checks in Truffle code were not executed. Let's turn the `-ea` flag on and fix all the code misbehaves.
2023-01-26 21:41:35 +00:00
James Dunkerley
359d4f56df
Initial Widget Dynamic Metadata (#4082)
Adjusts the old mock to use new attributes.
Expanded examples for dynamic widgets.
2023-01-25 14:16:10 +00:00
James Dunkerley
60f0e96047
Update starts_with, ends_with and contains to new API design (#4078)
- Updated `Text.starts_with`, `Text.ends_with` and `Text.contains` to new simpler API.
- Added a `Case_Sensitivity.Default` and adjusted `Table.distinct` to use it by default.
- Fixed a bug with `Data.fetch` on an HTTP error.
- Improved SQLite Case Sensitivity control in distinct to use collations.
2023-01-25 11:47:27 +00:00
Radosław Waśko
778d28fba3
Table with no columns is not valid, No_Output_Columns is always an error (#4073)
Implements https://www.pivotaltracker.com/story/show/184226020
2023-01-25 02:40:23 +00:00
Michael Mauderer
38906b39da
Implement Lazy Text Visualisation. (#3910)
Implements [#183453466](https://www.pivotaltracker.com/story/show/183453466).

https://user-images.githubusercontent.com/1428930/203870063-dd9c3941-ce79-4ce9-a772-a2014e900b20.mp4

# Important Notes
* the best laziness is used for `Text` type, which makes use of its internal representation to send data
* any type will first compute its default string representation and then send the content of that lazy to the IDE
* special handling of files and their content will be implemented in the future
* size of the displayed text can be updated dynamically based on best effort information: if the backend does not yet know the full width/height of the text, it can update the IDE at any time and this will be handled gracefully by updating the scrollbar position and sizes.
2023-01-24 20:55:36 +00:00
Dmitry Bushev
bf9508603f
Add Meta.get_annotation (#4049)
- add: `GeneralAnnotation` IR node for `@name expression` annotations
- update: compilation pipeline to process the annotation expressions
- update: rewrite `OverloadsResolution` compiler pass so that it keeps the order of module definitions
- add: `Meta.get_annotation` builtin function that returns the result of annotation expression
- misc: improvements (private methods, lazy arguments, build.sbt cleanup)
2023-01-24 21:28:33 +03:00
Radosław Waśko
d2e57edc8b
Add Table.cross_join and Table.zip to In-Memory Table (#4063)
Implements https://www.pivotaltracker.com/story/show/184239059
2023-01-23 13:19:52 +00:00
Pavel Marek
fcc2163ae3
All Enso objects are hasheable (#3878)
* Hash codes prototype

* Remove Any.hash_code

* Improve caching of hashcode in atoms

* [WIP] Add Hash_Map type

* Implement Any.hash_code builtin for primitives and vectors

* Add some values to ValuesGenerator

* Fix example docs on Time_Zone.new

* [WIP] QuickFix for HashCodeTest before PR #3956 is merged

* Fix hash code contract in HashCodeTest

* Add times and dates values to HashCodeTest

* Fix docs

* Remove hashCodeForMetaInterop specialization

* Introduce snapshoting of HashMapBuilder

* Add unit tests for EnsoHashMap

* Remove duplicate test in Map_Spec.enso

* Hash_Map.to_vector caches result

* Hash_Map_Spec is a copy of Map_Spec

* Implement some methods in Hash_Map

* Add equalsHashMaps specialization to EqualsAnyNode

* get and insert operations are able to work with polyglot values

* Implement rest of Hash_Map API

* Add test that inserts elements with keys with same hash code

* EnsoHashMap.toDisplayString use builder storage directly

* Add separate specialization for host objects in EqualsAnyNode

* Fix specialization for host objects in EqualsAnyNode

* Add polyglot hash map tests

* EconomicMap keeps reference to EqualsNode and HashCodeNode.

Rather than passing these nodes to `get` and `insert` methods.

* HashMapTest run in polyglot context

* Fix containsKey index handling in snapshots

* Remove snapshots field from EnsoHashMapBuilder

* Prepare polyglot hash map handling.

- Hash_Map builtin methods are separate nodes

* Some bug fixes

* Remove ForeignMapWrapper.

We would have to wrap foreign maps in assignments for this to be efficient.

* Improve performance of Hash_Map.get_builtin

Also, if_nothing parameter is suspended

* Remove to_flat_vector.

Interop API requires nested vector (our previous to_vector implementation). Seems that I have misunderstood the docs  the first time I read it.

- to_vector does not sort the vector by keys by default

* Fix polyglot hash maps method dispatch

* Add tests that effectively test hash code implementation.

Via hash map that behaves like a hash set.

* Remove Hashcode_Spec

* Add some polyglot tests

* Add Text.== tests for NFD normalization

* Fix NFD normalization bug in Text.java

* Improve performance of EqualsAnyNode.equalsTexts specialization

* Properly compute hash code for Atom and cache it

* Fix Text specialization in HashCodeAnyNode

* Add Hash_Map_Spec as part of all tests

* Remove HashMapTest.java

Providing all the infrastructure for all the needed Truffle nodes is no longer manageable.

* Remove rest of identityHashCode message implementations

* Replace old Map with Hash_Map

* Add some docs

* Add TruffleBoundaries

* Formatting

* Fix some tests to accept unsorted vector from Map.to_vector

* Delete Map.first and Map.last methods

* Add specialization for big integer hash

* Introduce proper HashCodeTest and EqualsTest.

- Use jUnit theories.
- Call nodes directly

* Fix some specializations for primitives in HashCodeAnyNode

* Fix host object specialization

* Remove Any.hash_code

* Fix import in Map.enso

* Update changelog

* Reformat

* Add truffle boundary to BigInteger.hashCode

* Fix performance of HashCodeTest - initialize DataPoints just once

* Fix MetaIsATest

* Fix ValuesGenerator.textual - Java's char is not Text

* Fix indent in Map_Spec.enso

* Add maps to datapoints in HashCodeTest

* Add specialization for maps in HashCodeAnyNode

* Add multiLevelAtoms to ValuesGenerator

* Provide a workaround for non-linear key inserts

* Fix specializations for double and BigInteger

* Cosmetics

* Add truffle boundaries

* Add allowInlining=true to some truffle boundaries.

Increases performance a lot.

* Increase the size of vectors, and warmup time for Vector.Distinct benchmark

* Various small performance fixes.

* Fix Geo_Spec tests to accept unsorted Map.to_vector

* Implement Map.remove

* FIx Visualization tests to accept unsorted Map.to_vector

* Treat java.util.Properties as Map

* Add truffle boundaries

* Invoke polyglot methods on java.util.Properties

* Ignore python tests if python lang is missing
2023-01-19 10:33:25 +01:00
Hubert Plociniczak
d463a43633
Resolve fully qualified names (#4056)
Added a separate pass, `FullyQualifiedNames`, that partially resolves fully qualified names. The pass only resolves the library part of the name and replaces it with a reference to the `Main` module.

There are 2 scenarios that could be potentially:
1) the code uses a fully qualified name to a component that has been
parsed/compiled
2) the code uses a fully qualified name to a component that has **not** be
imported

For the former case, it is sufficient to just check `PackageRepository` for the presence of the library name.
In the latter we have to ensure that the library has been already parsed and all its imports are resolved. That would require the reference to `Compiler` in the `FullyQualifiedNames` pass, which could then trigger a full compilation for missing library. Since it has some undesired consequences (tracking of dependencies becomes rather complex) we decided to exclude that scenario until it is really needed.

# Important Notes
With this change, one can use a fully qualified name directly.
e.g.
```
import Standard.Base
main =
Standard.Base.IO.println "Hello world!"
```
2023-01-18 20:19:36 +00:00
James Dunkerley
48e5ed9eea
Some little bits from Book Club week 1 (#4058)
- Add `get` to Table.
- Correct `Count Nothing` examples.
- Add `join` to File.
- Add `File_Format.all` listing all installed formats.
- Add some more ALIAS entries.
2023-01-18 11:46:13 +00:00
Radosław Waśko
8853053020
Division in Columns within InDB is integer based if both columns are integers (#4057)
Fixes https://www.pivotaltracker.com/story/show/184073099

# Important Notes
- Since now the only operator on columns for division, `/`, returns floats, it may be worth creating an additional `div` operator exposing integer division. But that will be done as a separate task aligning column operator APIs.
2023-01-17 20:29:25 +00:00