Commit Graph

638 Commits

Author SHA1 Message Date
Jaroslav Tulach
34c63e3b11
Ascribed types & atom_with_hole (#7348) 2023-07-24 07:02:58 +02:00
James Dunkerley
88f32d9b2a
Various small tickets... (#7367)
- Added `Text.length` into Text class so CB lists the built in.
- Added `File.starts_with` and tests for the built in method.
- Add `to_js_object` and `to_display_text` to `Regex`.
![image](https://github.com/enso-org/enso/assets/4699705/3b197c94-9c49-4bc5-a2cc-ce53b917942e)
- Add `to_js_object` and `to_display_text` to `Match`.
![image](https://github.com/enso-org/enso/assets/4699705/962ec4f2-324d-4f10-8ec0-932b093c6729)
- Remove the `bit_shift_l` alias from the built-ins.
- Add test and Enso wrapper for `Text.is_normalized`.
2023-07-23 09:04:11 +00:00
Radosław Waśko
56635c9a88
Add benchmarks comparing performance of Table operations 'vectorized' in Java vs performed in Enso (#7270)
The added benchmark is a basis for a performance investigation.

We compare the performance of the same operation run in Java vs Enso to see what is the overhead and try to get the Enso operations closer to the pure-Java performance.
2023-07-21 17:25:02 +00:00
Pavel Marek
cab6968340
Update GraalVM to 17.0.7 (23.0.0 JDK17) (#7176)
- Previous GraalVM update: https://github.com/enso-org/enso/pull/6750

Removed warnings:
- Remove deprecated `ConditionProfile.createCountingProfile()`.
- Add `@Shared` to some `@Cached` parameters (Truffle now emits warnings about potential `@Share` usage).
- Specialization method names should not start with execute
- Add limit attribute to some specialization methods
- Add `@NeverDefault` for some cached initializer expressions
- Add `@Idempotent` or `@NonIdempotent` where appropriate

BigInteger and potential Node inlining are tracked in follow-up issues.

# Important Notes
For `SDKMan` users:
```
sdk install java 17.0.7-graalce
sdk use java 17.0.7-graalce
```

For other users - download link can be found at https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-17.0.7

Release notes: https://www.graalvm.org/release-notes/JDK_17/

R component was dropped from the release 23.0.0, only `python` is available to install via `gu install python`.
2023-07-20 15:11:30 +00:00
GregoryTravis
8a829384ed
Add zone to Date_Times (#7325) 2023-07-19 11:04:50 +00:00
Jaroslav Tulach
a5ec6a9e51
Bench builder API (#7324)
Designing new `Bench` API to _collect benchmarks_ first and only execute them then. This is a minimal change to allow  implementation of #7323  - e.g. ability to invoke a _single benchmark_ via JMH harness.

# Important Notes
This is just the basic API skeleton. It can be enhanced, if the basic properties (allowing integration with JMH) are kept. It is not intent of this PR to make the API 100% perfect and usable. Neither it is goal of this PR to update existing benchmarks to use it (74ac8d7 changes only one of them to demonstrate _it all works_ somehow). It is however expected that once this PR is integrated, the newly written benchmarks (like the ones from #7270) are going to use (or even enhance) the new API.
2023-07-19 09:18:28 +00:00
GregoryTravis
2fb5c3710b
Add Fallback to Prim_Text_Helper.compile_regex; accept Regex in Text.parse_to_table (#7297)
This PR does three related things:
- Fails more gracefully when a non-string is passed to compile_regex
- Don't pass a non-string to compile_regex
- Allow a Regex param to parse_to_table
2023-07-18 19:55:56 +00:00
James Dunkerley
fd0bdc86dd
Fix issue with rename_columns and revert order of parameter change on select_columns. (#7321)
The Regex change introduced some issues.
Added a test for missed case in `rename_columns` where using vector of pairs.
Reverted parameter order change for `select_columns`.
2023-07-18 13:30:23 +00:00
Jaroslav Tulach
e59513065d
Ascribed Type checks for a -> b function types (#7298) 2023-07-17 09:55:30 +01:00
James Dunkerley
aaa235fbad
Add drop down for replace, remove Column_Selector (#7295)
- Add dropdowns for `replace` functions.
- Retire `Column_Selector` type.
- Add `select_blank_columns` and `remove_blank_columns` functions to table types.
- Allow Regex to be used to pick columns.
2023-07-14 17:30:52 +00:00
Radosław Waśko
866283c0a8
Improve error message on Filter_Condition missing arguments in Table.filter (#7290)
In #7148 I improved the error message when a `Filter_Condition` constructor without arguments is provided to `Vector.filter` and its friends. This PR applies the same check to the `Table.filter`.

This is useful, because when we select a Filter_Condition from a widget, initially it does not have all its arguments applied. This used to lead to confusing errors being reported to the user, now, a much clearer error is shown:

![image](https://github.com/enso-org/enso/assets/1436948/19140a7b-d6fc-4292-81d3-dc6d61135cb9)
2023-07-14 08:00:13 +00:00
Radosław Waśko
620cc361ce
Add date_diff, date_add and date_part to scalar Enso date-time values. (#7273)
Followup of #7221, adding `date_diff`, `date_add` and `date_part` to scalar Enso date-time values.
2023-07-13 15:17:21 +00:00
Radosław Waśko
ca68dd94da
Adding new Date/Time operations (-, date_add, date_diff, date_part) (#7221)
- Adds `Column.date_diff` for computing date/time difference as integer multiply of some unit.
- Adds `Column.date_add` for shifting date/time by a unit.
- Adds `Column.date_part` for extracting various parts of the date/time value as integer.
- Adds widgets for the 3 methods above whose content depends on the column value type.
- Adds shorthands: `Column.hour`, `Column.minute` and `Column.second` to extract these date parts.
- Extends `Time_Period` with support for milli-, micro- and nano- seconds; and adapts functions taking `Time_Period` to support these wherever possible.
2023-07-13 12:56:54 +00:00
Jaroslav Tulach
724a788500
Convert vector elements when needed (#7271)
Adds a test case demonstrating the _use of automatic conversions_ when processing vector elements.
2023-07-12 13:05:33 +00:00
James Dunkerley
0adab6c68c
Round on a column was always adding a warning (#7246)
- Only warn if outside allowed range.
- Added `is_infinite` to In-Memory column.
- Allow integer value type for `is_nan` and `is_infinite`.
2023-07-10 17:35:23 +00:00
GregoryTravis
345d6b9cb1
Add cross_join support to Database Table (#7234) 2023-07-10 16:29:37 +00:00
James Dunkerley
1fb60df61b
Fixes from the live demo. (#7243)
- Removed defaults from `cross_tab`. It caused an out-of-heap space error when it attempted to build a 205k x 205k table. Now has a hard limit of 10,000 columns - we can increase this once we have more concrete test data.
![image](https://github.com/enso-org/enso/assets/4699705/bc38d41c-56dc-41bd-8a7c-fa89ecfa7f79)

- Adjusted the dropdowns on `Aggregate_Column` for `columns` and `order_by` to be dropdowns as nested Vector editors are not supported.
![image](https://github.com/enso-org/enso/assets/4699705/f4a7c7cc-6a21-462c-a39e-65fbab82c367)

- Altered `Aggregate_Column` so `new_name` now `new_name:Text=""` and not taking `Nothing` anymore. Makes it appear correctly in IDE.
![image](https://github.com/enso-org/enso/assets/4699705/196a49ba-4274-44bb-b876-0372c8f62746)

- Added dropdowns for `fill_empty`, `fill_nothing` and `replace` on `Table`.
![image](https://github.com/enso-org/enso/assets/4699705/9ee5cec2-82d5-4452-b650-67015ac9fee5)

- Added `replace` to Database table throwing `Unsupport_Database_Operation`.
2023-07-09 18:03:05 +00:00
Hubert Plociniczak
9dcf48a3e0
Cache dataflow errors (#7193)
The logic in instrumentation specifically prohibited caching of panics (added in https://github.com/enso-org/enso/pull/1611).
Except it seems that the restriction was too aggressive because dataflow errors could be cached.
This indirectly fixes the problem with dataflow error in the context of a (temporary) change in execution context, which should keep around the old value.

Closes #7140.

After the fix:
[fix_7140.webm](https://github.com/enso-org/enso/assets/292128/b8d522db-985a-4e57-bbf1-a075d14d39a2)
2023-07-09 14:45:22 +00:00
GregoryTravis
bd26e95fd6
Add Table.replace; Change Text.replace to take a Text|Pattern, and remove the use_regex param. (#7223) 2023-07-06 16:13:11 +00:00
James Dunkerley
7749286c69
Tidy up the imports using script (#7220)
Ordering the imports to test a script.
2023-07-06 14:22:50 +00:00
GregoryTravis
6eb46afb40
Do not rename column on fill_nothing and add version to the Table allowing filling multiple (include fill_empty as well). (#7166)
Updated Column.fill_nothing and .fill_empty, and added the same to Table. (Both in-memory and db.)
2023-07-05 17:20:23 +00:00
Radosław Waśko
78545b4402
Add safepoints to standard libraries Java polyglot helpers (#7183)
Closes #7129
2023-07-05 14:12:13 +00:00
GregoryTravis
966f8b773a
Combine Regex and Pattern (#7172)
Merge Pattern into Regex.
2023-07-05 13:51:53 +00:00
Hubert Plociniczak
a140a04689
A dataflow error with warnings reports the former (#7205)
A dataflow error resulting from calling a value with warnings now
reports only the error rather than the warning.

Closes #7141.
2023-07-05 14:18:36 +02:00
Radosław Waśko
2d73277238
Fix a bug that somehow went under CI (#7204) 2023-07-05 08:54:27 +00:00
James Dunkerley
4fbe7e3830
Remove Array.new and Array.copy and move Vector functions to builtins. (#7147)
- Removed Array methods: `new`, `copy` and `new_[1234]`.
- New builtins for `Vector.insert`, `Vector.remove` and `Vector.flatten`.
- Replaced `Vector_Builder` use of `Array.copy` to a `Vector.Builder` approach.
2023-07-03 12:41:41 +00:00
Radosław Waśko
4ccf3566ce
Implement add_row_number for Database backends, fix primary key inference for SQLite (#7174)
Closes #6921 and also closes #7037
2023-07-03 11:51:42 +00:00
GregoryTravis
c866aa7fb5
parse_to_columns should generate at least one row for a non-match (#7171) 2023-06-30 18:10:33 +00:00
GregoryTravis
550d146493
Add round, ceil, floor, truncate to the In-Database Column type (#6988) 2023-06-30 16:47:40 +00:00
Radosław Waśko
6eac095579
Add support for Filter_Condition in any, all, find, partition and index_of (#7148)
Closes #6628
2023-06-30 16:06:01 +00:00
Pavel Marek
ebee8700ce
HashMapNode supports atoms with custom comparators (#7165)
Add proper handling for atoms with custom comparators into the hashing machinery.
2023-06-30 08:57:36 +00:00
Pavel Marek
60ef7e0ba3
Referencing self from static method raises compiler error (#7131)
Add a new `NoStaticInSelf` compiler pass that ensures that `self` keyword is used only from instance methods.
2023-06-28 19:04:32 +02:00
James Dunkerley
56688ec1e7
Minor fixes. (#7122)
Mostly stuff to tidy up the static methods in the CB.

- Remove default pattern from `parse_to_table` (caused IDE to freeze).
- Rename any `_` arguments to what they are.
- Merge `Date.now` into `Date.today`
- Merge the Interval constructors into a single constructor.
- Hide various methods.
2023-06-27 18:18:15 +00:00
Radosław Waśko
2bac9cc844
Execution Context integration for Database write operations (#7072)
Closes #6887
2023-06-27 15:51:21 +00:00
Jaroslav Tulach
477dd82670
Drop-down widgets for extension functions via UnresolvedSymbol (#7115)
Fixes #6955 by:
- using `visualisationModule` to specify the module where the visualization is to be used
- referring to method in `Meta.get_annotation` with `.method_name` - e.g. unresolved symbol notation
- evaluating arguments to `Meta.get_annotation` in the context of the user module (which can access the extension functions)
2023-06-27 15:19:42 +00:00
Dmitry Bushev
22259e696d
Add method call info for infix operators (#7090)
close #6374

In order to provide the method pointer information, the `IrToTruffle` pass sets the module name and the type name of the builtin node.
2023-06-27 13:11:12 +00:00
Hubert Plociniczak
4e5cb9cca6
Consistent naming of visualization (#7124) 2023-06-26 17:04:35 +00:00
Hubert Plociniczak
ae4666c4d3
Improve TCO in the presence of warnings (#7116)
Partially revert https://github.com/enso-org/enso/pull/6849, which introduced a regression in TCO in the presence of warnings. Rather than modifying the tail call status, `TailCallException` now propagates the extracted warnings and appends them to the final result.

Closes #7093

# Important Notes
Compared to the previous attempt we don't pay the penalty of adding the warnings or even checking for them because it is being dealt in a separate specialization.
2023-06-26 12:38:36 +00:00
Jaroslav Tulach
c4f19e7d66
Vector.map & State (#7078) 2023-06-26 13:13:41 +02:00
James Dunkerley
937651f696
Code Clean Up, Fix Weird Namespace, S3 List Objects and Read Object (#7114)
Mostly a tidy up as part of looking over the function catalogue for groups.
Sorted some whitespaces issues.
2023-06-24 23:18:58 +00:00
Pavel Marek
31aad1d9e6
Always call instance methods on Any (#7033)
Previously, static method calls on `Any` have not worked as expected. For example, `Any.to_text` returned Function instead of Text. That is because the function resolution for `Any.to_text` finds `Any.type.to_text` method on eigentype which expects two `self` arguments, but only one argument is provided.

Note that `Boolean.to_text` worked previously, and returned "Boolean" as expected. This is because the method resolution finds `Any.to_text` method that takes just one `self` argument.

This PR solves this issue by introducing special handling for static method dispatch on `Any`. Simply put, an additional `self` argument is prepended to the argument list.

# Important Notes
A new child node is introduced to `InvokeMethodNode`. This child node is a copy of the current `invokeFunctionNode` with one more `CallArgumentInfo` in its schema.
2023-06-23 14:46:42 +00:00
James Dunkerley
1859ccbab5
Improving widgets and other minor tweaks. (#7052)
- Removed `module` argument from `enso_project` (new `Project_Description.new` API).
- Removed the custom option from date and time parse/format dropdowns.
- The `format` dropdown uses the value to create the dropdown. (Screenshot below)
- Removed `StorageType` coalescing rules and replaced them with simpler logic in `ObjectStorage`.
- Update signature for `add_row_number` and add aliases.
2023-06-19 19:03:36 +00:00
James Dunkerley
760fb71798
First part of AWS S3 API, various small fixes. (#6973)
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
2023-06-15 16:20:13 +00:00
Radosław Waśko
dad57e6c7d
Implement remaining Update_Actions for update_database_table. (#7035)
Closes #6498
2023-06-15 08:48:22 +00:00
Pavel Marek
67821bf8df
Add compiler pass that discovers ambiguous imports (#6868)
Add a new compiler pass that analyses duplicated and ambiguous symbols from imports
2023-06-14 12:18:57 +02:00
Radosław Waśko
d9ed63fb89
Implement Insert update action for update_database_table. (#6990)
This adds the spec for all update actions, but implements the common input validation framework and `Insert`. Tests for remaining actions are marked as pending - these will be implemented in a subsequent PR.
2023-06-14 00:14:32 +00:00
Dmitry Bushev
6249c79ffd
Update sbt-java-formatter plugin (#7011)
Update java formatter plugin. The new version can remove unused imports.
2023-06-12 14:18:48 +00:00
GregoryTravis
fcc57e44e7
Implement addition and subtraction for Date_Period and Time_Period (#6956)
Date.+ should allow Date_Period, Time_Of_Day.+ should allow Time_Period and Date_TIme.+ should allow both.
Same for subtraction.
2023-06-07 12:46:19 +00:00
James Dunkerley
578ba59f1d
Use US Locale for Date and Time parsing and formatting (#6967)
Sorts out parsing and printing long form names of months and weekdays.
2023-06-06 21:44:25 +00:00
Radosław Waśko
1931e9e51f
Workaround for to_date_time type errors (#6964)
Related to #6912

It essentially solves it by removing any builtins that would take an EnsoDate/EnsoTimeOfDay/EnsoTimeZone and replacing them with Java utils that do the same operation.

This is not a proper solution - the builtin conversion is still invalid for the date/time types - but at this moment we may just no longer use the invalid conversion so it is much less of an issue. We still need to be aware of this if we want to introduce builtins taking date/time in the future.
2023-06-06 20:28:11 +00:00
GregoryTravis
912fbce97b
Reimplement Column.truncate, .ceil, and .floor as vectorized Java ops (#6941)
Reimplement these in Java.

Benchmarks:

Before:

Column.truncate floats average: 124.4ms
Column.ceil floats average: 121.47ms
Column.floor floats average: 120.18ms
Column.truncate ints average: 124.78ms
Column.ceil ints average: 120.41ms
Column.floor ints average: 102.35ms

After (boxed):

Column.truncate floats average: 3.75ms
Column.ceil floats average: 2.25ms
Column.floor floats average: 1.89ms
Column.truncate ints average: 2ms
Column.ceil ints average: 1.77ms
Column.floor ints average: 1.74ms

After (unboxed):
Column.truncate floats average: 3.32ms
Column.ceil floats average: 2.15ms
Column.floor floats average: 1.69ms
Column.truncate ints average: 1.74ms
Column.ceil ints average: 1.61ms
Column.floor ints average: 1.99ms
2023-06-06 18:07:12 +00:00
Radosław Waśko
b513839418
Refactor create_database_table into Connection.create_table and select_into_database_table, implement Set. (#6925)
First part for #6498 - refactoring of the upload infrastructure, in preparation for `update_database_table`.

Implemented a `Set` data structure which was long needed.

The APIs are added and an initial implementation is created, but it is not complete - but it has grown significantly already so the remaining implementation will be done as a separate PR.

Adds some basic ability for a function to ensure that it is only executed from within a transaction.
2023-06-06 10:36:05 +00:00
Jaroslav Tulach
f1bdcbb534
Documenting how to obtain static methods for a type (#6938)
Fixes #6748 by documenting how to obtain all static methods available on a `Type`.
2023-06-06 08:06:10 +00:00
GregoryTravis
3ffbe9cecf
Handle some edge cases in rounding (inexact representations and overflows) (#6922) 2023-06-05 17:21:13 +00:00
James Dunkerley
db96bd2e2c
Small fixes from book club. (#6933)
- Add the missing dropdowns for `Locale` and `Encoding`.
- Correct a few mismatched type signatures.
- Adjust `order_by` calls with a single `Sort_Column` to call in a Vector.
- Adjust parameter names for `transpose`.
- Fix for the table viz: escape HTML and `suppressFieldDotNotation`.
- Use `Filter_Condition.Equal True` for the default filter.
- Adjust `Data.fetch` to return the response on success when parse fails. Rename `parse` to `try_auto_parse`.
- Add various aliases for methods.
- Add tests for `Table.set` when using a `Vector`, `Range` or `Date_Range`.
- Add check for mismatched length on `Table.set`.

![image](https://github.com/enso-org/enso/assets/4699705/23ea0ba3-2b05-4af8-afd9-f35b55446c24)

![image](https://github.com/enso-org/enso/assets/4699705/8b0253e6-e9e8-490a-9607-0da51ab5a215)
2023-06-05 13:57:30 +00:00
Hubert Plociniczak
e9a92a1fb5
Throw panic on "no currying for conversions" (#6940)
Previously, a `RuntimeException` would be thrown when an attempt would be made to curry a conversion function. That is problematic for IDE where `executionFailed` means we can't enter functions due to lack of method pointers info.

Closes #6897.

![Screenshot from 2023-06-02 20-31-03](https://github.com/enso-org/enso/assets/292128/a6c77544-2c47-425c-8ce0-982d837dda5b)

# Important Notes
A more generic solution that allows to recover from execution failures will need a follow up.
2023-06-05 13:16:12 +00:00
Radosław Waśko
cfb2f2916e
Merge Column_Indexes_Out_Of_Range into Missing_Input_Columns. (#6901)
Implements #6869
2023-06-02 12:09:20 +00:00
Radosław Waśko
d44b1250b7
Implement Table.add_row_number (#6890)
Closes #5227

# Important Notes
- This lays first steps towards #6292 - we get pure Enso variants of MultiValueKey.
- Another part refactors `LongStorage` into `AbstractLongStorage` allowing it to provide alternative implementations of the underlying storage, in our case `LongRangeStorage` generating the values ad-hoc and `LongConstantStorage` - currently unused but in the future it can be adapted to support constant columns (once we implement similar facilities for other types).
2023-06-02 10:13:13 +00:00
James Dunkerley
343b5fb085
Execution control for Table.write and various widget tweaks... (#6835)
- Adds execution control to `Table.write`.
- Refactored the `Text.write` to make part reusable.
- Tidied up some legacy mess in tests.
- Add easier flow to go from `Text` to an `URI` to fetching data.
- Add decode functions to `Response` and `Response_Body`.
- Fix issue with 0 length regex matches (using same as Python and .Net approach).
- Add various ALIAS entries to make function discovery easier.
- Sort a lot of drop down and vector editors out (including switch to fully qualified names).
2023-06-01 22:10:03 +00:00
GregoryTravis
0337180384
Add rounding functions to the Column type (#6817) 2023-06-01 20:06:23 +00:00
GregoryTravis
7e53cd9af1
Add drop down for Locale like Encoding (#6654)
Add dropdowns for locale parameters for format and parse methods.
2023-05-31 12:43:20 +00:00
Jaroslav Tulach
c6cb937c0a
Runtime check of ascribed types (#6790) 2023-05-30 11:07:26 +02:00
Hubert Plociniczak
6eb4737330
Append warnings extracted before tail call execution (#6849)
Throwing `TailCallException` meant that exceptions that were extracted from the expression before the call was made could not be appended. This change catches the `TailCallException`, adds warnings to it and propagates it further, thus ensuring that we don't loose the information.

Closes #6765.

# Important Notes
Removed workarounds introduced in stdlib.
2023-05-29 12:44:15 +00:00
Jaroslav Tulach
e7ee2caa2e
Infrastructure for testing inter project imports and exports (#6840) 2023-05-26 17:33:54 +02:00
Radosław Waśko
c3e771c75c
Allow casting a Mixed column into a concrete type (#6777)
Follow-up of #6711

Closes #6838
2023-05-26 13:25:53 +00:00
Jaroslav Tulach
792cbc448c
Clearly select single specialization with enum dispatch pattern (#6819) 2023-05-24 20:42:24 +02:00
Pavel Marek
bd70ed6830
Update GraalVM to 22.3.1 JDK17 (#6750)
Upgrade GraalVM to 22.3.2 based on JDK17.

---------

Co-authored-by: somebody1234 <ehern.lee@gmail.com>
2023-05-24 10:39:08 +02:00
Pavel Marek
2b1e5cd017
Add a compiler pass to analyze non-existing imported symbols (#6726)
Add diagnosis for unresolved symbols in `from ... import sym1, sym2, ...` statements.

- Adds a new compiler pass, `ImportSymbolAnalysis`, that checks these statements and iterates through the symbols and checks if all the symbols can be resolved.
  - Works with `BindingsMap` metadata.
- Add `ImportExportTest` that creates various modules with various imports/exports and checks their generated `BindingMap`.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2023-05-22 10:41:15 +02:00
James Dunkerley
62fecfa474
Widgets, Vector as Column, Viz Fixes and Rename Columns (#6768)
- Fix couple of bugs in Table viz: rounding of bottom div, missing character, not including row count as an option.
- Add better JSON format for `Row`, add support for visualization in the Table viz both for `Vector Row` or `Row`.
- Fix some type signature errors.
- Move `Column_Format` to `Standard.Table.Internal`.
- Move `format_widget` to `File_Format.default_widget` and sort the signature of `Widget` methods.
- Added utility to make `Single_Choice` widgets.
- Added dropdown for delimiter on split methods.
- Removed `default_widget` from `Problem_Behavior` and `Filter_Condition`.
- Altered signature and widgets for table functions.
- Added `to_column` extension to allow easy conversion of Range and Vector to Column.
- Added `compute`, `compute_bulk`, `running` to Column to allow statistic computation.
- Added drop down for `Table.write` format parameter.
- Added drop down for `Table.rename_columns`.
- Added support for Vector of pairs for renaming columns.
- Added check when making a map from Vector if not 2 items.

![image](https://github.com/enso-org/enso/assets/4699705/beed257c-efe3-44a3-9e3a-041354701735)
2023-05-19 23:24:47 +00:00
Radosław Waśko
a9a464af37
Implement simple variants of parse for the Database backend (#6731)
Implements the simplest `parse` scenarios for the Database backend.

Before #6711 these could have been done by `cast`, but in #6711 the APIs were unified to only allow casting to the same set of types in both in-memory and Database. Converting Text to other types is supposed to be done by `parse` and not `cast`, so the ability to use `cast` for rudimentary parsing is removed in the Database backend to make it consistent with in-memory. But now it is lacking any, even simplest, Text->Int/Text->Date support. To alleviate that, the simple scenarios for `parse` are implemented (no support for format customization yet, will boil down to a cast under the hood).
2023-05-19 22:11:23 +00:00
GregoryTravis
4f71673718
Decimal/Integer .round and .int #6654 (#6743) 2023-05-19 19:23:17 +00:00
Radosław Waśko
447786a304
Implement cast for Table and Column (#6711)
Closes #6112
2023-05-19 10:00:20 +00:00
Jaroslav Tulach
8c370008bc
Meta.meta Integer . methods (#6740) 2023-05-18 11:40:44 +02:00
Dmitry Bushev
706791779b
SuggestionBuilder needs to send ascribedType of constructor parameters (#6655)
close #6611

Changelog:
- update: run compiler passes on the `ascribedType` field of the constructor arguments
- update: suggestion builder uses the type information attached to `ascribedType`
- feat: resolve qualified names in type signatures
2023-05-13 18:33:03 +00:00
Radosław Waśko
f5071a17fd
Improving widgets for take/drop (#6641)
Related to #6410

# Important Notes
- Updated some `Meta` methods (needed for error handling):
- `Meta.Type` now has `name` and `qualified_name`.
- `Meta.Constructor` has `declaring_type` allowing to get the type that this constructor is associated with.
2023-05-12 15:33:15 +00:00
Radosław Waśko
cd7fb73232
Add Date_Range (#6621)
Closes #6543
2023-05-11 16:03:02 +00:00
Jaroslav Tulach
1302b693d8
All Vector operations shall be applicable on java.util.ArrayList (#6642)
Fixes #6609 by
- e380e647af - running whole `Vector_Spec` on `java.util.ArrayList`
- 9b1229fe20 - introducing a node to handle interop values

# Important Notes
Contains additional DSL processor fix:
- 415623dcb9 - to not crash the compiler, but to properly report compiler error
2023-05-11 15:29:47 +00:00
Jaroslav Tulach
4e7191af46
Make Meta.get_annotation work for constructor (#6633)
Fixes #6612 by searching for the atom constructor on the `target` itself, not its type.
2023-05-10 15:42:28 +00:00
Hubert Plociniczak
cf3624c463
Limit the number of reported warnings (#6577)
Artifically limiting the number of reported warnings to 100. Also added benchmarks with random Ints to investigate perf issues when dealing with warnings (future task).
Ideally we would have a custom set-like collection that allows us internally to specify a maximal number of elements. But `EnsoHashMap` (and potentially `EnsoSet`) are still WIP when it comes to being PE-friendly.

The change also allows for checking if the limit for the number of reported warnings has been reached. It will visualize by adding an additional "Warnings limit reached." to the visualization.

The limit is configurable via `--warnings-limit` parameter to `run`.

Closes #6283.
2023-05-10 11:48:31 +00:00
GregoryTravis
7119811e7f
Add tests for Date.until, Date.next and Date.previous. (#6606) 2023-05-09 18:09:30 +00:00
Radosław Waśko
d8b926922a
Improve Non_Unique_Primary_Key error, split file format detection into read/write, improve SQLite format detection (#6604)
Closes #6437
Related to #6410

- Add example duplicate row to `Non_Unique_Primary_Key`.
- Ensure `File.read` fails if the file does not exist, always.
- Ensure SQLite fails if file is empty or nonexistent or malformed.
- Split file format detection into read and write modes, so that the read mode can depend on actual file _contents_.
2023-05-09 17:15:44 +00:00
GregoryTravis
d6f7cea923
tokenize_to_columns or parse_to_columns results in a single column we shouldn't add the 1 (#6607)
When generating new column names in tokenize/split_to_columns, if there's only one new column, use the deleted input column name and don't add disambiguating integers after it.
2023-05-09 15:56:33 +00:00
GregoryTravis
4ba8409def
Add format to the in-memory Column (#6538)
Add format to the in-memory Column

# Important Notes
Also updates .format in date types.
Some rearrangement of date formatting builtins / Java libraries.
2023-05-09 08:47:40 +00:00
James Dunkerley
bc0db18a6e
Small changes from Book Club issues (#6533)
- Add dropdown to tokenize and split `column`.
- Remove the custom `Join_Kind` dropdown.
- Adjust split and tokenize names to start numbering from 1, not 0.
- Add JS_Object serialization for Period.
- Add `days_until` and `until` to `Date`.
- Add `Date_Period.Day` and create `next` and `previous` on `Date`.
- Use simple names with `File_Format` dropdown.
- Avoid using `Main.enso` based imports in `Standard.Base.Data.Map` and `Standard.Base.Data.Text.Helpers`.
- Remove an incorrect import from `Standard.Database.Data.Table`.

From #6587:

A few small changes, lots of lines because this affected lots of tests:
- `Table.join` now defaults to `Join_Kind.Left_Outer`, to avoid losing rows in the left table unexpectedly. If the user really wants to have an Inner join, they can switch to it.
- `Table.join` now defaults to joining columns by name not by index - it looks in the right table for a column with the same name as the first column in left table.
- Missing Input Column errors now specify which table they refer to in the join.
- The unique name suffix in column renaming / default column names when loading from file is now a space instead of underscore.
2023-05-06 10:10:24 +00:00
GregoryTravis
f7282b7cff
"at_least_one" flag for tokenize_to_rows (#6539)
Add "at_least_one" flag for tokenize_to_rows.
2023-05-06 06:22:09 +00:00
Radosław Waśko
0be572af6a
Only Join_Kind.Inner removes the common-named columns (#6564)
As requested by @jdunkerley
2023-05-05 08:12:07 +00:00
Radosław Waśko
41a8257e8d
Separating Redshift connector from Database library into a new AWS library (#6550)
Related to #5777
2023-05-04 17:36:51 +00:00
James Dunkerley
bb8f910be9
Continued Execution Context work and some little fixes (#6506)
- Add `with_disabled` shortcut for `Context`.
- Protect `Image.write` behind `Context.Output`.
- Correct text on the `Forbidden_Operation` error message.
- Remove context overrides from tests.
- Add `File` operations tests with `Context.Output` disabled.
- Add tests for `Text.write` operations with `Context.Output` disabled.
- Use a better method to make `File_Format` dropdown widget.
- Fix bug in `Invalid_Format.to_display_text`.
2023-05-03 08:01:16 +00:00
Pavel Marek
3a42d0ce76
Reimplement enso_project as a proper builtin (#6352)
Remove the magical code generation of `enso_project` method from codegen phase and reimplement it as a proper builtin method.

The old behavior of `enso_project` was special, and violated the language semantics (regarding the `self` argument):
- It was implicitly declared in every module, so it could be called without a self argument.
- It can be called with explicit module as self argument, e.g. `Base.enso_project`, or `Visualizations.enso_project`.

Let's avoid implicit methods on modules and let's be explicit. Let's reimplement the `enso_project` as a builtin method. To comply with the language semantics, we will have to change the signature a bit:
- `enso_project` is a static method in the `Standard.Base.Meta.Enso_Project` module.
- It takes an optional `project` argument (instead of taking it as an explicit self argument).

Having the `enso_project` defined as a (shadowed) builtin method, we will automatically have suggestions created for it.

# Important Notes
- Truffle nodes are no longer generated in codegen phase for the `enso_project` method. It is a standard builtin now.
- The minimal import to use `enso_project` is now `from Standard.Base.Meta.Enso_Project import enso_project`.
- Tested implicitly by `org.enso.compiler.ExecCompilerTest#testInvalidEnsoProjectRef`.
2023-05-02 16:40:58 +00:00
Radosław Waśko
5eb9c3a843
Implement create_database_table for Database Table (#6467)
Closes #6327
2023-05-02 14:01:44 +00:00
Hubert Plociniczak
7e33300013
Make execution mode live default for CLI (#6496)
As requested in the design doc and in the ticket.

Closes #6495.
2023-05-02 09:58:41 +00:00
James Dunkerley
6b0c682b08
Add Execution Context control to Text.write (#6459)
- Adjusted `Context.is_enabled` to support default argument (moved built in so can have defaults).
- Made `environment` case-insensitive.
- Bug fix for play button.
- Short hand to execute within an enabled context.
- Forbid file writing if the Output context is disabled with a `Forbidden_Operation` error.
- Add temporary file support via `File.create_temporary_file` which is deleted on exit of JVM.
- Execution Context first pass in `Text.write`.
- Added dry run warning.
- Writes to a temporary file if disabled.
- Created a `DryRunFileManager` which will create and manage the temporary files.
- Added `format` dropdown to `File.read` and `Data.read`.
- Renamed `JSON_File` to `JSON_Format` to be consistent.

(still to unit test).
2023-04-29 08:39:18 +00:00
Radosław Waśko
cdd0065800
Change defaults for Connection.tables and ensure that Connection.query recognizes all available tables (#6443)
Closes #6398
2023-04-29 00:13:29 +00:00
Hubert Plociniczak
0dc9f3742c
Unbreak CI after outdated merge (#6466) 2023-04-28 14:35:11 +02:00
Hubert Plociniczak
ae3f9025e3
Invoke instance methods for Any overrides (#6441)
This change modifies method dispatch for methods that override Any's definitions. When an overrided method is invoked statically we call Any's method to stay consistent.
This change primarily addresses the plethora of problems related to `to_text` invocations. It does not attempt to completely modify method dispatch logic.

Closes #6300.
2023-04-28 07:18:37 +00:00
Hubert Plociniczak
c6790f1e9c
Report only unique warnings (#6372)
This change makes sure that reported warnings are unique, based on the value of internal clock tick and ignoring differences in reassignments.

Before:
![Screenshot from 2023-04-20 15-42-55](https://user-images.githubusercontent.com/292128/233415710-925c1045-37c7-49f5-9bc3-bfbfd30270a3.png)
After:
![Screenshot from 2023-04-20 15-27-27](https://user-images.githubusercontent.com/292128/233415807-8cb67bc2-ac37-4db7-924e-ae7619074b5b.png)

On the positive side, no further changes, like in LS, have to be done.


Closes #6257.
2023-04-28 07:16:00 +00:00
Radosław Waśko
462016a428
Create database table from memory (#6429) 2023-04-27 20:06:17 +00:00
James Dunkerley
0c7c3bdeaf
Fix for the massive number of warnings when renaming with invalid names. (#6450)
* Rename makeUnique overloads to avoid issue when Nothing is passed.
Suspend warnings when building the output table to avoid mass warning duplication.

* Add test for mixed invalid names.
Adjust so a single warning attached.

* PR comments.
2023-04-27 14:51:59 +01:00
Dmitry Bushev
9a0f739cb0
Visualization preprocessor should not catch errors (#6423)
related #6323

Fixes the following scenario:
- preprocessor function catches `ThreadInterruptedException` and returns it as a JSON
```json
{"error":"org.enso.interpreter.runtime.control.ThreadInterruptedException"}
```
- engine thinks that the visualization was computed successfully and just returns it to the user (without doing any fallback logic like retrying the computation)
- IDE displays blank visualization
2023-04-27 12:42:34 +00:00
James Dunkerley
0e51131809
Table Visualization and display text changes. (#6382)
- Missing tests from number parsing.
- Fix type signature on some warning methods.
- Fix warnings on `Standard.Database.Data.Table.parse_values`.
- Added test for `Nothing` and empty string on `use_first_row_as_names`.
- New API for `Number.format` taking a simple format string and `Locale`.
- Add ellipsis to truncated `Text.to_display_text`.
- Adjusted built-in `to_display_text` for numbers to not include type (but also to display BigInteger as value).
- Remove `Noise.Generator` interface type.
- Json: Added `to_display_text` to `JS_Object`.
- Time: Added `to_display_text` for `Date`, `Time_Of_Day`, `Date_Time`, `Duration` and `Period`.
- Text: Added `to_display_text` to `Locale`, `Case_Sensitivity`, `Encoding`, `Text_Sub_Range`, `Span`, `Utf_16_Span`.
- System: Added `to_display_text` to `File`, `File_Permissions`, `Process_Result` and `Exit_Code`.
- Network: Added `to_display_text` to `URI`, `HTTP_Status_Code` and `Header`.
- Added `to_display_text` to `Maybe`, `Regression`, `Pair`, `Range`, `Filter_Condition`.
- Added support for `to_js_object` and `to_display_text` to `Random_Number_Generator`.
- Verified all error types have `to_display_text`.
- Removed `BigInt`, `Date`, `Date_Time` and `Time_Of_Day` JS based rendering as using `to_display_text` now.
- Added support for rendering nested structures in the table viz.
2023-04-26 18:15:48 +00:00
GregoryTravis
255edda146
Add parse extensions to Text type. #6330 (#6404)
Add type-specific parse stubs to Text, e.g.:
Text.parse_json self -> Json =
Text.parse_url self -> Url =
Text.parse_number self -> Number =
2023-04-26 15:37:09 +00:00
GregoryTravis
dcfbf841b3
Replace Table should_equal with should_equal_verbose (#6405)
Table.should_equal_verbose shows the contents of the tables on failure; let's make this the standard comparison.
2023-04-26 14:01:49 +00:00
GregoryTravis
afd804d529
5127 Add Table.parse_to_columns to parse a single column to a set of columns. (#6383)
Implement Table.parse_to_columns
2023-04-24 15:21:38 +00:00
Radosław Waśko
f3873f9768
Infer SQLite types locally (#6381)
Closes #6208
2023-04-24 10:55:12 +00:00
Radosław Waśko
a43d524336
Add typechecks to Aggregate and Cross Tab (#6380)
Follow up of #6298 as it grew too much. Adds the needed typechecks to aggregate operations. Ensures that the DB operations report `Floating_Point_Equality` warning consistently with in-memory.
2023-04-24 08:55:54 +00:00
GregoryTravis
22f820feb7
Add Table.parse_text_to_table to convert Text to a Table. (#6294) 2023-04-21 17:43:19 +00:00
Radosław Waśko
8db2ad51a1
Adding typechecks to Column Operations (#6298)
Closes #6106
2023-04-21 12:20:12 +00:00
James Dunkerley
0350762386
Add replace, trim to Column. Better number parsing. (#6253)
- Add `replace` with same syntax as on `Text` to an in-memory `Column`.
- Add `trim` with same syntax as on `Text` to an in-memory `Column`.
- Add `trim` to in-database `Column`.
- Added `is_supported` to dialects and exposed the dialect consistently on the `Connection`.
- Add `write_table` support to `JSON_File` allowing `Table.write` to write JSON.
- Updated the parsing for integers and decimals:
- Support for currency symbols.
- Support for brackets for negative numbers.
- Automatic detection of decimal points and thousand separators.
- Tighter rules for scientific and thousand separated numbers.
- Remove `replace_text` from `Table`.
- Remove `write_json` from `Table`.
2023-04-20 16:04:59 +00:00
Pavel Marek
4076a64f33
NaN can be used as a key in Map (#6301)
`Number.nan` can be used as a key in `Map`. This PR basically implements the support for [JavaScript's Same Value Zero Equality](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Equality_comparisons_and_sameness#same-value-zero_equality) so that `Number.nan` can be used as a key in `Map`.

# Important Notes
- For NaN, it holds that `Meta.is_same_object Number.nan Number.nan`, and `Number.nan != Number.nan` - inspired by JS spec.
- `Meta.is_same_object x y` implies `Any.== x y`, except for `Number.nan`.
2023-04-20 09:25:36 +00:00
Hubert Plociniczak
6d3151f32d
UnresolvedSymbol is now accepted by Vector.sort (#6334)
`Vector.sort` does some custom method dispatch logic which always expected a function as `by` and `on` arguments. At the same time, `UnresolvedSymbol` is treated like a (to be resolved) `Function` and under normal circumstances there would be no difference between `_.foo` and `.foo` provided as arguments.

Rather than adding an additional phase that does some form of eta-expansion, to accomodate for this custom dispatch, this change only fixes the problem locally. We accept `Function` and `UnresolvedSymbol` and perform the resolution on the fly. Ideally, we would have a specialization on the latter but again, it would be dependent on the contents of the `Vector` so unclear if that is better.

Closes #6276,

# Important Notes
There was a suggestion to somehow modify our codegen to accomodate for this scenario but I went against it. In fact a lot of name literals have `isMethod` flag and that information is used in the passes but it should not control how (late) codegen is done. If we were to make this more generic, I would suggest maybe to add separate eta-expansion pass. But it could affect other things and could be potentially a significant change with limited potential initially, so potential future work item.
2023-04-20 07:58:58 +00:00
Hubert Plociniczak
f288198b4b
Change order of arguments when controlling context (#6357)
https://github.com/orgs/enso-org/discussions/6344 requested to change the order of arguments when controlling context permissions.

# Important Notes
The change brings it closer to the design doc but IMHO also a bit cumbersome to use (see changed tests) - applications involving default arguments don't play well when the last argument is not the default 🤷 .`
2023-04-19 14:00:27 +00:00
Hubert Plociniczak
4c1e73c864
Don't propagate warnings on suspended arguments (#6345)
* Don't propagate warnings on suspended arguments

In the current implementation, application of arguments with warnings
first extracts warnings, does the application and appends the warnings
to the result.
This process was however too eager if the suspended argument was a
literal (we don't know if it will be executed after all).

The change modifies method processor to take into account the
`@Suspend` annotation and not gather warnings before the application
takes place.

* PR review
2023-04-19 11:55:23 +02:00
Hubert Plociniczak
f720bd2516
Type of UnresolvedSymbol is Function (#6284)
`Meta.is_a` and `Meta.type_of` now recognize UnresolvedSymbol.
Closes #6277.
2023-04-17 13:44:15 +00:00
Jaroslav Tulach
413661b366
Splitting Gigantic EqualsNode into few Smaller Ones (#6280)
Splitting gigantic `EqualsNode` into few smaller ones
2023-04-17 11:29:54 +02:00
Pavel Marek
b42e910280
sort handles incomparable values (#5998)
* Update type ascriptions in some operators in Any

* Add @GenerateUncached to AnyToTextNode.

Will be used in another node with @GenerateUncached.

* Add tests for "sort handles incomparable types"

* Vector.sort handles incomparable types

* Implement sort handling for different comparators

* Comparison operators in Any do not throw Type_Error

* Fix some issues in Ordering_Spec

* Remove the remaining comparison operator overrides for numbers.

* Consolidate all sorting functionality into a single builtin node.

* Fix warnings attachment in sort

* PrimitiveValuesComparator handles other types than primitives

* Fix byFunc calling

* on function can be called from the builtin

* Fix build of native image

* Update changelog

* Add VectorSortTest

* Builtin method should not throw DataflowError.

If yes, the message is discarded (a bug?)

* TypeOfNode may not return only Type

* UnresolvedSymbol is not supported as `on` argument to Vector.sort_builtin

* Fix docs

* Fix bigint spec in LessThanNode

* Small fixes

* Small fixes

* Nothings and Nans are sorted at the end of default comparator group.

But not at the whole end of the resulting vector.

* Fix checking of `by` parameter - now accepts functions with default arguments.

* Fix changelog formatting

* Fix imports in DebuggingEnsoTest

* Remove Array.sort_builtin

* Add comparison operators to micro-distribution

* Remove Array.sort_builtin

* Replace Incomparable_Values by Type_Error in some tests

* Add on_incomparable argument to Vector.sort_builtin

* Fix after merge - Array.sort delegates to Vector.sort

* Add more tests for problem_behavior on Vector.sort

* SortVectorNode throws only Incomparable_Values.

* Delete Collections helper class

* Add test for expected failure for custom incomparable values

* Cosmetics.

* Fix test expecting different comparators warning

* isNothing is checked via interop

* Remove TruffleLogger from SortVectorNode

* Small review refactorings

* Revert "Remove the remaining comparison operator overrides for numbers."

This reverts commit 0df66b1080.

* Improve bench_download.py tool's `--compare` functionality.

- Output table is sorted by benchmark labels.
- Do not fail when there are different benchmark labels in both runs.

* Wrap potential interop values with `HostValueToEnsoNode`

* Use alter function in Vector_Spec

* Update docs

* Invalid comparison throws Incomparable_Values rather than Type_Error

* Number comparison builtin methods return Nothing in case of incomparables
2023-04-16 16:40:12 +02:00
Hubert Plociniczak
e163353d16
Deprecate Error's to_display_text builtin methods (#6275)
The primary motivation for this change was
https://github.com/enso-org/enso/issues/6248, which requested the possibility of defining `to_display_text` methods of common errors via regular method definitions. Until now one could only define them via builtins.

To be able to support that, polyglot invocation had to report `to_display_text` in the list of (invokable) members, which it didn't. Until now, it only considered fields of constructors and builtin methods. That is now fixed as indicated by the change in `Atom`.

Closes #6248.

# Important Notes
Once most of builtins have been translated to regular Enso code, it became apparent how the usage of `.` at the end of the message is not consistent and inflexible. The pure message should never follow with a dot or it makes it impossible to pretty print consistently for the purpose of error reporting. Otherwise we regularly end up with errors ending with `..` or worse. So I went medieval on the reasons for failures and removed all the dots.
The overall result is mostly the same except now we are much more consistent.

Finally, there was a bit of a good reason for using builtins as it simplified our testing.
Take for example `No_Such_Method.Error`. If we do not import `Errors.Common` module we only rely on builtin error types. The type obviously has the constructor but it **does not have** `to_display_text` in scope; the latter is no longer a builtin method but a regular method. This is not really a problem for users who will always import stdlib but our tests often don't. Hence the number of changes and sometimes lack of human-readable errors there.
2023-04-14 19:14:49 +00:00
GregoryTravis
4dcf5faddd
Add split and tokenize to the Table. (#6233)
Implement split and tokenize for tables.
2023-04-14 16:03:02 +00:00
Jaroslav Tulach
a74933d10f
Speed cascade of if statements up (#6255)
Fixes #5709. We have a test and a generic fix that improves inlining of every builtin. Everything seems to be faster.
2023-04-14 13:27:23 +00:00
James Dunkerley
4e92d76665
Add Warning methods onto Any to make working with them easier. (#6176)
Adds `has_warnings`, `remove_warnings` and `throw_on_warning` extension methods.
2023-04-12 15:08:01 +00:00
Jaroslav Tulach
5b3cf6f503
Consolidate Vector and Array methods (#6218)
Fixes #5011 by making sure the same methods that are on `Vector` are also available on `Array`.
2023-04-11 16:20:11 +00:00
Radosław Waśko
0f4f8a0542
Full-joins in SQLite (#6215)
Closes #5254

In #6189 the SQLite version was bumped to a newer release which has builtin support for Full and Right joins, so no workaround is no longer needed.
2023-04-06 16:49:14 +00:00
Hubert Plociniczak
3cce3b3caf
Replace IOContexts with execution env and contexts (#6171)
As per design, IOContexts controlled via type signatures are going away. They are replaced by explicit `Context.if_enabled` runtime checks that will be added to particular method implementations.

`production`/`development` `IOPermissions` are replaced with `live` and `design` execution enviornment. Currently, the `live` env has a hardcoded list of allowed contexts i.e. `Input` and `Output`.

# Important Notes
As per design PR-55. Closes #6129. Closes #6131.
2023-04-06 15:47:40 +00:00
Radosław Waśko
f5db35af07
Adjust {Table|Column}.parse to use Value_Type (#6213)
Closes #5660
2023-04-06 10:58:55 +00:00
Radosław Waśko
83b10a2088
Implement Table.union for Database backend (#6204)
Closes #5235
2023-04-06 08:40:34 +00:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Jaroslav Tulach
741b394b0d
Suspended atom fields are evaluated only once (#6151)
Implements #6134.

# Important Notes
One can define lazy atom fields as:
```haskell
type Lazy
Value ~x ~y
```
the evaluation of the `x` and `y` fields is then delayed until they are needed. The evaluation happens once. Then the computed value is kept in the atom for further use.
2023-04-05 23:46:08 +00:00
Jaroslav Tulach
4805193428
Text.to_display_text is (shortened) identity (#6174)
Fixes #5971.
2023-04-05 19:53:07 +00:00
James Dunkerley
a05cee2571
Add PRIVATE so function hidden from Component Browser and other tidying... (#6207)
Review the set of public functions. Mark many as PRIVATE so not in CB.
Some other small tidying.
2023-04-05 17:54:06 +00:00
GregoryTravis
d9bc5246ba
Remove old (Java) Regex library and replace with new (Truffle) library. (#6195)
Remove old (Java) Regex library and replace with new (Truffle) library.
2023-04-04 19:58:26 +00:00
GregoryTravis
fb77f42fd5
Update Text.split to take a Vector Text parameter (#6156)
Allows you to pass a vector of delimiters to `split`.
2023-04-04 14:44:47 +00:00
Jaroslav Tulach
519df66d39
Always try to resolve conversion for Any type (#6184)
Fixes #5898 by removing `Catch.panic` and speeding the `sieve.enso` benchmark from 1058 ms to 514 ms. Should there be no dedicated conversion, let's use one defined on `Any` type - e.g. defining a conversion `from(Any)` makes such a conversion is always available.
2023-04-04 09:24:35 +00:00
GregoryTravis
6766389cd7
#5123 Text.tokenize (#6150)
Implement Text.tokenize.
2023-04-03 20:13:49 +00:00
James Dunkerley
f26bcf6ab6
Small issues from working with Ned (#6160)
- `Process.run` now returns a `Process_Result` allowing the easy capture of stdout and stderr.
- Joining a column with a column name does not warn if adding just the prefix.
- Stop the table viz from changing case and adding spaces to the headers.
2023-04-03 13:01:42 +00:00
Radosław Waśko
6ddcb553e5
Date/time support for Postgres. Year/month/day operations on Columns. (#6153)
Closes #6115
2023-03-31 18:37:04 +00:00
Radosław Waśko
6f86115498
Proper implementation of Value Types in Table (#6073)
This is the first part of the #5158 umbrella task. It closes #5158, follow-up tasks are listed as a comment in the issue.

- Updates all prototype methods dealing with `Value_Type` with a proper implementation.
- Adds a more precise mapping from in-memory storage to `Value_Type`.
- Adds a dialect-dependent mapping between `SQL_Type` and `Value_Type`.
- Removes obsolete methods and constants on `SQL_Type` that were not portable.
- Ensures that in the Database backend, operation results are computed based on what the Database is meaning to return (by asking the Database about expected types of each operation).
- But also ensures that the result types are sane.
- While SQLite does not officially support a BOOLEAN affinity, we add a set of type overrides to our operations to ensure that Boolean operations will return Boolean values and will not be changed to integers as SQLite would suggest.
- Some methods in SQLite fallback to a NUMERIC affinity unnecessarily, so stuff like `max(text, text)` will keep the `text` type instead of falling back to numeric as SQLite would suggest.
- Adds ability to use custom fetch / builder logic for various types, so that we can support vendor specific types (for example, Postgres dates).

# Important Notes
- There are some TODOs left in the code. I'm still aligning follow-up tasks - once done I will try to add references to relevant tasks in them.
2023-03-31 16:16:18 +00:00
GregoryTravis
c8f5a91d6c
Implement Regular Expression split and update Text.split to the new API (#6116)
Re-implement split on top of Truffle regex.
2023-03-30 13:05:30 +00:00
GregoryTravis
6b9cbeacb2
Implement Regular Expression replace and update Text.replace to the new API (#5959)
Re-implement replace on top of Truffle regex.
2023-03-28 06:13:12 +00:00
James Dunkerley
58f2c7643f
Use new Enso Hash Codes and Comparable (#6060)
Enables `distinct`, `aggregate` and `cross_tab` to use the Enso hashing and equality operations.
Also, I rewired the way the ObjectComparators are obtained in polyglot code to be more consistent.

Add Comparator for `Day_Of_Week`, `Header`, `SQL_Type`, `Image` and `Matrix`.
Also, removed the custom `==` from these types as needed. (Closes #5626)
2023-03-24 15:02:25 +00:00
James Dunkerley
dd009fd1af
Tidy up the public module level statics (#6032)
Tidies up a lot of PUBLIC module statics - marked some as PRIVATE, made some methods of types.
2023-03-22 18:02:37 +00:00
James Dunkerley
546cb0c4ab
Deal with warnings attached to value when making widgets. (#5994)
- Fixes InvokeCallableNode to support warnings.
- Strips warnings from annotations in `get_widget_json`.
- Remove `get_full_annotations_json`.
- Fix warnings on Dialect.
2023-03-22 10:50:19 +00:00
Hubert Plociniczak
8c6fd60aaf
Detect conflicts between exported types and FQNs (#5986)
Exporting types named the same as the module where they are defined in `Main` modules of library components may lead to accidental name conflicts. This became apparent when trying to access `Problem_Behavior` module via a fully qualified name and the compiler rejected it. This is due to the fact that `Main` module exported `Error` type defined in `Standard.Base.Error` module, thus making it impossible to access any other submodules of `Standard.Base.Error` via a fully qualified name.

This change adds a warning to FullyQualifiedNames pass that detects any such future problems.
While only `Error` module was affected, it was widely used in the stdlib, hence the number of changes.

Closes #5902.

# Important Notes
I left out the potential conflict in micro-distribution, thus ensuring we actually detect and report the warning.
2023-03-21 21:09:41 +00:00
Jaroslav Tulach
b46be10f63
Introducing Meta.Type (#5956)
Fixing #5768 and #5765 and co. Introducing `Meta.Type` and giving it the desired methods.

# Important Notes
`Type` is no longer a `Meta.Atom`, but it has a dedicated `Meta.Type` representation.
2023-03-17 20:08:18 +00:00
GregoryTravis
e9600f5eac
Truffle regex: finish group accessors (#5883) 2023-03-15 19:15:09 +00:00
James Dunkerley
919e9474c5
Fix Location, a type signature issue, clean up warnings. (#5953)
Makes Location a normal type.
2023-03-15 18:19:28 +00:00
Jaroslav Tulach
888307f03f
Meta.get_constructor_fields works for Type (#5869)
Fixes #5805 by returning `[]` as list of fields of `Type`.

# Important Notes
`Type` is recognized as `Meta.is_atom` since #3671. However `Type` isn't an `Atom` internally. We have to provide special handling for it where needed.
2023-03-13 09:58:11 +00:00
Radosław Waśko
952beba8d1
Fix cross_tab column naming edge cases, add fill_empty (#5863)
Closes #5151 and adds some additional tests for `cross_tab` that verify duplicated and invalid names.

I decided that for empty or `Nothing` names, instead of replacing them with `Column` and implicitly losing connection with the value that was in the column, we should just error on such values.

To make handling of these easier, `fill_empty` was added allowing to easily replace the empty values with something else.

Also, `{is,fill}_missing` was renamed to `{is,fill}_nothing` to align with `Filter_Condition.Is_Nothing`.
2023-03-11 11:58:54 +00:00
Pavel Marek
5f7a4a5a39
Merge ordered and unordered comparators (#5845)
Merge _ordered_ and _unordered_ comparators into a single one.

# Important Notes
Comparator is now required to have only `compare` method:
```
type Comparator
comapre : T -> T -> (Ordering|Nothing)
hash : T -> Integer
```
2023-03-11 05:43:22 +00:00
James Dunkerley
7887fb8d40
Regex.Compile using Truffle Regex, update find, match and match_all (#5785) 2023-03-10 21:49:50 +00:00
Radosław Waśko
91ef8acf35
Review generated Column names (#5850)
Closes #5583 and closes #5157
2023-03-10 19:07:58 +00:00
Radosław Waśko
62e57f5557
Test some Mismatched Quote edge cases in Delimited reader (#5810)
Follow-up to #5113 - I add some more edge case tests as we discussed with @jdunkerley

When debugging some quoting issues, I also realised the current `Mismatched_Quote` error provided not enough information. So I amended it to at least include some context indicating which was the 'offending' cell.
2023-03-10 15:47:57 +00:00