Commit Graph

496 Commits

Author SHA1 Message Date
James Dunkerley
760fb71798
First part of AWS S3 API, various small fixes. (#6973)
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
2023-06-15 16:20:13 +00:00
Radosław Waśko
dad57e6c7d
Implement remaining Update_Actions for update_database_table. (#7035)
Closes #6498
2023-06-15 08:48:22 +00:00
Pavel Marek
67821bf8df
Add compiler pass that discovers ambiguous imports (#6868)
Add a new compiler pass that analyses duplicated and ambiguous symbols from imports
2023-06-14 12:18:57 +02:00
Radosław Waśko
d9ed63fb89
Implement Insert update action for update_database_table. (#6990)
This adds the spec for all update actions, but implements the common input validation framework and `Insert`. Tests for remaining actions are marked as pending - these will be implemented in a subsequent PR.
2023-06-14 00:14:32 +00:00
Dmitry Bushev
6249c79ffd
Update sbt-java-formatter plugin (#7011)
Update java formatter plugin. The new version can remove unused imports.
2023-06-12 14:18:48 +00:00
GregoryTravis
fcc57e44e7
Implement addition and subtraction for Date_Period and Time_Period (#6956)
Date.+ should allow Date_Period, Time_Of_Day.+ should allow Time_Period and Date_TIme.+ should allow both.
Same for subtraction.
2023-06-07 12:46:19 +00:00
James Dunkerley
578ba59f1d
Use US Locale for Date and Time parsing and formatting (#6967)
Sorts out parsing and printing long form names of months and weekdays.
2023-06-06 21:44:25 +00:00
Radosław Waśko
1931e9e51f
Workaround for to_date_time type errors (#6964)
Related to #6912

It essentially solves it by removing any builtins that would take an EnsoDate/EnsoTimeOfDay/EnsoTimeZone and replacing them with Java utils that do the same operation.

This is not a proper solution - the builtin conversion is still invalid for the date/time types - but at this moment we may just no longer use the invalid conversion so it is much less of an issue. We still need to be aware of this if we want to introduce builtins taking date/time in the future.
2023-06-06 20:28:11 +00:00
GregoryTravis
912fbce97b
Reimplement Column.truncate, .ceil, and .floor as vectorized Java ops (#6941)
Reimplement these in Java.

Benchmarks:

Before:

Column.truncate floats average: 124.4ms
Column.ceil floats average: 121.47ms
Column.floor floats average: 120.18ms
Column.truncate ints average: 124.78ms
Column.ceil ints average: 120.41ms
Column.floor ints average: 102.35ms

After (boxed):

Column.truncate floats average: 3.75ms
Column.ceil floats average: 2.25ms
Column.floor floats average: 1.89ms
Column.truncate ints average: 2ms
Column.ceil ints average: 1.77ms
Column.floor ints average: 1.74ms

After (unboxed):
Column.truncate floats average: 3.32ms
Column.ceil floats average: 2.15ms
Column.floor floats average: 1.69ms
Column.truncate ints average: 1.74ms
Column.ceil ints average: 1.61ms
Column.floor ints average: 1.99ms
2023-06-06 18:07:12 +00:00
Radosław Waśko
b513839418
Refactor create_database_table into Connection.create_table and select_into_database_table, implement Set. (#6925)
First part for #6498 - refactoring of the upload infrastructure, in preparation for `update_database_table`.

Implemented a `Set` data structure which was long needed.

The APIs are added and an initial implementation is created, but it is not complete - but it has grown significantly already so the remaining implementation will be done as a separate PR.

Adds some basic ability for a function to ensure that it is only executed from within a transaction.
2023-06-06 10:36:05 +00:00
Jaroslav Tulach
f1bdcbb534
Documenting how to obtain static methods for a type (#6938)
Fixes #6748 by documenting how to obtain all static methods available on a `Type`.
2023-06-06 08:06:10 +00:00
GregoryTravis
3ffbe9cecf
Handle some edge cases in rounding (inexact representations and overflows) (#6922) 2023-06-05 17:21:13 +00:00
James Dunkerley
db96bd2e2c
Small fixes from book club. (#6933)
- Add the missing dropdowns for `Locale` and `Encoding`.
- Correct a few mismatched type signatures.
- Adjust `order_by` calls with a single `Sort_Column` to call in a Vector.
- Adjust parameter names for `transpose`.
- Fix for the table viz: escape HTML and `suppressFieldDotNotation`.
- Use `Filter_Condition.Equal True` for the default filter.
- Adjust `Data.fetch` to return the response on success when parse fails. Rename `parse` to `try_auto_parse`.
- Add various aliases for methods.
- Add tests for `Table.set` when using a `Vector`, `Range` or `Date_Range`.
- Add check for mismatched length on `Table.set`.

![image](https://github.com/enso-org/enso/assets/4699705/23ea0ba3-2b05-4af8-afd9-f35b55446c24)

![image](https://github.com/enso-org/enso/assets/4699705/8b0253e6-e9e8-490a-9607-0da51ab5a215)
2023-06-05 13:57:30 +00:00
Hubert Plociniczak
e9a92a1fb5
Throw panic on "no currying for conversions" (#6940)
Previously, a `RuntimeException` would be thrown when an attempt would be made to curry a conversion function. That is problematic for IDE where `executionFailed` means we can't enter functions due to lack of method pointers info.

Closes #6897.

![Screenshot from 2023-06-02 20-31-03](https://github.com/enso-org/enso/assets/292128/a6c77544-2c47-425c-8ce0-982d837dda5b)

# Important Notes
A more generic solution that allows to recover from execution failures will need a follow up.
2023-06-05 13:16:12 +00:00
Radosław Waśko
cfb2f2916e
Merge Column_Indexes_Out_Of_Range into Missing_Input_Columns. (#6901)
Implements #6869
2023-06-02 12:09:20 +00:00
Radosław Waśko
d44b1250b7
Implement Table.add_row_number (#6890)
Closes #5227

# Important Notes
- This lays first steps towards #6292 - we get pure Enso variants of MultiValueKey.
- Another part refactors `LongStorage` into `AbstractLongStorage` allowing it to provide alternative implementations of the underlying storage, in our case `LongRangeStorage` generating the values ad-hoc and `LongConstantStorage` - currently unused but in the future it can be adapted to support constant columns (once we implement similar facilities for other types).
2023-06-02 10:13:13 +00:00
James Dunkerley
343b5fb085
Execution control for Table.write and various widget tweaks... (#6835)
- Adds execution control to `Table.write`.
- Refactored the `Text.write` to make part reusable.
- Tidied up some legacy mess in tests.
- Add easier flow to go from `Text` to an `URI` to fetching data.
- Add decode functions to `Response` and `Response_Body`.
- Fix issue with 0 length regex matches (using same as Python and .Net approach).
- Add various ALIAS entries to make function discovery easier.
- Sort a lot of drop down and vector editors out (including switch to fully qualified names).
2023-06-01 22:10:03 +00:00
GregoryTravis
0337180384
Add rounding functions to the Column type (#6817) 2023-06-01 20:06:23 +00:00
GregoryTravis
7e53cd9af1
Add drop down for Locale like Encoding (#6654)
Add dropdowns for locale parameters for format and parse methods.
2023-05-31 12:43:20 +00:00
Jaroslav Tulach
c6cb937c0a
Runtime check of ascribed types (#6790) 2023-05-30 11:07:26 +02:00
Hubert Plociniczak
6eb4737330
Append warnings extracted before tail call execution (#6849)
Throwing `TailCallException` meant that exceptions that were extracted from the expression before the call was made could not be appended. This change catches the `TailCallException`, adds warnings to it and propagates it further, thus ensuring that we don't loose the information.

Closes #6765.

# Important Notes
Removed workarounds introduced in stdlib.
2023-05-29 12:44:15 +00:00
Jaroslav Tulach
e7ee2caa2e
Infrastructure for testing inter project imports and exports (#6840) 2023-05-26 17:33:54 +02:00
Radosław Waśko
c3e771c75c
Allow casting a Mixed column into a concrete type (#6777)
Follow-up of #6711

Closes #6838
2023-05-26 13:25:53 +00:00
Jaroslav Tulach
792cbc448c
Clearly select single specialization with enum dispatch pattern (#6819) 2023-05-24 20:42:24 +02:00
Pavel Marek
bd70ed6830
Update GraalVM to 22.3.1 JDK17 (#6750)
Upgrade GraalVM to 22.3.2 based on JDK17.

---------

Co-authored-by: somebody1234 <ehern.lee@gmail.com>
2023-05-24 10:39:08 +02:00
Pavel Marek
2b1e5cd017
Add a compiler pass to analyze non-existing imported symbols (#6726)
Add diagnosis for unresolved symbols in `from ... import sym1, sym2, ...` statements.

- Adds a new compiler pass, `ImportSymbolAnalysis`, that checks these statements and iterates through the symbols and checks if all the symbols can be resolved.
  - Works with `BindingsMap` metadata.
- Add `ImportExportTest` that creates various modules with various imports/exports and checks their generated `BindingMap`.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2023-05-22 10:41:15 +02:00
James Dunkerley
62fecfa474
Widgets, Vector as Column, Viz Fixes and Rename Columns (#6768)
- Fix couple of bugs in Table viz: rounding of bottom div, missing character, not including row count as an option.
- Add better JSON format for `Row`, add support for visualization in the Table viz both for `Vector Row` or `Row`.
- Fix some type signature errors.
- Move `Column_Format` to `Standard.Table.Internal`.
- Move `format_widget` to `File_Format.default_widget` and sort the signature of `Widget` methods.
- Added utility to make `Single_Choice` widgets.
- Added dropdown for delimiter on split methods.
- Removed `default_widget` from `Problem_Behavior` and `Filter_Condition`.
- Altered signature and widgets for table functions.
- Added `to_column` extension to allow easy conversion of Range and Vector to Column.
- Added `compute`, `compute_bulk`, `running` to Column to allow statistic computation.
- Added drop down for `Table.write` format parameter.
- Added drop down for `Table.rename_columns`.
- Added support for Vector of pairs for renaming columns.
- Added check when making a map from Vector if not 2 items.

![image](https://github.com/enso-org/enso/assets/4699705/beed257c-efe3-44a3-9e3a-041354701735)
2023-05-19 23:24:47 +00:00
Radosław Waśko
a9a464af37
Implement simple variants of parse for the Database backend (#6731)
Implements the simplest `parse` scenarios for the Database backend.

Before #6711 these could have been done by `cast`, but in #6711 the APIs were unified to only allow casting to the same set of types in both in-memory and Database. Converting Text to other types is supposed to be done by `parse` and not `cast`, so the ability to use `cast` for rudimentary parsing is removed in the Database backend to make it consistent with in-memory. But now it is lacking any, even simplest, Text->Int/Text->Date support. To alleviate that, the simple scenarios for `parse` are implemented (no support for format customization yet, will boil down to a cast under the hood).
2023-05-19 22:11:23 +00:00
GregoryTravis
4f71673718
Decimal/Integer .round and .int #6654 (#6743) 2023-05-19 19:23:17 +00:00
Radosław Waśko
447786a304
Implement cast for Table and Column (#6711)
Closes #6112
2023-05-19 10:00:20 +00:00
Jaroslav Tulach
8c370008bc
Meta.meta Integer . methods (#6740) 2023-05-18 11:40:44 +02:00
Dmitry Bushev
706791779b
SuggestionBuilder needs to send ascribedType of constructor parameters (#6655)
close #6611

Changelog:
- update: run compiler passes on the `ascribedType` field of the constructor arguments
- update: suggestion builder uses the type information attached to `ascribedType`
- feat: resolve qualified names in type signatures
2023-05-13 18:33:03 +00:00
Radosław Waśko
f5071a17fd
Improving widgets for take/drop (#6641)
Related to #6410

# Important Notes
- Updated some `Meta` methods (needed for error handling):
- `Meta.Type` now has `name` and `qualified_name`.
- `Meta.Constructor` has `declaring_type` allowing to get the type that this constructor is associated with.
2023-05-12 15:33:15 +00:00
Radosław Waśko
cd7fb73232
Add Date_Range (#6621)
Closes #6543
2023-05-11 16:03:02 +00:00
Jaroslav Tulach
1302b693d8
All Vector operations shall be applicable on java.util.ArrayList (#6642)
Fixes #6609 by
- e380e647af - running whole `Vector_Spec` on `java.util.ArrayList`
- 9b1229fe20 - introducing a node to handle interop values

# Important Notes
Contains additional DSL processor fix:
- 415623dcb9 - to not crash the compiler, but to properly report compiler error
2023-05-11 15:29:47 +00:00
Jaroslav Tulach
4e7191af46
Make Meta.get_annotation work for constructor (#6633)
Fixes #6612 by searching for the atom constructor on the `target` itself, not its type.
2023-05-10 15:42:28 +00:00
Hubert Plociniczak
cf3624c463
Limit the number of reported warnings (#6577)
Artifically limiting the number of reported warnings to 100. Also added benchmarks with random Ints to investigate perf issues when dealing with warnings (future task).
Ideally we would have a custom set-like collection that allows us internally to specify a maximal number of elements. But `EnsoHashMap` (and potentially `EnsoSet`) are still WIP when it comes to being PE-friendly.

The change also allows for checking if the limit for the number of reported warnings has been reached. It will visualize by adding an additional "Warnings limit reached." to the visualization.

The limit is configurable via `--warnings-limit` parameter to `run`.

Closes #6283.
2023-05-10 11:48:31 +00:00
GregoryTravis
7119811e7f
Add tests for Date.until, Date.next and Date.previous. (#6606) 2023-05-09 18:09:30 +00:00
Radosław Waśko
d8b926922a
Improve Non_Unique_Primary_Key error, split file format detection into read/write, improve SQLite format detection (#6604)
Closes #6437
Related to #6410

- Add example duplicate row to `Non_Unique_Primary_Key`.
- Ensure `File.read` fails if the file does not exist, always.
- Ensure SQLite fails if file is empty or nonexistent or malformed.
- Split file format detection into read and write modes, so that the read mode can depend on actual file _contents_.
2023-05-09 17:15:44 +00:00
GregoryTravis
d6f7cea923
tokenize_to_columns or parse_to_columns results in a single column we shouldn't add the 1 (#6607)
When generating new column names in tokenize/split_to_columns, if there's only one new column, use the deleted input column name and don't add disambiguating integers after it.
2023-05-09 15:56:33 +00:00
GregoryTravis
4ba8409def
Add format to the in-memory Column (#6538)
Add format to the in-memory Column

# Important Notes
Also updates .format in date types.
Some rearrangement of date formatting builtins / Java libraries.
2023-05-09 08:47:40 +00:00
James Dunkerley
bc0db18a6e
Small changes from Book Club issues (#6533)
- Add dropdown to tokenize and split `column`.
- Remove the custom `Join_Kind` dropdown.
- Adjust split and tokenize names to start numbering from 1, not 0.
- Add JS_Object serialization for Period.
- Add `days_until` and `until` to `Date`.
- Add `Date_Period.Day` and create `next` and `previous` on `Date`.
- Use simple names with `File_Format` dropdown.
- Avoid using `Main.enso` based imports in `Standard.Base.Data.Map` and `Standard.Base.Data.Text.Helpers`.
- Remove an incorrect import from `Standard.Database.Data.Table`.

From #6587:

A few small changes, lots of lines because this affected lots of tests:
- `Table.join` now defaults to `Join_Kind.Left_Outer`, to avoid losing rows in the left table unexpectedly. If the user really wants to have an Inner join, they can switch to it.
- `Table.join` now defaults to joining columns by name not by index - it looks in the right table for a column with the same name as the first column in left table.
- Missing Input Column errors now specify which table they refer to in the join.
- The unique name suffix in column renaming / default column names when loading from file is now a space instead of underscore.
2023-05-06 10:10:24 +00:00
GregoryTravis
f7282b7cff
"at_least_one" flag for tokenize_to_rows (#6539)
Add "at_least_one" flag for tokenize_to_rows.
2023-05-06 06:22:09 +00:00
Radosław Waśko
0be572af6a
Only Join_Kind.Inner removes the common-named columns (#6564)
As requested by @jdunkerley
2023-05-05 08:12:07 +00:00
Radosław Waśko
41a8257e8d
Separating Redshift connector from Database library into a new AWS library (#6550)
Related to #5777
2023-05-04 17:36:51 +00:00
James Dunkerley
bb8f910be9
Continued Execution Context work and some little fixes (#6506)
- Add `with_disabled` shortcut for `Context`.
- Protect `Image.write` behind `Context.Output`.
- Correct text on the `Forbidden_Operation` error message.
- Remove context overrides from tests.
- Add `File` operations tests with `Context.Output` disabled.
- Add tests for `Text.write` operations with `Context.Output` disabled.
- Use a better method to make `File_Format` dropdown widget.
- Fix bug in `Invalid_Format.to_display_text`.
2023-05-03 08:01:16 +00:00
Pavel Marek
3a42d0ce76
Reimplement enso_project as a proper builtin (#6352)
Remove the magical code generation of `enso_project` method from codegen phase and reimplement it as a proper builtin method.

The old behavior of `enso_project` was special, and violated the language semantics (regarding the `self` argument):
- It was implicitly declared in every module, so it could be called without a self argument.
- It can be called with explicit module as self argument, e.g. `Base.enso_project`, or `Visualizations.enso_project`.

Let's avoid implicit methods on modules and let's be explicit. Let's reimplement the `enso_project` as a builtin method. To comply with the language semantics, we will have to change the signature a bit:
- `enso_project` is a static method in the `Standard.Base.Meta.Enso_Project` module.
- It takes an optional `project` argument (instead of taking it as an explicit self argument).

Having the `enso_project` defined as a (shadowed) builtin method, we will automatically have suggestions created for it.

# Important Notes
- Truffle nodes are no longer generated in codegen phase for the `enso_project` method. It is a standard builtin now.
- The minimal import to use `enso_project` is now `from Standard.Base.Meta.Enso_Project import enso_project`.
- Tested implicitly by `org.enso.compiler.ExecCompilerTest#testInvalidEnsoProjectRef`.
2023-05-02 16:40:58 +00:00
Radosław Waśko
5eb9c3a843
Implement create_database_table for Database Table (#6467)
Closes #6327
2023-05-02 14:01:44 +00:00
Hubert Plociniczak
7e33300013
Make execution mode live default for CLI (#6496)
As requested in the design doc and in the ticket.

Closes #6495.
2023-05-02 09:58:41 +00:00
James Dunkerley
6b0c682b08
Add Execution Context control to Text.write (#6459)
- Adjusted `Context.is_enabled` to support default argument (moved built in so can have defaults).
- Made `environment` case-insensitive.
- Bug fix for play button.
- Short hand to execute within an enabled context.
- Forbid file writing if the Output context is disabled with a `Forbidden_Operation` error.
- Add temporary file support via `File.create_temporary_file` which is deleted on exit of JVM.
- Execution Context first pass in `Text.write`.
- Added dry run warning.
- Writes to a temporary file if disabled.
- Created a `DryRunFileManager` which will create and manage the temporary files.
- Added `format` dropdown to `File.read` and `Data.read`.
- Renamed `JSON_File` to `JSON_Format` to be consistent.

(still to unit test).
2023-04-29 08:39:18 +00:00