Commit Graph

121 Commits

Author SHA1 Message Date
Radosław Waśko
2bac9cc844
Execution Context integration for Database write operations (#7072)
Closes #6887
2023-06-27 15:51:21 +00:00
Hubert Plociniczak
4e5cb9cca6
Consistent naming of visualization (#7124) 2023-06-26 17:04:35 +00:00
Jaroslav Tulach
6abcbcb00c
Moving construction of IR into runtime parser project (#7096) 2023-06-24 06:34:21 +02:00
Jaroslav Tulach
5f572d0680
Avoid dependency on BindingsMap from IR (#7055) 2023-06-19 10:28:31 +02:00
Jaroslav Tulach
9307715900
Lock Enso4Igv NPM dependencies (#7053) 2023-06-19 09:48:13 +02:00
James Dunkerley
760fb71798
First part of AWS S3 API, various small fixes. (#6973)
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
2023-06-15 16:20:13 +00:00
Dmitry Bushev
35ef358121
Update sqlite jdbc dependency (#7031)
# Important Notes
Update `sqlite-jdbc`
2023-06-15 11:00:18 +00:00
Dmitry Bushev
48f0c6f5e8
Scala 2.13.11 and libraries update (#7010)
Update Scala and libraries.
2023-06-14 13:15:57 +00:00
Jaroslav Tulach
067ed0cb21
Recognize root of Enso repository as VSCode project (#6942) 2023-06-13 14:23:52 +02:00
Dmitry Bushev
6249c79ffd
Update sbt-java-formatter plugin (#7011)
Update java formatter plugin. The new version can remove unused imports.
2023-06-12 14:18:48 +00:00
Pavel Marek
bd70ed6830
Update GraalVM to 22.3.1 JDK17 (#6750)
Upgrade GraalVM to 22.3.2 based on JDK17.

---------

Co-authored-by: somebody1234 <ehern.lee@gmail.com>
2023-05-24 10:39:08 +02:00
Pavel Marek
2b1e5cd017
Add a compiler pass to analyze non-existing imported symbols (#6726)
Add diagnosis for unresolved symbols in `from ... import sym1, sym2, ...` statements.

- Adds a new compiler pass, `ImportSymbolAnalysis`, that checks these statements and iterates through the symbols and checks if all the symbols can be resolved.
  - Works with `BindingsMap` metadata.
- Add `ImportExportTest` that creates various modules with various imports/exports and checks their generated `BindingMap`.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2023-05-22 10:41:15 +02:00
Radosław Waśko
41a8257e8d
Separating Redshift connector from Database library into a new AWS library (#6550)
Related to #5777
2023-05-04 17:36:51 +00:00
Pavel Marek
4076a64f33
NaN can be used as a key in Map (#6301)
`Number.nan` can be used as a key in `Map`. This PR basically implements the support for [JavaScript's Same Value Zero Equality](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Equality_comparisons_and_sameness#same-value-zero_equality) so that `Number.nan` can be used as a key in `Map`.

# Important Notes
- For NaN, it holds that `Meta.is_same_object Number.nan Number.nan`, and `Number.nan != Number.nan` - inspired by JS spec.
- `Meta.is_same_object x y` implies `Any.== x y`, except for `Number.nan`.
2023-04-20 09:25:36 +00:00
Pavel Marek
b42e910280
sort handles incomparable values (#5998)
* Update type ascriptions in some operators in Any

* Add @GenerateUncached to AnyToTextNode.

Will be used in another node with @GenerateUncached.

* Add tests for "sort handles incomparable types"

* Vector.sort handles incomparable types

* Implement sort handling for different comparators

* Comparison operators in Any do not throw Type_Error

* Fix some issues in Ordering_Spec

* Remove the remaining comparison operator overrides for numbers.

* Consolidate all sorting functionality into a single builtin node.

* Fix warnings attachment in sort

* PrimitiveValuesComparator handles other types than primitives

* Fix byFunc calling

* on function can be called from the builtin

* Fix build of native image

* Update changelog

* Add VectorSortTest

* Builtin method should not throw DataflowError.

If yes, the message is discarded (a bug?)

* TypeOfNode may not return only Type

* UnresolvedSymbol is not supported as `on` argument to Vector.sort_builtin

* Fix docs

* Fix bigint spec in LessThanNode

* Small fixes

* Small fixes

* Nothings and Nans are sorted at the end of default comparator group.

But not at the whole end of the resulting vector.

* Fix checking of `by` parameter - now accepts functions with default arguments.

* Fix changelog formatting

* Fix imports in DebuggingEnsoTest

* Remove Array.sort_builtin

* Add comparison operators to micro-distribution

* Remove Array.sort_builtin

* Replace Incomparable_Values by Type_Error in some tests

* Add on_incomparable argument to Vector.sort_builtin

* Fix after merge - Array.sort delegates to Vector.sort

* Add more tests for problem_behavior on Vector.sort

* SortVectorNode throws only Incomparable_Values.

* Delete Collections helper class

* Add test for expected failure for custom incomparable values

* Cosmetics.

* Fix test expecting different comparators warning

* isNothing is checked via interop

* Remove TruffleLogger from SortVectorNode

* Small review refactorings

* Revert "Remove the remaining comparison operator overrides for numbers."

This reverts commit 0df66b1080.

* Improve bench_download.py tool's `--compare` functionality.

- Output table is sorted by benchmark labels.
- Do not fail when there are different benchmark labels in both runs.

* Wrap potential interop values with `HostValueToEnsoNode`

* Use alter function in Vector_Spec

* Update docs

* Invalid comparison throws Incomparable_Values rather than Type_Error

* Number comparison builtin methods return Nothing in case of incomparables
2023-04-16 16:40:12 +02:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Pavel Marek
c7da5f82c2
Engine benchmark visualization tool can compare two bench runs (#6198)
Add `--compare <bench-run-id-1> <bench-run-id-2>` option to `bench_download.py` script, that prints the difference of benchmark scores for two benchmark runs from GH as a table.

# Important Notes
I tried to add an option to compare two or more branches and visualize them in the graphs, but gave that up after some struggling.
2023-04-05 19:03:50 +00:00
Pavel Marek
3ca0a17d66
Add engine benchmark analysis tool (#5852)
Add Engine benchmark analysis tool - a python script for downloading benchmark data, and Enso project for the analysis. I have also included benchmark data for 02/2022.

Related issues and discussions:
- https://github.com/enso-org/enso/issues/5714
- https://github.com/enso-org/enso/issues/5165
- https://github.com/enso-org/enso/discussions/5718
2023-03-28 15:56:24 +00:00
Dmitry Bushev
aa97787603
Update ZIO library (#6072)
close #6069

Changelog:
- update: ZIO major version bump
2023-03-28 07:58:59 +00:00
Kaz Wesley
e171fba301
New documentation parser (#5917)
Implement new Enso documentation parser; remove old Scala Enso parser.

Performance: Total time parsing documentation is now ~2ms.

# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2023-03-15 15:43:51 +00:00
Jaroslav Tulach
0778c85094
Fixing the node.js dependencies, so the VSCode plugin can be built again (#5738)
Fixing node.js dependencies as:
```bash
enso/tools/enso4igv$ mvn clean install -Pvsix
```
was broken. Making sure the VSIX build is part of the _actions workflow_.

# Important Notes
To reproduce/verify remove `enso/tools/enso4igv/node_modules` first and try to build.
2023-02-28 13:43:26 +00:00
Jaroslav Tulach
f53696eda4
sbt runEngineDistribution ...args... to build and execute the engine (#5609)
Automating the assembly of the engine and its execution into a single task. If you are modifying standard libraries, engine sources or Enso tests, you can launch `sbt` and then just:
```
sbt:enso> runEngineDistribution --run test/Tests/src/Data/Maybe_Spec.enso
[info] Engine package created at built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev
[info] Executing built-distribution/enso-engine-...-dev/bin/enso --run test/Tests/src/Data/Maybe_Spec.enso
Maybe:  [5/5, 30ms]
- should have a None variant [14ms]
- should have a Some variant [5ms]
- should provide the `maybe` function [4ms]
- should provide `is_some` [2ms]
- should provide `is_none` [3ms]
5 tests succeeded.
0 tests failed.unEngineDistribution 4s
0 tests skipped.
```
the [runEngineDistribution](3a581f29ee/docs/CONTRIBUTING.md (running-enso)) `sbt` input task makes sure all your sources are properly compiled and only then executes your enso source. Everything ready at a single press of Enter.

# Important Notes
To debug in chrome dev tools, just add `--inspect`:
```
sbt:enso> runEngineDistribution --inspect --run test/Tests/src/Data/Maybe_Spec.enso
E.g. in Chrome open: devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/7JsgjXlntK8
```
everything gets build and one can just attach the Enso debugger.
2023-02-16 21:37:12 +00:00
Pavel Marek
3e8467c204
Update GraalVM to 22.3.1 (#5602)
Updates the engine to GraalVM 22.3.1 version, which contains fixes for:
- Chrome inspector issues - 38eb3b5932

# Important Notes
- Update to GraalVM 22.3.1
- Remove host object wrapping workaround
2023-02-14 15:51:17 +00:00
Michael Mauderer
349cc210e0
Bump rustc to nightly-2023-01-12 (#4053)
Bump rustc nightly-2022-08-30 and fix new errors and lints.
https://www.pivotaltracker.com/story/show/184229094
2023-02-02 23:05:25 +00:00
Michał Wawrzyniec Urbańczyk
b8a4a70f10
Cloud fixes (#4114)
This PR aims to fix current issues with cloud IDE.

1. Backend image:
  * bumping the system version to avoid glibc version mismatch on parser;
  * explicitly installing required GraalVM components;

2. Frontend upload:
  * update to follow the new file naming;
  * uploading the whole shaders subtree.
2023-02-02 20:34:45 +01:00
Paweł Buchowski
9002df986e
Fix missing /home/ensodev directory (#4103)
The new jgit integration tries to create config file in ~/.config/jgit/config. We are creating default ensodev with explicit `-M, --no-create-home`. As long as there wasn't any strong reasoning behind that this commit changes it to fix problems with initialising jgit.

Error that this PR fixes:

```
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]: [error] [2023-01-30T11:21:47.185Z] [org.eclipse.jgit.util.FS] Cannot save config file 'FileBasedConfig[/home/ensodev/.config/jgit/config]'
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]: java.io.IOException: Some(Creating directories for /home/ensodev/.config/jgit failed)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at org.eclipse.jgit.util.FileUtils.mkdirs(FileUtils.java:413)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at org.eclipse.jgit.internal.storage.file.LockFile.lock(LockFile.java:140)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at org.eclipse.jgit.storage.file.FileBasedConfig.save(FileBasedConfig.java:184)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at org.eclipse.jgit.util.FS$FileStoreAttributes.saveToConfig(FS.java:761)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at org.eclipse.jgit.util.FS$FileStoreAttributes.lambda$5(FS.java:443)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Jan 30 11:21:47 ip-172-31-0-83.eu-west-1.compute.internal enso_runtime[3853]:   at java.base/java.lang.Thread.run(Thread.java:829)
```
2023-01-31 10:39:20 +00:00
Jaroslav Tulach
f54464881f
enso4igv can open engine/language-server & co. projects (#4098)
In order to investigate `engine/language-server` project, I need to be able to open its sources in IGV and NetBeans.

# Important Notes
By adding same Java source (this time `package-info.java`) and compiling with our Frgaal compiler the necessary `.enso-sources*` files are generated for `engine/language-server` and then the `enso4igv` plugin can open them and properly understand their compile settings.

![Logical View of language-server project](https://user-images.githubusercontent.com/26887752/215472696-ec9801f3-4692-4bdb-be92-c4d2ab552e60.png)

In addition to that this PR enhances the _"logical view"_ presentation of the project by including all source roots found under `src/*/*`.
2023-01-31 08:40:04 +00:00
Wojciech Daniło
ce5b078130
Dependency cleaning (#4092) 2023-01-27 23:39:37 +01:00
Wojciech Daniło
da84e34b9a
Shaders precompilation (#4003) 2023-01-27 01:09:09 +01:00
Jaroslav Tulach
0465aa3080
Basic VSCode support for Enso language and development (#4014)
Basic VSCode support for Enso language and development

# Important Notes
See the [building instructions](https://github.com/enso-org/enso/pull/4014/files#diff-1f944b52bce988a17f27b2cdd35e0efe16a1df8b86ee6ced94c0a286033f1ab3R154).
2023-01-06 14:18:20 +00:00
Jaroslav Tulach
c8179d8a9b
Basic IGV Scala Support (#4015)
Many engine sources are written in Scala. IGV doesn't have any support for Scala by default. This PR adds syntax coloring and debugging support for our `.scala` files.
2023-01-02 08:34:52 +00:00
Jaroslav Tulach
0041b649eb
IGV can jump to JMH sources & more (#4008)
Improvements to behavior and visual appearance of IGV Enso integration.
2022-12-30 05:30:32 +00:00
Dmitry Bushev
bc9211434b
Update GraalVM Docker image version (#3904)
The image versioning has changed and  now includes `os_version` prefix https://github.com/graalvm/container/pkgs/container/graalvm-ce

```
ghcr.io/graalvm/$IMAGE_NAME[:][$os_version][-$java_version][-$version][-$build_number]
```
2022-11-24 08:00:11 +00:00
Jaroslav Tulach
402ebb2f8e
Upgrade to GraalVM 22.3.0 (#3663)
Upgrading to GraalVM 22.3.0.

# Important Notes
- Removed all deprecated `FrameSlot`, and replaced them with frame indexes - integers.
- Add more information to `AliasAnalysis` so that it also gathers these indexes.
- Add quick build mode option to `native-image` as default for non-release builds
- `graaljs` and `native-image` should now be downloaded via `gu` automatically, as dependencies.
- Remove `engine-runner-native` project - native image is now build straight from `engine-runner`.
- We used to have `engine-runner-native` without `sqldf` in classpath as a workaround for an internal native image bug.
- Fixed chrome inspector integration, such that it shows values of local variables both for current stack frame and caller stack frames.
- There are still many issues with the debugging in general, for example, when there is a polyglot value among local variables, a `NullPointerException` is thrown and no values are displayed.
- Removed some deprecated `native-image` options
- Remove some deprecated Truffle API method calls.
2022-11-23 14:30:48 +00:00
Hubert Plociniczak
2a38edecd4
Simple HTTPBin to replace Go equivalent (#3887)
1-to-1 translation of the HTTPBin expected by our testsuite using Java's HttpServer.
Can be started from SBT via
```
sbt:enso> simple-httpbin/run <hostname> <port>
```

# Important Notes
@mwu-tow this will mean we can ditch Go dependency completely and replace it with the above call.
2022-11-18 11:27:27 +00:00
Jaroslav Tulach
9cfd6aae26
Proper classpath of engine sources in Enso4Igv plugin (#3810)
This PR modifies `sbt` to record options sent to [frgaal](http://frgaal.org) compiler. Then it reads these options (especially exact classpath used during compilation) from IGV or NetBeans.

# Important Notes
![Open project in IGV](https://user-images.githubusercontent.com/26887752/201684275-b3ee7a37-7b55-4290-b426-75df0280ba32.png)
2022-11-15 07:05:53 +00:00
Hubert Plociniczak
85d4337f26
Project save backed by git (#3851)
This change adds support for Version Controlled projects in language server.
Version Control supports operations:
- `init` - initialize VCS for a project
- `save` - commit all changes to the project in VCS
- `restore` - ability to restore project to some past `save`
- `status` - show the status of the project from VCS' perspective
- `list` - show a list of requested saves

# Important Notes
Behind the scenes, Enso's VCS uses git (or rather [jGit](https://www.eclipse.org/jgit/)) but nothing stops us from using a different implementation as long as it conforms to the establish API.
2022-11-14 17:32:39 +00:00
James Dunkerley
45276b243d
Expanding Derived Columns and Expression Syntax (#3782)
- Added expression ANTLR4 grammar and sbt based build.
- Added expression support to `set` and `filter` on the Database and InMemory `Table`.
- Added expression support to `aggregate` on the Database and InMemory `Table`.
- Removed old aggregate functions (`sum`, `max`, `min` and `mean`) from `Column` types.
- Adjusted database `Column` `+` operator to do concatenation (`||`) when text types.
- Added power operator `^` to both `Column` types.
- Adjust `iif` to allow for columns to be passed for `when_true` and `when_false` parameters.
- Added `is_present` to database `Column` type.
- Added `coalesce`, `min` and `max` functions to both `Column` types performing row based operation.
- Added support for `Date`, `Time_Of_Day` and `Date_Time` constants in database.
- Added `read` method to InMemory `Column` returning `self` (or a slice).

# Important Notes
- Moved approximate type computation to `SQL_Type`.
- Fixed issue in `LongNumericOp` where it was always casting to a double.
- Removed `head` from InMemory Table (still has `first` method).
2022-11-08 15:57:59 +00:00
James Dunkerley
f0f6deef2a
Load the File_Format types via a ServiceLoader (#3813)
Moves the File.read method into the `File` type.
Uses the ServiceLoader to find all types for the File_Format.
2022-10-24 09:55:18 +00:00
Michał Wawrzyniec Urbańczyk
ad69eeb4ad
Build script merge (#3743)
Merged the build script into main repository. Some related cleanups.
2022-10-10 23:38:48 +02:00
Wojciech Daniło
61546a7ade
Wip/wdanilo/widgets 182746060 (#3678) 2022-10-04 04:51:27 +02:00
Radosław Waśko
551100af3b
Add Table.distinct function to In-Memory table (#3684)
Implements https://www.pivotaltracker.com/story/show/182307143

# Important Notes
- Modified standard library Java helpers dependencies so that `std-table` module depends on `std-base`, as a provided dependency. This is allowed, because `std-table` is used by the `Standard.Table` Enso module which depends on `Standard.Base` which ensures that the `std-base` is loaded onto the classpath, thus whenever `std-table` is loaded by `Standard.Table`, so is `std-base`. Thus we can rely on classes from `std-base` and its dependencies being _provided_ on the classpath. Thanks to that we can use utilities like `Text_Utils` also in `std-table`, avoiding code duplication. Additional advantage of that is that we don't need to specify ICU4J as a separate dependency for `std-table`, since it is 'taken' from `std-base` already - so we avoid including it in our build packages twice.
2022-09-07 12:28:41 +00:00
Dmitry Bushev
c2a5db0063
Update apply-edit benchmark to use lazy visualization (#3642)
This PR adds an ability to check that new API works. There is a separate task to integrate it in IDE
2022-08-15 07:11:51 +00:00
Hubert Plociniczak
8575b76b0a
Support pattern matching on constants (#3641)
This change adds support for matching on constants by:
1) extending parser to allow literals in patterns
2) generate branch node for literals

Related to https://www.pivotaltracker.com/story/show/182743559
2022-08-12 13:18:58 +00:00
Dmitry Bushev
5e114acbb5
Update Scala to 2.13.8 (#3631)
Update Scala compiler and libraries.
2022-08-08 19:32:55 +00:00
Dmitry Bushev
0701e762ea
Add Language Server benchmarking tool (#3578) 2022-07-22 14:12:52 +00:00
James Dunkerley
2527a7bdb2
Update SQLite, PostgreSQL and Redshift drivers (#3571)
Updated the SQLite, PostgreSQL and Redshift drivers.

# Important Notes
Updated the API for Redshift and proved able to connect without the ini file workaround.
2022-07-11 18:39:16 +00:00
Hubert Plociniczak
22fc03ca3d
Fix here leftovers (#3567)
Hooking up Reporting_Stream_Encoder_Spec reveals missing renaming.
Also removed one more `here.` that wasn't run in tests.

Kudos to @radeusgd for finding those!
2022-07-07 12:02:18 +00:00
Michał Wawrzyniec Urbańczyk
43a893cae6
Bump the build script (#3535) 2022-07-01 03:58:14 +02:00
Jaroslav Tulach
479f74cc84
Enso Integration with Ideal Graph Visualizer (#3533)
This PR adds sources for Enso language support in IGV (and NetBeans). The support is based on TextMate grammar shown in the editor and registration of the Enso language so IGV can find it. Then this PR adds new GitHub Actions workflow file to build the project using Maven.
2022-06-23 04:43:49 +00:00