Add support for recovering from GL context loss. When the context is restored, the loading spinner is shown until shaders finish recompiling.
[vokoscreenNG-2023-08-25_09-39-11.webm](https://github.com/enso-org/enso/assets/1047859/cfa90ec5-72a1-41e6-bafa-177fa5e85fb2)
*While the context is missing, the loading spinner is rendered in the 0% state. (This condition will not normally be observed, except momentarily, as the browser should restore the context immediately if it is lost while the page is visible.) When we receive a new context, the spinner switches to the 90% state until restoration completes. Restoration is fast, as we don't need to do much work except recompiling shaders.*
# Important Notes
- A new debug hotkey, Ctrl+Alt+Shift+X, causes context loss for testing. Pressing it a second time causes context restoration.
- `Texture` is still a CPU-bound texture. It now uses the "immutable" `texStorage/texSubImage` API, which is a ["preferred alternative"](https://registry.khronos.org/webgl/specs/latest/2.0/#3.7.6) to the `texImage` API because it can be more efficient.
- The type for texture uniforms is now `Uniform<Option<Texture>>`. Texture uniforms are decoupled from the context.
- A new `ContextLost` error type can be returned by functions that cannot complete if the context is lost.
- Fix some crashes that could occur when context was lost.
- Clarify ownership of some rendering-related types: Externalize, and where possible eliminate, `Rc/RefCell`s.
- Closes#7238
- Aligns `update_database_table` to a more consistent and clearer API - `update_rows`.
- Adds a `truncate_table` helper function, to pair up with `drop_table`. Both are `PRIVATE` for now.
- Adds tests for NULLs in keys in `update_rows` and `delete_rows`.
- The behaviour is sometimes unexpected, so instead these fail with `Null_Values_In_Key_Columns`.
- Adds a workaround for https://github.com/oracle/graal/issues/7359
- Adds a workaround for a related bug where a stack frame has no name (its `rootNode.getName() == null`).
- I could not track down this bug to provide a neat repro.
Fixes#7468
The fix is pretty simple: we reuse the existing functionality for importing stuff and generating expressions. It fixes issues with `Nothing` or `Report_Unmatched` types.
https://github.com/enso-org/enso/assets/6566674/4e7addf9-2175-4f2a-a571-4ef823de5cb0
While debugging, I found it easier to work with a suggestion database when exported to some external format. Hence, I implemented serde serialization support for database entries and also a new debug shortcut <kbd>ctrl</kbd>+<kbd>shift</kbd>+<kbd>u</kbd> to dump all entries to the console.
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.
Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.
Most important differences have been outlined in `docs/infrastructure/logging.md`.
# Important Notes
Addresses:
- #7253
- #6739
- Closes#7633
- Moves `Round_Spec.enso` from published `Standard.Test` into our `test/Tests` project; the `Table_Tests` that depend on it, simply `import enso_dev.Tests`.
- Changes the layout of the local libraries directory:
- It used to be `root/<namespace>/<name>`.
- Now it is `root/<dir>` - the namespace and name are now read from `package.yaml` instead.
- Adds the parent directory of the current project to the default `ENSO_LIBRARY_PATH`.
- It is treated as a secondary path, so the default `ENSO_HOME/lib` still takes precedence.
- This allows projects to reference and load 'sibling' projects easily - the only requirement is for the project to enable `prefer-local-libraries: true` or add the other local project to its edition. The edition resolution logic is **not changed**.
- Closes#6111
- Aligns semantics of handling Mixed columns.
- Now, if an operation like `iif` or `fill_nothing` is given a `Mixed` column, the result will also be `Mixed` regardless of the `inferred_precise_value_type`.
- Enables a few old tests that were pending but could be enabled since the types work is advanced enough.
Closes#6261
- Adds support for copy-pasting nodes with `cmd + C` and `cmd + V` shortcuts.
- Only a single, currently selected node will be copied. Adding support for multiple node copies seems easy, though (but was out of the scope of the task).
- We use a custom data format for clipboard content. Node's metadata is also copied, so opened visualizations are preserved. However, the visualization's size is not preserved, as we do not store this info in metadata.
- For custom format to work, we use a pretty new feature called [Clipboard pickling](https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/explainer.md), but it is available in Electron and in most browsers already.
- Pasting plain text from other applications (or from Enso, if the code is copied in edit mode) is supported and is currently enabled. There are some security concerns related to this, though. I will create a separate issue/discussion for that.
- Undo/redo works as you expect.
- New node is pasted at the cursor position.
https://github.com/enso-org/enso/assets/6566674/7a04d941-19f7-4a39-9bce-0e554af50ba3
- Closes#5159
- Now data downloaded from the database can keep the type much closer to the original type (like string length limits or smaller integer types).
- Cast also exposes these types.
- The integers are still all stored as 64-bit Java `long`s, we just check their bounds. Changing underlying storage for memory efficiency may come in the future: #6109
- Fixes#7565
- Fixes#7529 by checking for arithmetic overflow in in-memory integer arithmetic operations that could overflow. Adds a documentation note saying that the behaviour for Database backends is unspecified and depends on particular database.
close#7604
After moving the rename action to the dashboard, IDE is unaware of the new project name. PR implements a new `refactoring/projectRenamed` notification that is sent from the server to clients and informs them about the changed project name.
# Important Notes
https://github.com/enso-org/enso/assets/357683/7c62726d-217e-4e69-8e48-568e0b7b8c34
This was meant to be a trivial change, but actually a dirty workaround needed to be applied. Because enter is used to both open searcher and accept input/entry, both actions were fired at once. I fixed it by debouncing opening searcher event (so the searcher will be opened only once key event handling is over)
Fixes#7213 by reacting to new `isBigInteger` and `asBigInteger` messages. Adjusts tests to require `BigInteger` when appropriate. `EnsoBigInteger` no longer extends `Number`. Enso tests to come.
Fixes#6552Fixes#6910Fixes#6872
Implementation of new node design. Includes many changes related to stylesheet update handling and per-style FRP construction, as well as refactoring of scene layers used by graph editor. Some additional components were migrated to use `Rectangle` shape and new mouse handling events. Fixed text rendering, where random thin lines appeared at the borders of glyph sprites. Refined edge layout to match new node sizes and not leave any visible gaps between line segments.
The node colors are currently randomly selected from predefined list. Later this will be improved to use group information from the suggestion database, once that is fully migrated to use the documentation tags, thus removing the dependency on the execution context.
https://github.com/enso-org/enso/assets/919491/aa687e53-a2fa-4e95-a15f-132c05e6337a
<img width="653" alt="image" src="https://github.com/enso-org/enso/assets/919491/30f3e897-62fc-40ea-b57b-124ac923bafd">
Fixes#7336 in a quick way.
Next to the old way of defining groups, the library can just add `GROUP` tag to some entities, and it will be added to the group specified in tag's description.
The group name may be qualified (with project name, like `Standard.Base.Input/Output`) or just name - in the latter case, IDE will assume a group defined in the same library as the entity.
Also moved some entities from "export" list in package.yaml to GROUP tag to give an example. I didn't move all of those, as I assume the library team will reorganize those groups anyway.
### Important Notes
@jdunkerley @radeusgd @GregoryTravis When you will start specifying groups in tags, remember that:
* The groups still belongs to a concrete project; if some entity outside a project wants to be added to its group, the "qualified" name should be specified. See `Table.new` example in this PR.
* If the group name does not reflect any group in package.yaml **the tag is ignored**.
* A single entity may be only in a single group. If it's specified in both package.yaml and in tag, the tag takes precedence.
---------
Co-authored-by: Ilya Bogdanov <fumlead@gmail.com>
Fixes#7198Fixes#7318https://github.com/enso-org/enso/assets/3919101/4aead1e2-de01-4b6e-aa12-403af0b3c677
This PR changes the way components are kept in the controllers to allow mixing different groups when filtering. On this occasion, the code was greatly simplified:
* Instead of identifying entries by section, group and entry ID we have just a single EntryId representing position on the list. This way the view was simplified.
* Removed support for headers in Component Grid (but the Grid View still has this feature).
* Removed remnants of the old searcher and "actions".
Also, this PR fixes#7201. I decided that the top modules will have full path (namespace, library and module name), so they will be displayed as `Standard.Base.Data` instead of just `Data` (so it's clear we're browsing part of the standard library.
### Important Notes
The searcher's breadcrumbs controller is in not very nice state, but it will be revised anyway, as the breadcrumbs will be synchronized with documentation panel in the new design.
- Previous GraalVM update: https://github.com/enso-org/enso/pull/6750
Removed warnings:
- Remove deprecated `ConditionProfile.createCountingProfile()`.
- Add `@Shared` to some `@Cached` parameters (Truffle now emits warnings about potential `@Share` usage).
- Specialization method names should not start with execute
- Add limit attribute to some specialization methods
- Add `@NeverDefault` for some cached initializer expressions
- Add `@Idempotent` or `@NonIdempotent` where appropriate
BigInteger and potential Node inlining are tracked in follow-up issues.
# Important Notes
For `SDKMan` users:
```
sdk install java 17.0.7-graalce
sdk use java 17.0.7-graalce
```
For other users - download link can be found at https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-17.0.7
Release notes: https://www.graalvm.org/release-notes/JDK_17/
R component was dropped from the release 23.0.0, only `python` is available to install via `gu install python`.
This PR does three related things:
- Fails more gracefully when a non-string is passed to compile_regex
- Don't pass a non-string to compile_regex
- Allow a Regex param to parse_to_table
- Add dropdowns for `replace` functions.
- Retire `Column_Selector` type.
- Add `select_blank_columns` and `remove_blank_columns` functions to table types.
- Allow Regex to be used to pick columns.
close#7194
Changelog:
- add: `/projects/{project_id}/enso_project` HTTP endpoint returning an `.enso-project` archive structure
- update: archive enso project to a `.enso-project` `.tar.gz` archive
- update: make project `path` a required field
- Adds `Column.date_diff` for computing date/time difference as integer multiply of some unit.
- Adds `Column.date_add` for shifting date/time by a unit.
- Adds `Column.date_part` for extracting various parts of the date/time value as integer.
- Adds widgets for the 3 methods above whose content depends on the column value type.
- Adds shorthands: `Column.hour`, `Column.minute` and `Column.second` to extract these date parts.
- Extends `Time_Period` with support for milli-, micro- and nano- seconds; and adapts functions taking `Time_Period` to support these wherever possible.
Closes#7047
Adds an ability to resize visualizations by dragging a special (invisible) shape along the bottom and right borders of visualizations.
- Visualizations are aligned to the left border of the node now.
- Default visualization width now equals to the node's width (default height is the same)
- Changing the width of the node also changes visualization width, but only if no manual drag-resizing was applied
- Visualization size is preserved when reopening visualization (but it is not saved in project metadata)
- No visual indication that resizing is possible exist, it will be implemented in #7049https://github.com/enso-org/enso/assets/6566674/2f2525e8-cf10-4c92-953a-b69eb97a954a
Adds a new bare-bones AI searcher that can be triggered with `cmd+tab`. It will interpret the searcher input as a prompt to an AI model and replace the created node with the suggestion that was computed.
https://github.com/enso-org/enso/assets/1428930/f8403533-54ba-4ea5-9d3c-6bdf3cf336b5
Implements the first step of #7099.
# Important Notes
Contains some refactoring that allows us to have multiple controllers side by side. So QA testing should make sure that the Component Browser Searcher is still working as before.
`executionFailed` instead is sent when an evaulation finishes with a a critical failure or a non-critical error.
The PR tries to miniminally modify the change in the messages exchange so as to avoid a major redesign at this point.
Closes#7002.
# Important Notes
Unblocks IDE which will need to modify to this new setup.
The change adds logic that will attempt a few retries when executing `gu` (GraalVM updater) commands. Previously, if it failed, it failed. Retries should help with the most common case - occassional network hiccups.
Closes#6880.
# Important Notes
Note that I don't use an external library for retries on purpose. Didn't want to introduce a yet another dependency for this tiny functionality.
Previously, static method calls on `Any` have not worked as expected. For example, `Any.to_text` returned Function instead of Text. That is because the function resolution for `Any.to_text` finds `Any.type.to_text` method on eigentype which expects two `self` arguments, but only one argument is provided.
Note that `Boolean.to_text` worked previously, and returned "Boolean" as expected. This is because the method resolution finds `Any.to_text` method that takes just one `self` argument.
This PR solves this issue by introducing special handling for static method dispatch on `Any`. Simply put, an additional `self` argument is prepended to the argument list.
# Important Notes
A new child node is introduced to `InvokeMethodNode`. This child node is a copy of the current `invokeFunctionNode` with one more `CallArgumentInfo` in its schema.
Fixes#6816.
The code for adding imports for dropdown widgets was unified with CB. The code was moved from the searcher controller to the graph controller.
Also, I changed the signature for a few `lookup_*` methods of the suggestion database, because I have always found it weird that they return `Option` instead of `Result`. They now work nicely with the surrounding code.
https://github.com/enso-org/enso/assets/6566674/49125f8d-096e-4cca-a922-4811ed717a4d
- Removed `module` argument from `enso_project` (new `Project_Description.new` API).
- Removed the custom option from date and time parse/format dropdowns.
- The `format` dropdown uses the value to create the dropdown. (Screenshot below)
- Removed `StorageType` coalescing rules and replaced them with simpler logic in `ObjectStorage`.
- Update signature for `add_row_number` and add aliases.
Request Timeouts started plaguing IDE due to numerous `executionContext/***Visualization` requests. While caused by a bug they revealed a bigger problem in the Language Server when serving large amounts of requests:
1) Long and short lived jobs are fighting for various locks. Lock contention leads to some jobs waiting for a longer than desired leading to unexpected request timeouts. Increasing timeout value is just delaying the problem.
2) Requests coming from IDE are served almost instantly and handled by various commands. Commands can issue further jobs that serve request. We apparently have and always had a single-thread thread pool for serving such jobs, leading to immediate thread starvation.
Both reasons increase the chances of Request Timeouts when dealing with a large number of requests. For 2) I noticed that while we used to set the `enso-runtime-server.jobParallelism` option descriptor key to some machine-dependent value (most likely > 1), the value set would **only** be available for instrumentation. `JobExecutionEngine` where it is actually used would always get the default, i.e. a single-threaded ThreadPool. This means that this option descriptor was simply misused since its introduction. Moved that option to runtime options so that it can be set and retrieved during normal operation.
Adding parallelism intensified problem 1), because now we could execute multiple jobs and they would compete for resources. It also revealed a scenario for a yet another deadlock scenario, due to invalid order of lock acquisition. See `ExecuteJob` vs `UpsertVisualisationJob` order for details.
Still, a number of requests would continue to randomly timeout due to lock contention. It became apparent that
`Attach/Modify/Detach-VisualisationCmd` should not wait until a triggered `UpsertVisualisationJob` sends a response to the client; long and short lived jobs will always compete for resources and we cannot guarantee that they will not timeout that way. That is why the response is sent immediately from the command handler and not from the job executed after it.
This brings another problematic scenario:
1. `AttachVisualisationCmd` is executed, response sent to the client, `UpsertVisualisationJob` scheduled.
2. In the meantime `ModifyVisualisationCmd` comes and fails; command cannot find the visualization that will only be added by `UpsertVisualisationJob`, which might have not yet been scheduled to run.
Remedied that by checking visualisation-related jobs that are still in progress. It also allowed for cancelling jobs which results wouldn't be used anyway (`ModifyVisualisationCmd` sends its own `UpsertVisualisationJob`). This is not a theoretical scenario, it happened frequently on IDE startup.
This change does not fully solve the rather problematic setup of numerous locks, which are requested by short and long lived jobs. A better design should still be investigated. But it significantly reduces the chances of Request Timeouts which IDE had to deal with.
With this change I haven't been able to experience Request Timeouts for relatively modest projects anymore.
I added the possibility of logging wait times for locks to better investigate further problems.
Closes#7005
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
There was an inherent race condition between edit, close & open commands which could not be prevented solely using locks. `EditFileCmd` triggered `EnsureCompiledJob` which was applying edits collected over time. At the same `CloseFileCmd` and `OpenFileCmd` were executed asynchronously and required locks on compilation unit and file lock.
Additionally, open file was resetting the module's runtime source irrespective of any edits that could already have been applied with the asynchronous execution in `EnsureCompiledJob`. This was visible especially during early manipulation of the project when open/close was performed due to a bug in IDE (#6843).
Now commands can be run either synchronously or asynchronously. Only that way can we ensure that `close` & `open` commands finish by the time any editions are being applied to module's sources.
Closes#6841.
# Important Notes
In the given video, `"foo"` would be greyed out because it would never be part of the module's (runtime) sources. Therefore no IR would be generated for it or instrumentation, meaning it would be present in `expressionUpdates` information necessary for IDE.
[Kazam_screencast_00014.webm](https://github.com/enso-org/enso/assets/292128/226a17b8-729a-415a-803f-003a9695b2f1)
First part for #6498 - refactoring of the upload infrastructure, in preparation for `update_database_table`.
Implemented a `Set` data structure which was long needed.
The APIs are added and an initial implementation is created, but it is not complete - but it has grown significantly already so the remaining implementation will be done as a separate PR.
Adds some basic ability for a function to ensure that it is only executed from within a transaction.
Closes#5227
# Important Notes
- This lays first steps towards #6292 - we get pure Enso variants of MultiValueKey.
- Another part refactors `LongStorage` into `AbstractLongStorage` allowing it to provide alternative implementations of the underlying storage, in our case `LongRangeStorage` generating the values ad-hoc and `LongConstantStorage` - currently unused but in the future it can be adapted to support constant columns (once we implement similar facilities for other types).
- Adds execution control to `Table.write`.
- Refactored the `Text.write` to make part reusable.
- Tidied up some legacy mess in tests.
- Add easier flow to go from `Text` to an `URI` to fetching data.
- Add decode functions to `Response` and `Response_Body`.
- Fix issue with 0 length regex matches (using same as Python and .Net approach).
- Add various ALIAS entries to make function discovery easier.
- Sort a lot of drop down and vector editors out (including switch to fully qualified names).
* Remove unused code: project management in component browser
* Encapsulate internal FRP logic of project list
* Collapse some code paths
* Open project passed on command line through presenter
A project name or ID that is passed on the command line was initialised
in the controller setup, before the presenters and views are set up.
Now, we fully initialise the IDE before opening a project so we have
control over the view while a project is being opened.
* Show a spinner in all cases of opening a project
* Let root presenter open/close projects when switching projects
* Change spinner to make progress over a fixed period
* Resolve issues when Project Manager API isn't available
* Bump wasm size limit
Fixes#6754. The issue mentions being able to create and delete nodes when the full-screen visualisation is active. Besides those, you now also can't:
- collapse nodes into a function
- enter a function
- exit a function
Fixes#5203
This PR changes behavior when `text/applyChange` returned error.
Before we always assumed that the change was _not_ applied, and tried to send full synchronization still assuming old file content. But this was not the case on some errors (timeouts for example). Now we instead reopen the file (getting its actual content) and then make a full invalidation.
Also added a shortcut allowing manual file reloading, what may be useful in some kinds of error (and also allowed me testing of reopening file in the application).
# Important Notes
The unit tests of sending text updates were improved: now we actually check if all expected messages are emitted from the IDE.
Fixes#6521: Picking a function from the CB that's defined in the main module now resolves to `Main.<func-name>` instead of `<project-name>.<func-name>`.
Note that, when collapsing nodes to a function, this referral style was already used, so this is just a change in the behaviour of the CB.
The change adds an additional field to `ExpressionUpdates` messages sent by `ProgramExecutionSupport` to indicate if the type of value (or its method pointer) has changed and therefore would potentially require a suggestions' update.
Prior to #3729 that check was done during the instrumentation. However we still want to continue to support "pending expression" functionality therefore `SuggestionsHandler` will use the additional information to filter only the required expression updates.
Most of the changes are related to adapting our tests to the new field.
Closes#6706.
# Important Notes
The associated project now loads and navigates smoothly.
Also attaching a screenshot from the project that illustrates that pending functionality continues to work:
[Kazam_screencast_00006.webm](https://github.com/enso-org/enso/assets/292128/35918841-f84f-4e1c-b1b0-40e45d97e111)
Fixes#6260: The shortcut to open the full-screen visualisation is now `shift-space` so it doesn't interfere with the `space` shortcut to toggle the mini-visualisation.
Fixes#5088. Adds a ensoGL spinner for visualizations waiting on data.
https://user-images.githubusercontent.com/1428930/236801655-67a0ffed-da5d-4e27-8797-cd8126cb86d9.mp4
# Important Notes
This spinner will not show up for the duration where visualizations are processing data on the frontend. If this is a concern, visualization need to implement heir own loading spinner, or we need to provide a unified API for them to keep the spinner visible.
Artifically limiting the number of reported warnings to 100. Also added benchmarks with random Ints to investigate perf issues when dealing with warnings (future task).
Ideally we would have a custom set-like collection that allows us internally to specify a maximal number of elements. But `EnsoHashMap` (and potentially `EnsoSet`) are still WIP when it comes to being PE-friendly.
The change also allows for checking if the limit for the number of reported warnings has been reached. It will visualize by adding an additional "Warnings limit reached." to the visualization.
The limit is configurable via `--warnings-limit` parameter to `run`.
Closes#6283.
Fixes#6377: The fact that `ctrl-r` was restoring the project to potentially the initial state was confusing. This PR changes that shortcut to `ctrl-shift-r`, as discussed in #6377.
Note that:
- `ctrl-shift-r` was already taken by the re-execution shortcut, so that got the `ctrl-alt` modifiers instead, along with the interruption shortcut for consistency.
- `ctrl-alt-shift-r` was already taken by a shortcut to refresh the whole Electron app, so that wasn't available.
Add format to the in-memory Column
# Important Notes
Also updates .format in date types.
Some rearrangement of date formatting builtins / Java libraries.
Closes#5075: dragging or scrolling while a visualisation is in full screen caused the camera to move in the graph editor.
The problem was that clicking on the visualisation triggered some FRP node that indicated that the project list should be closed, which then indicated that the navigator shouldn't be disabled. However, the FRP code in the graph editor interpreted "shouldn't be disabled" as "should be enabled", ignoring the fact that there's also a full-screen visualisation, which should always disable the navigator.
https://user-images.githubusercontent.com/607786/235908932-0b579d69-5fd8-438e-a82b-02678d8e6156.mp4
Closes#6196.
Three things were going wrong:
- Not directly contributing, but adding confusion was the fact that padding of 5px was added in two different places. Since the 10px we've had up until now looked better, especially given the size of the rounded corners, I've kept it at 10px, but only applied in one place.
- The main issue was that the length the scrollbars scroll over didn't take padding into account. At the same time, I changed the `max` and `thumb_size` variables to the coordinate system of the content. This is also how they're being used in `ScrollArea`, which is the only other place where `Scrollbar`s are being used.
- The line height of text grid entries was set to the default of 1.2. That's the default line height in browsers, which is great for multi-line text and elements whose height is greater than the line height. In this case, however, where the height and the font size are set to the same value, the default setting of 1.2 pushes the text below the allotted space.
https://user-images.githubusercontent.com/607786/234297411-8c0b3851-5977-4ca5-b3b4-5b0782510e14.mp4
Remove the magical code generation of `enso_project` method from codegen phase and reimplement it as a proper builtin method.
The old behavior of `enso_project` was special, and violated the language semantics (regarding the `self` argument):
- It was implicitly declared in every module, so it could be called without a self argument.
- It can be called with explicit module as self argument, e.g. `Base.enso_project`, or `Visualizations.enso_project`.
Let's avoid implicit methods on modules and let's be explicit. Let's reimplement the `enso_project` as a builtin method. To comply with the language semantics, we will have to change the signature a bit:
- `enso_project` is a static method in the `Standard.Base.Meta.Enso_Project` module.
- It takes an optional `project` argument (instead of taking it as an explicit self argument).
Having the `enso_project` defined as a (shadowed) builtin method, we will automatically have suggestions created for it.
# Important Notes
- Truffle nodes are no longer generated in codegen phase for the `enso_project` method. It is a standard builtin now.
- The minimal import to use `enso_project` is now `from Standard.Base.Meta.Enso_Project import enso_project`.
- Tested implicitly by `org.enso.compiler.ExecCompilerTest#testInvalidEnsoProjectRef`.
- Adjusted `Context.is_enabled` to support default argument (moved built in so can have defaults).
- Made `environment` case-insensitive.
- Bug fix for play button.
- Short hand to execute within an enabled context.
- Forbid file writing if the Output context is disabled with a `Forbidden_Operation` error.
- Add temporary file support via `File.create_temporary_file` which is deleted on exit of JVM.
- Execution Context first pass in `Text.write`.
- Added dry run warning.
- Writes to a temporary file if disabled.
- Created a `DryRunFileManager` which will create and manage the temporary files.
- Added `format` dropdown to `File.read` and `Data.read`.
- Renamed `JSON_File` to `JSON_Format` to be consistent.
(still to unit test).
Rewrites node input component. Now the input is composed of multiple widget components arranged in a tree of views with automatic layout. That allows creating complex UI elements on top of the node itself, and further widget positions will be automatically adapted to that. The tree roughly follow the span tree, as it is built by consuming its nodes and eagerly creating widgets from them. The tree is rebuilt every time the expression changes, but that rebuild process reuses as much previously created widgets as possible, and only updates their configuration as needed. Each widget type can have its own configuration options that can be passed to it from the parent, or assigned based on configuration received from the language server.
<img width="773" alt="image" src="https://user-images.githubusercontent.com/919491/233439310-9c39ea88-19bc-43da-9baf-1bb176e2724e.png">
# Important Notes
For now, all span-tree updates are sent over to the shared Frp endpoint of the whole tree, so there is no mechanism for intercepting them by the parent widgets. One idea would be to use existing bubbling/capturing events on widget display objects for that purpose, but I think existing implementation is simpler and more convenient, and we can always easily change that if we have a use for it.
There are some issues with performance due to much more display objects being created on the graph. Expect it to be a little worse, especially at initialization time.
Integrate the UI for electing the Execution Environment with the Language Server and unify existing uses. Implements #5930 + actual integration instead of just mocking it.
https://user-images.githubusercontent.com/1428930/232919438-6e1e295a-34fe-4756-86a4-5f5d8f718fa0.mp4
# Important Notes
The console output is only emitted as part of the `INFO` level. A better check would be to look at the messages sent to the backend in the developer console.
- Add `replace` with same syntax as on `Text` to an in-memory `Column`.
- Add `trim` with same syntax as on `Text` to an in-memory `Column`.
- Add `trim` to in-database `Column`.
- Added `is_supported` to dialects and exposed the dialect consistently on the `Connection`.
- Add `write_table` support to `JSON_File` allowing `Table.write` to write JSON.
- Updated the parsing for integers and decimals:
- Support for currency symbols.
- Support for brackets for negative numbers.
- Automatic detection of decimal points and thousand separators.
- Tighter rules for scientific and thousand separated numbers.
- Remove `replace_text` from `Table`.
- Remove `write_json` from `Table`.
While Enso runs single-threaded, its `ResourceManager` required additional asynchronous thread to execute its _"finalizers"_. What has been necessary back then is no longer needed since _GraalVM 21.1_. GraalVM now provides support for submitting `ThreadLocalAction` that gets then picked and executed via `TruffleSafepoint` locations. This PR uses such mechanism to _"inject"_ finalizer execution into already running Enso evaluation thread.
Requiring more than one thread has complicated Enso's co-existence with other Truffle language. For example Graal.js is strictly singlethreaded and used to refuse (simple) co-existence with Enso. By allowing Enso to perform all its actions in a single thread, the synergy with Graal.js becomes better.
* Update type ascriptions in some operators in Any
* Add @GenerateUncached to AnyToTextNode.
Will be used in another node with @GenerateUncached.
* Add tests for "sort handles incomparable types"
* Vector.sort handles incomparable types
* Implement sort handling for different comparators
* Comparison operators in Any do not throw Type_Error
* Fix some issues in Ordering_Spec
* Remove the remaining comparison operator overrides for numbers.
* Consolidate all sorting functionality into a single builtin node.
* Fix warnings attachment in sort
* PrimitiveValuesComparator handles other types than primitives
* Fix byFunc calling
* on function can be called from the builtin
* Fix build of native image
* Update changelog
* Add VectorSortTest
* Builtin method should not throw DataflowError.
If yes, the message is discarded (a bug?)
* TypeOfNode may not return only Type
* UnresolvedSymbol is not supported as `on` argument to Vector.sort_builtin
* Fix docs
* Fix bigint spec in LessThanNode
* Small fixes
* Small fixes
* Nothings and Nans are sorted at the end of default comparator group.
But not at the whole end of the resulting vector.
* Fix checking of `by` parameter - now accepts functions with default arguments.
* Fix changelog formatting
* Fix imports in DebuggingEnsoTest
* Remove Array.sort_builtin
* Add comparison operators to micro-distribution
* Remove Array.sort_builtin
* Replace Incomparable_Values by Type_Error in some tests
* Add on_incomparable argument to Vector.sort_builtin
* Fix after merge - Array.sort delegates to Vector.sort
* Add more tests for problem_behavior on Vector.sort
* SortVectorNode throws only Incomparable_Values.
* Delete Collections helper class
* Add test for expected failure for custom incomparable values
* Cosmetics.
* Fix test expecting different comparators warning
* isNothing is checked via interop
* Remove TruffleLogger from SortVectorNode
* Small review refactorings
* Revert "Remove the remaining comparison operator overrides for numbers."
This reverts commit 0df66b1080.
* Improve bench_download.py tool's `--compare` functionality.
- Output table is sorted by benchmark labels.
- Do not fail when there are different benchmark labels in both runs.
* Wrap potential interop values with `HostValueToEnsoNode`
* Use alter function in Vector_Spec
* Update docs
* Invalid comparison throws Incomparable_Values rather than Type_Error
* Number comparison builtin methods return Nothing in case of incomparables
As per design, IOContexts controlled via type signatures are going away. They are replaced by explicit `Context.if_enabled` runtime checks that will be added to particular method implementations.
`production`/`development` `IOPermissions` are replaced with `live` and `design` execution enviornment. Currently, the `live` env has a hardcoded list of allowed contexts i.e. `Input` and `Output`.
# Important Notes
As per design PR-55. Closes#6129. Closes#6131.
Enso will now associate with two file extensions:
* `.enso` — Enso source file.
* If the source file belongs to a project under the Project Manager-managed directory, it will be opened.
* If the source file belongs to a project located elsewhere, it will be imported into the PM-managed directory and opened;
* Otherwise, opening the `.enseo` file will fail. (e.g., loose source file without any project)
* `.enso-project` — Enso project bundle, i.e., `tar.gz` archive containing a compressed Enso project directory.
* it will be imported under the PM-managed directory; a unique directory name shall be generated if needed.
### Important Notes
On Windows, the NSIS installer is expected to handle the file associations.
On macOS, the file associations are expected to be set up after the first time Enso is started,
On Linux, the file associations are not supported yet.
Implements #6134.
# Important Notes
One can define lazy atom fields as:
```haskell
type Lazy
Value ~x ~y
```
the evaluation of the `x` and `y` fields is then delayed until they are needed. The evaluation happens once. Then the computed value is kept in the atom for further use.
This is the first part of the #5158 umbrella task. It closes#5158, follow-up tasks are listed as a comment in the issue.
- Updates all prototype methods dealing with `Value_Type` with a proper implementation.
- Adds a more precise mapping from in-memory storage to `Value_Type`.
- Adds a dialect-dependent mapping between `SQL_Type` and `Value_Type`.
- Removes obsolete methods and constants on `SQL_Type` that were not portable.
- Ensures that in the Database backend, operation results are computed based on what the Database is meaning to return (by asking the Database about expected types of each operation).
- But also ensures that the result types are sane.
- While SQLite does not officially support a BOOLEAN affinity, we add a set of type overrides to our operations to ensure that Boolean operations will return Boolean values and will not be changed to integers as SQLite would suggest.
- Some methods in SQLite fallback to a NUMERIC affinity unnecessarily, so stuff like `max(text, text)` will keep the `text` type instead of falling back to numeric as SQLite would suggest.
- Adds ability to use custom fetch / builder logic for various types, so that we can support vendor specific types (for example, Postgres dates).
# Important Notes
- There are some TODOs left in the code. I'm still aligning follow-up tasks - once done I will try to add references to relevant tasks in them.
Instrumentation of calls involving warning values never really worked because:
1) newly created nodes didn't set the UUID of their children
2) the instrumentable wrappers always had an empty (i.e. null) UUID and
they never referred `get`/`setId` calls to their delegates
On the surface, everything worked fine. Except when one actually relied on the instrumentation of values with warnings for proper setup. Then no instrumentation (replacement of nodes) was performed due to empty UUID (as required by `hasTag` of `FunctionCallInstrumentationNode`).
Closes#6045. Discovered in #5893.
Implements #5933: adding tooltips to the buttons next to nodes.
To make the UI consistent, I've added tooltips to the `ToggleButton` class directly, since whenever you have an icon button, it seems helpful to have a tooltip.
`ToggleButton` is only used for the profiling button in the top-right corner and the buttons next to nodes. The output context switch button [isn't implemented yet](https://github.com/enso-org/enso/issues/5929), but once it is, adding a tooltip should be one-liner.
Exporting types named the same as the module where they are defined in `Main` modules of library components may lead to accidental name conflicts. This became apparent when trying to access `Problem_Behavior` module via a fully qualified name and the compiler rejected it. This is due to the fact that `Main` module exported `Error` type defined in `Standard.Base.Error` module, thus making it impossible to access any other submodules of `Standard.Base.Error` via a fully qualified name.
This change adds a warning to FullyQualifiedNames pass that detects any such future problems.
While only `Error` module was affected, it was widely used in the stdlib, hence the number of changes.
Closes#5902.
# Important Notes
I left out the potential conflict in micro-distribution, thus ensuring we actually detect and report the warning.
Fixing #5768 and #5765 and co. Introducing `Meta.Type` and giving it the desired methods.
# Important Notes
`Type` is no longer a `Meta.Atom`, but it has a dedicated `Meta.Type` representation.
Implements #5640 and #5650
It made sense for me to implement those two together, as I wanted to make sure that the necessary widget API changes will support custom entry values for both dynamic and static data.
- Added support for custom dropdown labels defined on the method annotations
- Added shortening of static dropdown values, which resolves
| dynamic dropdown - custom labels | static dropdown - automatic shortening |
|-|-|
|||
# Important Notes
During implementation I had multiple data update order issues caused by FRP network forming a diamond shape. Two inputs that are often updated together were combined with `all` combinator, and that was further fed into the dropdown. This caused two updates to propagate through the whole network, and one of them was immediately outdated. To fix this and similar future scenarios, I've added an `next_tick` FRP node. It buffers the incoming events until the next browser microtask, preserving only the last received event. Currently if it is called inside a `requestAnimationFrame` callback, the effects of that processing will only be rendered in the next frame. Later this can be mitigated by delaying the rendering logic until the microtask queue is empty.
The `logAvailableComponentsForDebugging` will check and install all necessary components of GraalVM for every mentioned version. While not harmful, it adds up to startup time.
Additionally added an option in language server startup to skip installation of GraalVM components. The latter is already performed by project-manager when opening the project and it is unnecessary to do it twice. Due to LS' architecture this configuration has to be passed around via multiple configs.
Finally, skipped the attempt to install Python component on Windows - this is not supported by GraalVM atm.
Closes#5749.
# Important Notes
The impact of this problem could be really felt the more versions of Enso and GraalVM one had since it would go through all of them.
Implement new Enso documentation parser; remove old Scala Enso parser.
Performance: Total time parsing documentation is now ~2ms.
# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2nd PR for IDE/Cloud authorization with cognito. This PR introduces boilerplate react app + some amplify code to fetch the access token + username of the currently logged in user, if they are already authenticated.
Registration + Login + Set Username + Forgot Password flows are to be added in next PRs to keep the changes reviewable.
Closes#5151 and adds some additional tests for `cross_tab` that verify duplicated and invalid names.
I decided that for empty or `Nothing` names, instead of replacing them with `Column` and implicitly losing connection with the value that was in the column, we should just error on such values.
To make handling of these easier, `fill_empty` was added allowing to easily replace the empty values with something else.
Also, `{is,fill}_missing` was renamed to `{is,fill}_nothing` to align with `Filter_Condition.Is_Nothing`.
Merge _ordered_ and _unordered_ comparators into a single one.
# Important Notes
Comparator is now required to have only `compare` method:
```
type Comparator
comapre : T -> T -> (Ordering|Nothing)
hash : T -> Integer
```
Added support for named arguments in IDE.
https://user-images.githubusercontent.com/919491/223681303-4c716639-d06e-4e33-aa22-6ebca2801b01.mp4
Named arguments are now recognized in node expressions. The function argument placeholders are rendered around series of named arguments. Insertion and deletion of arguments either by connection dragging or by widget selection will cause arguments around to be rewritten into appropriate form, such that the meaning of the expression doesn't change. We no longer need to introduce any wildcards (`_`) in argument positions when editing an argument list of a resolved method.
For unresolved function calls, the old behaviour remains, as we don't have data about argument names or their desired order.
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.
Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.
Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.
Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.
Finally fix an SBT warning when generating library cache.
Closes https://github.com/enso-org/enso/issues/5763
# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
This mainly fixes#5627 in both scenarios: when the initProtocol method takes too long time and also when we run GUI before LS listens for new connections.
If the IDE initialization fails (e.g. due to above two reasons), it is retried 3 times, waiting some time.
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.
When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.
The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.
Part of https://github.com/enso-org/enso/issues/5568 work.
- Updates the `rename_columns` API.
- Add `first_row`, `second_row` and `last_row` to the Table types.
- New option for reading only last row of ResultSet.
Closes#5340
This PR adds matching searched component browser entries by alias. Now the searcher input is also matched to the `ALIAS` tags of a component, and the best match is used for filtering and sorting the components in the component browser. The alias match scores are reduced by a factor to give them a lower priority when sorting filtered entries in the component browser.
Multiple aliases for a single entry can be obtained from either multiple `ALIAS` tags in the documentation, or comma-separated aliases inside one `ALIAS` tag.
When the searcher input matches one of the entry's aliases the entry in the component browser is displayed as `alias (label)`.
https://user-images.githubusercontent.com/117099775/220571385-d6c2aba6-f13b-4517-9cdf-fe146eeb751a.mp4
Rename is_match + match to match + find (respectively), and remove all non-regexp functionality.
Regexp flags and Match_Mode are also no longer supported by these methods.
Remove regex support from .locate and .locate_all; regex functionality is moved to .match and .match_all where appropriate. This is in preparation for simplifying regex support across the board.
Also change Matching_Mode types to a single type with two variants.
Note: the matcher parameter to .locate and .locate_all has been replaced by a case_sensitivity parameter, of type Case_Sensitivity, which differs in that it also has a Default option. Default is treated as Sensitive.
Closes#5102
This PR improves searching entries in the component browser. Now the searcher input is also matched to the code that a component would generate, and the best match of the two is used for filtering and sorting the components in the component browser.
https://user-images.githubusercontent.com/117099775/219328904-c7a067d5-4998-4ee5-8475-d4974cd7bff5.mp4
#### Entry name formatting
Additionally, the component entry's displayed name format is changed to show the method's name first, followed by the type name in parentheses. This formatting fits better in the narrow columns of the component browser.
Automating the assembly of the engine and its execution into a single task. If you are modifying standard libraries, engine sources or Enso tests, you can launch `sbt` and then just:
```
sbt:enso> runEngineDistribution --run test/Tests/src/Data/Maybe_Spec.enso
[info] Engine package created at built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev
[info] Executing built-distribution/enso-engine-...-dev/bin/enso --run test/Tests/src/Data/Maybe_Spec.enso
Maybe: [5/5, 30ms]
- should have a None variant [14ms]
- should have a Some variant [5ms]
- should provide the `maybe` function [4ms]
- should provide `is_some` [2ms]
- should provide `is_none` [3ms]
5 tests succeeded.
0 tests failed.unEngineDistribution 4s
0 tests skipped.
```
the [runEngineDistribution](3a581f29ee/docs/CONTRIBUTING.md (running-enso)) `sbt` input task makes sure all your sources are properly compiled and only then executes your enso source. Everything ready at a single press of Enter.
# Important Notes
To debug in chrome dev tools, just add `--inspect`:
```
sbt:enso> runEngineDistribution --inspect --run test/Tests/src/Data/Maybe_Spec.enso
E.g. in Chrome open: devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/7JsgjXlntK8
```
everything gets build and one can just attach the Enso debugger.
- Updated `Widget.Vector_Editor` ready for use by IDE team.
- Added `get` to `Row` to make API more aligned.
- Added `first_column`, `second_column` and `last_column` to `Table` APIs.
- Adjusted `Column_Selector` and associated methods to have simpler API.
- Removed `Column` from `Aggregate_Column` constructors.
- Added new `Excel_Workbook` type and added to `Excel_Section`.
- Added new `SQLiteFormatSPI` and `SQLite_Format`.
- Added new `IamgeFormatSPI` and `Image_Format`.
Add `Comparator` type class emulation for all types. Migrate all the types in stdlib to this new `Comparator` API. The main documentation is in `Ordering.enso`.
Fixes these pivotals:
- https://www.pivotaltracker.com/story/show/183945328
- https://www.pivotaltracker.com/story/show/183958734
- https://www.pivotaltracker.com/story/show/184380208
# Important Notes
- The new Comparator API forces users to specify both `equals` and `hash` methods on their custom comparators.
- All the `compare_to` overrides were replaced by definition of a custom _ordered_ comparator.
- All the call sites of `x.compare_to y` method were replaced with `Ordering.compare x y`.
- `Ordering.compare` is essentially a shortcut for `Comparable.from x . compare x y`.
- The default comparator for `Any` is `Default_Unordered_Comparator`, which just forwards to the builtin `EqualsNode` and `HashCodeNode` nodes.
- For `x`, one can get its hash with `Comparable.from x . hash x`.
- This makes `hash` as _hidden_ as possible. There are no other public methods to get a hash code of an object.
- Comparing `x` and `y` can be done either by `Ordering.compare x y` or `Comparable.from x . compare x y` instead of `x.compare_to y`.
An exception encountered during serialization prevents engine from continuing because it enters an infinite loop(!).
# Important Notes
The aim of this PR is to make it possible for engine to recover from the serialization failures. Any failure would mean that we enter an infinite loop in deserialization which is in turn waiting for the serialization to finish (which will never happen).
In this particular case FQNs are [referencing concrete modules](https://github.com/enso-org/enso/issues/5037). A separate PR will address that.
Expressions returning polyglot values were not reporting the type of the result because we have to do additional magic that infers the correct Enso type. Since this is exactly what `TypeOfNode` does, I re-used the logic.
Straightforward solution failed in tests because of assertions:
```
[enso] WARNING: Execution of function main failed (Invalid library usage. Cached library must be adopted by a RootNode before it is executed.).
java.lang.AssertionError: Invalid library usage. Cached library must be adopted by a RootNode before it is executed.
```
That is why this PR replaces `ExecutionEventListener` with `ExecutionEventNodeFactory`.
# Important Notes
Usage of `TypeOfNode` for programs that **do not** import stdlib means that we report types that do not involve stdlib e.g.
`Standard.Builtins.Main.Integer` instead of `Standard.Base.Data.Numbers.Integer`. While surprising, this is correct and I would say desirable. While reviewing the code, notice the difference in expectations in our runtime tests.
Start `project-manager` with following options to provide first 20s of the startup sequence:
```
$ project-manager --profiling-events-log-path=start.log --profiling-path=start.npss --profiling-time=20
```
once the `start.log` and `start.npss` files are generated (next to each other), open them in GraalVM's VisualVM:
```
$ graalvm/bin/jvisualvm --openfile start.npss
```
analyze.
Implementation of https://www.pivotaltracker.com/story/show/184012743https://user-images.githubusercontent.com/919491/214082311-cf49e43c-1d1f-4654-903c-a4224cd954d8.mp4
This is also a step towards more general widget support. The widget metadata is queried using `Meta.get_annotation` method through a dedicated visualization. For now only `Single_Choice` case is handled, and always all suggestions are is returned.
# Important Notes
There are limitations as to which node segments receive a widget. Only chain method calls are supported now (`thing.method` syntax), and only outside of lambda scope. Widgets in lambdas will require support for visualisations of lambda subexpressions, which is currently missing in the engine. The IDE technically tries to place the widgets there, but the data never arrives. It should work once the engine support is added.
This PR includes a mock for `Meta.get_annotation` call that only supports `Table.at` method. Real implementation is a separate task that is already in progress.
- New `set` function design - takes a `Column` and works with that more easily and supports control of `Set_Mode`.
- New simple `parse` API on `Column`.
- Separated expression support for `filter` to new `filter_by_expression` on `Table`.
- New `compute` function allowing creation of a column from an expression.
- Added case sensitivity argument to `Column` based on `starts_with`, `ends_with` and `contains`.
- Added case sensitivity argument to `Filter_Condition` for `Starts_With`, `Ends_With`, `Contains` and `Not_Contains`.
- Fixed the issue in JS Table visualisation where JavaScript date was incorrectly set.
- Some dynamic dropdown expressions - experimenting with ways to use them.
- Fixed issue with `.pretty` that wasn't escaping `\`.
- Changed default Postgres DB to `postgres`.
- Fixed SQLite support for starts_with, ends_with and contains to be consistent (using GLOB not LIKE).
When a large long would be passed to a host call expecting a double, it would crash with a
```
Cannot convert '<some long>'(language: Java, type: java.lang.Long) to Java type 'double': Invalid or lossy primitive coercion
```
That is unlikely to be expected by users. It also came up in the Statistics examples during Sum. One could workaround it by forcing the conversion manually with `.to_decimal` but it is not a permanent solution.
Instead this change adds a custom type mapping from Long to Double that will do it behind the scenes with no user interaction. The mapping kicks in only for really large longs.
# Important Notes
Note that the _safe_ range is hardcoded in Truffle and it is not accessible in enso packages. Therefore a simple c&p for that max safe long value was necessary.
https://github.com/enso-org/enso/pull/3764 introduced static wrappers for instance methods. Except it had a limitation to only be allowed for types with at least a single constructor.
That excluded builtin types as well which, by default, don't have them. This limitation is problematic for Array/Vector consolidation and makes builtin types somehow second-citizens.
This change lifts the limitation for builtin types only. Note that we do want to share the implementation of the generated builtin methods. At the same time due to the additional argument we have to adjust the starting index of the arguments.
This change avoids messing with the existing dispatch logic, to avoid unnecessary complexity.
As a result it is now possible to call builtin types' instance methods, statically:
```
arr = Array.new_1 42
Array.length arr
```
That would previously lead to missing method exception in runtime.
# Important Notes
The only exception is `Nothing`. Primarily because it requires `Nothing` to have a proper eigentype (`Nothing.type`) which would messed up a lot of existing logic for no obvious benefit (no more calling of `foo=Nothing` in parameters being one example).
- Updated `Text.starts_with`, `Text.ends_with` and `Text.contains` to new simpler API.
- Added a `Case_Sensitivity.Default` and adjusted `Table.distinct` to use it by default.
- Fixed a bug with `Data.fetch` on an HTTP error.
- Improved SQLite Case Sensitivity control in distinct to use collations.
Implements [#183453466](https://www.pivotaltracker.com/story/show/183453466).
https://user-images.githubusercontent.com/1428930/203870063-dd9c3941-ce79-4ce9-a772-a2014e900b20.mp4
# Important Notes
* the best laziness is used for `Text` type, which makes use of its internal representation to send data
* any type will first compute its default string representation and then send the content of that lazy to the IDE
* special handling of files and their content will be implemented in the future
* size of the displayed text can be updated dynamically based on best effort information: if the backend does not yet know the full width/height of the text, it can update the IDE at any time and this will be handled gracefully by updating the scrollbar position and sizes.
- add: `GeneralAnnotation` IR node for `@name expression` annotations
- update: compilation pipeline to process the annotation expressions
- update: rewrite `OverloadsResolution` compiler pass so that it keeps the order of module definitions
- add: `Meta.get_annotation` builtin function that returns the result of annotation expression
- misc: improvements (private methods, lazy arguments, build.sbt cleanup)
Introduces unboxed (and arity-specialized) storage schemes for Atoms. It results in improvements both in memory consumption and runtime.
Memory wise: instead of using an array, we now use object fields. We also enable unboxing. This cuts a good few pointers in an unboxed object. E.g. a quadruple of integers is now 64 bytes (4x8 bytes for long fields + 16 bytes for layout and constructor pointers + 16 bytes for a class header). It used to be 168 bytes (4x24 bytes for boxed Longs + 16 bytes for array header + 32 bytes for array contents + 8 bytes for constructor ptr + 16 bytes for class header), so we're saving 104 bytes a piece. In the least impressive scenarios (all-boxed fields) we're saving 8 bytes per object (saving 16 bytes for array header, using 8 bytes for the new layout field). In the most-benchmarked case (list of longs), we save 32 bytes per cons-cell.
Time wise:
All list-summing benchmarks observe a ~2x speedup. List generation benchmarks get ~25x speedups, probably both due to less GC activity and better allocation characteristics (only allocating one object per Cons, rather than Cons + Object[] for fields). The "map-reverse" family gets a neat 10x speedup (part of the work is reading, which is 2x faster, the other is allocating, which is now 25x faster, we end up with 10x when combined).
This PR adds new sections to the component browser section navigator bar. The sections are based on the namespaces from which libraries are imported. Selecting a namespace section from the navigator bar highlights the modules from that namespace. Selecting a module from a different namespace switches the navigator bar indicator to the correct namespace category. The currently selected namespace is also shown as the root of the breadcrumbs.
https://user-images.githubusercontent.com/117099775/212144719-2470a99d-8d8c-47ca-ab50-5bac65468090.mp4
For a new project where only the `Standard` namespace exists the only visible change is the breadcrumbs. Adding for example additional modules in the project `src` folder will create them in the `local` namespace, this namespace will show up as a separate navigator section.
* Hash codes prototype
* Remove Any.hash_code
* Improve caching of hashcode in atoms
* [WIP] Add Hash_Map type
* Implement Any.hash_code builtin for primitives and vectors
* Add some values to ValuesGenerator
* Fix example docs on Time_Zone.new
* [WIP] QuickFix for HashCodeTest before PR #3956 is merged
* Fix hash code contract in HashCodeTest
* Add times and dates values to HashCodeTest
* Fix docs
* Remove hashCodeForMetaInterop specialization
* Introduce snapshoting of HashMapBuilder
* Add unit tests for EnsoHashMap
* Remove duplicate test in Map_Spec.enso
* Hash_Map.to_vector caches result
* Hash_Map_Spec is a copy of Map_Spec
* Implement some methods in Hash_Map
* Add equalsHashMaps specialization to EqualsAnyNode
* get and insert operations are able to work with polyglot values
* Implement rest of Hash_Map API
* Add test that inserts elements with keys with same hash code
* EnsoHashMap.toDisplayString use builder storage directly
* Add separate specialization for host objects in EqualsAnyNode
* Fix specialization for host objects in EqualsAnyNode
* Add polyglot hash map tests
* EconomicMap keeps reference to EqualsNode and HashCodeNode.
Rather than passing these nodes to `get` and `insert` methods.
* HashMapTest run in polyglot context
* Fix containsKey index handling in snapshots
* Remove snapshots field from EnsoHashMapBuilder
* Prepare polyglot hash map handling.
- Hash_Map builtin methods are separate nodes
* Some bug fixes
* Remove ForeignMapWrapper.
We would have to wrap foreign maps in assignments for this to be efficient.
* Improve performance of Hash_Map.get_builtin
Also, if_nothing parameter is suspended
* Remove to_flat_vector.
Interop API requires nested vector (our previous to_vector implementation). Seems that I have misunderstood the docs the first time I read it.
- to_vector does not sort the vector by keys by default
* Fix polyglot hash maps method dispatch
* Add tests that effectively test hash code implementation.
Via hash map that behaves like a hash set.
* Remove Hashcode_Spec
* Add some polyglot tests
* Add Text.== tests for NFD normalization
* Fix NFD normalization bug in Text.java
* Improve performance of EqualsAnyNode.equalsTexts specialization
* Properly compute hash code for Atom and cache it
* Fix Text specialization in HashCodeAnyNode
* Add Hash_Map_Spec as part of all tests
* Remove HashMapTest.java
Providing all the infrastructure for all the needed Truffle nodes is no longer manageable.
* Remove rest of identityHashCode message implementations
* Replace old Map with Hash_Map
* Add some docs
* Add TruffleBoundaries
* Formatting
* Fix some tests to accept unsorted vector from Map.to_vector
* Delete Map.first and Map.last methods
* Add specialization for big integer hash
* Introduce proper HashCodeTest and EqualsTest.
- Use jUnit theories.
- Call nodes directly
* Fix some specializations for primitives in HashCodeAnyNode
* Fix host object specialization
* Remove Any.hash_code
* Fix import in Map.enso
* Update changelog
* Reformat
* Add truffle boundary to BigInteger.hashCode
* Fix performance of HashCodeTest - initialize DataPoints just once
* Fix MetaIsATest
* Fix ValuesGenerator.textual - Java's char is not Text
* Fix indent in Map_Spec.enso
* Add maps to datapoints in HashCodeTest
* Add specialization for maps in HashCodeAnyNode
* Add multiLevelAtoms to ValuesGenerator
* Provide a workaround for non-linear key inserts
* Fix specializations for double and BigInteger
* Cosmetics
* Add truffle boundaries
* Add allowInlining=true to some truffle boundaries.
Increases performance a lot.
* Increase the size of vectors, and warmup time for Vector.Distinct benchmark
* Various small performance fixes.
* Fix Geo_Spec tests to accept unsorted Map.to_vector
* Implement Map.remove
* FIx Visualization tests to accept unsorted Map.to_vector
* Treat java.util.Properties as Map
* Add truffle boundaries
* Invoke polyglot methods on java.util.Properties
* Ignore python tests if python lang is missing
Added a separate pass, `FullyQualifiedNames`, that partially resolves fully qualified names. The pass only resolves the library part of the name and replaces it with a reference to the `Main` module.
There are 2 scenarios that could be potentially:
1) the code uses a fully qualified name to a component that has been
parsed/compiled
2) the code uses a fully qualified name to a component that has **not** be
imported
For the former case, it is sufficient to just check `PackageRepository` for the presence of the library name.
In the latter we have to ensure that the library has been already parsed and all its imports are resolved. That would require the reference to `Compiler` in the `FullyQualifiedNames` pass, which could then trigger a full compilation for missing library. Since it has some undesired consequences (tracking of dependencies becomes rather complex) we decided to exclude that scenario until it is really needed.
# Important Notes
With this change, one can use a fully qualified name directly.
e.g.
```
import Standard.Base
main =
Standard.Base.IO.println "Hello world!"
```
**Vector**
- Adjusted `Vector.sort` to be `Vector.sort order on by`.
- Adjusted other sort to use `order` for direction argument.
- Added `insert`, `remove`, `index_of` and `last_index_of` to `Vector`.
- Added `start` and `if_missing` arguments to `find` on `Vector`, and adjusted default is `Not_Found` error.
- Added type checking to `+` on `Vector`.
- Altered `first`, `second` and `last` to error with `Index_Out_Of_Bounds` on `Vector`.
- Removed `sum`, `exists`, `head`, `init`, `tail`, `rest`, `append`, `prepend` from `Vector`.
**Pair**
- Added `last`, `any`, `all`, `contains`, `find`, `index_of`, `last_index_of`, `reverse`, `each`, `fold` and `reduce` to `Pair`.
- Added `get` to `Pair`.
**Range**
- Added `first`, `second`, `index_of`, `last_index_of`, `reverse` and `reduce` to `Range`.
- Added `at` and `get` to `Range`.
- Added `start` and `if_missing` arguments to `find` on `Range`.
- Simplified `last` and `length` of `Range`.
- Removed `exists` from `Range`.
**List**
- Added `second`, `find`, `index_of`, `last_index_of`, `reverse` and `reduce` to `Range`.
- Added `at` and `get` to `List`.
- Removed `exists` from `List`.
- Made `all` short-circuit if any fail on `List`.
- Altered `is_empty` to not compute the length of `List`.
- Altered `first`, `tail`, `head`, `init` and `last` to error with `Index_Out_Of_Bounds` on `List`.
**Others**
- Added `first`, `second`, `last`, `get` to `Text`.
- Added wrapper methods to the Random_Number_Generator so you can get random values more easily.
- Adjusted `Aggregate_Column` to operate on the first column by default.
- Added `contains_key` to `Map`.
- Added ALIAS to `row_count` and `order_by`.
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/184023445
Added a dropdown widget to graph node for all span tree nodes that have tag values present. When an option is selected, the controller receives a partial expression update, which targets specific crumbs of the expression (similar to how edge endpoint updates work).
https://user-images.githubusercontent.com/919491/210219931-8ae418fd-3ac4-44a5-abea-9e670f15cdf9.mp4
# Important Notes
Right now the dropdown widget is recreated every time the node is edited, including a dropdown option being selected. This causes it to close every time. I wanted to get around that by diffing span trees, but I wasn't able to do it in useful way. Additionally, current implementation of node input expression view heavily relies on being reinitialized from scratch every time. This led to more necessary changes than I was comfortable with for this task. I believe it will be easier to implement it as part of more complete widget support, especially after dynamic data support, as we will have proper widget type information.
Most of the problems with accessing `ArrayOverBuffer` have been resolved by using `CoerceArrayNode` (https://github.com/enso-org/enso/pull/3817). In `Array.sort` we still however specialized on Array which wasn't compatible with `ArrayOverBuffer`. Similarly sorting JS or Python arrays wouldn't work.
Added a specialization to `Array.sort` to deal with that case. A generic specialization (with `hasArrayElements`) not only handles `ArrayOverBuffer` but also polyglot arrays coming from JS or Python. We could have an additional specialization for `ArrayOverBuffer` only (removed in the last commit) that returns `ArrayOverBuffer` rather than `Array` although that adds additional complexity which so far is unnecessary.
Also fixed an example in `Array.enso` by providing a default argument.
Compiler performed name resolution of literals in type signatures but would silently fail to report any problems.
This meant that wrong names or forgotten imports would sneak in to stdlib.
This change introduces 2 main changes:
1) failed name resolutions are appended in `TypeNames` pass
2) `GatherDiagnostics` pass also collects and reports failures from type
signatures IR
Updated stdlib so that it passes given the correct gatekeepers in place.
VCS restore operation was correctly restoring the state of projects to the requested commit. Unfortunately, after the operation file system was becoming out-of-sync with language server's buffers (and IDE's content versions).
A few important changes are introduced here that complicate the interaction between components:
1) `vcs restore` returns an actual diff between the current state and the
requested commit
2) the response is forwarded to buffer registry first rather than to the client
3) the diff is used to identify appropriate collaborative editors and
notify them about the need to reload buffers from file system
4) all clients of affected open buffers are notified of the change via
`text/didChange` notification. If a file was removed and there were open buffers for it, clients will be notified via `file/event` and editor will be stopped
5) only then the client is notified about a successful restore operation
This PR addresses one of the two problems reported in https://www.pivotaltracker.com/story/show/184097084.
# Important Notes
We need to make sure that IDE correctly responds to `text/didChange` notifications.
`NestedPatternMatch` pass desugared complex patterns in a very inefficient way resulting in an exponential generation of the number of `case` IR (and Truffle) nodes. Every failed nested pattern would copy all the remaining patterns of the original case expression, in a desugared form. While the execution itself of such deeply nested `case` expression might not have many problems, the time spent in compilation phases certainly was a blocker.
This change desugars deeply nested into individual cases with a fallthrough logic. However the fallthrough logic is implemented directly in Truffle nodes, rather than via IR. That way we can generate much simpler IR for nested patterns.
Consider a simple case of
```
case x of
Cons (Cons a b) Nil -> a + b
Cons a Nil -> a
_ -> 0
```
Before the change, the compiler would generate rather large IR even for those two patterns:
```
case x of
Cons w y -> case w of
Cons a b -> case y of
Nil -> a + b
_ -> case x of
Cons a z -> case z of
Nil -> a
_ -> case x of
_ -> 0
_ -> 0
_ -> case x of
Cons a z -> case z of
Nil -> a
_ -> case x of
_ -> 0
_ -> 0
Cons a z -> case z of
Nil -> a
_ -> case x of
_ -> 0
_ -> 0
```
Now we generate simple patterns with fallthrough semantics and no catch-all branches:
```
case x of
Cons w y -> case w of
Cons a b -> case y of ## fallthrough on failed match ##
Nil -> a + b ## fallthrough on failed match ##
Cons a z -> case z of
Nil -> a ## fallthrough on failed match ##
_ -> 0
```
# Important Notes
If you wonder how much does it improve, then @radeusgd's example in https://www.pivotaltracker.com/story/show/183971366/comments/234688327 used to take at least 8 minutes to compile and run.
Now it takes 5 seconds from cold start.
Also, the example in the benchmark includes compilation time on purpose (that was the main culprit of the slowdown).
For the old implementation I had to kill it after 15 minutes as it still wouldn't finish a single compilation.
Now it runs 2 seconds or less.
Bonus points: This PR will also fix problem reported in https://www.pivotaltracker.com/story/show/184071954 (duplicate errors for nested patterns)
`Any.==` is a builtin method. The semantics is the same as it used to be, except that we no longer assume `x == y` iff `Meta.is_same_object x y`, which used to be the case and caused failures in table tests.
# Important Notes
Measurements from `EqualsBenchmarks` shows that the performance of `Any.==` for recursive atoms increased by roughly 20%, and the performance for primitive types stays roughly the same.
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/184023380
Dropdown component. Planned to be used in nodes as a single and multiple selection widget, both for static and dynamically loaded values. Initial support is focused on static data, with limited support for dynamic sources. Notably, loading states are not supported yet. Full support for that is planned to be added later with widget lazy-loading.
- Supports single and multiple selections.
- Dedicated API for providing a static list of all entries.
- Range-based query API for dynamically loading data as it is scrolled (only basic support - will need more work for proper async lazy-loading).
- Internal entry cache and query batching to avoid querying data one by one (the batching for now is very basic, will have to be improved for proper lazy-loading).
- Automatic dropdown width adjustment based on the entry label lengths, up to a set max allowed value.
- Open and close animation.
- Keyboard support for focusing and selecting entries.
# Important Notes
Implementing the dropdown on top of grid-view have uncovered some assumptions around grid-view layers. It was assumed to always be a part of the component browser. Removing that assumption required a mechanism for propagating camera update information through layer tree. This is now implemented using a `camera_parent` layer field. Ideally each layer should simply have at most a single parent, and camera inheritance would follow that. That refactor turned out to be quite involved, so right now the simpler temporary solution is introduced in order to not delay this PR further.
Implements `getMetaObject` and related messages from Truffle interop for Enso values and types. Turns `Meta.is_a` into builtin and re-uses the same functionality.
# Important Notes
Adds `ValueGenerator` testing infrastructure to provide unified access to special Enso values and builtin types that can be reused by other tests, not just `MetaIsATest` and `MetaObjectTest`.
Use JavaScript to parse and serialise to JSON. Parses to native Enso object.
- `.to_json` now returns a `Text` of the JSON.
- Json methods now `parse`, `stringify` and `from_pairs`.
- New `JSON_Object` representing a JavaScript Object.
- `.to_js_object` allows for types to custom serialize. Returning a `JS_Object`.
- Default JSON format for Atom now has a `type` and `constructor` property (or method to call for as needed to deserialise).
- Removed `.into` support for now.
- Added JSON File Format and SPI to allow `Data.read` to work.
- Added `Data.fetch` API for easy Web download.
- Default visualization for JS Object trunctes, and made Vector default truncate children too.
Fixes defect where types with no constructor crashed on `to_json` (e.g. `Matching_Mode.Last.to_json`.
Adjusted default visualisation for Vector, so it doesn't serialise an array of arrays forever.
Likewise, JS_Object default visualisation is truncated to a small subset.
New convention:
- `.get` returns `Nothing` if a key or index is not present. Takes an `other` argument allowing control of default.
- `.at` error if key or index is not present.
- `Nothing` gains a `get` method allowing for easy propagation.
This PR provides a visual indication of whether the project's current state differs from the most recent snapshot saved in the VCS. The project name displayed in the IDE changes to a darker text to indicate that the VCS snapshot is outdated, and back to a lighter text when the current project state corresponds to the last saved VCS snapshot.
https://user-images.githubusercontent.com/117099775/208088438-20dfc2aa-2a7d-47bf-bc12-3d3dff7a4974.mp4
The outdated project snapshot indicator is set when:
* A node is moved.
* A node is added or removed.
* The text editor is used to edit the text.
* The project is auto-saved, and the auto-saved project state does not correspond to the last saved snapshot in the VCS.
The outdated project snapshot indicator is cleared when:
* A new project snapshot is successfully saved using `ctrl+s`.
* The project is auto-saved, and the auto-saved project state is confirmed to correspond to the last saved snapshot in the VCS. This occurs, for example, when a project change is undone and the project is reverted to the last saved snapshot state.
The auto-save events do not occur immediately after a project change but have a short delay, thus the VCS status update is affected by the same delay when triggered by an auto-save event.
This removes the special handling of polyglot exceptions and allows matching on Java exceptions in the same way as for any other types.
`Polyglot_Error`, `Panic.catch_java` and `Panic.catch_primitive` are gone
The change mostly deals with the backslash of removing `Polyglot_Error` and two `Panic` methods.
`Panic.catch` was implemented as a builtin instead of delegating to `Panic.catch_primitive` builtin that is now gone.
This fixes https://www.pivotaltracker.com/story/show/182844611
- Aligned `compare_to` so returns `Type_Error` if `that` is wrong type for `Text`, `Ordering` and `Duration`.
- Add `empty_object`, `empty_array`. `get_or_else`, `at`, `field_names` and `length` to `Json`.
- Fix `Json` serialisation of NaN and Infinity (to "null").
- Added `length`, `at` and `to_vector` to Pair (allowing it to be treated as a Vector).
- Added `running_fold` to the `Vector` and `Range`.
- Added `first` and `last` to the `Vector.Builder`.
- Allow `order_by` to take a single `Sort_Column` or have a mix of `Text` and `Sort_Column.Name` in a `Vector`.
- Allow `select_columns_helper` to take a `Text` value. Allows for a single field in group_by in cross_tab.
- Added `Patch` and `Custom` to HTTP_Method.
- Added running `Statistic` calculation and moved more of the logic from Java to Enso. Performance seems similar to pure Java version now.
This is an enhancement of the `Slider` component implemented in #3852. It adds the following features:
* Tooltips and precision change hints
* Selectable slider limit behaviors
* Textual slider value editing
* Vertical slider layout
#### Tooltips
An information tooltip can now be added to a slider, it is shown when the mouse hovers over the component. Additionally, a pop-up indicating the slider's precision appears when the slider's precision has been adjusted.
https://user-images.githubusercontent.com/117099775/206148098-3b4dc059-18aa-4200-9ee0-5d4382363810.mp4
#### Slider limits
The previous slider implementation clamped the adjusted value to the slider's minimum/maximum limits. Now the following behaviors are available:
* Hard limits: Clamp the value to a range within the slider's limits.
* Soft limits: The value can extend beyond the slider's limits. When this occurs, an overflow indicator will be displayed on the side of the limit that is exceeded.
* Adaptive limits: The value can extend beyond the slider's limits. When this occurs, the exceeded limit will temporarily be adjusted to double the slider's range. This will be performed iteratively until the value falls within the extended limits. When a limit is extended and the value is adjusted to fit a smaller range, the extended limit will be iteratively halved until only the necessary range is covered. The slider's extended limits will never shrink to a range smaller than the original range.
These behaviors can be set to the lower and upper limits of a slider independently.
https://user-images.githubusercontent.com/117099775/206148139-6149c91d-ef49-4e2d-97f6-71084f52591c.mp4
#### Textual editing
The slider's value can now be entered through a text input field. Double-click to edit the slider's current value. To confirm the edit press `enter`, or press `escape` to cancel the edit. If an invalid value is entered on confirmation the slider will revert to its value before the edit. The slider's precision will be adjusted based on the number of decimal places of the value entered.
https://user-images.githubusercontent.com/117099775/206148170-d3fa4c82-6e73-4b1c-9be9-cb99979f7b70.mp4
#### Vertical layout
The slider component now supports a vertical layout. In this case value adjustment is performed by a vertical mouse movement, and a horizontal movement adjusts the slider's precision. The slider's track now fills the component in a vertical direction, and the slider's label is displayed near the top end of the component.
https://user-images.githubusercontent.com/117099775/206148211-0f176aaf-bc1b-45e2-afd7-0d28391aafcb.mp4
#### Scroll bar mode
The slider component supports two indicator modes:
* `Track`: The component is filled with a colored bar from the lower limit (empty) to the upper limit (full) dependent on the slider's value.
* `Thumb`: The component contains a rounded indicator that moves along the slider from one end to the other, indicating the slider's value proportionally to the slider's limits. The width of the indicator is configurable.
In addition, the value text, text entry, and precision adjustment can be turned off to provide a scroll bar appearance when used with the `Thumb` indicator.
https://user-images.githubusercontent.com/117099775/206148261-ae291073-85e9-4082-9f91-39b65fecdc0f.mp4
#### Example scene shortcuts
The example scene contains two shortcuts in order to evaluate the dynamic addition and removal of the slider components:
* `CTRL+D` drops all the slider components that are added to the scene.
* `CTRL+A` adds a new set of example slider components to the scene.
Allow arbitrary expression evaluation in the chromeinspector console. Moreover, allow modifications of any variable in any stack frame.
# Important Notes
- Implement inline parsing in `EnsoLanguage.parse(InlineParsingRequest)`.
- Debugging experience is affected by this [bug in Truffle](https://github.com/oracle/graal/issues/5513), which causes NPEs when a host object gets into chromeinspector. I tried to implement a workaround, but it does not work all the time. Nevertheless, it should not matter that much - if there is a NPE in the debugger, you can just ignore it, as it should be concealed in the debugger and should not be propagted outside. See comments in the `docs/debugger`.
* Sequence literal (Vector) should preserve warnings
When Vector was created via a sequence literal, we simply dropped any
associated any warnings associated with it.
This change propagates Warnings during the creation of the Vector.
Ideally, it would be sufficient to propagate warnings from the
individual elements to the underlying storage but doesn't go well with
`Vector.fromArray`.
* update changelog
* Array-like structures preserver warnings
Added a WarningsLibrary that exposes `hasWarnings` and `getWarnings`
messages. That way we can have a single storage that defines how to
extract warnings from an Array and the others just delegate to it.
This simplifies logic added to sequence literals to handle warnings.
* Ensure polyglot method calls are warning-free
Since warnings are no longer automatically extracted from Array-like
structures, we delay the operation until an actual polyglot method call
is performed.
Discovered a bug in `Warning.detach_selected_warnings` which was missing
any usage or tests.
* nits
* Support multi-dimensional Vectors with warnings
* Propagate warnings from case branches
* nit
* Propagate all vector warnings when reading element
Previously, accessing an element of an Array-like structure would only
return warnings of that element or of the structure itself.
Now, accessing an element also returns warnings from all its elements as
well.
This PR is a draft PR while I learn EnsoGL. The eventual goal is to implement the projects list portion of the cloud dashboard in this PR. This PR will implement part of https://www.pivotaltracker.com/n/projects/2539513/stories/183557950
### Important Notes
This PR is still really rough and contains a lot of hacks & hard-coded values. The FRP usage is also likely to be suboptimal and need fixing.
Save a snapshot of the project directory to the VCS on `ctrl+s`. If the operation fails because the VCS was not initialized previously, it will try to initialize the VCS first and then save a snapshot.
# Important Notes
Note that one cannot
```
import Standard.Table as Table_Module
```
because of the 2-component name restriction that gets desugared to `Standard.Table.Main` and we have to write
```
import Standard.Table.Main as Table_Module
```
in a few places. Once we move `Json.to_table` extension this can be improved.
It appears that when were doing
`from XYZ import all`
the module `XYZ` was also being taken into account during name resolution.
This was unfortunate and became problematic when one had a type with the same name defined in it.
During pattern matching one could not simply do
```
from XYZ import all
...
case ... of
_ : XYZ -> ...
```
since the compiler would complain that we try to pattern match on a type but give it a module.
The module is now excluded from the name resolution, when importing everything from the module.
It appears that this "feature" was used in a number of our tests, so they had to be adapted.
This fixes task 4 in https://www.pivotaltracker.com/story/show/183833055
- Adds transpose and cross_tab to the In-Memory table.
- Cross Tab is built on top of aggregate and hence allows for expressions and has same error trapping as in aggregate.
# Important Notes
Only basic tests have been implemented. Error and warning tests will be added as a follow up task.
By default all polyglot symbols that have been imported were always exported. This means that importing a module that had some polyglot imports brought them into the scope automatically. This didn't follow our desired semantics.
Fixes task 3 in https://www.pivotaltracker.com/story/show/183833055.
This `Slider` component allows adjusting a numeric value with the mouse. The value is increased or decreased by clicking on the component and dragging it to the left or right.
The `Slider` has a configurable default value. `Ctrl`+clicking on the component resets its value to that default. When the value is moved away from the default, the value is printed in **bold**.
The `Slider` precision is increased or decreased by clicking the component and dragging upward or downward. This precision influences how quickly the value changes when the mouse moves horizontally, the steps in which the value is incremented or decremented, and the number of digits used to display the value. There is a margin around the component within which the precision is not changed. Beyond this margin, the precision is increased or decreased in powers of 10 (e.g. `0.1` -> `0.01` -> `0.001` when moving the mouse downwards, or `0.1` -> `1.0` -> `10.0` when moving the mouse upwards). The margin and distance between consecutive steps along the vertical axis are configurable.
The value of the `Slider` is limited to a configurable range, and cannot be adjusted beyond that range. A colored bar fills the component to indicate the current value within the range.
#### Video demonstration
https://user-images.githubusercontent.com/117099775/202244982-2f6f419d-7281-41f6-8607-7e492ad25b46.mp4
#### Future additions
This is the first iteration of the `Slider` component. Additional features are planned for the future:
* Textual editing of the value.
* Improved visual feedback on precision changes.
* Additional out-of-range behaviors.
Upgrading to GraalVM 22.3.0.
# Important Notes
- Removed all deprecated `FrameSlot`, and replaced them with frame indexes - integers.
- Add more information to `AliasAnalysis` so that it also gathers these indexes.
- Add quick build mode option to `native-image` as default for non-release builds
- `graaljs` and `native-image` should now be downloaded via `gu` automatically, as dependencies.
- Remove `engine-runner-native` project - native image is now build straight from `engine-runner`.
- We used to have `engine-runner-native` without `sqldf` in classpath as a workaround for an internal native image bug.
- Fixed chrome inspector integration, such that it shows values of local variables both for current stack frame and caller stack frames.
- There are still many issues with the debugging in general, for example, when there is a polyglot value among local variables, a `NullPointerException` is thrown and no values are displayed.
- Removed some deprecated `native-image` options
- Remove some deprecated Truffle API method calls.
Previously, when exporting the same module multiple times only the first statement would count and the rest would be discarded by the compiler.
This change allows for multiple exports of the same module e.g.,
```
export project.F1
from project.F1 export foo
```
Multiple exports may however lead to conflicts when combined with hiding names. Added logic in `ImportResolver` to detect such scenarios.
This fixes https://www.pivotaltracker.com/n/projects/2539304/stories/183092447
# Important Notes
Added a bunch of scenarios to simulate pos and neg results.
- Allow `Map` to store a `Nothing` key (fixes `Vector.distinct` with a `Nothing`).
- Add `column_names` method to `Table` as a shorthand.
- Return data flow error when comparing with Nothing (not a Panic or a Polyglot exception).
- Allow milli and micro second for DateTime and Time Of Day
# Important Notes
- Added a load of tests for the various comparison operators to Numbers_Spec.
It appears that we were always adding builtin methods to the scope of the module and the builtin type that shared the same name.
This resulted in some methods being accidentally available even though they shouldn't.
This change treats differently builtins of types and modules and introduces auto-registration feature for builtins.
By default all builtin methods are registered with a type, unless explicitly defined in the annotation property.
Builtin methods that are auto-registered do not have to be explicitly defined and are registered with the underlying type.
Registration correctly infers the right type, depending whether we deal with static or instance methods.
Builtin methods that are not auto-registered have to be explicitly defined **always**. Modules' builtin methods are the prime example.
# Important Notes
Builtins now carry information whether they are static or not (inferred from the lack of `self` parameter).
They also carry a `autoRegister` property to determine if a builtin method should be automatically registered with the type.
This change adds support for Version Controlled projects in language server.
Version Control supports operations:
- `init` - initialize VCS for a project
- `save` - commit all changes to the project in VCS
- `restore` - ability to restore project to some past `save`
- `status` - show the status of the project from VCS' perspective
- `list` - show a list of requested saves
# Important Notes
Behind the scenes, Enso's VCS uses git (or rather [jGit](https://www.eclipse.org/jgit/)) but nothing stops us from using a different implementation as long as it conforms to the establish API.
- Added expression ANTLR4 grammar and sbt based build.
- Added expression support to `set` and `filter` on the Database and InMemory `Table`.
- Added expression support to `aggregate` on the Database and InMemory `Table`.
- Removed old aggregate functions (`sum`, `max`, `min` and `mean`) from `Column` types.
- Adjusted database `Column` `+` operator to do concatenation (`||`) when text types.
- Added power operator `^` to both `Column` types.
- Adjust `iif` to allow for columns to be passed for `when_true` and `when_false` parameters.
- Added `is_present` to database `Column` type.
- Added `coalesce`, `min` and `max` functions to both `Column` types performing row based operation.
- Added support for `Date`, `Time_Of_Day` and `Date_Time` constants in database.
- Added `read` method to InMemory `Column` returning `self` (or a slice).
# Important Notes
- Moved approximate type computation to `SQL_Type`.
- Fixed issue in `LongNumericOp` where it was always casting to a double.
- Removed `head` from InMemory Table (still has `first` method).
This PR adds `Period` type, which is a date-only complement to `Duration` builtin type.
# Important Notes
- `Period` replaces `Date_Period`, and `Time_Period`.
- Added shorthand constructors for `Duration` and `Period`. For example: `Period.days 10` instead of `Period.new days=10`.
- `Period` can be compared to other `Period` in some cases, other cases throw an error.
Define start of Enso epoch as 15th of October 1582 - start of the Gregorian calendar.
# Important Notes
- Some (Gregorian) calendar related functionalities within `Date` and `Date_Time` now produces a warning if the receiving Date/Date_Time is before the epoch start, e.g., `week_of_year`, `is_leap_year`, etc.
1. Changes how we do monadic state – rather than a haskelly solution, we now have an implicit env with mutable data inside. It's better for the JVM. It also opens the possibility to have state ratained on exceptions (previously not possible) – both can now be implemented.
2. Introduces permission check system for IO actions.
Moved loading of Builtin Types and Methods to a static initializer. That way the information is available at Native Image build time and one does not have to update a corresponding entry in `reflect-config` which would be a real pain when we start using it in anger.
Fixes https://www.pivotaltracker.com/story/show/183374932
Most of the time, rather than defining the type of the parameter of the builtin, we want to accept every Array-like object i.e. Vector, Array, polyglot Array etc.
Rather than writing all possible combinations, and likely causing bugs on the way anyway as we already saw, one should use `CoerceArrayNode` to convert to Java's `Object[]`.
Added various test cases to illustrate the problem.
The main culprit of a Vector slowdown (when compared to Array) was the normalization of the index when accessing the elements. Turns out that the Graal was very persistent on **not** inlining that particular fragment and that was degrading the results in benchmarks.
Being unable to force it to do it (looks like a combination of thunk execution and another layer of indirection) we resorted to just moving the normalization to the builtin method. That makes Array and Vector perform roughly the same.
Moved all handling of invalid index into the builtin as well, simplifying the Enso implementation. This also meant that `Vector.unsafe_at` is now obsolete.
Additionally, added support for negative indices in Array, to behave in the same way as for Vector.
# Important Notes
Note that this workaround only addresses this particular perf issue. I'm pretty sure we will have more of such scenarios.
Before the change `averageOverVector` benchmark averaged around `0.033 ms/op` now it does consistently `0.016 ms/op`, similarly to `averageOverArray`.
Trying to invoke a foreign method with non-installed language (either not enabled in the Truffle `Context`, or not installed in the GraalVM distribution) results in `Polyglot_Error`, rather than crashing the entire engine.
- Reimplement the `Duration` type to a built-in type.
- `Duration` is an interop type.
- Allow Enso method dispatch on `Duration` interop coming from different languages.
# Important Notes
- The older `Duration` type should now be split into new `Duration` builtin type and a `Period` type.
- This PR does not implement `Period` type, so all the `Period`-related functionality is currently not working, e.g., `Date - Period`.
- This PR removes `Integer.milliseconds`, `Integer.seconds`, ..., `Integer.years` extension methods.
This PR introduced an overhauled Component List Panel implementation, making use of the efficient EnsoGL grid view component. Also, it delivers a couple of new features:
* A part of the new design: there are no more section headers in grid, instead groups are "glued" together. The local scope section is under "popular" (old "favorites").
* The keyboard management inside grid works.
* there is a mouse hover highlight
* selecting the lowest entry in section when jumping with navigation bar.
* accepting input as-is with cmd/ctrl + Enter.
https://user-images.githubusercontent.com/3919101/194561890-fffb9b41-2f0d-4357-8d9a-5038a6bcb023.mp4
### Important Notes
**What is not implemented:**
* [Focus management between panels.](https://www.pivotaltracker.com/story/show/180872763) The grid is always focused. To accept the current input, use ctrl+Enter shortcut.
* [Proper handling of selection when having empty space on the right and pressing right arrow.](https://www.pivotaltracker.com/story/show/183487880)
* When entering a module, its name is not added to the input as described in the design doc. Will be a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
**Known issues**
* [the selection, especially in the local scope section, has sometimes an undesirable offset](https://www.pivotaltracker.com/story/show/183487730). The cause is known, but not so easy to fix.
* The inserted nodes are often producing errors. The Browser's inherits the outdated understanding of the language from old Node Searcher, and it does not include new form of imports, static methods etc. Those all will be fixed as a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
* The performance is improved, but still not ideal, due to problems in [text areas](https://www.pivotaltracker.com/story/show/183406745).
* To scroll the documentation panel, you must first click on it.
This change brings by-type pattern matching to Enso.
One can pattern match on Enso types as well as on polyglot types.
For example,
```
case x of
_ : Integer -> ...
_ : Text -> ...
_ -> ...
```
as well as Java's types
```
case y of
_ : ArrayList -> ...
_ : List -> ...
_ : AbstractList -> ...
_ -> ..
```
It is no longer possible to match a value with a corresponding type constructor.
For example
```
case Date.now of
Date -> ...
```
will no longer match and one should match on the type (`_ : Date`) instead.
```
case Date of
Date -> ...
```
is fine though, as requested in the ticket.
The change required further changes to `type_of` logic which wasn't dealing well with polyglot values.
Implements https://www.pivotaltracker.com/story/show/183188846
# Important Notes
~I discovered late in the game that nested patterns involving type patterns, such as `Const (f : Foo) tail -> ...` are not possible due to the old parser logic.
I would prefer to add it in a separate PR because this one is already getting quite large.~ This is now supported!
