PR adds special kind of jobs for visualizations. It should
- prevent cancelling visualization jobs when the program is re-executed (resulting in the fact that visualization is not showing up)
- omit unnecessary executions of visualization jobs (the case when the user goes through menu items in the component browser)
I skipped the tests because testing of the last scenario involves indeterminism. I.e. when preparing the test, we can't control which of submitted visualization jobs will be actually executed, and which will be cancelled.
When running the profiling run-graph and flamegraph demo scenes, if a profile file is not found in the directory served over http, fall back to generating demo data.
Implements https://www.pivotaltracker.com/story/show/183402892
# Important Notes
- Fixes inconsistent `compare_to` vs `==` behaviour in date/time types and adds test for that.
- Adds test for `Table.order_by` on dates and custom types.
- Fixes an issue with `Table.order_by` for custom types.
- Unifies how incomparable objects are reported by `Table.order_by` and `Vector.sort`.
- Adds benchmarks comparing `Table.order_by` and `Vector.sort` performance.
When nodes get invalidated in the cache, they have to be recomputed. Let the IDE know which of the nodes are pending by sending `Api.ExpressionUpdate.Payload.Pending` message.
# Important Notes
This PR introduces new `Api.ExpressionUpdate.Payload.Pending` message. This message is delivered before re-computation of nodes. Later `Api.ExpressionUpdate.Payload.Value` or other is sent to notify the IDE that a value for given node is available.
Trivial implementation of of the `Api.ExpressionUpdate.Payload.Pending` message in the IDE is provided by this PR to (improperly) visualize pending node status - further improvements needed in follow up PRs.
Each builtin `makeFunction` method has to be currently registered for reflection. Adding registration of `SliceVectorMethodGen` to make following example work ag
```bash
enso$ sbt engine-runner-native/buildNativeImage
enso$ ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
```
# Important Notes
Regression caused by #3724 - Once the above code is executed in the CI, we'll discover such breakages before integration.
Makes statics static. A type and its instances have different methods defined on them, as it should be. Constructors are now scoped in types, and can be imported/exported.
# Important Notes
The method of fixing stdlib chosen here is to just not. All the conses are exported to make all old code work. All such instances are marked with `TODO Dubious constructor export` so that it can be found and fixed.
This change implements a simple `type_of` method that returns a type of a given value, including for polyglot objects.
The change also allows for pattern matching on various time-related instances. It is a nice-to-have on its own, but it was primarily needed here to write some tests. For equality checks on types we currently can't use `==` due to a known _feature_ which essentially does wrong dispatching. This will be improved in the upcoming statics PR so we agreed that there is no point in duplicating that work and we can replace it later.
Also, note that this PR changes `Meta.is_same_object`. Comparing types revealed that it was wrong when comparing polyglot wrappers over the same value.
This PR adds two new labels to interact with CI systems:
* `CI: Keep up to date` tells mergify bot to auto-update the PR against develop but do not merge it.
* `CI: No changelog needed` is meant to replace old `[ci no changelog needed]`
* `CI: Clean build required` marks PR as being incompatible with main development branch and will ensure that runners are cleaned before and after running jobs related to this PR.
Note that currently only first label is supported, others will follow with the build script update. However they need to be merged before, so I am able to develop and test them.
This PR disables the wasm-opt optimization in the crates that can be used as WASM entry points. Unfortunately, wasm-pack does not allow for disabling wasm-opt through a command line flag, so we have to disable it by setting an appropriate flag in each Cargo.toml.
Use an `ArraySlice` to slice `Vector`.
Avoids memory copying for the slice function.
# Important Notes
| Test | Ref | New |
| --- | --- | --- |
| New Vector | 71.9 | 71.0 |
| Append Single | 26.0 | 27.7 |
| Append Large | 15.1 | 14.9 |
| Sum | 156.4 | 165.8 |
| Drop First 20 and Sum | 171.2 | 165.3 |
| Drop Last 20 and Sum | 170.7 | 163.0 |
| Filter | 76.9 | 76.9 |
| Filter With Index | 166.3 | 168.3 |
| Partition | 278.5 | 273.8 |
| Partition With Index | 392.0 | 393.7 |
| Each | 101.9 | 102.7 |
- Note: the performance of New and Append has got slower from previous tests.
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720
real 0m4.503s
user 0m9.248s
sys 0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720
real 0m0.176s
user 0m0.042s
sys 0m0.038s
```
# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.
Will need to add the above command to CI.
Implements https://www.pivotaltracker.com/story/show/183082087
# Important Notes
- Removed unnecessary invocations of `Error.throw` improving performance of `Vector.distinct`. The time of the `add_work_days and work_days_until should be consistent with each other` test suite came down from 15s to 3s after the changes.
When a GridView is navigated using the keyboard, scroll it to display the newly selected entry.
https://www.pivotaltracker.com/story/show/182593635
#### Visuals
See below for a video demonstrating automatic scrolling of the GridView when arrow keys are pressed on the keyboard. The first video below shows the default scrolling behavior in a GridView without headers.
Note:
- When the Grid View is scrolled, the mouse hover highlight moves away from the mouse position. This is not a new regression, the behavior is the same in the `develop` branch (e.g. when scrolling using the mouse).
https://user-images.githubusercontent.com/273837/190183984-91f7808c-3606-43f8-bcda-ac4d5f84e00f.mov
The video below shows the behavior in a GridView with headers when the GridView is first scrolled to its top-left corner. The following guidance from the Design Doc is enabled and showcased:
> Users can change the selected component by pressing the arrow keys. The Focus does not move up if it does not have to (in most cases, the focus is located in the second row from the bottom). Instead, the component list scrolls down if there are enough entries.
https://user-images.githubusercontent.com/273837/189151546-e50aaf22-6f4d-41cb-809f-60038305745f.mov
The next video shows the behavior in the same GridView as in the previous example when the GridView is first scrolled away from any of its boundary entries. Notably, scrolling happens only when the selection is moved using the keyboard arrow keys, not when changing the selection using the mouse. This behavior is based on a discussion with @wdanilo on Discord.
https://user-images.githubusercontent.com/273837/189151974-d992be93-f61f-4e9f-9f4c-dfe260bbec5b.mov
Show the visualization when a newly created node is being edited. Hide it after the editing is finished, unless the user explicitly changed the state of the visualization.
https://www.pivotaltracker.com/story/show/182634059
#### Visuals
The following video shows the behavior of the visualization when a new node is created by the user and is being edited:
- By default, the visualization is shown when the node becomes edited and disappears when the node's expression is accepted.
- If the user clicks the visualization toggle button once while editing the node, the visualization disappears and stays disabled after the node's expression is accepted.
- If the user clicks the visualization toggle button twice while editing the node, the visualization is visible and stays enabled after the node's expression is accepted.
This PR is not concerned with the correctness of the contents of the visualization window. This is in line with the current behavior on the `develop` branch, where clicking the visualization toggle button while editing a newly created node shows `null` content in the visualization (even if the edited expression is not null).
https://user-images.githubusercontent.com/273837/191294970-62143983-764a-446c-9616-9765de0923bf.mov
This PR contains an entry definition for Grid View to be used inside Component List Panel View. The Example grid view with the entry definition may be seen on new_component_list_panel_view debug scene.
https://user-images.githubusercontent.com/3919101/190663278-23c35ab0-f426-4001-8128-df7147aafb9e.mp4
# Important Notes
* The styling is not detailed yet due to time constraints (I want to move to integration this grid view to Component Panel List ASAP) and the fact that I could not get new mplus1 font working with text Area.
* Implementing this required adding a "contour offset" feature to the Grid View.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181870304).
This PR implements Controller-level support for entering modules in the component browser.
Clicking headers/entries opens the module, filling the Local Scope and Submodules sections.
Please ignore multiple visual issues with the component browser on the screen - this PR does not cause them, and they will be fixed soon when we change the implementation of the component list panel to Grid View.
https://user-images.githubusercontent.com/6566674/190933275-c544ea8d-529d-4b49-820f-d36874344572.mp4
# Important Notes
- We will implement the actual breadcrumbs integration in a separate PR: https://www.pivotaltracker.com/story/show/182675703
- The FRP changes in the component list panel code are temporary. The GridView-based implementation will later replace most of the boilerplate.
This PR reenables code signing and notarization on macOS.
[ci no changelog needed]
# Important Notes
* electron-builder has been bumped, mostly to avoid missing Python issue. A workaround for a regression with Windows installer is provided as a patch.
Repairing the constructor name following the types work. Some general tiding up as well.
- Remove `Standard.Database.Data.Column.Aggregate_Column_Builder`.
- Remove `Standard.Database.Data.Dialect.Dialect.Dialect_Data`.
- Remove unused imports and update some type definitions.
- Rename `Postgres.Postgres_Data` => `Postgres_Options.Postgres`.
- Rename `Redshift.Redshift_Data` => `Redshift_Options.Redshift`.
- Rename `SQLite.SQLite_Data` => `SQLite_Options.SQLite`.
- Rename `Credentials.Credentials_Data` => `Credentials.Username_And_Password`.
- Rename `Sql` to `SQL` across the board.
- Merge `Standard.Database.Data.Internal` into `Standard.Database.Internal`.
- Move dialects into `Internal` and merge the function in `Helpers` into `Base_Generator`.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181445628).
This PR implements a Breadcrumbs panel for the new component browser.
The Breadcrumbs is a horizontal list of text labels separated by a special icon and has an optional ellipsis icon at the end.
It is implemented using the new GridView component.
Video:
Demo of adding new breadcrumbs, scrolling behavior, and selecting breadcrumbs with the mouse.
https://user-images.githubusercontent.com/6566674/189199432-77807cef-00dc-4abe-b95c-b17a536f59f6.mp4
Demo of selecting breadcrumbs with keyboard shortcuts:
https://user-images.githubusercontent.com/6566674/189199603-53e55335-73ba-4ed7-8291-4455144c06aa.mp4
# Important Notes
- This PR implements an old interaction of the design of the component browser. The new design of the breadcrumbs can not be easily integrated into the current look of the component browser, so we would need to update styles later. It should be a relatively simple task. *The implementation uses color from the new design though. (but not fonts and sizes)*
- I found a bug in the grid view implementation that causes panics at runtime in some conditions. The reason is triggering FRP endpoints while constructing new entries. This issue is fixed in the PR.
Turns that if you import a two-part import we had special code that would a) add Main submodule b) add an explicit rename.
b) is problematic because sometimes we only want to import specific names.
E.g.,
```
from Bar.Foo import Bar, Baz
```
would be translated to
```
from Bar.Foo.Main as Foo import Bar, Baz
```
and it should only be translated to
```
from Bar.Foo.Main import Bar, Baz
```
This change detects this scenario and does not add renames in that case.
Fixes [183276486](https://www.pivotaltracker.com/story/show/183276486).
Resolves https://www.pivotaltracker.com/story/show/183285801
@JaroslavTulach suggested the current implementation may not handle these correctly, which suggests that the logic is not completely trivial - so I added a test to ensure that it works as we'd expect. Fortunately, it did work - but it's good to keep the tests to avoid regressions.
Changes following Marcin's work. Should be back to very similar public API as before.
- Add an "interface" type: `Standard.Base.System.File_Format.File_Format`.
- All `File_Format` types now have a `can_read` method to decide if they can read a file.
- Move `Standard.Table.IO.File_Format.Text.Text_Data` to `Standard.Base.System.File_Format.Plain_Text_Format.Plain_Text`.
- Move `Standard.Table.IO.File_Format.Bytes` to `Standard.Base.System.File_Format.Bytes`.
- Move `Standard.Table.IO.File_Format.Infer` to `Standard.Base.System.File_Format.Infer`. **(doesn't belong here...)**
- Move `Standard.Table.IO.File_Format.Unsupported_File_Type` to `Standard.Base.Error.Common.Unsupported_File_Type`.
- Add `Infer`, `File_Format`, `Bytes`, `Plain_Text`, `Plain_Text_Format` to `Standard.Base` exports.
- Fold extension methods of `Standard.Base.Meta.Unresolved_Symbol` into type.
- Move `Standard.Table.IO.File_Format.Auto` to `Standard.Table.IO.Auto_Detect.Auto_Detect`.
- Added a `types` Vector of all the built in formats.
- `Auto_Detect` asks each type if they `can_read` a file.
- Broke up and moved `Standard.Table.IO.Excel` into `Standard.Table.Excel`:
- Moved `Standard.Table.IO.File_Format.Excel.Excel_Data` to `Standard.Table.Excel.Excel_Format.Excel_Format.Excel`.
- Renamed `Sheet` to `Worksheet`.
- Internal types `Reader` and `Writer` providing the actual read and write methods.
- Created `Standard.Table.Delimited` with similar structure to `Standard.Table.Excel`:
- Moved `Standard.Table.IO.File_Format.Delimited.Delimited_Data` to `Standard.Table.Delimited.Delimited_Format.Delimited_Format.Delimited`.
- Moved `Standard.Table.IO.Quote_Style` to `Standard.Table.Delimited.Quote_Style`.
- Moved the `Reader` and `Writer` internal types into here. Renamed methods to have unique names.
- Add `Aggregate_Column`, `Auto_Detect`, `Delimited`, `Delimited_Format`, `Excel`, `Excel_Format`, `Sheet_Names`, `Range_Names`, `Worksheet` and `Cell_Range` to `Standard.Table` exports.
IR cache never really took into account a situation when a binding from the imported module has changed. In other words, it would continue to happily use the serialized metadata without noticing that it changed.
This change forces cache invalidation when any of the imported modules was invalidated (or rather not loaded from cache).
# Important Notes
Added simple test infrastructure that simulates file modifications that would trigger the initial cache invalidation.
If they succeed, cache invalidation is propagated thus causing an error.
Add support for moving the selection in a Grid View using the keyboard.
https://www.pivotaltracker.com/story/show/182585789
#### Visuals
See below for videos showcasing GridView selection keyboard navigation in the `grid_view` debug scene. In the videos, messages in the Developer Console can be observed. When a keypress would result in the selection being moved out of the GridView, the selection is not moved and a message is emitted in the Developer Console instead, showcasing an FRP output signal emitted on such event. Please note that the videos are recorded with the tracing level changed to `DEBUG`. In a default build, the tracing level is set to `WARN`, and the messages visible in the videos are not displayed in the Developer Console.
https://user-images.githubusercontent.com/273837/188483972-89d79f7b-1303-457b-869f-282e0809a755.movhttps://user-images.githubusercontent.com/273837/188484294-e9b6461c-a84f-4817-9447-d792f2ebdbb5.mov
The following video shows moving the selection between "regular" entries and header entries. It also shows a current usability limitation of the selection keyboard navigation feature, such that the Grid is not scrolled when the selection leaves the visible part of the Grid, and the selection may thus disappear from view.
https://user-images.githubusercontent.com/273837/188485238-29a82b27-de2f-4cf8-a2e7-ff8c3f41478d.mov
# Important Notes
- Keyboard navigation only works when a GridView has focus.
- Selection keyboard navigation only works if the selection was already set to some entry beforehand.
- If keyboard navigation would move selection outside of the grid, the selection movement is canceled and an FRP event is emitted.
`Vector` type is now a builtin type. This requires a bunch of additional builtin methods for its creation:
- Use `Vector.from_array` to convert any array-like structure into a `Vector` [by copy](f628b28f5f)
- Use (already existing) `Vector.from_polyglot_array` to convert any array-like structure into a `Vector` **without** copying
- Use (already existing) `Vector.fill 1 item` to create a singleton `Vector`
Additional, for pattern matching purposes, we had to implement a `VectorBranchNode`. Use following to match on `x` being an instance of `Vector` type:
```
import Standard.Base.Data.Vector
size = case x of
Vector.Vector -> x.length
_ -> 0
```
Finally, `VectorLiterals` pass that transforms `[1,2,3]` to (roughly)
```
a1 = 1
a2 = 2
a3 = 3
Vector (Array (a1,a2, a3))
```
had to be modified to generate
```
a1 = 1
a2 = 2
a3 = 3
Vector.from_array (Array (a1, a2, a3))
```
instead to accomodate to the API changes. As of 025acaa676 all the known CI checks passes. Let's start the review.
# Important Notes
Matching in `case` statement is currently done via `Vector_Data`. Use:
```
case x of
Vector.Vector_Data -> True
```
until a better alternative is found.
The goal of this request is to simplify hello world and other trivial Enso programs. No need to learn any standard library functions, enough to write:
```
main = "Hello World!"
```
and the result is going to be printed:
```bash
enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run hello.enso
'Hello World!'
```
the result is only printed, if it is not `Nothing`. E.g. if the last statement is `IO.print ...` (which returns `Nothing`), then no value is printed at the end by the launcher.
Many engine PRs modify builtins or other engine internals and then they are subject to [incremental CI runtime errors](https://www.pivotaltracker.com/n/projects/2539304/stories/182868680) as outdated `IR` caches from global space at `~/.local/share/enso/cache/ir/Standard/Builtins/0.0.0-dev/0.0.0-dev/Standard/Builtins` are read in.
This PR provides solution for that problem by explicitly defining `IR.Module` `serialVersionUID`. By changing the `serialVersionUID` one prevents previously saved `IR` caches to be loaded into the running process. Change the `serialVersionUID` whenever you see errors caused by reading outdated `IR` caches in the CI.
# Important Notes
Whenever one needs to avoid loading previous `IR` caches, go to `case class IR.Module` and change the `@SerialVersionUID(3692L)` to **number of your pull request**.
