* added polyfill globals plugin to fix issue with missing types like Buffer that was affecting nightly releases;
* fixed exit code propagation for Windows build script wrapper;
* bumped the build script and refreshed the generated workflows.
Includes https://github.com/enso-org/ci-build/pull/8
Avoid long lines when using `dom().set_inner_text` - rather split the long lines to 1024 chunks and insert them as individual `<div>` elements.
# Important Notes
I was testing the behavior on following program:
```
from Standard.Base import all
import Standard.Base.Data.Statistics
import Standard.Visualization
main =
number1 = 200000
operator1 = 0.up_to number1 . to_vector . map .noise
operator2 = operator1.sort
operator3 = operator2.to_text
```
before my change the visualization of `operator3` was blank. With my change it gets filled with data.
This PR does not add anything visual for the user: it's a first part of making efficient Component Browser using fresh Grid View implementation.
The Layout is an intermediate model, which keeps the information how the groups are laid out in the Component browser, and allows querying for group entry by location and location of specific group entry (or header).
The layout is meant for the new design, where there are no section separators and Local Scope section is below Favorites.
# Important Notes
The new structure is not used in action, but tested with unit tests.
First of all this PR demonstrates how to implement _lazy visualization_:
- one needs to write/enhance Enso visualization libraries - this PR adds two optional parameters (`bounds` and `limit`) to `process_to_json_text` function.
- the `process_to_json_text` can be tested by standard Enso test harness which this PR also does
- then one has to modify JavaScript on the IDE side to construct `setPreprocessor` expression using the optional parameters
The idea of _scatter plot lazy visualization_ is to limit the amount of points the IDE requests. Initially the limit is set to `limit=1024`. The `Scatter_Plot.enso` then processes the data and selects/generates the `limit` subset. Right now it includes `min`, `max` in both `x`, `y` axis plus randomly chosen points up to the `limit`.
The D3 visualization widget is capable of _zooming in_. When that happens the JavaScript widget composes new expression with `bounds` set to the newly visible area. By calling `setPreprocessor` the engine recomputes the visualization data, filters out any data outside of the `bounds` and selects another `limit` points from the new data. The IDE visualization then updates itself to display these more detailed data. Users can zoom-in to see the smallest detail where the number of points gets bellow `limit` or they can select _Fit all_ to see all the data without any `bounds`.
# Important Notes
Randomly selecting `limit` samples from the dataset may be misleading. Probably implementing _k-means clustering_ (where `k=limit`) would generate more representative approximation.
This PR reenables code signing on Windows.
Each Windows package built on CI should be now signed.
Additionally, some refactorings were done around electron-builder config, so it is easier to use outside the build script and offers more configuration options.
So far, when opening the searcher with no node selected, an empty input without an associated AST node was created. This input was manipulated by the user and the final expression from the input was used to create a new node when the user confirmed their input. This PR changes this, so that when the searcher is opened without a selected node, a new AST node is created right away with some placeholder content, and this node is updated when the user confirms their input.
The only change visible to the user, is that if the text editor is opened during editing, a new node will appear in the source when the searcher is opened to create a new node. All other behaviour should stay the same.
[ci no changelog needed]
This PR fixes a [regression](https://www.pivotaltracker.com/story/show/182972359) found in the `interface` debug the scene. It was caused by multiple `Navigator`s that were present in the demo scene and conflict with each other.
Also, this PR includes a fix for the invalid logic of the new component browser. We disable `Navigator` when the mouse hovers the component browser. Still, due to a mistake in the code, the component browser was considered visible at all times and therefore blocked the navigator in certain mouse positions.
To reproduce this bug (before this PR):
1. Open a default project
2. Place the mouse somewhere in the middle of the screen (near one of the nodes)
3. Try panning or zooming the scene (you won't be able because of this bug)
Add functionality to revert a node that was changed during editing through the searcher.
Note that currently, nodes are not edited by the searcher, except upon confirmation, Thus this functionality results in a no-op at the moment.
# Important Notes
[ci no changelog needed]
This PR replaces webpack with esbuild, as our bundler.
The change leads to out-of-the-box ~5x improvement in bundling times, reducing the latency in watch-based workflows.
Along with this a new development server (with live reload capacity) has been introduced to support watch command.
[ci no changelog needed]
### Important Notes
* workflow for checking docs has been removed because it was using outdated prettier version and caused troubles; while the same check is performed in a better way by the GUI/Lint job.
* introduced little more typescript in the scripts in place of js, usually with minimal changes.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181870555)
This PR changes the relative position of the edited node in such a way that it is left-aligned to the component browser window. This change reflects the most recent version of the [design doc](https://github.com/enso-org/design/blob/main/epics/component-browser/design.md#overview)
<img width="1157" alt="Screenshot 2022-08-08 at 19 15 47" src="https://user-images.githubusercontent.com/6566674/183454192-81960e0a-ab69-43a4-b7df-d13320a9d16d.png">
As an additional change, the FRP implementation of the `Camera2d` was extended with a new output (screen dimensions) and fixed. With the old implementation, there was a possibility of panic at runtime because of non-exclusive borrows of `RefCell`. The FRP event for camera position was emited inside the scope with a mutable `RefCell` borrow. Any attempt to borrow the camera one more time (e.g., by calling one of the getters, such as `zoom()`) caused panic at runtime.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/182713338).
This PR fixes the implementation of the entries highlight (selection box) in the component list panel of the component browser. The previous behavior was caused by the dumb implementation of keeping the selection inside the borders of the component list panel. Now the selection is correctly clipped and is hovering over entries at all times.
Screencasts:
https://user-images.githubusercontent.com/6566674/182196585-02ab5ec4-7d12-4d7a-8563-ac95aec3c9f4.mp4https://user-images.githubusercontent.com/6566674/182196768-9d28e83d-6c77-4ef6-abb0-98d50c4e74b7.mp4
# Important Notes
- This PR does not change the fact that after opening the component browser, the selection hovers over a random entry. See a [dedicated task](#182729573) for that.
- The selection box shape was moved from the `component_group` module because it depends heavily on list panel view implementation.
- The selection was removed from the `component_group` demo scene. We have a `component_list_panel_view` demo scene instead.
Show custom icons in Component Browser for entries that have a non-empty `Icon` section in their docs with the section's body containing a name of a predefined icon.
https://www.pivotaltracker.com/story/show/182584336
#### Visuals
A screenshot of a couple custom icons in the Component Browser:
<img width="346" alt="Screenshot 2022-07-27 at 15 55 33" src="https://user-images.githubusercontent.com/273837/181265249-d57f861f-8095-4933-9ef6-e62644e11da3.png">
# Important Notes
- The PR assigns icon names to four items in the standard library, but only three of them are shown in the Component Browser because of [a parsing bug in the Engine](https://www.pivotaltracker.com/story/show/182781673).
- Icon names are assigned only to four items in the standard library because only two currently predefined icons match entries in the currently defined Virtual Component Groups. Adjusting the definitions of icons and Virtual Component Groups is covered by [a different task](https://www.pivotaltracker.com/story/show/182584311).
- A bug in the documentation of the Enso protocol message `DocSection` is fixed. A `text` field in the `Tag` interface is renamed to `body` (this is the field name used in Engine).
This is a partial workaround for https://www.pivotaltracker.com/story/show/182713407
Because some entries' names are too long to be displayed inside the Component List Panel, the user may have no clue what the method is. Therefore, we add a title to each doc page.
There is also code introduced for displaying name of modules when submodule header is selected. However, it does not work, and it's hard to fix without significant changes in the Component List Panel API and implementation. Because those will be altered when implementing [Efficient Component List Panel](https://www.pivotaltracker.com/story/show/182561072), fixing is postponed until then.
# Important Notes
The code contains some significant refactoring. Removed long functions obtaining entries from the component::List - instead you can get group, and then entry from group. They return an option, which is turned to Result in the presenter (where we actually take advantage of having Result instead of Option).
Filter the Virtual Component Groups (a.k.a. "Favorites Data Science Tools") in the `component::List` (a.k.a. Hierarchical Action List) to only contain components with IDs listed in the Engine's response to a `search/completion` request.
This completes the "Virtual Component Groups filtered by input type" task (linked below) because the Engine's response to a `search/completion` request contains IDs of components filtered by input node type and `this` type.
https://www.pivotaltracker.com/story/show/182661634
#### Visuals
See below for a video showing the list of Favorites filtered by the type of the input node. Please note that the video also displays a few known issues that are present in the existing code and not introduced by this PR:
- "Opening the Component Browser 2nd or later time flashes its last contents from the previous time" - reported as [issue 15 in PR 3530](https://github.com/enso-org/enso/pull/3530#pullrequestreview-1035698205).
- The text of all the entries in the Component Browser does not show immediately, but the entries appear one by one instead (this is related to the performance of the current implementation of Component Browser and Text Area).
https://user-images.githubusercontent.com/273837/179000801-65ee7388-dde6-44b9-90fb-7453b4fb788c.mov
A screenshot showing the default, unfiltered list of Favorites when no input node is selected:
<img width="440" alt="Screenshot 2022-07-14 at 15 58 26" src="https://user-images.githubusercontent.com/273837/179000404-f14773a3-35a9-4e7a-877d-fcbb477b4769.png">
**Note**: This PR also contains content of previous Grid View PR. We decided to discard the previous, because this one did some refactoring of old one, and it's not a big addition.
Added a scrollable::GridView component, which just embeds the GridView in ScrollArea. Also, re-worked the idea of text layers.
https://user-images.githubusercontent.com/3919101/179020359-512ee127-c333-4f86-bff5-f1cb4154e03c.mp4
This PR contains all work for finishing integration of first Component List Panel in the IDE:
* It adds a stub for the whole Component Browser View. The documentation panel is re-used from the old searcher.
* It has the presenter implementation, integrating the view with Hierarchical Component List from the controller.
* It extends the View API, so the integration is possible, making use of Component Group Set wrapper.
* The selection integration was also merged into this PR, because it depended on the API extension mentioned above. However, we should avoid such practice in the future.
https://user-images.githubusercontent.com/3919101/177816427-8c4285b4-8941-4048-a400-52f4acf77a9f.mp4
# Important Notes
There are some known issues, to-be-fixed in the future.
* The performance is bad. It should be improved with new text::Area, and the decent one shall come with [GridView inside component browser](https://www.pivotaltracker.com/story/show/182561072)
* There is no keyboard navigation. It should also be delivered with [GridView](https://www.pivotaltracker.com/story/show/182561072).
* The Favorites section is not [filtered out by node source type](https://www.pivotaltracker.com/story/show/182661634).
Implements https://www.pivotaltracker.com/story/show/182633483
Adds functionality to the Searcher Controller to save the expression of a node to its metadata, while it is edited. In later PRs this will be used to restore the expression, if an edit is aborted. This will be needed, as the expression will be changed within the source while editing to allow the visualization preview to be shown. As of this PR, the information is only stored, not used yet.
Demo Video
------------------
Note how the metadata of the node includes a `edit_status` field, when the project is saved during editing, and how it is cleared when saving without an edit happening.
https://user-images.githubusercontent.com/1428930/178740111-d0bbaa43-00b0-42e3-9fbb-d9249f07ae35.mp4
# Important Notes
[ci no changelog needed]
Visualization within the UI is no longer working since the removal of the `here` keyword.
This PR fixes the JavaScript calls.
# Important Notes
There are various lines of rust code which still have `here.` within them and these will need to be fixed at some point.
If a node created by the user gets placed off-screen, the screen's camera is panned to make the node visible.
https://www.pivotaltracker.com/story/show/181188687
#### Visuals
A screencast showing a number of node creation scenarios when the camera is panned to the newly created node, including when zoomed out.
https://user-images.githubusercontent.com/273837/177169716-50a12b0a-c742-4b01-9766-56206e7938b9.mov
# Important Notes
- Camera is panned also if the node is only partially visible, or if there's not enough free space visible around the node. The specific amount of free space that needs to be visible around a newly created node is configured in the theme.
- If the screen area is so small that the node cannot be fully fit in it (either horizontally or vertically), showing the left and top boundaries of the node's area takes priority over showing the corresponding opposite edges.
Implement simple placeholder icons for all entry kinds supported in the Suggestion Database. The icons are planned to be used in Component Browser as default icons for entries. This is intended to allow visually distinguishing different entry kinds.
The following additional fixes and tweaks are applied:
- Icons previously using only 1 color from the theme now use the color provided through shape parameters instead.
- The `data_science` and `network` icons now use only the 2 colors provided through shape parameters.
- The `join` icon has its shape and colors modified and uses only the 2 colors provided through shape parameters.
- The demo scene now parametrizes icon shapes using colors from the Component Browser Design Doc.
https://www.pivotaltracker.com/story/show/182584322
#### Visuals
Original contents of the demo scene before the PR:
<img width="2197" alt="x-orig" src="https://user-images.githubusercontent.com/273837/176669422-ee2e14c7-9ef4-42fd-acb7-ae3be6b68587.png">
Final contents of the demo scene after the PR:
<img width="2201" alt="x2-final" src="https://user-images.githubusercontent.com/273837/176668720-6f1685fd-f7e6-44d7-85f5-f6a6d6789644.png">
Make the local scope components available to the Component Browser through `component::List` (previously known as "Hierarchical Action List"). (See the [Local Scope Section in the Design Doc](https://github.com/enso-org/design/blob/main/epics/component-browser/design.md#local-scope-section) for more details.)
https://www.pivotaltracker.com/story/show/181869186
# Important Notes
- Note: the PR description does not include a screencast due to the changes in this PR not having any visual effect at this moment. The result of this PR's changes would be only observed in the Component Browser, but the Component Browser is not merged yet. As described in the task's Acceptance Criteria, the changes in this PR are currently only covered by tests.
- The `component::group::Data::new_empty_visible` constructor was renamed to `from_name_and_id` and changed to set the `visible` field to a default value. All known code paths appear to call the `update_sorting_and_visibility` method before checking the value of the `visible` field, so setting the value to `true` when creating the object does not seem needed.
[ci no changelog needed]
A semi-manual s/this/self appied to the whole standard library.
Related to https://www.pivotaltracker.com/story/show/182328601
In the compiler promoted to use constants instead of hardcoded
`this`/`self` whenever possible.
# Important Notes
The PR **does not** require explicit `self` parameter declaration for methods as this part
of the design is still under consideration.
[Task link](https://www.pivotaltracker.com/story/show/182194574).
[ci no changelog needed]
This PR implements a new selection box that will replace an old (not really working) one in the component browser. The old selection box wasn't working well with the headers of the component groups, so we were forced to make a much harder implementation.
The new implementation duplicates some visual components and places them in a separate layer. Then, a rectangular mask cuts off everything that is not "selected". This way:
- We have more control over what the selected entries should look like.
- We can easily support the multi-layer structure of the component groups with headers.
- We avoid problems with nested masks that our renderer doesn't support at the moment.
To be more precise, we duplicate the following:
- Background of the component group becomes the "fill" of the selection.
- Entries text and icons - we can alter them easily.
- Header background and header text. By placing them in separate scene layers we ensure correct rendering order.
https://user-images.githubusercontent.com/6566674/173657899-1067f538-4329-44f9-9dc2-78c8a4708b5a.mp4
# Important Notes
- This PR implements the base of our future selection mechanism, selecting entries with a mouse and keyboard still has several issues that will be fixed in the future tasks.
- The scrolling behavior will also be improved in future tasks. Right we only restrict the selection box position so that it never leaves the borders of the component group.
- I added a new function to `add` shapes to new layers in a non-exclusive way (we had only `add_exclusive`) before. I have no idea how we didn't use this feature before even though we mention it a lot in the docs.
- The demo scene restricts the position of the selection box for one-column component groups but does not for the wide component group.
Put information about Virtual Component Groups in the Hierarchical Actions List.
https://www.pivotaltracker.com/story/show/181865548
# Important Notes
- This PR implements the subtask 2 of 2 in the ["Virtual Component Groups in the Hierarchical Action List" task](https://www.pivotaltracker.com/story/show/181865548).
- Note: the PR description does not include a screencast due to the changes in this PR not having any visual effect at this moment. The result of this PR's changes would be only observed in the Component Browser, but the Component Browser is not merged yet. As described in the task's Acceptance Criteria, the changes in this PR are currently only covered by tests.
- Manual integration testing with the Engine showed that a response to an `executionContext/getComponentGroups` request is non-empty only after an `executionContext/executionComplete` message is received by the IDE. (See also [a discussion on Discord](https://discord.com/channels/401396655599124480/983669600854106112).) This was not known by the IDE team or documented before, and the existing code was modified to take this into account. The protocol docs are expected to be updated by the Engine team.
- A list of component groups looked up in the suggestion database is cached in the node searcher as an optimization.
[ci no changelog needed]
Remove a `Symbol`from the `SymbolRegistry` when its `SpriteSystem` is dropped.
This fixes the remaining buffer leak (after #3504) in https://www.pivotaltracker.com/story/show/181943457
# Important Notes
- The `SymbolRegistry` now assigns unique `SymbolId`s, so that we can tell if a `SymbolId` refers to a `Symbol` that has already been unregistered (this shouldn't happen, but it's not statically-obvious that it doesn't, so if it occurs we shouldn't misbehave).
- Also fix a bug in how `buffer_count` was tracked (we were decrementing more than incrementing!).
### Pull Request Description
Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`):
- `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`.
- `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental.
(See key in first chart for the settings defining these profiles.)
We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes:
- `release`: A profile that supports fast developer iteration, while offering realistic performance.
- `production`: A maximally-optimized profile, for nightly builds and actual releases.
Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags.
### Performance details
As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster.
<details>
<summary>Methodology (click to show)</summary>
I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data:
Build the application with profiling:
`./run.sh ide build --profiling-level=debug`
Run the `open_project` workflow repeatedly:
`for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done`
For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV:
`echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do`
(Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...)
The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data.
</details>
#### Build times
In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds.
(To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
[ci no changelog needed]
This PR implements a new helper for the future Component Browser - `component_group::multi::Wrapper`. It propagates FRP events from multiple component groups and ensures that only a single component group is focused at all times.
See the updated component group demo scene (console logs shows propagated FRP events from all component groups):
https://user-images.githubusercontent.com/6566674/172359141-8ea6f1ba-e357-4c1b-852a-adb4d5207e03.mp4
- Fixed a `define_endpoints_2!` macro. FRP endpoints for `focus` events weren't connected properly.
- List View now uses an overlay shape to catch mouse events, it allows much easier implementation of `is_header_selected` in the component group.
This PR introduces a new structure delivered by Searcher Controller: The Component List.
The Component list is hierarchical, and its structure reflects how the components are displayed in Component Browser - only "submodule" section at this point, other sections will be covered in next tasks.
This does not introduce anything visual; the structures are tested in unit tests.
Fix a memory leak introduced by #3451. Found this to be part of the cause of [the buffer leaks](https://www.pivotaltracker.com/story/show/181943457/comments/231558434).
The problem with this use of `define_endpoints_2` is,
- The FRP network contains (strong) references to the `Model`.
- The `Model` has a strong reference to `api::Private`, which owns the FRP network.
Thus we never free the `Model`.
