This PR is a draft PR while I learn EnsoGL. The eventual goal is to implement the projects list portion of the cloud dashboard in this PR. This PR will implement part of https://www.pivotaltracker.com/n/projects/2539513/stories/183557950
### Important Notes
This PR is still really rough and contains a lot of hacks & hard-coded values. The FRP usage is also likely to be suboptimal and need fixing.
Save a snapshot of the project directory to the VCS on `ctrl+s`. If the operation fails because the VCS was not initialized previously, it will try to initialize the VCS first and then save a snapshot.
This PR fixes the `code_to_insert` method of entry to insert valid code according to the newest language version. Also created a separate method for getting imports required by given entry.
Now, method entries do not add imports (except when they are extensions), and are insterted with place for this type `_.method`. Static methods and constructors are inserted with the type name, and proper import for type is added.
There are some additional work done:
* The ReferentName and NormalizedName were removed, as we are now case-sensitive.
* All QualifiedName structures were replaced with new one in `name` module, as there is no longer functional difference between type qualified name and module qualified name.
* The QualifiedName structure removes "Main" module segment where it is not necessary, thus simplifying our code base and avoiding potential issues.
* Added macro `mock_suggestion_database` which should make creating consistent mocks of SuggestionDatabase much simpler.
* Fixed bug where the visualization preview show no value for some time.
https://user-images.githubusercontent.com/3919101/202750275-0d378d5f-1482-4637-bdcd-c428a9eac0d4.mp4
# Important Notes
The tests in controller/searcher.rs file are not of the best quality, but those will be overhauled anyway when implementing my next task.
This is a part 1 of the fix for https://discordapp.com/channels/401396655599124480/1041669067188219914/1041669067188219914
Every time the node would have a connection going into a lambda body, the entire lambda in the destination node was replaced with the input variable. That because the lambda for some reason is not decomposed into span tree, and the presenter created a connection going to the port spanned over the entire lambda, and then thought this was a connection created by the user.
Such connections, going into "inside" of the span tree, should not break nodes expression, so they are not displayed at all after this fix. The proper fix will be making span-tree lambda decomposition, but it will be a next PR.
Fixes regression about inaccessible node output port. https://www.pivotaltracker.com/n/projects/2539304/stories/183792368
Additionally, a related issue related to nodes not being visible after initial load has been uncovered and fixed in the process.
# Important Notes
Previously the output shape size was only updated when node size itself was updated. When node's expression was changed, the output port view is recreated from scratch, but the current node size has not been propagated to the newly created port. In some cases the node size was changed shortly after, masking the bug. In other cases, the newly created shape size was never set.
The port size management has been moved from model method into frp network, and the initial size value is emitted during network initialization.
Fixes [#183511669](https://www.pivotaltracker.com/story/show/183511669)
This PR improves the `ImportInfo` structure from Double Representation crate so it provides information what names are brought into the scope with this import. The from_ast method also recognized [various ways of defining imports](https://github.com/enso-org/enso/blob/develop/docs/syntax/imports.md)
# Important Notes
The parser seems to not recognize properly imports with aliases. I have not fixed that, as we expect a new parser to be merged very soon.
Fixes https://www.pivotaltracker.com/story/show/182926584
[Task link](https://www.pivotaltracker.com/story/show/183426449)
This PR fixes an IDE freeze introduced by https://github.com/enso-org/enso/pull/3732 and reimplements reverting edited nodes to their previous state.
The cause of the IDE freeze is quite interesting. A detailed investigation is available [here](https://gist.github.com/vitvakatu/785e34881368b8cfda61715d7543cbd0).
The graph editor needs to update the Presenter state only if the user is editing the node. Before this PR, the graph editor notified the Presenter with a visual representation of the node content instead of code expression. It caused inconsistency between the states of the controller and Presenter and caused severe performance issues.
https://user-images.githubusercontent.com/6566674/195831224-6d6e8258-e347-48b4-890a-d89c7300bc39.mp4
# Important Notes
- ~~There is a more complex alternative solution – it requires refactoring of the `component::node::input::area` module. The Presenter can be notified with `expression.code` changes, not `expression.viz_code`. I found a simpler solution (`.gate(&edit_mode)`), which has the same effect but does not require additional refactoring.~~ Said solution is implemented in a separate commit
When hovering the mouse pointer over the Marketplace button on the left bar of the Component Browser, show a caption informing that the Marketplace will be available soon.
https://www.pivotaltracker.com/story/show/182613789
#### Visuals
The video below demonstrates the caption shown when hovering the Marketplace button on the left bar of the Component Browser. It shows the caption disappearing after a hardcoded time, or when the mouse pointer is moved away from the button.
https://user-images.githubusercontent.com/273837/196195809-45a712e1-ad86-47d8-99ff-1475a0b74c6e.mov
# Important Notes
- The "Label" visual component was fixed. Previously, the width calculation of the background was not synchronized correctly with the text width. As a result, a zero-width background was displayed when a Label was shown for the first time.
This PR implements displaying a currently active section name as a first crumb in the breadcrumbs panel. Sections are called `Popular`, `Modules` and `Local`.
https://user-images.githubusercontent.com/6566674/194551276-90bd7d6b-8509-43ec-b3c0-11c35fda9063.mp4
# Important Notes
This PR also contains a fix for [this bug](https://www.pivotaltracker.com/story/show/183499312). It was caused by mistake in the FRP implementation of the breadcrumbs. You could only trigger this bug after code changes in the `animation/loop.rs` module. It is not possible to see it otherwise. Still, the code was not correct, and now it is fixed.
This PR introduced an overhauled Component List Panel implementation, making use of the efficient EnsoGL grid view component. Also, it delivers a couple of new features:
* A part of the new design: there are no more section headers in grid, instead groups are "glued" together. The local scope section is under "popular" (old "favorites").
* The keyboard management inside grid works.
* there is a mouse hover highlight
* selecting the lowest entry in section when jumping with navigation bar.
* accepting input as-is with cmd/ctrl + Enter.
https://user-images.githubusercontent.com/3919101/194561890-fffb9b41-2f0d-4357-8d9a-5038a6bcb023.mp4
### Important Notes
**What is not implemented:**
* [Focus management between panels.](https://www.pivotaltracker.com/story/show/180872763) The grid is always focused. To accept the current input, use ctrl+Enter shortcut.
* [Proper handling of selection when having empty space on the right and pressing right arrow.](https://www.pivotaltracker.com/story/show/183487880)
* When entering a module, its name is not added to the input as described in the design doc. Will be a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
**Known issues**
* [the selection, especially in the local scope section, has sometimes an undesirable offset](https://www.pivotaltracker.com/story/show/183487730). The cause is known, but not so easy to fix.
* The inserted nodes are often producing errors. The Browser's inherits the outdated understanding of the language from old Node Searcher, and it does not include new form of imports, static methods etc. Those all will be fixed as a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
* The performance is improved, but still not ideal, due to problems in [text areas](https://www.pivotaltracker.com/story/show/183406745).
* To scroll the documentation panel, you must first click on it.
There was a regression introduced by PR #3678 with text synchronization between IDE and the Engine. This PR fixes it.
It was reproducible as an error log about version mismatch in a case when the metadata becomes shorter (e.g. on removing node).
# Important Notes
[ci no changelog needed]
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/182675703)
This PR implements the actual integration of the breadcrumbs panel with the component list panel. A special breadcrumbs controller (`controller::searcher::breadcrumbs`) is tracking the currently opened module with a list of its parents. The searcher presenter uses the API of the controller to sync the displayed list of breadcrumbs with the controller state.
https://user-images.githubusercontent.com/6566674/193064122-7d3fc4d6-9148-4ded-a73e-767ac9ac83f8.mp4
# Important Notes
- There is an `All` breadcrumb displayed at all times at the beginning of the list. It will be replaced with a section name as part of [Section Title on Component Browser's Breadcrumbs Panel](https://www.pivotaltracker.com/story/show/182610561) task.
- I changed the implementation of `project::main_module_id`, `project::QualifiedName::main_module`, and `API::main_module` so that they are logically connected which each other.
- I adjusted the Breadcrumbs View to avoid "appearance" animation glitches when opening new modules. `set_entries` was replaced with the `set_entries_from` endpoint.
When nodes get invalidated in the cache, they have to be recomputed. Let the IDE know which of the nodes are pending by sending `Api.ExpressionUpdate.Payload.Pending` message.
# Important Notes
This PR introduces new `Api.ExpressionUpdate.Payload.Pending` message. This message is delivered before re-computation of nodes. Later `Api.ExpressionUpdate.Payload.Value` or other is sent to notify the IDE that a value for given node is available.
Trivial implementation of of the `Api.ExpressionUpdate.Payload.Pending` message in the IDE is provided by this PR to (improperly) visualize pending node status - further improvements needed in follow up PRs.
This PR disables the wasm-opt optimization in the crates that can be used as WASM entry points. Unfortunately, wasm-pack does not allow for disabling wasm-opt through a command line flag, so we have to disable it by setting an appropriate flag in each Cargo.toml.
Show the visualization when a newly created node is being edited. Hide it after the editing is finished, unless the user explicitly changed the state of the visualization.
https://www.pivotaltracker.com/story/show/182634059
#### Visuals
The following video shows the behavior of the visualization when a new node is created by the user and is being edited:
- By default, the visualization is shown when the node becomes edited and disappears when the node's expression is accepted.
- If the user clicks the visualization toggle button once while editing the node, the visualization disappears and stays disabled after the node's expression is accepted.
- If the user clicks the visualization toggle button twice while editing the node, the visualization is visible and stays enabled after the node's expression is accepted.
This PR is not concerned with the correctness of the contents of the visualization window. This is in line with the current behavior on the `develop` branch, where clicking the visualization toggle button while editing a newly created node shows `null` content in the visualization (even if the edited expression is not null).
https://user-images.githubusercontent.com/273837/191294970-62143983-764a-446c-9616-9765de0923bf.mov
This PR contains an entry definition for Grid View to be used inside Component List Panel View. The Example grid view with the entry definition may be seen on new_component_list_panel_view debug scene.
https://user-images.githubusercontent.com/3919101/190663278-23c35ab0-f426-4001-8128-df7147aafb9e.mp4
# Important Notes
* The styling is not detailed yet due to time constraints (I want to move to integration this grid view to Component Panel List ASAP) and the fact that I could not get new mplus1 font working with text Area.
* Implementing this required adding a "contour offset" feature to the Grid View.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181870304).
This PR implements Controller-level support for entering modules in the component browser.
Clicking headers/entries opens the module, filling the Local Scope and Submodules sections.
Please ignore multiple visual issues with the component browser on the screen - this PR does not cause them, and they will be fixed soon when we change the implementation of the component list panel to Grid View.
https://user-images.githubusercontent.com/6566674/190933275-c544ea8d-529d-4b49-820f-d36874344572.mp4
# Important Notes
- We will implement the actual breadcrumbs integration in a separate PR: https://www.pivotaltracker.com/story/show/182675703
- The FRP changes in the component list panel code are temporary. The GridView-based implementation will later replace most of the boilerplate.
Repairing the constructor name following the types work. Some general tiding up as well.
- Remove `Standard.Database.Data.Column.Aggregate_Column_Builder`.
- Remove `Standard.Database.Data.Dialect.Dialect.Dialect_Data`.
- Remove unused imports and update some type definitions.
- Rename `Postgres.Postgres_Data` => `Postgres_Options.Postgres`.
- Rename `Redshift.Redshift_Data` => `Redshift_Options.Redshift`.
- Rename `SQLite.SQLite_Data` => `SQLite_Options.SQLite`.
- Rename `Credentials.Credentials_Data` => `Credentials.Username_And_Password`.
- Rename `Sql` to `SQL` across the board.
- Merge `Standard.Database.Data.Internal` into `Standard.Database.Internal`.
- Move dialects into `Internal` and merge the function in `Helpers` into `Base_Generator`.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181445628).
This PR implements a Breadcrumbs panel for the new component browser.
The Breadcrumbs is a horizontal list of text labels separated by a special icon and has an optional ellipsis icon at the end.
It is implemented using the new GridView component.
Video:
Demo of adding new breadcrumbs, scrolling behavior, and selecting breadcrumbs with the mouse.
https://user-images.githubusercontent.com/6566674/189199432-77807cef-00dc-4abe-b95c-b17a536f59f6.mp4
Demo of selecting breadcrumbs with keyboard shortcuts:
https://user-images.githubusercontent.com/6566674/189199603-53e55335-73ba-4ed7-8291-4455144c06aa.mp4
# Important Notes
- This PR implements an old interaction of the design of the component browser. The new design of the breadcrumbs can not be easily integrated into the current look of the component browser, so we would need to update styles later. It should be a relatively simple task. *The implementation uses color from the new design though. (but not fonts and sizes)*
- I found a bug in the grid view implementation that causes panics at runtime in some conditions. The reason is triggering FRP endpoints while constructing new entries. This issue is fixed in the PR.
Avoid long lines when using `dom().set_inner_text` - rather split the long lines to 1024 chunks and insert them as individual `<div>` elements.
# Important Notes
I was testing the behavior on following program:
```
from Standard.Base import all
import Standard.Base.Data.Statistics
import Standard.Visualization
main =
number1 = 200000
operator1 = 0.up_to number1 . to_vector . map .noise
operator2 = operator1.sort
operator3 = operator2.to_text
```
before my change the visualization of `operator3` was blank. With my change it gets filled with data.
This PR does not add anything visual for the user: it's a first part of making efficient Component Browser using fresh Grid View implementation.
The Layout is an intermediate model, which keeps the information how the groups are laid out in the Component browser, and allows querying for group entry by location and location of specific group entry (or header).
The layout is meant for the new design, where there are no section separators and Local Scope section is below Favorites.
# Important Notes
The new structure is not used in action, but tested with unit tests.
First of all this PR demonstrates how to implement _lazy visualization_:
- one needs to write/enhance Enso visualization libraries - this PR adds two optional parameters (`bounds` and `limit`) to `process_to_json_text` function.
- the `process_to_json_text` can be tested by standard Enso test harness which this PR also does
- then one has to modify JavaScript on the IDE side to construct `setPreprocessor` expression using the optional parameters
The idea of _scatter plot lazy visualization_ is to limit the amount of points the IDE requests. Initially the limit is set to `limit=1024`. The `Scatter_Plot.enso` then processes the data and selects/generates the `limit` subset. Right now it includes `min`, `max` in both `x`, `y` axis plus randomly chosen points up to the `limit`.
The D3 visualization widget is capable of _zooming in_. When that happens the JavaScript widget composes new expression with `bounds` set to the newly visible area. By calling `setPreprocessor` the engine recomputes the visualization data, filters out any data outside of the `bounds` and selects another `limit` points from the new data. The IDE visualization then updates itself to display these more detailed data. Users can zoom-in to see the smallest detail where the number of points gets bellow `limit` or they can select _Fit all_ to see all the data without any `bounds`.
# Important Notes
Randomly selecting `limit` samples from the dataset may be misleading. Probably implementing _k-means clustering_ (where `k=limit`) would generate more representative approximation.
So far, when opening the searcher with no node selected, an empty input without an associated AST node was created. This input was manipulated by the user and the final expression from the input was used to create a new node when the user confirmed their input. This PR changes this, so that when the searcher is opened without a selected node, a new AST node is created right away with some placeholder content, and this node is updated when the user confirms their input.
The only change visible to the user, is that if the text editor is opened during editing, a new node will appear in the source when the searcher is opened to create a new node. All other behaviour should stay the same.
[ci no changelog needed]
This PR fixes a [regression](https://www.pivotaltracker.com/story/show/182972359) found in the `interface` debug the scene. It was caused by multiple `Navigator`s that were present in the demo scene and conflict with each other.
Also, this PR includes a fix for the invalid logic of the new component browser. We disable `Navigator` when the mouse hovers the component browser. Still, due to a mistake in the code, the component browser was considered visible at all times and therefore blocked the navigator in certain mouse positions.
To reproduce this bug (before this PR):
1. Open a default project
2. Place the mouse somewhere in the middle of the screen (near one of the nodes)
3. Try panning or zooming the scene (you won't be able because of this bug)
Add functionality to revert a node that was changed during editing through the searcher.
Note that currently, nodes are not edited by the searcher, except upon confirmation, Thus this functionality results in a no-op at the moment.
# Important Notes
[ci no changelog needed]
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181870555)
This PR changes the relative position of the edited node in such a way that it is left-aligned to the component browser window. This change reflects the most recent version of the [design doc](https://github.com/enso-org/design/blob/main/epics/component-browser/design.md#overview)
<img width="1157" alt="Screenshot 2022-08-08 at 19 15 47" src="https://user-images.githubusercontent.com/6566674/183454192-81960e0a-ab69-43a4-b7df-d13320a9d16d.png">
As an additional change, the FRP implementation of the `Camera2d` was extended with a new output (screen dimensions) and fixed. With the old implementation, there was a possibility of panic at runtime because of non-exclusive borrows of `RefCell`. The FRP event for camera position was emited inside the scope with a mutable `RefCell` borrow. Any attempt to borrow the camera one more time (e.g., by calling one of the getters, such as `zoom()`) caused panic at runtime.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/182713338).
This PR fixes the implementation of the entries highlight (selection box) in the component list panel of the component browser. The previous behavior was caused by the dumb implementation of keeping the selection inside the borders of the component list panel. Now the selection is correctly clipped and is hovering over entries at all times.
Screencasts:
https://user-images.githubusercontent.com/6566674/182196585-02ab5ec4-7d12-4d7a-8563-ac95aec3c9f4.mp4https://user-images.githubusercontent.com/6566674/182196768-9d28e83d-6c77-4ef6-abb0-98d50c4e74b7.mp4
# Important Notes
- This PR does not change the fact that after opening the component browser, the selection hovers over a random entry. See a [dedicated task](#182729573) for that.
- The selection box shape was moved from the `component_group` module because it depends heavily on list panel view implementation.
- The selection was removed from the `component_group` demo scene. We have a `component_list_panel_view` demo scene instead.
Show custom icons in Component Browser for entries that have a non-empty `Icon` section in their docs with the section's body containing a name of a predefined icon.
https://www.pivotaltracker.com/story/show/182584336
#### Visuals
A screenshot of a couple custom icons in the Component Browser:
<img width="346" alt="Screenshot 2022-07-27 at 15 55 33" src="https://user-images.githubusercontent.com/273837/181265249-d57f861f-8095-4933-9ef6-e62644e11da3.png">
# Important Notes
- The PR assigns icon names to four items in the standard library, but only three of them are shown in the Component Browser because of [a parsing bug in the Engine](https://www.pivotaltracker.com/story/show/182781673).
- Icon names are assigned only to four items in the standard library because only two currently predefined icons match entries in the currently defined Virtual Component Groups. Adjusting the definitions of icons and Virtual Component Groups is covered by [a different task](https://www.pivotaltracker.com/story/show/182584311).
- A bug in the documentation of the Enso protocol message `DocSection` is fixed. A `text` field in the `Tag` interface is renamed to `body` (this is the field name used in Engine).
This is a partial workaround for https://www.pivotaltracker.com/story/show/182713407
Because some entries' names are too long to be displayed inside the Component List Panel, the user may have no clue what the method is. Therefore, we add a title to each doc page.
There is also code introduced for displaying name of modules when submodule header is selected. However, it does not work, and it's hard to fix without significant changes in the Component List Panel API and implementation. Because those will be altered when implementing [Efficient Component List Panel](https://www.pivotaltracker.com/story/show/182561072), fixing is postponed until then.
# Important Notes
The code contains some significant refactoring. Removed long functions obtaining entries from the component::List - instead you can get group, and then entry from group. They return an option, which is turned to Result in the presenter (where we actually take advantage of having Result instead of Option).
Filter the Virtual Component Groups (a.k.a. "Favorites Data Science Tools") in the `component::List` (a.k.a. Hierarchical Action List) to only contain components with IDs listed in the Engine's response to a `search/completion` request.
This completes the "Virtual Component Groups filtered by input type" task (linked below) because the Engine's response to a `search/completion` request contains IDs of components filtered by input node type and `this` type.
https://www.pivotaltracker.com/story/show/182661634
#### Visuals
See below for a video showing the list of Favorites filtered by the type of the input node. Please note that the video also displays a few known issues that are present in the existing code and not introduced by this PR:
- "Opening the Component Browser 2nd or later time flashes its last contents from the previous time" - reported as [issue 15 in PR 3530](https://github.com/enso-org/enso/pull/3530#pullrequestreview-1035698205).
- The text of all the entries in the Component Browser does not show immediately, but the entries appear one by one instead (this is related to the performance of the current implementation of Component Browser and Text Area).
https://user-images.githubusercontent.com/273837/179000801-65ee7388-dde6-44b9-90fb-7453b4fb788c.mov
A screenshot showing the default, unfiltered list of Favorites when no input node is selected:
<img width="440" alt="Screenshot 2022-07-14 at 15 58 26" src="https://user-images.githubusercontent.com/273837/179000404-f14773a3-35a9-4e7a-877d-fcbb477b4769.png">
**Note**: This PR also contains content of previous Grid View PR. We decided to discard the previous, because this one did some refactoring of old one, and it's not a big addition.
Added a scrollable::GridView component, which just embeds the GridView in ScrollArea. Also, re-worked the idea of text layers.
https://user-images.githubusercontent.com/3919101/179020359-512ee127-c333-4f86-bff5-f1cb4154e03c.mp4
This PR contains all work for finishing integration of first Component List Panel in the IDE:
* It adds a stub for the whole Component Browser View. The documentation panel is re-used from the old searcher.
* It has the presenter implementation, integrating the view with Hierarchical Component List from the controller.
* It extends the View API, so the integration is possible, making use of Component Group Set wrapper.
* The selection integration was also merged into this PR, because it depended on the API extension mentioned above. However, we should avoid such practice in the future.
https://user-images.githubusercontent.com/3919101/177816427-8c4285b4-8941-4048-a400-52f4acf77a9f.mp4
# Important Notes
There are some known issues, to-be-fixed in the future.
* The performance is bad. It should be improved with new text::Area, and the decent one shall come with [GridView inside component browser](https://www.pivotaltracker.com/story/show/182561072)
* There is no keyboard navigation. It should also be delivered with [GridView](https://www.pivotaltracker.com/story/show/182561072).
* The Favorites section is not [filtered out by node source type](https://www.pivotaltracker.com/story/show/182661634).
Implements https://www.pivotaltracker.com/story/show/182633483
Adds functionality to the Searcher Controller to save the expression of a node to its metadata, while it is edited. In later PRs this will be used to restore the expression, if an edit is aborted. This will be needed, as the expression will be changed within the source while editing to allow the visualization preview to be shown. As of this PR, the information is only stored, not used yet.
Demo Video
------------------
Note how the metadata of the node includes a `edit_status` field, when the project is saved during editing, and how it is cleared when saving without an edit happening.
https://user-images.githubusercontent.com/1428930/178740111-d0bbaa43-00b0-42e3-9fbb-d9249f07ae35.mp4
# Important Notes
[ci no changelog needed]
Visualization within the UI is no longer working since the removal of the `here` keyword.
This PR fixes the JavaScript calls.
# Important Notes
There are various lines of rust code which still have `here.` within them and these will need to be fixed at some point.
If a node created by the user gets placed off-screen, the screen's camera is panned to make the node visible.
https://www.pivotaltracker.com/story/show/181188687
#### Visuals
A screencast showing a number of node creation scenarios when the camera is panned to the newly created node, including when zoomed out.
https://user-images.githubusercontent.com/273837/177169716-50a12b0a-c742-4b01-9766-56206e7938b9.mov
# Important Notes
- Camera is panned also if the node is only partially visible, or if there's not enough free space visible around the node. The specific amount of free space that needs to be visible around a newly created node is configured in the theme.
- If the screen area is so small that the node cannot be fully fit in it (either horizontally or vertically), showing the left and top boundaries of the node's area takes priority over showing the corresponding opposite edges.
Implement simple placeholder icons for all entry kinds supported in the Suggestion Database. The icons are planned to be used in Component Browser as default icons for entries. This is intended to allow visually distinguishing different entry kinds.
The following additional fixes and tweaks are applied:
- Icons previously using only 1 color from the theme now use the color provided through shape parameters instead.
- The `data_science` and `network` icons now use only the 2 colors provided through shape parameters.
- The `join` icon has its shape and colors modified and uses only the 2 colors provided through shape parameters.
- The demo scene now parametrizes icon shapes using colors from the Component Browser Design Doc.
https://www.pivotaltracker.com/story/show/182584322
#### Visuals
Original contents of the demo scene before the PR:
<img width="2197" alt="x-orig" src="https://user-images.githubusercontent.com/273837/176669422-ee2e14c7-9ef4-42fd-acb7-ae3be6b68587.png">
Final contents of the demo scene after the PR:
<img width="2201" alt="x2-final" src="https://user-images.githubusercontent.com/273837/176668720-6f1685fd-f7e6-44d7-85f5-f6a6d6789644.png">
Make the local scope components available to the Component Browser through `component::List` (previously known as "Hierarchical Action List"). (See the [Local Scope Section in the Design Doc](https://github.com/enso-org/design/blob/main/epics/component-browser/design.md#local-scope-section) for more details.)
https://www.pivotaltracker.com/story/show/181869186
# Important Notes
- Note: the PR description does not include a screencast due to the changes in this PR not having any visual effect at this moment. The result of this PR's changes would be only observed in the Component Browser, but the Component Browser is not merged yet. As described in the task's Acceptance Criteria, the changes in this PR are currently only covered by tests.
- The `component::group::Data::new_empty_visible` constructor was renamed to `from_name_and_id` and changed to set the `visible` field to a default value. All known code paths appear to call the `update_sorting_and_visibility` method before checking the value of the `visible` field, so setting the value to `true` when creating the object does not seem needed.
[ci no changelog needed]
A semi-manual s/this/self appied to the whole standard library.
Related to https://www.pivotaltracker.com/story/show/182328601
In the compiler promoted to use constants instead of hardcoded
`this`/`self` whenever possible.
# Important Notes
The PR **does not** require explicit `self` parameter declaration for methods as this part
of the design is still under consideration.
[Task link](https://www.pivotaltracker.com/story/show/182194574).
[ci no changelog needed]
This PR implements a new selection box that will replace an old (not really working) one in the component browser. The old selection box wasn't working well with the headers of the component groups, so we were forced to make a much harder implementation.
The new implementation duplicates some visual components and places them in a separate layer. Then, a rectangular mask cuts off everything that is not "selected". This way:
- We have more control over what the selected entries should look like.
- We can easily support the multi-layer structure of the component groups with headers.
- We avoid problems with nested masks that our renderer doesn't support at the moment.
To be more precise, we duplicate the following:
- Background of the component group becomes the "fill" of the selection.
- Entries text and icons - we can alter them easily.
- Header background and header text. By placing them in separate scene layers we ensure correct rendering order.
https://user-images.githubusercontent.com/6566674/173657899-1067f538-4329-44f9-9dc2-78c8a4708b5a.mp4
# Important Notes
- This PR implements the base of our future selection mechanism, selecting entries with a mouse and keyboard still has several issues that will be fixed in the future tasks.
- The scrolling behavior will also be improved in future tasks. Right we only restrict the selection box position so that it never leaves the borders of the component group.
- I added a new function to `add` shapes to new layers in a non-exclusive way (we had only `add_exclusive`) before. I have no idea how we didn't use this feature before even though we mention it a lot in the docs.
- The demo scene restricts the position of the selection box for one-column component groups but does not for the wide component group.
Put information about Virtual Component Groups in the Hierarchical Actions List.
https://www.pivotaltracker.com/story/show/181865548
# Important Notes
- This PR implements the subtask 2 of 2 in the ["Virtual Component Groups in the Hierarchical Action List" task](https://www.pivotaltracker.com/story/show/181865548).
- Note: the PR description does not include a screencast due to the changes in this PR not having any visual effect at this moment. The result of this PR's changes would be only observed in the Component Browser, but the Component Browser is not merged yet. As described in the task's Acceptance Criteria, the changes in this PR are currently only covered by tests.
- Manual integration testing with the Engine showed that a response to an `executionContext/getComponentGroups` request is non-empty only after an `executionContext/executionComplete` message is received by the IDE. (See also [a discussion on Discord](https://discord.com/channels/401396655599124480/983669600854106112).) This was not known by the IDE team or documented before, and the existing code was modified to take this into account. The protocol docs are expected to be updated by the Engine team.
- A list of component groups looked up in the suggestion database is cached in the node searcher as an optimization.
[ci no changelog needed]
Remove a `Symbol`from the `SymbolRegistry` when its `SpriteSystem` is dropped.
This fixes the remaining buffer leak (after #3504) in https://www.pivotaltracker.com/story/show/181943457
# Important Notes
- The `SymbolRegistry` now assigns unique `SymbolId`s, so that we can tell if a `SymbolId` refers to a `Symbol` that has already been unregistered (this shouldn't happen, but it's not statically-obvious that it doesn't, so if it occurs we shouldn't misbehave).
- Also fix a bug in how `buffer_count` was tracked (we were decrementing more than incrementing!).
### Pull Request Description
Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`):
- `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`.
- `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental.
(See key in first chart for the settings defining these profiles.)
We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes:
- `release`: A profile that supports fast developer iteration, while offering realistic performance.
- `production`: A maximally-optimized profile, for nightly builds and actual releases.
Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags.
### Performance details
As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster.
<details>
<summary>Methodology (click to show)</summary>
I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data:
Build the application with profiling:
`./run.sh ide build --profiling-level=debug`
Run the `open_project` workflow repeatedly:
`for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done`
For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV:
`echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do`
(Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...)
The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data.
</details>
#### Build times
In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds.
(To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
[ci no changelog needed]
This PR implements a new helper for the future Component Browser - `component_group::multi::Wrapper`. It propagates FRP events from multiple component groups and ensures that only a single component group is focused at all times.
See the updated component group demo scene (console logs shows propagated FRP events from all component groups):
https://user-images.githubusercontent.com/6566674/172359141-8ea6f1ba-e357-4c1b-852a-adb4d5207e03.mp4
- Fixed a `define_endpoints_2!` macro. FRP endpoints for `focus` events weren't connected properly.
- List View now uses an overlay shape to catch mouse events, it allows much easier implementation of `is_header_selected` in the component group.
This PR introduces a new structure delivered by Searcher Controller: The Component List.
The Component list is hierarchical, and its structure reflects how the components are displayed in Component Browser - only "submodule" section at this point, other sections will be covered in next tasks.
This does not introduce anything visual; the structures are tested in unit tests.
Fix a memory leak introduced by #3451. Found this to be part of the cause of [the buffer leaks](https://www.pivotaltracker.com/story/show/181943457/comments/231558434).
The problem with this use of `define_endpoints_2` is,
- The FRP network contains (strong) references to the `Model`.
- The `Model` has a strong reference to `api::Private`, which owns the FRP network.
Thus we never free the `Model`.
Define some workflows for batch-mode profiling.
Implemented:
- collapse nodes
- create node
- enter collapsed node
- new project
- open visualization
They can currently be built and run with a command like:
`./run.sh ide build --profiling-level=debug && dist/ide/linux-unpacked/enso --entry-point profile --workflow create_node --save-profile out.json`
And the data can be displayed with:
`dist/ide/linux-unpacked/enso --entry-point profiling_run_graph --load-profile out.json`
Demo of recording and viewing a profile with a command-line one-liner:
https://user-images.githubusercontent.com/1047859/169954795-2d9520ca-84f9-45d2-b83a-5063ebe6f718.mp4
See: https://www.pivotaltracker.com/story/show/182195399.
# Important Notes
- When defining workflows, two helpers are enough to allow us to tell when the action is really done: `Fixture::compile_new_shaders`, and `Fixture::backend_execution`. Often, it is appropriate to await both, but it depends on the task.
- The shader compiler is now driven by a `Controller`; while the `Compiler` is reset if context is lost, the `Controller`'s state survives context loss.
- A new `--load-profile` option supports specifying a profile by path when running `profiling_run_graph`.
- Drop the `with_same_start` profiler interface; we ended up preferring a child profiler convention, and this interface was not implemented compatibly with the stricter data model we've had since the introduction of `profiler::data`.
- Fix the noisy `rustfmt` output.
* The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD).
* Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`.
* Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted.
* Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.)
* If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue.
* Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work.
* Whitelisting dependabot from changelog checks, so PRs created by it are mergeable.
* Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds.
* Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax.
* Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there.
Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
This PR contains minimal integration with new engine's method and an integration test printing the method's return value. It was written as a part of https://www.pivotaltracker.com/story/show/181743571
# Important Notes
The test requires 2022.1.1-nightly.2022-04-26 engine version or later.
Add a method in `SuggestionDatabase` allowing to find a suggestion entry by a fully qualified path, working faster than a linear search through all the entries in the `SuggestionDatabase`.
https://www.pivotaltracker.com/story/show/181852566
# Important Notes
- **Testing:** when testing the PR with the newest currently available nightly version of the Engine, you may observe the following warnings in the Chrome JS Developer Console (the numerical values may differ):
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Nothing.Nothing.is_nothing was overwritten with 768.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Numbers.Integer.up_to was overwritten with 936.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Numbers.Integer.down_to was overwritten with 937.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Text.Text.== was overwritten with 971.
This is a result of bugs in the Standard Library. Those are planned to be addressed by:
- #3480 short-term (to fix the specific bugs currently present in the Standard Library),
- https://www.pivotaltracker.com/story/show/182283983 long-term (to improve the Engine such that it disallows introducing this category of bugs in the Standard Library in the future).
(For more details, see also: https://discord.com/channels/401396655599124480/978929754138877962.)
As a result of the bugs mentioned above, the Engine is responding with some invalid replies. In case of such invalid replies, warnings are emitted in the JS Dev Console. Other than the warnings, the code is expected to work correctly with the Engine for the cases when the Engine returns correct data.
- A `HashMapTree` was used for storing the map. A quick back-of-the-envelope estimation of memory usage of a simpler alternative (a one-level `HashMap`) would put the alternative at ~1MB (~10k entities × averaged ~100 bytes per entity path), which was considered too much in a discussion with @farmaazon.
- The `HashMapTree::remove` method deletes a whole subtree of a `HashMapTree`. An alternative removal method was implemented for use in `suggestion_database::QualifiedNameToIdMap` to better match `HashMap` entry removal semantics.
- In case of path collisions, a warning is emitted.
- Paths are treated case-sensitively.
- The new method of the `SuggestionDatabase` type is currently only used in unit tests. This matches the explicit requirement in the Task's description.
- JS Developer Console logging was enabled for the `tracing` package. The `WARN` level was picked to match the default level enabled in the "old" logging infrastructure.
[ci no changelog needed]
This PR extends the Component Group Entry with icon and option to highlight the text. Here the convert has highlighted "con".
https://user-images.githubusercontent.com/3919101/169046537-4f8b823c-322e-40dc-8abb-24d1d7092341.mp4
### Important Notes
Although this PR includes effort for adjusting Component Group style to better reflect the design, it is not entirely finished: the selection still works badly and will be fixed in another PR.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181725003)
This PR implements a fully visible component group header while scrolling the group (using the ScrollArea).
The header moves in sync with scrolling movements (using new `set_header_pos` FRP input), so it looks like the component group is scrolled. ScrollArea masks the "scrolled" entries above the header. This design allows a fully visible header even though our renderer doesn't support nested layers masking yet.
The screencast:
https://user-images.githubusercontent.com/6566674/168320360-2c2017b2-0ef5-42ce-9c79-82b9641c1d73.mp4
The most recent one, with the updated demo scene from develop:
https://user-images.githubusercontent.com/6566674/168555268-8552c4b0-f887-4388-89a1-e65ddf668be6.mp4
# Important Notes
- I fixed the API of the list view so now it supports non-hardcoded scene layers (previously it did not). I also believe it was implemented incorrectly.
- I've found a [pretty weird bug](https://www.pivotaltracker.com/story/show/182193824): the component group inside the ScrollArea is invisible unless I add some arbitrary shape to the scroll area content. I use a `transparent_circle` for this purpose in the demo scene. The bug is probably related to masking the sublayers, though I wasn't able to reproduce it properly on a simpler example.
- The selection box is removed from the demo scene as agreed with @farmaazon . The correct implementation has proven to be much harder than I expected, and we will implement another approach in a separate PR.
- I also modified the `shadow::Parameters` so that it uses `Var`s instead of plain values.
The change promotes static methods of `Ref`, `get` and `put`, to be
methods of `Ref` type.
The change also removes `Ref` module from the default namespace.
Had to mostly c&p functional dispatch for now, in order for the methods
to be found. Will auto-generate that code as part of builtins system.
Related to https://www.pivotaltracker.com/story/show/182138899
* Extends the instrumentation of the code base and upgrades some FRPs to the newer API macro.
* Extends the run-graph demo scene to specify a profile via URL without recompilation.
* Fixes labels in the flame graph demo scene.
* Fixes an issue with loading profiles that contains escaped characters.
# Important Notes
* no longer contains the upgrade of the `text::View` to `define_endpoints_2`. This should be fixed as part of the text rendering rewrite.
[ci no changelog needed]
Parametrize the colors used in a Component Group view based on a single color passed to an FRP input.
Customizing the colors of a Component Group will be needed for the larger Component Group List panel. This customization will work as a visual hint for the User, helping them to distinguish different Component Groups in the panel. A single input color will be configured for every Component Group in the `package.yaml` file (see the Design Doc). Therefore, all shades of the color required by the Component Group view must be calculated from this single input color.
https://www.pivotaltracker.com/story/show/181725039
#### Visuals
The following screencast of the `component_group` debug scene shows how all required shades of color are calculated from a single input color. It also shows a new "dimmed" display mode of the Component Group. The debug scene does not support selecting entries in a "dimmed" Component Group, as this is not required by the Design Doc.
https://user-images.githubusercontent.com/273837/168074651-bf3d5ea5-99b0-4b69-9934-ad8565ffc54e.mov
The following is a screenshot of the Node Searcher, to demonstrate that it still works correctly:
<img width="623" alt="Screenshot 2022-05-09 at 17 13 01" src="https://user-images.githubusercontent.com/273837/167441109-e9a47b5a-45a2-4172-85ed-c593e43e02d6.png">
# Important Notes
- A new type `Params` was added in the `list_view::entry::Entry` trait. This was needed to allow passing FRP information to entries separately for every ListView instance.
- Note: `style_prefix` and `max_width_px` parameters of the `list_view::entry::Entry::new` function may get moved into the new `Params` type in the future. To save time, this was not attempted in this PR, as agreed with @farmaazon.
[ci no changelog needed]
