Implements:
- UUIDs: https://www.pivotaltracker.com/story/show/182931137
- Comments: https://www.pivotaltracker.com/story/show/182981779
- Type annotations and signatures: https://www.pivotaltracker.com/story/show/182497454
- Fix getter names (https://github.com/enso-org/enso/pull/3627#discussion_r940887460).
# Important Notes
- I can't fully test UUIDs; I have tested that the data obtained in Rust matches my understanding of how the format is supposed to work. What remains to be tested is that the data in Java matches the way the old parser handles the format. So @JaroslavTulach, let me know if you see any cases where I'm not returning the same values.
- This implementation of type annotations and signatures accepts any expression in type context. It would probably be nice to narrow this down at some point, but for now I have no design info on what specifically should be allowed in type expressions; this implementation should be at least an incremental improvement.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/182955595)
This PR implements variable column widths in the new Grid View component. We need this feature to quickly implement various parts of the UI, including the breadcrumbs panel of the component browser.
There are two ways to change the width of the specific column:
1. "From the outside", using the `set_column_width` endpoint of the Grid View
2. "From the inside", using the `override_column_width` endpoint of the EntryFrp.
Both ways work similarly, but the latter is helpful for our breadcrumbs implementation, as it allows for entry to decide on the width of the column by its content.
See the screencast with three grid views. The top-left one has every even column shrunk by GridView API. Every grid view has a second column extended by EntryFrp API.
https://user-images.githubusercontent.com/6566674/185060985-7b7df076-c659-41fa-977a-22875493f8d4.mp4
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181870555)
This PR changes the relative position of the edited node in such a way that it is left-aligned to the component browser window. This change reflects the most recent version of the [design doc](https://github.com/enso-org/design/blob/main/epics/component-browser/design.md#overview)
<img width="1157" alt="Screenshot 2022-08-08 at 19 15 47" src="https://user-images.githubusercontent.com/6566674/183454192-81960e0a-ab69-43a4-b7df-d13320a9d16d.png">
As an additional change, the FRP implementation of the `Camera2d` was extended with a new output (screen dimensions) and fixed. With the old implementation, there was a possibility of panic at runtime because of non-exclusive borrows of `RefCell`. The FRP event for camera position was emited inside the scope with a mutable `RefCell` borrow. Any attempt to borrow the camera one more time (e.g., by calling one of the getters, such as `zoom()`) caused panic at runtime.
Based on usage; I believe this handles every case in current `.enso` files.
# Important Notes
- `import` is a built-in macro, so an import statement parses as a `MultiSegmentApp`.
- Every `import` syntax will have a segment whose leading keyword is `import`; however `import` macros can be identified more efficiently by looking at only the first keyword. A `MultiSegmentApp` is an import if and only if its first keyword is in the set { "polyglot", "from", "import" }.
Show custom icons in Component Browser for entries that have a non-empty `Icon` section in their docs with the section's body containing a name of a predefined icon.
https://www.pivotaltracker.com/story/show/182584336
#### Visuals
A screenshot of a couple custom icons in the Component Browser:
<img width="346" alt="Screenshot 2022-07-27 at 15 55 33" src="https://user-images.githubusercontent.com/273837/181265249-d57f861f-8095-4933-9ef6-e62644e11da3.png">
# Important Notes
- The PR assigns icon names to four items in the standard library, but only three of them are shown in the Component Browser because of [a parsing bug in the Engine](https://www.pivotaltracker.com/story/show/182781673).
- Icon names are assigned only to four items in the standard library because only two currently predefined icons match entries in the currently defined Virtual Component Groups. Adjusting the definitions of icons and Virtual Component Groups is covered by [a different task](https://www.pivotaltracker.com/story/show/182584311).
- A bug in the documentation of the Enso protocol message `DocSection` is fixed. A `text` field in the `Tag` interface is renamed to `body` (this is the field name used in Engine).
This PR adds a new variant of selection, where the mouse-hovered entry is highlighted and may be selected by clicking.
In the video below, we have three grid views with slightly different settings:
* In the left-top corner, both hover and selection highlight is just a shape under the label. Such a grid view does not require additional layers (when compared to non-selectable grid view).
* In the left-bottom corner the hover is normal shape, but selection is a _masked layer_ which allows us to have different text color. This setting requires three more layers to render.
* In the right-top corner, both hover and selection are displayed in the masked layer, creating 6 additional layers.
https://user-images.githubusercontent.com/3919101/181514178-f243bfeb-f2dd-4507-adc3-5344ae0579b7.mp4
Provide a JNI dynamic-library interface to `enso_parser`.
# Important Notes
- The library can be built with: `cargo build -p enso-parser-jni`.
- A new `org.enso.syntax2.Parser` API is implemented on top of the JNI interface provided by `enso-parser-jni`.
- We are using the `jni` crate, since apparently Java cannot just call C-ABI functions. The crate is not well-maintained. I came across an obviously-unsound `safe` function, and found it was reported over a year ago, with a PR to fix: jni-rs/jni-rs#303. However our needs are simple. We can't trust any safety guarantees they imply, but I think we are unlikely to encounter any logic bugs using the basic bindings.
**Note**: This PR also contains content of previous Grid View PR. We decided to discard the previous, because this one did some refactoring of old one, and it's not a big addition.
Added a scrollable::GridView component, which just embeds the GridView in ScrollArea. Also, re-worked the idea of text layers.
https://user-images.githubusercontent.com/3919101/179020359-512ee127-c333-4f86-bff5-f1cb4154e03c.mp4
This PR contains all work for finishing integration of first Component List Panel in the IDE:
* It adds a stub for the whole Component Browser View. The documentation panel is re-used from the old searcher.
* It has the presenter implementation, integrating the view with Hierarchical Component List from the controller.
* It extends the View API, so the integration is possible, making use of Component Group Set wrapper.
* The selection integration was also merged into this PR, because it depended on the API extension mentioned above. However, we should avoid such practice in the future.
https://user-images.githubusercontent.com/3919101/177816427-8c4285b4-8941-4048-a400-52f4acf77a9f.mp4
# Important Notes
There are some known issues, to-be-fixed in the future.
* The performance is bad. It should be improved with new text::Area, and the decent one shall come with [GridView inside component browser](https://www.pivotaltracker.com/story/show/182561072)
* There is no keyboard navigation. It should also be delivered with [GridView](https://www.pivotaltracker.com/story/show/182561072).
* The Favorites section is not [filtered out by node source type](https://www.pivotaltracker.com/story/show/182661634).
implement simple variable assignments and function definitions.
This implements:
- https://www.pivotaltracker.com/story/show/182497122
- https://www.pivotaltracker.com/story/show/182497144 (the code blocks are not created yet, but the function declaration is recognized.)
# Important Notes
- Introduced S-expression-based tests, and pretty-printing-roundtrip testing.
- Started writing tests for TypeDef based on the examples in the issue. None of them parse successfully.
- Fixed Number tokenizing.
- Moved most contents of parser's `main.rs` to `lib.rs` (fixes a warning).
Implement generation of Java AST types from the Rust AST type definitions, with support for deserializing in Java syntax trees created in Rust.
### New Libraries
#### `enso-reflect`
Implements a `#[derive(Reflect)]` macro to enable runtime analysis of datatypes. Macro interface includes helper attributes; **the Rust types and the `reflect` attributes applied to them fully determine the Java types** ultimately produced (by `enso-metamodel`). This is the most important API, as it is used in the subject crates (`enso-parser`, and dependencies with types used in the AST). [Module docs](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/reflect/macros/src/lib.rs).
#### `enso-metamodel`
Provides data models for data models in Rust/Java/Meta (a highly-abstracted language-independent model--I have referred to it before as the "generic representation", but that was an overloaded term).
The high-level interface consists of operations on data models, and between them. For example, the only operations needed by [the binary that drives datatype transpilation](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/parser/generate-java/src/main.rs) are: `rust::to_meta`, `java::from_meta`, `java::transform::optional_to_null`, `java::to_syntax`.
The low-level interface consists of direct usage of the datatypes; this is used by [the module that implements some serialization overrides](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/parser/generate-java/src/serialization.rs) (so that the Java interface to `Code` references can produce `String`s on demand based on serialized offset/length pairs). The serialization override mechanism is based on customizing, not replacing, the generated deserialization methods, so as to be as robust as possible to changes in the Rust source or in the transpilation process.
### Important Notes
- Rust/Java serialization is exhaustively tested for structural compatibility. A function [`metamodel::meta::serialization::testcases`](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/metamodel/src/meta/serialization.rs) uses `reflect`-derived data to generate serialized representations of ASTs to use as test cases. Its should-accept cases cover every type a tree can contain; it also produces a representative set of should-reject cases. A Rust `#[test]` confirms that these cases are accepted/rejected as expected, and generated Java tests (see Binaries below) check the generated Java deserialization code against the same test cases.
- Deserializing `Code` is untested. The mechanism is in place (in Rust, we serialize only the offset/length of the `Cow`; in Java, during deserialization we obtain a context object holding a buffer for all string data; the accessor generated in Java uses the buffer and the offset/length to return `String`s), but it will be easier to test once we have implemented actually parsing something and instantiating the `Cow`s with source code.
- `#[tagged_enum]` [now supports](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/shapely/macros/src/tagged_enum.rs#L36-L51) control over what is done with container-level attributes; they can be applied to the container and variants (default), only to the container, or only to variants.
- Generation of `sealed` classes is supported, but currently disabled by `TARGET_VERSION` in `metamodel::java::syntax` so that tests don't require Java 15 to run. (The same logic is run either way; there is a shallow difference in output.)
### Binaries
The `enso-parser-generate-java` crate defines several binaries:
- `enso-parser-generate-java`: Performs the transpilation; after integration, this will be invoked by the build script.
- `java-tests`: Generates the Java code that tests format deserialization; after integration this command will be invoked by the build script, and its Java output compiled and run during testing.
- `graph-rust`/`graph-meta`/`graph-java`: Produce GraphViz representations of data models in different typesystems; these are for developing and understanding model transformations.
Until integration, a **script regenerates the Java and runs the format tests: `./tools/parser_generate_java.sh`**. The generated code can be browsed in `target/generated_java`.
If a node created by the user gets placed off-screen, the screen's camera is panned to make the node visible.
https://www.pivotaltracker.com/story/show/181188687
#### Visuals
A screencast showing a number of node creation scenarios when the camera is panned to the newly created node, including when zoomed out.
https://user-images.githubusercontent.com/273837/177169716-50a12b0a-c742-4b01-9766-56206e7938b9.mov
# Important Notes
- Camera is panned also if the node is only partially visible, or if there's not enough free space visible around the node. The specific amount of free space that needs to be visible around a newly created node is configured in the theme.
- If the screen area is so small that the node cannot be fully fit in it (either horizontally or vertically), showing the left and top boundaries of the node's area takes priority over showing the corresponding opposite edges.
Implement simple placeholder icons for all entry kinds supported in the Suggestion Database. The icons are planned to be used in Component Browser as default icons for entries. This is intended to allow visually distinguishing different entry kinds.
The following additional fixes and tweaks are applied:
- Icons previously using only 1 color from the theme now use the color provided through shape parameters instead.
- The `data_science` and `network` icons now use only the 2 colors provided through shape parameters.
- The `join` icon has its shape and colors modified and uses only the 2 colors provided through shape parameters.
- The demo scene now parametrizes icon shapes using colors from the Component Browser Design Doc.
https://www.pivotaltracker.com/story/show/182584322
#### Visuals
Original contents of the demo scene before the PR:
<img width="2197" alt="x-orig" src="https://user-images.githubusercontent.com/273837/176669422-ee2e14c7-9ef4-42fd-acb7-ae3be6b68587.png">
Final contents of the demo scene after the PR:
<img width="2201" alt="x2-final" src="https://user-images.githubusercontent.com/273837/176668720-6f1685fd-f7e6-44d7-85f5-f6a6d6789644.png">
[Task link](https://www.pivotaltracker.com/story/show/182194574).
[ci no changelog needed]
This PR implements a new selection box that will replace an old (not really working) one in the component browser. The old selection box wasn't working well with the headers of the component groups, so we were forced to make a much harder implementation.
The new implementation duplicates some visual components and places them in a separate layer. Then, a rectangular mask cuts off everything that is not "selected". This way:
- We have more control over what the selected entries should look like.
- We can easily support the multi-layer structure of the component groups with headers.
- We avoid problems with nested masks that our renderer doesn't support at the moment.
To be more precise, we duplicate the following:
- Background of the component group becomes the "fill" of the selection.
- Entries text and icons - we can alter them easily.
- Header background and header text. By placing them in separate scene layers we ensure correct rendering order.
https://user-images.githubusercontent.com/6566674/173657899-1067f538-4329-44f9-9dc2-78c8a4708b5a.mp4
# Important Notes
- This PR implements the base of our future selection mechanism, selecting entries with a mouse and keyboard still has several issues that will be fixed in the future tasks.
- The scrolling behavior will also be improved in future tasks. Right we only restrict the selection box position so that it never leaves the borders of the component group.
- I added a new function to `add` shapes to new layers in a non-exclusive way (we had only `add_exclusive`) before. I have no idea how we didn't use this feature before even though we mention it a lot in the docs.
- The demo scene restricts the position of the selection box for one-column component groups but does not for the wide component group.
Remove a `Symbol`from the `SymbolRegistry` when its `SpriteSystem` is dropped.
This fixes the remaining buffer leak (after #3504) in https://www.pivotaltracker.com/story/show/181943457
# Important Notes
- The `SymbolRegistry` now assigns unique `SymbolId`s, so that we can tell if a `SymbolId` refers to a `Symbol` that has already been unregistered (this shouldn't happen, but it's not statically-obvious that it doesn't, so if it occurs we shouldn't misbehave).
- Also fix a bug in how `buffer_count` was tracked (we were decrementing more than incrementing!).
[ci no changelog needed]
This PR implements a new helper for the future Component Browser - `component_group::multi::Wrapper`. It propagates FRP events from multiple component groups and ensures that only a single component group is focused at all times.
See the updated component group demo scene (console logs shows propagated FRP events from all component groups):
https://user-images.githubusercontent.com/6566674/172359141-8ea6f1ba-e357-4c1b-852a-adb4d5207e03.mp4
- Fixed a `define_endpoints_2!` macro. FRP endpoints for `focus` events weren't connected properly.
- List View now uses an overlay shape to catch mouse events, it allows much easier implementation of `is_header_selected` in the component group.
This PR introduces a new structure delivered by Searcher Controller: The Component List.
The Component list is hierarchical, and its structure reflects how the components are displayed in Component Browser - only "submodule" section at this point, other sections will be covered in next tasks.
This does not introduce anything visual; the structures are tested in unit tests.
- Change dev profile settings. Improves build performance; will not affect anything else. Details below.
- Introduce script for benchmarking various incremental builds. Usage is explained in the script comments.
- Add a line to `intervals` showing total main-thread CPU work logged in a profile; this can be used to compare the results of optimizations (I'll be starting a discussion informed by that data separately; this change just enables the tooling to report it).
Define some workflows for batch-mode profiling.
Implemented:
- collapse nodes
- create node
- enter collapsed node
- new project
- open visualization
They can currently be built and run with a command like:
`./run.sh ide build --profiling-level=debug && dist/ide/linux-unpacked/enso --entry-point profile --workflow create_node --save-profile out.json`
And the data can be displayed with:
`dist/ide/linux-unpacked/enso --entry-point profiling_run_graph --load-profile out.json`
Demo of recording and viewing a profile with a command-line one-liner:
https://user-images.githubusercontent.com/1047859/169954795-2d9520ca-84f9-45d2-b83a-5063ebe6f718.mp4
See: https://www.pivotaltracker.com/story/show/182195399.
# Important Notes
- When defining workflows, two helpers are enough to allow us to tell when the action is really done: `Fixture::compile_new_shaders`, and `Fixture::backend_execution`. Often, it is appropriate to await both, but it depends on the task.
- The shader compiler is now driven by a `Controller`; while the `Compiler` is reset if context is lost, the `Controller`'s state survives context loss.
- A new `--load-profile` option supports specifying a profile by path when running `profiling_run_graph`.
- Drop the `with_same_start` profiler interface; we ended up preferring a child profiler convention, and this interface was not implemented compatibly with the stricter data model we've had since the introduction of `profiler::data`.
- Fix the noisy `rustfmt` output.
There are two dirty flags in layers: depth_order_dirty and element_depth_order_dirty - one marking changed in Layer, second marking change in one of sublayers. The depth_order_dirty has a proper callback for setting element_depth_order_dirty of its parent. However, the latter did not propagate up.
I fixed it by adding callback for element_depth_order_dirty which sets the depth_order_dirty of the parent.
# Important Notes
* The question to @wdanilo : is it possible, that I can propagate dirty directly to element_depth_order_dirty, without setting depth_order_dirty? As far as I understand the code, it would also work (and we would omit some unnecessary updates).
* I tried to leave some logs, but I don't feel how to do that: the tooling I used was very specific, only the concrete ids of symbols and layers were logged, and I don't know how to generalize it.
Add a method in `SuggestionDatabase` allowing to find a suggestion entry by a fully qualified path, working faster than a linear search through all the entries in the `SuggestionDatabase`.
https://www.pivotaltracker.com/story/show/181852566
# Important Notes
- **Testing:** when testing the PR with the newest currently available nightly version of the Engine, you may observe the following warnings in the Chrome JS Developer Console (the numerical values may differ):
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Nothing.Nothing.is_nothing was overwritten with 768.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Numbers.Integer.up_to was overwritten with 936.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Numbers.Integer.down_to was overwritten with 937.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Text.Text.== was overwritten with 971.
This is a result of bugs in the Standard Library. Those are planned to be addressed by:
- #3480 short-term (to fix the specific bugs currently present in the Standard Library),
- https://www.pivotaltracker.com/story/show/182283983 long-term (to improve the Engine such that it disallows introducing this category of bugs in the Standard Library in the future).
(For more details, see also: https://discord.com/channels/401396655599124480/978929754138877962.)
As a result of the bugs mentioned above, the Engine is responding with some invalid replies. In case of such invalid replies, warnings are emitted in the JS Dev Console. Other than the warnings, the code is expected to work correctly with the Engine for the cases when the Engine returns correct data.
- A `HashMapTree` was used for storing the map. A quick back-of-the-envelope estimation of memory usage of a simpler alternative (a one-level `HashMap`) would put the alternative at ~1MB (~10k entities × averaged ~100 bytes per entity path), which was considered too much in a discussion with @farmaazon.
- The `HashMapTree::remove` method deletes a whole subtree of a `HashMapTree`. An alternative removal method was implemented for use in `suggestion_database::QualifiedNameToIdMap` to better match `HashMap` entry removal semantics.
- In case of path collisions, a warning is emitted.
- Paths are treated case-sensitively.
- The new method of the `SuggestionDatabase` type is currently only used in unit tests. This matches the explicit requirement in the Task's description.
- JS Developer Console logging was enabled for the `tracing` package. The `WARN` level was picked to match the default level enabled in the "old" logging infrastructure.
[ci no changelog needed]
This PR extends the Component Group Entry with icon and option to highlight the text. Here the convert has highlighted "con".
https://user-images.githubusercontent.com/3919101/169046537-4f8b823c-322e-40dc-8abb-24d1d7092341.mp4
### Important Notes
Although this PR includes effort for adjusting Component Group style to better reflect the design, it is not entirely finished: the selection still works badly and will be fixed in another PR.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181725003)
This PR implements a fully visible component group header while scrolling the group (using the ScrollArea).
The header moves in sync with scrolling movements (using new `set_header_pos` FRP input), so it looks like the component group is scrolled. ScrollArea masks the "scrolled" entries above the header. This design allows a fully visible header even though our renderer doesn't support nested layers masking yet.
The screencast:
https://user-images.githubusercontent.com/6566674/168320360-2c2017b2-0ef5-42ce-9c79-82b9641c1d73.mp4
The most recent one, with the updated demo scene from develop:
https://user-images.githubusercontent.com/6566674/168555268-8552c4b0-f887-4388-89a1-e65ddf668be6.mp4
# Important Notes
- I fixed the API of the list view so now it supports non-hardcoded scene layers (previously it did not). I also believe it was implemented incorrectly.
- I've found a [pretty weird bug](https://www.pivotaltracker.com/story/show/182193824): the component group inside the ScrollArea is invisible unless I add some arbitrary shape to the scroll area content. I use a `transparent_circle` for this purpose in the demo scene. The bug is probably related to masking the sublayers, though I wasn't able to reproduce it properly on a simpler example.
- The selection box is removed from the demo scene as agreed with @farmaazon . The correct implementation has proven to be much harder than I expected, and we will implement another approach in a separate PR.
- I also modified the `shadow::Parameters` so that it uses `Var`s instead of plain values.
* Extends the instrumentation of the code base and upgrades some FRPs to the newer API macro.
* Extends the run-graph demo scene to specify a profile via URL without recompilation.
* Fixes labels in the flame graph demo scene.
* Fixes an issue with loading profiles that contains escaped characters.
# Important Notes
* no longer contains the upgrade of the `text::View` to `define_endpoints_2`. This should be fixed as part of the text rendering rewrite.
[ci no changelog needed]
Parametrize the colors used in a Component Group view based on a single color passed to an FRP input.
Customizing the colors of a Component Group will be needed for the larger Component Group List panel. This customization will work as a visual hint for the User, helping them to distinguish different Component Groups in the panel. A single input color will be configured for every Component Group in the `package.yaml` file (see the Design Doc). Therefore, all shades of the color required by the Component Group view must be calculated from this single input color.
https://www.pivotaltracker.com/story/show/181725039
#### Visuals
The following screencast of the `component_group` debug scene shows how all required shades of color are calculated from a single input color. It also shows a new "dimmed" display mode of the Component Group. The debug scene does not support selecting entries in a "dimmed" Component Group, as this is not required by the Design Doc.
https://user-images.githubusercontent.com/273837/168074651-bf3d5ea5-99b0-4b69-9934-ad8565ffc54e.mov
The following is a screenshot of the Node Searcher, to demonstrate that it still works correctly:
<img width="623" alt="Screenshot 2022-05-09 at 17 13 01" src="https://user-images.githubusercontent.com/273837/167441109-e9a47b5a-45a2-4172-85ed-c593e43e02d6.png">
# Important Notes
- A new type `Params` was added in the `list_view::entry::Entry` trait. This was needed to allow passing FRP information to entries separately for every ListView instance.
- Note: `style_prefix` and `max_width_px` parameters of the `list_view::entry::Entry::new` function may get moved into the new `Params` type in the future. To save time, this was not attempted in this PR, as agreed with @farmaazon.
[ci no changelog needed]
* Extends the instrumentation of the code base and upgrades some FRPs to the newer API macro.
* Extends the run-graph demo scene to specify a profile via URL without recompilation.
* Fixes labels in the flame graph demo scene.
* Fixes an issue with loading profiles that contains escaped characters.
# Important Notes
[ci no changelog needed]
Implement a command that launches the application, runs a series of steps (a "workflow"), writes a profile to a file, and exits.
See: [#181775808](https://www.pivotaltracker.com/story/show/181775808)
# Important Notes
- The command to capture run and profile is used like: `./run profile --workflow=new_project --save-profile=out.json`. Defining some more workflows (collapse nodes, create node and edit value) comes next; they are implemented with the same infrastructure as the integration-tests.
- The `--save-profile` option can also be used when profiling interactively; when the option is provided, capturing a profile with the hotkey will write a file instead of dumping the data to the devtools console.
- If the IDE panics, the error message is now printed to the console that invoked the process, as well as the devtools console. (If a batch workflow fails, this allows us to see why.)
- New functionality (writing profile files, quitting on command, logging to console) relies on Electron APIs. These APIs are implemented in `index.js`, bridged to the render process in `preload.js`, and wrapped for use in Rust in a `debug_api` crate.
* New JSON profile format.
* Use string-table optimization for labels in JSON format.
* Use TimeOffset header to render beanpoles
* Log RPC messages sent to the backend.
* Display RPC requests on graph
* Simplify metadata-logging interface.
Implements a visualization that is integrated with our GUI profiling visualization for the multiprocess data implemented in #3395https://user-images.githubusercontent.com/1428930/165915395-c850c7b2-1cc5-4eb0-8f21-37565d113b1e.mp4
The visualization shows a horizontal line for Engine, Language Server and GUI and renders arrows for each message passed between them. Information about the message is revealed on hover.
# Important Notes
* this PR refactors the tooltip mechanism. Note that this has not been in active use anywhere else, as tooltips for node received a custom implementation and the tooltip that was previously implemented was used nowhere else yet.
[ci no changelog needed]
* The List View component was refactored: it allows for hiding the internal selection widget, and exposes information where the widget should be placed. This allows us to create selection widget in component list panel, so it can be animated between component groups and sections.
* Fixed some warnings when checking WASM code.
* Adjusted the style of Component Group View a little, so it better reflects the design doc. Still not ideal, because the list_view has some weird design regarding padding, but I don't want to stuck in some bigger refactoring.
I will add a video in a few minutes.
# Important Notes
https://user-images.githubusercontent.com/3919101/165507826-60329f9e-7de3-4eb2-9271-292e45568cb2.mov
[ci no changelog needed]
This is fixed copy of already reviewed #3384
[Task link](https://www.pivotaltracker.com/story/show/181413200)
This PR implements content clipping for the ScrollArea component.
List of changes:
- Implemented `InstanceWithAttachedLayer` abstraction that allows creating additional sublayers for our components. In the future, this abstraction can be used for text rendering as well (right now text rendering requires additional hardcoded layers).
- Fixed `complex-shape-system` demo scene by removing `node_searcher_mask` layer.
- Fixed `SublayersModel::remove` - it was not clearing the `layer_placement` hashmap.
- Implemented disabling the wheel scrolling in `Navigator`, and refactored it to reduce the number of functions arguments by introducing a `NavigatorSettings` struct.
Video (`scroll_area` demo):
https://user-images.githubusercontent.com/6566674/164506455-e177a7a7-9f1c-4f50-888f-112423cebbe4.mp4
# Important Notes
- `InstanceWithAttachedLayer` is implemented in such a way that it allows an extension in the future - namely to use it to simplify text rendering. The implementation might be simplified though.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181413200)
This PR implements content clipping for the ScrollArea component.
List of changes:
- Implemented `InstanceWithAttachedLayer` abstraction that allows creating additional sublayers for our components. In the future, this abstraction can be used for text rendering as well (right now text rendering requires additional hardcoded layers).
- Fixed `complex-shape-system` demo scene by fixing `node_searcher_mask` layer.
- Fixed `SublayersModel::remove` - it was not clearing the `layer_placement` hashmap.
- Implemented disabling the wheel scrolling in `Navigator`, and refactored it to reduce the number of functions arguments by introducing a `NavigatorSettings` struct.
Video (`scroll_area` demo):
https://user-images.githubusercontent.com/6566674/164506455-e177a7a7-9f1c-4f50-888f-112423cebbe4.mp4
# Important Notes
- `InstanceWithAttachedLayer` is implemented in such a way that it allows an extension in the future - namely to use it to simplify text rendering. The implementation might be simplified though.
See: [#181837344](https://www.pivotaltracker.com/story/show/181837344).
I've separated this PR from some deeper changes I'm making to the profile format, because the changeset was getting too complex. The new APIs and tools in this PR are fully-implemented, except the profile format is too simplistic--it doesn't currently support headers that are needed to determine the relative timings of events from different processes.
- Adds basic support for profile files containing data collected by multiple processes.
- Implements `api_events_to_profile`, a tool for converting backend message logs (#3392) to the `profiler` format so they can be merged with frontend profiles (currently they can be merged with `cat`, but the next PR will introduce a merge tool).
- Introduces `message_beanpoles`, a simple tool that diagrams timing relationships between frontend and backend messages.
### Important Notes
- All TODOs introduced here will be addressed in the next PR that defines the new format.
- Introduced a new crate, `enso_profiler_enso_data`, to be used by profile consumers that need to refer to Enso application datatypes to interpret metadata.
- Introduced a `ProfileBuilder` abstraction for writing the JSON profile format; partially decouples the runtime event log structures from the format definition.
- Introducing the conversion performed for `ProfilerBuilder` uncovered that the `.._with_same_start!` low-level `profiler` APIs don't currently work; they return `Started<_>` profilers, but that is inconsistent with the stricter data model that I introduced when I implemented `profiler_data`; they need to return profilers in a created, unstarted state. Low-level async profilers have not been a priority, but once #3382 merges we'll have a way to render their data, which will be really useful because async profilers capture *why* we're doing things. I'll bring up scheduling this in the next performance meeting.
Add logging of EnsoGL performance stats to the profiling framework. Also extends the visualization in the debug scene to show an overview of the performance stats. We now render a timeline of blocks that indicate by their colour the rough FPS range we are in:
https://user-images.githubusercontent.com/1428930/162433094-57fbb61a-b502-43bb-8815-b7fc992d3862.mp4
# Important Notes
[ci no changelog needed]
Needs to be merged after https://github.com/enso-org/enso/pull/3382 as it requires some changes about metadata logging from there. That is why this PR is currently still in draft mode and based on that branch.
* Profiling: intervals tool
* devtools profile generator
* docs
* ignore open intervals, because devtools does
* intervals tool: show interval counts
* Re-add a doc comment. Was accidentally deleted.
* Implement review.
* DURATION_FLOOR_MS
* Rename methods that apply a profile's contents to self
* Description of chrome format
* Link to profile.json format documentation is design doc
* Implement profiling-level switch described in design doc
* lint
In this branch:
* The workaround for cursor-not-being-updated-after-closing-searcher bug (discovered while testing #3278) is reverted.
* The proper fix was introduced: created an abstraction for EnsoGL component, which, when dropping, will not immediately drop the FRP network and model, but instead put it into the Garbage Collector. The Collector ensures, that all "component hiding" effects and events will be handled, and drops FRP network and model only after that.
* I run clippy for wasm32 target out of curiosity. There was one warning, and I fixed it on this branch.
When a new node is created with the <kbd>TAB</kbd> key or by clicking the `(+)` on-screen button while multiple nodes are selected, place the new node below all the selected nodes. (Previously, the new node was placed below the node that was selected earliest.)
Additionally, when placing a new node below an existing non-error node with a visualization enabled, place the new node below the visualization. (Previously, the new node was placed to the left of the visualization.)
https://www.pivotaltracker.com/story/show/180887079
#### Visuals
The following screencast demonstrates the feature on various arrangements of selected nodes, with visualization enabled and disabled.
https://user-images.githubusercontent.com/273837/159971452-148aa4d7-c0f3-4b48-871a-a2783989f403.mov
The following screencast demonstrates that new nodes created by double-clicking an output port of a node with visualization enabled are now placed below the visualization:
https://user-images.githubusercontent.com/273837/160107733-e3f7d0f9-0161-49d1-8cbd-06e18c843a20.mov
# Important Notes
- Some refactorings that were needed for this PR were ported from the #3301 PR:
- the code responsible for calculating the positions of new nodes was moved to a separate module (`new_node_position`);
- the `free_place_finder` module was made a submodule of the `new_node_position` module, due to the latter being its only user.
Use a new algorithm for placement of new nodes in cases when:
- a) there is no selected node, and the `TAB` key is pressed while the mouse pointer is near an existing node (especially in an area below an existing node);
- b) a connection is dragged out from an existing node and dropped near the node (especially in an area below the node).
In both cases mentioned above, the new node will now be placed in a location suggested by an internal algorithm, aligned to existing nodes. Specifically, the placement algorithm used is similar to when pressing `TAB` with a node selected.
For more details, see: https://www.pivotaltracker.com/story/show/181076066
# Important Notes
- Visible visualizations enabled with the "eye icon" button are treated as part of a node. (In case of nodes with errors, visualizations are not visible, and are not treated as part of a node.)
[ci no changelog needed]
This PR adds a few simple unit tests for GraphEditor, that can be used as an example of native Unit Tests.
Covered:
1. Creating nodes
- By internal API
- By using a TAB shortcut
- By using (+) button
- By dropping edge
2. Connecting two nodes with an edge
Some APIs were extended to allow their testing.
Usage of `glyph::System` in `text/component/area` was disabled by conditional compilation, as this code can't be used in native code due to JS dependencies.
* Profiling application details
Add enough profiling to account for every missed frame during startup.
See https://www.pivotaltracker.com/story/show/181499507
* Build ActiveInterval hierarchy in profiler_data
* update doctests / await_!
* docs/formatting/naming
* more graph modes
* increase WASM size
Due to new render-profile-flamegraph scene. We should remove these from the main release WASM blob one way or another.
* lint
* fix a test
* Organization (feedback)
* Add @wdanilo to Cargo.lock CODEOWNERS
As discussed after my previous PR got stuck waiting for Cargo.lock review.
* fix doctests
* Update docs. Removed a limitation.
@akavel spotted a compilation error, when building test for graph_editor crate. The cause was that:
* prelude without serde still added serde derivatives in im_string_newtype
* and the graph_editor needs serde from prelude anyway (because it wants to have serializable ImStrings).
In this PR two things are implemented:
1. Node Searcher zoom factor (and therefore its size) is fixed no matter how you move the main camera. The node searcher is also positioned directly below currently edited node at all times.
2. Node growth/shrink animation when you start/finish node editing. After animation end the edited node zoom factor is also fixed and matches the zoom factor of the node searcher.
See attached video with different ways of editing/creating nodes:
https://user-images.githubusercontent.com/6566674/157348758-2880aa2b-494d-46e6-8eee-a22be84081ed.mp4
#### Technical details
1. Added several additional scene layers for separate rendering: `node_searcher`, `node_searcher_text`, `edited_node`, `edited_node_text`. Searcher is always rendered by `node_searcher` camera, edited node moves between its usual layers and `edited_node` layer. Because text rendering uses different API, all node components were modified to support change of the layer.
2. Also added `node_searcher` DOM layer, because documentation is implemented as a DOM object.
3. Added two FRP endpoints for `ensogl::Animation`: `on_end` and `set_value`. These endpoints are useful while implementing growth/shrink animation.
4. Added FRP endpoints for the `Camera2d`: `position` and `zoom` outputs. This allows to synchronize cameras easily using FRP networks.
5. Growth/shrink animation implemented in GraphEditor by blending two animations, similar to Node Snapping implementation. However, shrinking animation is a bit tricky to implement correctly, as we must always return node back to the `main` scene layer after editing is done.
* Creating a new node with the (+) button (#3278)
[The Task](https://www.pivotaltracker.com/story/show/180887253)
A new (+) button on the left-bottom corner appeared. It may be clicked to open searcher in the middle of the scene, as an alternative to tab key.
https://user-images.githubusercontent.com/3919101/154514279-7972ed6a-0203-47cb-9a09-82dba948cf2f.mp4
* The window_control_buttons::common was extracted to separate crate `ensogl-component-button` almost without change.
* This includes a severe refactoring of adding nodes in general in the Graph Editor. The whole responsibility of adding new nodes (and starting their editing) was moved to Graph Editor - the Project View only reacts for GE events to show searcher properly.
* The status bar was moved from the bottom-left corner to the middle-top of the scene. It does not collide with (+) button, and plays "notification" role anyway.
* The `interface` debug scene was buggy. The problem was with one expression's span-tree. When I replaced it, the scene works.
* I've removed "new searcher" API, as it is completely outdated.
* I've changed code owners of integration tests to GUI team, as it is the team writing mostly the integration tests (int rust)
* Fix regression #181528359
* Add docs & remove unused function
* Fix & enable native Rust tests
* Fix formatting
Co-authored-by: Adam Obuchowicz <adam.obuchowicz@enso.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
[ci no changelog needed]
This PR reverts commit [0836ce741d](0836ce741d) because of the spotted regression:
To reproduce:
1. Open a default project.
2. Without doing anything else, cmd + click on any node to edit it.
3. Abort editing by pressing escape.
4. Top-most node disappears (it is actually removed from scene)
If you start editing the bottom node - you will also see a visible regression in node searcher's position.
See thread https://discord.com/channels/401396655599124480/950730235719065620/950731247909478410 for details.
Add an API to create a flame graph from profiling data. Also adds a demo scene showcasing the functionality that generates some profiling data by measuring dummy function calls and rendering a flame graph for the dummy data (see video for the result).
Not that the functionality is not yet exposed user-facing in the GUI itself, but only as API and demo scene, therefore [ci no changelog needed]
https://user-images.githubusercontent.com/1428930/155118977-ecac0628-777c-48bd-9aa7-30ee6aef1976.mp4
# Important Notes
* Change from the initial design: labels are shown on the flame graph instead of as a tooltip. This is because tooltips are currently only implemented in the graph editor and would require some additional refactoring (probably taking the better part of a day).
* re-instated the behaviour that logs are shown in the JS console if development mode is active.
API for storing metadata.
See: https://www.pivotaltracker.com/story/show/181149277
# Important Notes
**New APIs**:
- Storing metadata is implemented with `profiler::MetadataLogger`.
- A full metadata storage/retrieval example is in [the top-level doctests](https://github.com/enso-org/enso/blob/wip/kw/profiling-metadata-api/lib/rust/profiler/data/src/lib.rs) for profiler::data, a crate which implements an API for profiling data consumers (it abstracts away the low-level details of the event log, and checks its invariants in the process) [after review of this new API here I'll open a PR to add it to the design doc].
**Implementation**:
- `profiler::Event` is parameterized by a metadata type, so that different types of metadata can be dependency-injected into it.
- A data consumer defines its metadata type as an enum of all the kinds of metadata it is interested in.
- Producing the metadata enum is accomplished without defining its type (which would require dependencies from around the app): A `MetadataLogger` internally use a serialization helper `Variant` to serialize its variant of the metadata enum without knowledge of the other possible variants.
**Performance impact**: still in the low ns/measurement range, comparable to pushing to a vec.
*Note*: `LocalVecBuilder` is currently present under the name `Log`, which is accurate but probably too overloaded. I'd like to find the right name for it, document it with examples, and move it to its own crate under data-structures, but I don't want doing that to hold up this PR.
[The Task](https://www.pivotaltracker.com/story/show/180887253)
A new (+) button on the left-bottom corner appeared. It may be clicked to open searcher in the middle of the scene, as an alternative to tab key.
https://user-images.githubusercontent.com/3919101/154514279-7972ed6a-0203-47cb-9a09-82dba948cf2f.mp4
# Important Notes
* The window_control_buttons::common was extracted to separate crate `ensogl-component-button` almost without change.
* This includes a severe refactoring of adding nodes in general in the Graph Editor. The whole responsibility of adding new nodes (and starting their editing) was moved to Graph Editor - the Project View only reacts for GE events to show searcher properly.
* The status bar was moved from the bottom-left corner to the middle-top of the scene. It does not collide with (+) button, and plays "notification" role anyway.
* The `interface` debug scene was buggy. The problem was with one expression's span-tree. When I replaced it, the scene works.
* I've removed "new searcher" API, as it is completely outdated.
* I've changed code owners of integration tests to GUI team, as it is the team writing mostly the integration tests (int rust)
[Task link](#181181203).
This is a reincarnation of PR [3273](https://github.com/enso-org/enso/pull/3273).
The maximum zoom factor of Graph Editor is limited to 1.0x. It is not possible to zoom in from the default camera position.
Debug Mode (activated with `ctrl-shift-d` shortcut) allows to zoom up to 100.0x (the previous behavior of Graph Editor).
If you enable Debug Mode, then zoom in and disable Debug Mode - you won't see the immediate change of zoom factor back to 1.0x. But it will "jump" (with animation) back once you make a zoom in/out event with your controls.
Video:
https://user-images.githubusercontent.com/6566674/154037310-1d166737-353e-4ae6-aca1-f7840571ab16.mp4
# Important Notes
This is a reincarnation of PR [3273](https://github.com/enso-org/enso/pull/3273). There are two changes since that PR:
1. Fixed bug with GeoMap zooming described [here](https://github.com/enso-org/enso/pull/3290). This is done by restricting `ZoomEvent` API so that it will never contain `amount` which is equal to `0.0`.
2. A few refactoring changes from https://github.com/enso-org/enso/pull/3289 to simplify code a bit.
This change makes EnsoGL runtime stats be always collected, even when EnsoGL `Monitor` panel is not visible. Those stats are intended to be used in the future by a profiling framework.
**Performance impact:** Continuous collection of stats introduces an overhead of two Web Performance API `now()` calls in each frame of the main rendering loop, plus a small number of simple arithmetic calculations. This is assumed to be a negligible and acceptable overhead.
#### Visuals
A screenshot of the Monitor panel in full `ide` after applying the PR, taken in IDE built with `./run dist`:
<img width="991" alt="Screenshot 2022-02-14 at 16 11 42" src="https://user-images.githubusercontent.com/273837/153891378-8a2fb333-34ce-46ce-99df-7d796817310c.png">
A recording, also in IDE built with `./run dist`; note that FPS is impacted by the act of recording itself:
https://user-images.githubusercontent.com/273837/154104016-49a12e23-1210-4477-9743-ec1611e5b4ed.movhttps://www.pivotaltracker.com/story/show/181093601
# Important Notes
- Responsibility for controlling how `Stats` gathering and calculation is performed at various points in the main rendering loop was removed from `Monitor` - the `Monitor`'s purpose is only to display existing data, it should not influence how the data is collected.
- Two previously existing distinct `Monitor` structs were merged into one, to avoid confusion; after previous refactorings, the remaining `stats::Monitor` did not have much useful code anyway.
- In `stats` package, refactoring was done, to make `StatsData` a "dumb", data-only type, and to move the logic related to stats collection and frame tracking to other helper types.
[ci no changelog needed]
Fix the `scroll-area` example scene, so that a red circle inside the scroll area, and a gray rounded rectangle representing the background of the scroll area, are now both visible.
Previously, adding just a `Sprite` (instead of a corresponding `SpriteSystem`) to the scene resulted in the underlying `Symbol` not being added to a visible `Layer`. (Notably, `Sprite::display_object()` is not the same as `Sprite::symbol.display_object()`.)
https://www.pivotaltracker.com/story/show/180459079
This reverts commit e69e8078c3.
The removed code (stats aggregation and `stats` demo scene) will not be needed by the profiling framework, as discussed on chat.
(reverts changes introduced as part of https://www.pivotaltracker.com/story/show/181093920)
* profiling instrumentation
* Support native testing with mock impl of `mod js`
* Add benchmarks
* Wrapper: support methods.
* `#[profile]`: work in any context
* feature-gate lineno info that breaks IDE
* Support async; more docs; add perf analysis
* docs & formatting
This change adds utility code for calculating summaries from multiple samples (snapshots) of EnsoGL runtime stats values.
This internal feature is expected to be used by Enso IDE performance profiling tools, which are planned to be added in the near future.
https://www.pivotaltracker.com/story/show/181093920
A demo scene named `stats` was added, showcasing how to perform calculations using the new tools. Currently, the summary calculations in the scene work only when the EnsoGL stats Monitor Panel is visible; this is planned to be improved in a future task (https://www.pivotaltracker.com/story/show/181093601).
- Note: the stats aggregation code is intended to be later used in Enso IDE's main rendering loop, so it needs to have very good performance characteristics.
- Due to that, `Accumulator` was designed to only use simple addition arithmetic, and be constant-memory once created.
Fix EnsoGL runtime statistics collection and processing, so that all values in a single vertical slice of the Monitor Panel graph (accessible through `control-option-tilde` shortcut) represent stats for the same animation/rendering frame.
Before the change, for any vertical slice of the graph, the FPS value was shown for the previous (`i - 1`) rendering frame, whereas the other values were shown for the current (`i`) rendering frame.
The alignment of the stats will become even more important in the future, when the values will be included in more detailed profiling reports, which is planned.
The GUI that is impacted (if in a subtle way) by the changed code is the performance monitor panel, accessible with a `ctrl-option-tilde` keyboard shortcut in the IDE.
https://www.pivotaltracker.com/story/show/181140499
The goal of this change is to decouple the code calculating EnsoGL runtime stats (notably, FPS, frame time, and WASM memory usage) from `Sampler` trait implementations (this trait is intended only to be responsible for post-processing stats data for display/view purposes).
Apart from the general improvement in separation of responsibilities, a longer-term goal for this change is to make it possible to access all the stats in any code that doesn't need the `Sampler` functionalities (expected to be needed for https://www.pivotaltracker.com/story/show/181093832 and https://www.pivotaltracker.com/story/show/181093601).
The main loop's code is performance-critical, and this has implications
on tasks design and planning. Knowing this earlier would help me avoid
some major workflow inefficiencies while working on tasks touching this
code.
The old JS-based Welcome Screen was removed and replaced with the Rusty one.
Co-authored-by: Adam Obuchowicz <adam.obuchowicz@enso.org>
Co-authored-by: Adam Obuchowicz <adam.obuchowicz@luna-lang.org>
