Fixes regression about inaccessible node output port. https://www.pivotaltracker.com/n/projects/2539304/stories/183792368
Additionally, a related issue related to nodes not being visible after initial load has been uncovered and fixed in the process.
# Important Notes
Previously the output shape size was only updated when node size itself was updated. When node's expression was changed, the output port view is recreated from scratch, but the current node size has not been propagated to the newly created port. In some cases the node size was changed shortly after, masking the bug. In other cases, the newly created shape size was never set.
The port size management has been moved from model method into frp network, and the initial size value is emitted during network initialization.
Fixes https://www.pivotaltracker.com/story/show/182926584
[Task link](https://www.pivotaltracker.com/story/show/183426449)
This PR fixes an IDE freeze introduced by https://github.com/enso-org/enso/pull/3732 and reimplements reverting edited nodes to their previous state.
The cause of the IDE freeze is quite interesting. A detailed investigation is available [here](https://gist.github.com/vitvakatu/785e34881368b8cfda61715d7543cbd0).
The graph editor needs to update the Presenter state only if the user is editing the node. Before this PR, the graph editor notified the Presenter with a visual representation of the node content instead of code expression. It caused inconsistency between the states of the controller and Presenter and caused severe performance issues.
https://user-images.githubusercontent.com/6566674/195831224-6d6e8258-e347-48b4-890a-d89c7300bc39.mp4
# Important Notes
- ~~There is a more complex alternative solution – it requires refactoring of the `component::node::input::area` module. The Presenter can be notified with `expression.code` changes, not `expression.viz_code`. I found a simpler solution (`.gate(&edit_mode)`), which has the same effect but does not require additional refactoring.~~ Said solution is implemented in a separate commit
This PR introduced an overhauled Component List Panel implementation, making use of the efficient EnsoGL grid view component. Also, it delivers a couple of new features:
* A part of the new design: there are no more section headers in grid, instead groups are "glued" together. The local scope section is under "popular" (old "favorites").
* The keyboard management inside grid works.
* there is a mouse hover highlight
* selecting the lowest entry in section when jumping with navigation bar.
* accepting input as-is with cmd/ctrl + Enter.
https://user-images.githubusercontent.com/3919101/194561890-fffb9b41-2f0d-4357-8d9a-5038a6bcb023.mp4
### Important Notes
**What is not implemented:**
* [Focus management between panels.](https://www.pivotaltracker.com/story/show/180872763) The grid is always focused. To accept the current input, use ctrl+Enter shortcut.
* [Proper handling of selection when having empty space on the right and pressing right arrow.](https://www.pivotaltracker.com/story/show/183487880)
* When entering a module, its name is not added to the input as described in the design doc. Will be a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
**Known issues**
* [the selection, especially in the local scope section, has sometimes an undesirable offset](https://www.pivotaltracker.com/story/show/183487730). The cause is known, but not so easy to fix.
* The inserted nodes are often producing errors. The Browser's inherits the outdated understanding of the language from old Node Searcher, and it does not include new form of imports, static methods etc. Those all will be fixed as a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
* The performance is improved, but still not ideal, due to problems in [text areas](https://www.pivotaltracker.com/story/show/183406745).
* To scroll the documentation panel, you must first click on it.
Show the visualization when a newly created node is being edited. Hide it after the editing is finished, unless the user explicitly changed the state of the visualization.
https://www.pivotaltracker.com/story/show/182634059
#### Visuals
The following video shows the behavior of the visualization when a new node is created by the user and is being edited:
- By default, the visualization is shown when the node becomes edited and disappears when the node's expression is accepted.
- If the user clicks the visualization toggle button once while editing the node, the visualization disappears and stays disabled after the node's expression is accepted.
- If the user clicks the visualization toggle button twice while editing the node, the visualization is visible and stays enabled after the node's expression is accepted.
This PR is not concerned with the correctness of the contents of the visualization window. This is in line with the current behavior on the `develop` branch, where clicking the visualization toggle button while editing a newly created node shows `null` content in the visualization (even if the edited expression is not null).
https://user-images.githubusercontent.com/273837/191294970-62143983-764a-446c-9616-9765de0923bf.mov
Repairing the constructor name following the types work. Some general tiding up as well.
- Remove `Standard.Database.Data.Column.Aggregate_Column_Builder`.
- Remove `Standard.Database.Data.Dialect.Dialect.Dialect_Data`.
- Remove unused imports and update some type definitions.
- Rename `Postgres.Postgres_Data` => `Postgres_Options.Postgres`.
- Rename `Redshift.Redshift_Data` => `Redshift_Options.Redshift`.
- Rename `SQLite.SQLite_Data` => `SQLite_Options.SQLite`.
- Rename `Credentials.Credentials_Data` => `Credentials.Username_And_Password`.
- Rename `Sql` to `SQL` across the board.
- Merge `Standard.Database.Data.Internal` into `Standard.Database.Internal`.
- Move dialects into `Internal` and merge the function in `Helpers` into `Base_Generator`.
Avoid long lines when using `dom().set_inner_text` - rather split the long lines to 1024 chunks and insert them as individual `<div>` elements.
# Important Notes
I was testing the behavior on following program:
```
from Standard.Base import all
import Standard.Base.Data.Statistics
import Standard.Visualization
main =
number1 = 200000
operator1 = 0.up_to number1 . to_vector . map .noise
operator2 = operator1.sort
operator3 = operator2.to_text
```
before my change the visualization of `operator3` was blank. With my change it gets filled with data.
First of all this PR demonstrates how to implement _lazy visualization_:
- one needs to write/enhance Enso visualization libraries - this PR adds two optional parameters (`bounds` and `limit`) to `process_to_json_text` function.
- the `process_to_json_text` can be tested by standard Enso test harness which this PR also does
- then one has to modify JavaScript on the IDE side to construct `setPreprocessor` expression using the optional parameters
The idea of _scatter plot lazy visualization_ is to limit the amount of points the IDE requests. Initially the limit is set to `limit=1024`. The `Scatter_Plot.enso` then processes the data and selects/generates the `limit` subset. Right now it includes `min`, `max` in both `x`, `y` axis plus randomly chosen points up to the `limit`.
The D3 visualization widget is capable of _zooming in_. When that happens the JavaScript widget composes new expression with `bounds` set to the newly visible area. By calling `setPreprocessor` the engine recomputes the visualization data, filters out any data outside of the `bounds` and selects another `limit` points from the new data. The IDE visualization then updates itself to display these more detailed data. Users can zoom-in to see the smallest detail where the number of points gets bellow `limit` or they can select _Fit all_ to see all the data without any `bounds`.
# Important Notes
Randomly selecting `limit` samples from the dataset may be misleading. Probably implementing _k-means clustering_ (where `k=limit`) would generate more representative approximation.
This PR contains all work for finishing integration of first Component List Panel in the IDE:
* It adds a stub for the whole Component Browser View. The documentation panel is re-used from the old searcher.
* It has the presenter implementation, integrating the view with Hierarchical Component List from the controller.
* It extends the View API, so the integration is possible, making use of Component Group Set wrapper.
* The selection integration was also merged into this PR, because it depended on the API extension mentioned above. However, we should avoid such practice in the future.
https://user-images.githubusercontent.com/3919101/177816427-8c4285b4-8941-4048-a400-52f4acf77a9f.mp4
# Important Notes
There are some known issues, to-be-fixed in the future.
* The performance is bad. It should be improved with new text::Area, and the decent one shall come with [GridView inside component browser](https://www.pivotaltracker.com/story/show/182561072)
* There is no keyboard navigation. It should also be delivered with [GridView](https://www.pivotaltracker.com/story/show/182561072).
* The Favorites section is not [filtered out by node source type](https://www.pivotaltracker.com/story/show/182661634).
Visualization within the UI is no longer working since the removal of the `here` keyword.
This PR fixes the JavaScript calls.
# Important Notes
There are various lines of rust code which still have `here.` within them and these will need to be fixed at some point.
If a node created by the user gets placed off-screen, the screen's camera is panned to make the node visible.
https://www.pivotaltracker.com/story/show/181188687
#### Visuals
A screencast showing a number of node creation scenarios when the camera is panned to the newly created node, including when zoomed out.
https://user-images.githubusercontent.com/273837/177169716-50a12b0a-c742-4b01-9766-56206e7938b9.mov
# Important Notes
- Camera is panned also if the node is only partially visible, or if there's not enough free space visible around the node. The specific amount of free space that needs to be visible around a newly created node is configured in the theme.
- If the screen area is so small that the node cannot be fully fit in it (either horizontally or vertically), showing the left and top boundaries of the node's area takes priority over showing the corresponding opposite edges.
Remove a `Symbol`from the `SymbolRegistry` when its `SpriteSystem` is dropped.
This fixes the remaining buffer leak (after #3504) in https://www.pivotaltracker.com/story/show/181943457
# Important Notes
- The `SymbolRegistry` now assigns unique `SymbolId`s, so that we can tell if a `SymbolId` refers to a `Symbol` that has already been unregistered (this shouldn't happen, but it's not statically-obvious that it doesn't, so if it occurs we shouldn't misbehave).
- Also fix a bug in how `buffer_count` was tracked (we were decrementing more than incrementing!).
[ci no changelog needed]
This PR implements a new helper for the future Component Browser - `component_group::multi::Wrapper`. It propagates FRP events from multiple component groups and ensures that only a single component group is focused at all times.
See the updated component group demo scene (console logs shows propagated FRP events from all component groups):
https://user-images.githubusercontent.com/6566674/172359141-8ea6f1ba-e357-4c1b-852a-adb4d5207e03.mp4
- Fixed a `define_endpoints_2!` macro. FRP endpoints for `focus` events weren't connected properly.
- List View now uses an overlay shape to catch mouse events, it allows much easier implementation of `is_header_selected` in the component group.
Fix a memory leak introduced by #3451. Found this to be part of the cause of [the buffer leaks](https://www.pivotaltracker.com/story/show/181943457/comments/231558434).
The problem with this use of `define_endpoints_2` is,
- The FRP network contains (strong) references to the `Model`.
- The `Model` has a strong reference to `api::Private`, which owns the FRP network.
Thus we never free the `Model`.
Define some workflows for batch-mode profiling.
Implemented:
- collapse nodes
- create node
- enter collapsed node
- new project
- open visualization
They can currently be built and run with a command like:
`./run.sh ide build --profiling-level=debug && dist/ide/linux-unpacked/enso --entry-point profile --workflow create_node --save-profile out.json`
And the data can be displayed with:
`dist/ide/linux-unpacked/enso --entry-point profiling_run_graph --load-profile out.json`
Demo of recording and viewing a profile with a command-line one-liner:
https://user-images.githubusercontent.com/1047859/169954795-2d9520ca-84f9-45d2-b83a-5063ebe6f718.mp4
See: https://www.pivotaltracker.com/story/show/182195399.
# Important Notes
- When defining workflows, two helpers are enough to allow us to tell when the action is really done: `Fixture::compile_new_shaders`, and `Fixture::backend_execution`. Often, it is appropriate to await both, but it depends on the task.
- The shader compiler is now driven by a `Controller`; while the `Compiler` is reset if context is lost, the `Controller`'s state survives context loss.
- A new `--load-profile` option supports specifying a profile by path when running `profiling_run_graph`.
- Drop the `with_same_start` profiler interface; we ended up preferring a child profiler convention, and this interface was not implemented compatibly with the stricter data model we've had since the introduction of `profiler::data`.
- Fix the noisy `rustfmt` output.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181725003)
This PR implements a fully visible component group header while scrolling the group (using the ScrollArea).
The header moves in sync with scrolling movements (using new `set_header_pos` FRP input), so it looks like the component group is scrolled. ScrollArea masks the "scrolled" entries above the header. This design allows a fully visible header even though our renderer doesn't support nested layers masking yet.
The screencast:
https://user-images.githubusercontent.com/6566674/168320360-2c2017b2-0ef5-42ce-9c79-82b9641c1d73.mp4
The most recent one, with the updated demo scene from develop:
https://user-images.githubusercontent.com/6566674/168555268-8552c4b0-f887-4388-89a1-e65ddf668be6.mp4
# Important Notes
- I fixed the API of the list view so now it supports non-hardcoded scene layers (previously it did not). I also believe it was implemented incorrectly.
- I've found a [pretty weird bug](https://www.pivotaltracker.com/story/show/182193824): the component group inside the ScrollArea is invisible unless I add some arbitrary shape to the scroll area content. I use a `transparent_circle` for this purpose in the demo scene. The bug is probably related to masking the sublayers, though I wasn't able to reproduce it properly on a simpler example.
- The selection box is removed from the demo scene as agreed with @farmaazon . The correct implementation has proven to be much harder than I expected, and we will implement another approach in a separate PR.
- I also modified the `shadow::Parameters` so that it uses `Var`s instead of plain values.
The change promotes static methods of `Ref`, `get` and `put`, to be
methods of `Ref` type.
The change also removes `Ref` module from the default namespace.
Had to mostly c&p functional dispatch for now, in order for the methods
to be found. Will auto-generate that code as part of builtins system.
Related to https://www.pivotaltracker.com/story/show/182138899
* Extends the instrumentation of the code base and upgrades some FRPs to the newer API macro.
* Extends the run-graph demo scene to specify a profile via URL without recompilation.
* Fixes labels in the flame graph demo scene.
* Fixes an issue with loading profiles that contains escaped characters.
# Important Notes
* no longer contains the upgrade of the `text::View` to `define_endpoints_2`. This should be fixed as part of the text rendering rewrite.
[ci no changelog needed]
* Extends the instrumentation of the code base and upgrades some FRPs to the newer API macro.
* Extends the run-graph demo scene to specify a profile via URL without recompilation.
* Fixes labels in the flame graph demo scene.
* Fixes an issue with loading profiles that contains escaped characters.
# Important Notes
[ci no changelog needed]
Implement a command that launches the application, runs a series of steps (a "workflow"), writes a profile to a file, and exits.
See: [#181775808](https://www.pivotaltracker.com/story/show/181775808)
# Important Notes
- The command to capture run and profile is used like: `./run profile --workflow=new_project --save-profile=out.json`. Defining some more workflows (collapse nodes, create node and edit value) comes next; they are implemented with the same infrastructure as the integration-tests.
- The `--save-profile` option can also be used when profiling interactively; when the option is provided, capturing a profile with the hotkey will write a file instead of dumping the data to the devtools console.
- If the IDE panics, the error message is now printed to the console that invoked the process, as well as the devtools console. (If a batch workflow fails, this allows us to see why.)
- New functionality (writing profile files, quitting on command, logging to console) relies on Electron APIs. These APIs are implemented in `index.js`, bridged to the render process in `preload.js`, and wrapped for use in Rust in a `debug_api` crate.
This PR replaces hard-coded `@Builtin_Method` and `@Builtin_Type` nodes in Builtins with an automated solution
that a) collects metadata from such annotations b) generates `BuiltinTypes` c) registers builtin methods with corresponding
constructors.
The main differences are:
1) The owner of the builtin method does not necessarily have to be a builtin type
2) You can now mix regular methods and builtin ones in stdlib
3) No need to keep track of builtin methods and types in various places and register them by hand (a source of many typos or omissions as it found during the process of this PR)
Related to #181497846
Benchmarks also execute within the margin of error.
### Important Notes
The PR got a bit large over time as I was moving various builtin types and finding various corner cases.
Most of the changes however are rather simple c&p from Builtins.enso to the corresponding stdlib module.
Here is the list of the most crucial updates:
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/builtin/Builtins.java` - the core of the changes. We no longer register individual builtin constructors and their methods by hand. Instead, the information about those is read from 2 metadata files generated by annotation processors. When the builtin method is encountered in stdlib, we do not ignore the method. Instead we lookup it up in the list of registered functions (see `getBuiltinFunction` and `IrToTruffle`)
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java` has now information whether it corresponds to the builtin type or not.
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/RuntimeStubsGenerator.scala` - when runtime stubs generator encounters a builtin type, based on the @Builtin_Type annotation, it looks up an existing constructor for it and registers it in the provided scope, rather than creating a new one. The scope of the constructor is also changed to the one coming from stdlib, while ensuring that synthetic methods (for fields) also get assigned correctly
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/IrToTruffle.scala` - when a builtin method is encountered in stdlib we don't generate a new function node for it, instead we look it up in the list of registered builtin methods. Note that Integer and Number present a bit of a challenge because they list a whole bunch of methods that don't have a corresponding method (instead delegating to small/big integer implementations).
During the translation new atom constructors get initialized but we don't want to do it for builtins which have gone through the process earlier, hence the exception
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor not only generates the actual code fpr nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor no longer generates only (root) nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/TypeProcessor.java` - Similar to MethodProcessor but handles @Builtin_Type annotations. It doesn't, **yet**, generate any builtin objects. It also collects the names, as present in stdlib, if any, so that we can generate the names automatically (see generated `types/ConstantsGen.java`)
- `engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin` - various classes annotated with @BuiltinType to ensure that the atom constructor is always properly registered for the builitn. Note that in order to support types fields in those, annotation takes optional `params` parameter (comma separated).
- `engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala` - drop manual creation of test list which seemed to be a relict of the old design
Implements a visualization that is integrated with our GUI profiling visualization for the multiprocess data implemented in #3395https://user-images.githubusercontent.com/1428930/165915395-c850c7b2-1cc5-4eb0-8f21-37565d113b1e.mp4
The visualization shows a horizontal line for Engine, Language Server and GUI and renders arrows for each message passed between them. Information about the message is revealed on hover.
# Important Notes
* this PR refactors the tooltip mechanism. Note that this has not been in active use anywhere else, as tooltips for node received a custom implementation and the tooltip that was previously implemented was used nowhere else yet.
[ci no changelog needed]
* The List View component was refactored: it allows for hiding the internal selection widget, and exposes information where the widget should be placed. This allows us to create selection widget in component list panel, so it can be animated between component groups and sections.
* Fixed some warnings when checking WASM code.
* Adjusted the style of Component Group View a little, so it better reflects the design doc. Still not ideal, because the list_view has some weird design regarding padding, but I don't want to stuck in some bigger refactoring.
I will add a video in a few minutes.
# Important Notes
https://user-images.githubusercontent.com/3919101/165507826-60329f9e-7de3-4eb2-9271-292e45568cb2.mov
In this branch:
* The workaround for cursor-not-being-updated-after-closing-searcher bug (discovered while testing #3278) is reverted.
* The proper fix was introduced: created an abstraction for EnsoGL component, which, when dropping, will not immediately drop the FRP network and model, but instead put it into the Garbage Collector. The Collector ensures, that all "component hiding" effects and events will be handled, and drops FRP network and model only after that.
* I run clippy for wasm32 target out of curiosity. There was one warning, and I fixed it on this branch.
When a new node is created with the <kbd>TAB</kbd> key or by clicking the `(+)` on-screen button while multiple nodes are selected, place the new node below all the selected nodes. (Previously, the new node was placed below the node that was selected earliest.)
Additionally, when placing a new node below an existing non-error node with a visualization enabled, place the new node below the visualization. (Previously, the new node was placed to the left of the visualization.)
https://www.pivotaltracker.com/story/show/180887079
#### Visuals
The following screencast demonstrates the feature on various arrangements of selected nodes, with visualization enabled and disabled.
https://user-images.githubusercontent.com/273837/159971452-148aa4d7-c0f3-4b48-871a-a2783989f403.mov
The following screencast demonstrates that new nodes created by double-clicking an output port of a node with visualization enabled are now placed below the visualization:
https://user-images.githubusercontent.com/273837/160107733-e3f7d0f9-0161-49d1-8cbd-06e18c843a20.mov
# Important Notes
- Some refactorings that were needed for this PR were ported from the #3301 PR:
- the code responsible for calculating the positions of new nodes was moved to a separate module (`new_node_position`);
- the `free_place_finder` module was made a submodule of the `new_node_position` module, due to the latter being its only user.
Use a new algorithm for placement of new nodes in cases when:
- a) there is no selected node, and the `TAB` key is pressed while the mouse pointer is near an existing node (especially in an area below an existing node);
- b) a connection is dragged out from an existing node and dropped near the node (especially in an area below the node).
In both cases mentioned above, the new node will now be placed in a location suggested by an internal algorithm, aligned to existing nodes. Specifically, the placement algorithm used is similar to when pressing `TAB` with a node selected.
For more details, see: https://www.pivotaltracker.com/story/show/181076066
# Important Notes
- Visible visualizations enabled with the "eye icon" button are treated as part of a node. (In case of nodes with errors, visualizations are not visible, and are not treated as part of a node.)
[ci no changelog needed]
This PR adds a few simple unit tests for GraphEditor, that can be used as an example of native Unit Tests.
Covered:
1. Creating nodes
- By internal API
- By using a TAB shortcut
- By using (+) button
- By dropping edge
2. Connecting two nodes with an edge
Some APIs were extended to allow their testing.
Usage of `glyph::System` in `text/component/area` was disabled by conditional compilation, as this code can't be used in native code due to JS dependencies.
* Profiling application details
Add enough profiling to account for every missed frame during startup.
See https://www.pivotaltracker.com/story/show/181499507
* Build ActiveInterval hierarchy in profiler_data
* update doctests / await_!
* docs/formatting/naming
* more graph modes
* increase WASM size
Due to new render-profile-flamegraph scene. We should remove these from the main release WASM blob one way or another.
* lint
* fix a test
* Organization (feedback)
* Add @wdanilo to Cargo.lock CODEOWNERS
As discussed after my previous PR got stuck waiting for Cargo.lock review.
* fix doctests
* Update docs. Removed a limitation.
Double-clicking a node's output port or clicking the port with a right mouse button (RMB) creates a new node aligned to the clicked node.
#### Visuals
The screencast below demonstrates the following features:
- double-clicking the left mouse button on a node's output port;
- clicking the right mouse button on a node's output port;
- alignment of the nodes created as a result of the actions described above;
- corner case: double-clicking (and RMB-clicking) on output ports of a "collapsed" ("enterable") node;
- double-clicking on a "collapsed" ("enterable") node still allows entering the node when done over an area of the node that is not the node's output port;
- basic support for nodes with multiple output ports (shown on the `interface` demo scene).
https://user-images.githubusercontent.com/273837/158991856-e0faa5f0-9d2f-44bd-bddd-ba314977db6e.mov
The supplementary screencast below demonstrates that double-clicking or RMB-clicking a node's output port cancels the action of dragging a new connection from a node.
https://user-images.githubusercontent.com/273837/158998097-100aed42-37ff-4467-939f-2b755ef0d3dc.movhttps://www.pivotaltracker.com/story/show/181076145
# Important Notes
- The "double-clicking a node" shortcut was previously used to allow entering a "collapsed" node (for example, a node created by pressing the `cmd+g` keyboard shortcut after selecting a group of nodes). This PR keeps that functionality when the user double-clicks on a node, as long as the mouse is not positioned over the node's output ports.
- The support for nodes with multiple output ports is currently very basic. The information about a port (`Crumb`) is passed into the `create_node` function, but it is not passed further to `NodeSource`. The Node Searcher currently does not support passing port information through `NodeSource`.
