See: [#181837344](https://www.pivotaltracker.com/story/show/181837344).
I've separated this PR from some deeper changes I'm making to the profile format, because the changeset was getting too complex. The new APIs and tools in this PR are fully-implemented, except the profile format is too simplistic--it doesn't currently support headers that are needed to determine the relative timings of events from different processes.
- Adds basic support for profile files containing data collected by multiple processes.
- Implements `api_events_to_profile`, a tool for converting backend message logs (#3392) to the `profiler` format so they can be merged with frontend profiles (currently they can be merged with `cat`, but the next PR will introduce a merge tool).
- Introduces `message_beanpoles`, a simple tool that diagrams timing relationships between frontend and backend messages.
### Important Notes
- All TODOs introduced here will be addressed in the next PR that defines the new format.
- Introduced a new crate, `enso_profiler_enso_data`, to be used by profile consumers that need to refer to Enso application datatypes to interpret metadata.
- Introduced a `ProfileBuilder` abstraction for writing the JSON profile format; partially decouples the runtime event log structures from the format definition.
- Introducing the conversion performed for `ProfilerBuilder` uncovered that the `.._with_same_start!` low-level `profiler` APIs don't currently work; they return `Started<_>` profilers, but that is inconsistent with the stricter data model that I introduced when I implemented `profiler_data`; they need to return profilers in a created, unstarted state. Low-level async profilers have not been a priority, but once #3382 merges we'll have a way to render their data, which will be really useful because async profilers capture *why* we're doing things. I'll bring up scheduling this in the next performance meeting.
In this branch:
* The workaround for cursor-not-being-updated-after-closing-searcher bug (discovered while testing #3278) is reverted.
* The proper fix was introduced: created an abstraction for EnsoGL component, which, when dropping, will not immediately drop the FRP network and model, but instead put it into the Garbage Collector. The Collector ensures, that all "component hiding" effects and events will be handled, and drops FRP network and model only after that.
* I run clippy for wasm32 target out of curiosity. There was one warning, and I fixed it on this branch.
Use a new algorithm for placement of new nodes in cases when:
- a) there is no selected node, and the `TAB` key is pressed while the mouse pointer is near an existing node (especially in an area below an existing node);
- b) a connection is dragged out from an existing node and dropped near the node (especially in an area below the node).
In both cases mentioned above, the new node will now be placed in a location suggested by an internal algorithm, aligned to existing nodes. Specifically, the placement algorithm used is similar to when pressing `TAB` with a node selected.
For more details, see: https://www.pivotaltracker.com/story/show/181076066
# Important Notes
- Visible visualizations enabled with the "eye icon" button are treated as part of a node. (In case of nodes with errors, visualizations are not visible, and are not treated as part of a node.)
API for storing metadata.
See: https://www.pivotaltracker.com/story/show/181149277
# Important Notes
**New APIs**:
- Storing metadata is implemented with `profiler::MetadataLogger`.
- A full metadata storage/retrieval example is in [the top-level doctests](https://github.com/enso-org/enso/blob/wip/kw/profiling-metadata-api/lib/rust/profiler/data/src/lib.rs) for profiler::data, a crate which implements an API for profiling data consumers (it abstracts away the low-level details of the event log, and checks its invariants in the process) [after review of this new API here I'll open a PR to add it to the design doc].
**Implementation**:
- `profiler::Event` is parameterized by a metadata type, so that different types of metadata can be dependency-injected into it.
- A data consumer defines its metadata type as an enum of all the kinds of metadata it is interested in.
- Producing the metadata enum is accomplished without defining its type (which would require dependencies from around the app): A `MetadataLogger` internally use a serialization helper `Variant` to serialize its variant of the metadata enum without knowledge of the other possible variants.
**Performance impact**: still in the low ns/measurement range, comparable to pushing to a vec.
*Note*: `LocalVecBuilder` is currently present under the name `Log`, which is accurate but probably too overloaded. I'd like to find the right name for it, document it with examples, and move it to its own crate under data-structures, but I don't want doing that to hold up this PR.
* profiling instrumentation
* Support native testing with mock impl of `mod js`
* Add benchmarks
* Wrapper: support methods.
* `#[profile]`: work in any context
* feature-gate lineno info that breaks IDE
* Support async; more docs; add perf analysis
* docs & formatting
This change adds utility code for calculating summaries from multiple samples (snapshots) of EnsoGL runtime stats values.
This internal feature is expected to be used by Enso IDE performance profiling tools, which are planned to be added in the near future.
https://www.pivotaltracker.com/story/show/181093920
A demo scene named `stats` was added, showcasing how to perform calculations using the new tools. Currently, the summary calculations in the scene work only when the EnsoGL stats Monitor Panel is visible; this is planned to be improved in a future task (https://www.pivotaltracker.com/story/show/181093601).
- Note: the stats aggregation code is intended to be later used in Enso IDE's main rendering loop, so it needs to have very good performance characteristics.
- Due to that, `Accumulator` was designed to only use simple addition arithmetic, and be constant-memory once created.
This PR adds `json-rpc` crate — a library facilitating writing clients using JSON-RPC 2.0 protocol.
This library is meant to be used in implementation of File Manager and, in future, of Language Server clients.
The library is agnostic about `Transport` — but the interface has been designed in compliance with web-sys websockets, as this will be primary platform.
The RPC clients implemented on top of this library are expected to provide Future-based asynchronous API.
Client is designed to work in a single-thread environment,
Implements #426.
AST lives in `ast` package, that relies significantly on `ast-macros` to generate boilerplate.
Additional `macro-utils` library was split out from `shapely-macros` and `ast-macros`.
The implementation was contributed by @wdanilo , I basically just did some refactoring, documenting and testing.
### Important Notes
* AST is known to be incomplete structurally, finishing it is #336
* AST is missing a number of necessary functions, some of them explicitly marked as FIXME in the code, finishing them is #338
* while I have written some tests, they are not yet part of CI — I want to this smart way (i.e. allowing tests that rely on parser), it is tracked as #340
* AST JSON serialization is incompatible with Scala, solving this is #297
* there is some non-deterministic issue with CI on Windows — I need to look into this closer but it seems to not be related to any Rust parts
