Some small improvements relating to rendering:
- Add a debug option: `-debug.pixel-read-period`. This can be used to measure the performance impact of checking the pointer location on different hardware. [On my development box, it makes no difference to performance.] (Closes#5490).
- Unbind pixel pack buffers after each use. This is recommended practice. It has no performance impact on my machine, and allows SpectorJS to run (`-debug.enable-spector`). (Closes#5941).
Also, simplify the profiling CLI: the `profile.load-profile` and `profile.save-profile` options have been renamed to `profile.load`/`profile.save`; `profile.save` now has a default filename, so you can capture a profile at any time in Electron with Ctrl+Alt+P and it will be written to `profile.json`.
Somebody forgot to apply `./run fmt` before committing to develop, so now we have a lot of whitespace changes in files.
# Important Notes
See https://github.com/enso-org/enso/issues/5166
close#5881
Changelog
- add: ProtocolFactory object that initializes and returns the protocol object
- update: Add protocol initialization to the initialization component
close#5911
In interactive mode, perform writing IR caches in the background jobs queue. Background jobs execution is delayed until the first execution is complete.
Right now, we use the cursor position to determine the target position for dropped items. However, it seems that during dragging of files, we do not always receive mouse events, thus cannot update the cursor position. To avoid this, this PR refactors the functionality to use the location of the drop event, instead of the last known cursor position.
Fixes#5237.
https://user-images.githubusercontent.com/1428930/224735951-9cd6ff62-a749-4ff3-8437-c0bee3c0dd05.mp4
Fixes#5826.
# Important Notes
- Change frontend representation of negation.
- Fix a precedence issue: The `.` operators in -1.x and -1.2 must have different precedences.
- Remove a no-longer-needed special case from backend translation.
- Add tests for this case after all translations.
3rd PR for IDE/Cloud authorization with cognito. This PR introduces registration templates + flows + amplify wrappers for registering & confirming user registration.
Login + Set Username + Forgot Password flows are to be added in next PRs to keep the changes reviewable.
Implements #5640 and #5650
It made sense for me to implement those two together, as I wanted to make sure that the necessary widget API changes will support custom entry values for both dynamic and static data.
- Added support for custom dropdown labels defined on the method annotations
- Added shortening of static dropdown values, which resolves
| dynamic dropdown - custom labels | static dropdown - automatic shortening |
|-|-|
|||
# Important Notes
During implementation I had multiple data update order issues caused by FRP network forming a diamond shape. Two inputs that are often updated together were combined with `all` combinator, and that was further fed into the dropdown. This caused two updates to propagate through the whole network, and one of them was immediately outdated. To fix this and similar future scenarios, I've added an `next_tick` FRP node. It buffers the incoming events until the next browser microtask, preserving only the last received event. Currently if it is called inside a `requestAnimationFrame` callback, the effects of that processing will only be rendered in the next frame. Later this can be mitigated by delaying the rendering logic until the microtask queue is empty.
The `logAvailableComponentsForDebugging` will check and install all necessary components of GraalVM for every mentioned version. While not harmful, it adds up to startup time.
Additionally added an option in language server startup to skip installation of GraalVM components. The latter is already performed by project-manager when opening the project and it is unnecessary to do it twice. Due to LS' architecture this configuration has to be passed around via multiple configs.
Finally, skipped the attempt to install Python component on Windows - this is not supported by GraalVM atm.
Closes#5749.
# Important Notes
The impact of this problem could be really felt the more versions of Enso and GraalVM one had since it would go through all of them.
Implement new Enso documentation parser; remove old Scala Enso parser.
Performance: Total time parsing documentation is now ~2ms.
# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2nd PR for IDE/Cloud authorization with cognito. This PR introduces boilerplate react app + some amplify code to fetch the access token + username of the currently logged in user, if they are already authenticated.
Registration + Login + Set Username + Forgot Password flows are to be added in next PRs to keep the changes reviewable.
Adds a common project that allows sharing code between the `runtime` and `std-bits`.
Due to classpath separation and the way it is compiled, the classes will be duplicated - we will have one copy for the `runtime` classpath and another copy as a small JAR for `Standard.Base` library.
This is still much better than having the code duplicated - now at least we have a single source of truth for the shared implementations.
Due to the copying we should not expand this project too much, but I encourage to put here any methods that would otherwise require us to copy the code itself.
This may be a good place to put parts of the hashing logic to then allow sharing the logic between the `runtime` and the `MultiValueKey` in the `Table` library (cc: @Akirathan).
close#5070
Changelog:
- Include the original exception to log expressions
- Enable logging of Akka Actors' lifecycle events on debug logging level
- Decrease the severity of interruption log messages because interruptions are part of the workflow. The computation can be interrupted at any time, and still be recomputed after. Warnings are just misleading in this case.
Closes#5854
Switches dropdown activation indicator to a triangle shape, and moved it to the horizontal center of a port.
# Important Notes
Modified triangle SDF to be exact. That way the grow operation behaves as expected, rounding the corners. Other than that, it produces the same bound shape at 0 distance.
Precompute MSDFs for all ASCII glyphs; after this, we no longer spend any time on MSDF computations when loading or interacting with the example projects.
Also shader precompilation (during build) is now parallel; if you have many cores and an SSD, it's now practically instant.
Closes#5722.
# Important Notes
- The *dynamic-assets* mechanism now used for MSDF data and shaders is versatile, and could be used to pre-seed any other computation-intensive runtime caches.
Fixes#5807
When implementing [cached icons in the Component Browser](#5779), I assumed pixel_ratio may be only integral (1.0 on normal machines and 2.0 on Retina), but I forgot Windows by default scales its UI by 1.25, breaking the icons entirely.
Added support for named arguments in IDE.
https://user-images.githubusercontent.com/919491/223681303-4c716639-d06e-4e33-aa22-6ebca2801b01.mp4
Named arguments are now recognized in node expressions. The function argument placeholders are rendered around series of named arguments. Insertion and deletion of arguments either by connection dragging or by widget selection will cause arguments around to be rewritten into appropriate form, such that the meaning of the expression doesn't change. We no longer need to introduce any wildcards (`_`) in argument positions when editing an argument list of a resolved method.
For unresolved function calls, the old behaviour remains, as we don't have data about argument names or their desired order.
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.
Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.
Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.
Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.
Finally fix an SBT warning when generating library cache.
Closes https://github.com/enso-org/enso/issues/5763
# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
This PR changes build script's `ide watch` and `ide start` commands, so they don't use `electron-builder` to package. Instead, they invoke `electron` directly, significantly reducing time overhead.
`ide watch` will now start Electron process, while continuously rebuilding gui and the client in the background. Changes can be puilled by reloading within the electron, or closing the electron and letting it start once again. To stop, the script should be interrupted with `Ctrl+C`.
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.
When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.
The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.
Part of https://github.com/enso-org/enso/issues/5568 work.
Fixes#5188
Added a new method `ShapeOps::recolorize` which changes color depending on values on r, g, b channels. It should be explained more in the docs. It will allow us using colored cached icons in the Component Browser.
Fixes#5023
This PR adds the ability to add a parameter to shapes defined, with `shape!` macro being a reference to a cached shape.
The API and results may be read [in the example scene](33b6f5937e/lib/rust/ensogl/example/cached-shape/src/lib.rs)
It also contains many other changes, required to have it working:
* We render cached shapes to texture in a different mode than normal shapes: the alpha channel is replaced with information about signed distance. That allows us using cached shapes as normal shapes, i.e. translate them, add to other shapes etc.
* We initialize and arrange shapes as a part of Word initialization, not in pass.
* We keep and blend colors in RGBA instead of LCHA - this is preparation for replacing colors in the next task, and also speeds up our shaders a bit.
The code was refactored in the process: the cached-shape related things were moved to a single module.
Visualizations closing right after opening was caused by the GUI being unresponsive during loading of some visualizations. This caused the timer for measuring the time between space bar press and space bar release to be inflated. The delayed events triggered the "visualization preview mode”, thus closing the visualization has it seemed that the space bar was held down, even though the events just arrived with some delay.
The problem is mitigated by considering the number of frames that have passed between the space and down and the space bar up event, instead of just the wall clock time. If the number of frames is too low, this indicates that frames were dropped to the time is inflated.
Fixes https://github.com/enso-org/enso/issues/5223
Fixes an error in our scoring algorithm for computing match scores. It now correctly computes scores for patterns that are trailing the target text and ranks patterns at the end of the target text higher than patterns in the middle of the target text.
Closes #4965 (for now).
See also Discussion https://github.com/enso-org/enso/discussions/5649
In cloud we want to allow users to create new project from the template. List of templates is a bit outdated and doesn't contain all from the https://github.com/enso-org/project-templates. This PR simply adds missing ones
A combination of commands triggered separate compilation that only recompiled part of builtins. A mechanism that workaround annotation processor's problems with separate compilation was updated to include changes from https://github.com/enso-org/enso/pull/4111. This was pretty tough to find given the rather unusual circumstances in CI.
# Important Notes
This eliminates the _random_ failures in CI related to separate compilation.
To reproduce a specific set of steps has to be executed:
```
sbt> all buildEngineDistribution engine-runner/assembly runtime/Benchmark/compile language-server/Benchmark/compile searcher/Benchmark/compile
(exit sbt)
sbt> test
```
Expressions returning polyglot values were not reporting the type of the result because we have to do additional magic that infers the correct Enso type. Since this is exactly what `TypeOfNode` does, I re-used the logic.
Straightforward solution failed in tests because of assertions:
```
[enso] WARNING: Execution of function main failed (Invalid library usage. Cached library must be adopted by a RootNode before it is executed.).
java.lang.AssertionError: Invalid library usage. Cached library must be adopted by a RootNode before it is executed.
```
That is why this PR replaces `ExecutionEventListener` with `ExecutionEventNodeFactory`.
# Important Notes
Usage of `TypeOfNode` for programs that **do not** import stdlib means that we report types that do not involve stdlib e.g.
`Standard.Builtins.Main.Integer` instead of `Standard.Base.Data.Numbers.Integer`. While surprising, this is correct and I would say desirable. While reviewing the code, notice the difference in expectations in our runtime tests.
Implementation of https://www.pivotaltracker.com/story/show/184012743https://user-images.githubusercontent.com/919491/214082311-cf49e43c-1d1f-4654-903c-a4224cd954d8.mp4
This is also a step towards more general widget support. The widget metadata is queried using `Meta.get_annotation` method through a dedicated visualization. For now only `Single_Choice` case is handled, and always all suggestions are is returned.
# Important Notes
There are limitations as to which node segments receive a widget. Only chain method calls are supported now (`thing.method` syntax), and only outside of lambda scope. Widgets in lambdas will require support for visualisations of lambda subexpressions, which is currently missing in the engine. The IDE technically tries to place the widgets there, but the data never arrives. It should work once the engine support is added.
This PR includes a mock for `Meta.get_annotation` call that only supports `Table.at` method. Real implementation is a separate task that is already in progress.
This PR contains the first implementation of `cached_shape!` macro, which should help us with reducing draw calls in our application.
```rust
mod icon1 {
use super::*;
ensogl_core::cached_shape! { 32 x 32;
() {
let shape = Circle(16.px()).fill(color::Rgba::green());
shape.into()
}
}
}
mod icon2 {
use super::*;
ensogl_core::cached_shape! { 202 x 312;
() {
let shape = Rect((200.px(), 310.px())).fill(color::Rgba::red());
shape.into()
}
}
}
```
The above code creates two cached shapes. They are similar to normal shapes (created with `shape!` macro), except that:
1. they do not allow for any parametrization
2. They are rendered at the application start to the special texture with cached shapes.
The texture will be used in next PRs to cache all Component Browser icons on the texture and draw all of them just by single, fast draw call. In the future, more shapes can be cached, further reducing draw calls and making them simple.
# Important Notes
The results are presented in `cached_shapes` debug scene: there are two shapes displayed and a scaled cached texture is displayed in the background.
In order to investigate `engine/language-server` project, I need to be able to open its sources in IGV and NetBeans.
# Important Notes
By adding same Java source (this time `package-info.java`) and compiling with our Frgaal compiler the necessary `.enso-sources*` files are generated for `engine/language-server` and then the `enso4igv` plugin can open them and properly understand their compile settings.
In addition to that this PR enhances the _"logical view"_ presentation of the project by including all source roots found under `src/*/*`.
https://github.com/enso-org/enso/pull/3764 introduced static wrappers for instance methods. Except it had a limitation to only be allowed for types with at least a single constructor.
That excluded builtin types as well which, by default, don't have them. This limitation is problematic for Array/Vector consolidation and makes builtin types somehow second-citizens.
This change lifts the limitation for builtin types only. Note that we do want to share the implementation of the generated builtin methods. At the same time due to the additional argument we have to adjust the starting index of the arguments.
This change avoids messing with the existing dispatch logic, to avoid unnecessary complexity.
As a result it is now possible to call builtin types' instance methods, statically:
```
arr = Array.new_1 42
Array.length arr
```
That would previously lead to missing method exception in runtime.
# Important Notes
The only exception is `Nothing`. Primarily because it requires `Nothing` to have a proper eigentype (`Nothing.type`) which would messed up a lot of existing logic for no obvious benefit (no more calling of `foo=Nothing` in parameters being one example).
This PR implements HTML generation from documentation IR for all suggestion database entries and replaces the old documentation panel with a newer one.
Additional adjustments to the looks of the documentation would be applied separately in a future PR. This PR focuses on the fastest possible delivery of a usable documentation panel. We want to test it in real-world use cases and gather feedback for future improvements.
Documentation demo scene with mocked data:
https://user-images.githubusercontent.com/6566674/213436313-88753ed8-346f-423e-956e-7db39f5dc266.mp4
Component browser with actual engine-provided data:
https://user-images.githubusercontent.com/6566674/213436375-d0ec074b-f7a6-4deb-a7de-3adee999cc86.mp4
# Important Notes
- Fixed language protocol data structures.
- Scrolling to the selected method is also implemented here.
- Also, the selected item is highlighted with yellow.
- Only some pieces of information we have are displayed. For example, we don't display return types for methods or types of arguments.
- A bunch of code related to previous implementation is removed, but probably not all of it.
Implements [#183453466](https://www.pivotaltracker.com/story/show/183453466).
https://user-images.githubusercontent.com/1428930/203870063-dd9c3941-ce79-4ce9-a772-a2014e900b20.mp4
# Important Notes
* the best laziness is used for `Text` type, which makes use of its internal representation to send data
* any type will first compute its default string representation and then send the content of that lazy to the IDE
* special handling of files and their content will be implemented in the future
* size of the displayed text can be updated dynamically based on best effort information: if the backend does not yet know the full width/height of the text, it can update the IDE at any time and this will be handled gracefully by updating the scrollbar position and sizes.
Introduces unboxed (and arity-specialized) storage schemes for Atoms. It results in improvements both in memory consumption and runtime.
Memory wise: instead of using an array, we now use object fields. We also enable unboxing. This cuts a good few pointers in an unboxed object. E.g. a quadruple of integers is now 64 bytes (4x8 bytes for long fields + 16 bytes for layout and constructor pointers + 16 bytes for a class header). It used to be 168 bytes (4x24 bytes for boxed Longs + 16 bytes for array header + 32 bytes for array contents + 8 bytes for constructor ptr + 16 bytes for class header), so we're saving 104 bytes a piece. In the least impressive scenarios (all-boxed fields) we're saving 8 bytes per object (saving 16 bytes for array header, using 8 bytes for the new layout field). In the most-benchmarked case (list of longs), we save 32 bytes per cons-cell.
Time wise:
All list-summing benchmarks observe a ~2x speedup. List generation benchmarks get ~25x speedups, probably both due to less GC activity and better allocation characteristics (only allocating one object per Cons, rather than Cons + Object[] for fields). The "map-reverse" family gets a neat 10x speedup (part of the work is reading, which is 2x faster, the other is allocating, which is now 25x faster, we end up with 10x when combined).
This is a followup to https://github.com/enso-org/enso/pull/4046#pullrequestreview-1248164069.
- Compiler job cancelation is now handled - the cache entry will be cleared or the job will be resumed if there is still demand.
- The compilation awaiting process has been rewritten to not use polling. The "read cache" job is now only used to dispatch known valid reads. It is always completed within a single run cycle.
- Changed shader struct to use `ImString` for storing code, so it can be cheaply cloned. This is now common, as we are cloning it from cache.
`@` should not be legal to use as a binary operator. I accepted it in the parser because it occurred in the .enso sources, but it was actually used to create a syntax error to test error recovery.
See: https://www.pivotaltracker.com/story/show/184054024
While doing regular node manipulation in editior, noticed a number of situations when
```
java.lang.IndexOutOfBoundsException: None
at java.base/java.lang.Character.offsetByCodePoints(Character.java:8699)
at java.base/java.lang.String.offsetByCodePoints(String.java:820)
at org.enso.text.buffer.CodePointView$Ops$.drop(CodePointView.scala:98)
at org.enso.text.buffer.CodePointView$Ops$.drop(CodePointView.scala:57)
at org.enso.text.buffer.Node.drop(Tree.scala:218)
at org.enso.text.buffer.Rope.dropWith(Rope.scala:86)
at org.enso.text.buffer.CodePointView.drop(CodePointView.scala:30)
at org.enso.text.editing.RopeTextEditor$.cutOutTail(RopeTextEditor.scala:42)
...
```
would be thrown. Further text edits would simply be rejected requiring a complete restart. I doubt we should propagate
`IndexOutOfBoundsException`. Instead it is safer to just apply the edit to the end of the rope.
* Hash codes prototype
* Remove Any.hash_code
* Improve caching of hashcode in atoms
* [WIP] Add Hash_Map type
* Implement Any.hash_code builtin for primitives and vectors
* Add some values to ValuesGenerator
* Fix example docs on Time_Zone.new
* [WIP] QuickFix for HashCodeTest before PR #3956 is merged
* Fix hash code contract in HashCodeTest
* Add times and dates values to HashCodeTest
* Fix docs
* Remove hashCodeForMetaInterop specialization
* Introduce snapshoting of HashMapBuilder
* Add unit tests for EnsoHashMap
* Remove duplicate test in Map_Spec.enso
* Hash_Map.to_vector caches result
* Hash_Map_Spec is a copy of Map_Spec
* Implement some methods in Hash_Map
* Add equalsHashMaps specialization to EqualsAnyNode
* get and insert operations are able to work with polyglot values
* Implement rest of Hash_Map API
* Add test that inserts elements with keys with same hash code
* EnsoHashMap.toDisplayString use builder storage directly
* Add separate specialization for host objects in EqualsAnyNode
* Fix specialization for host objects in EqualsAnyNode
* Add polyglot hash map tests
* EconomicMap keeps reference to EqualsNode and HashCodeNode.
Rather than passing these nodes to `get` and `insert` methods.
* HashMapTest run in polyglot context
* Fix containsKey index handling in snapshots
* Remove snapshots field from EnsoHashMapBuilder
* Prepare polyglot hash map handling.
- Hash_Map builtin methods are separate nodes
* Some bug fixes
* Remove ForeignMapWrapper.
We would have to wrap foreign maps in assignments for this to be efficient.
* Improve performance of Hash_Map.get_builtin
Also, if_nothing parameter is suspended
* Remove to_flat_vector.
Interop API requires nested vector (our previous to_vector implementation). Seems that I have misunderstood the docs the first time I read it.
- to_vector does not sort the vector by keys by default
* Fix polyglot hash maps method dispatch
* Add tests that effectively test hash code implementation.
Via hash map that behaves like a hash set.
* Remove Hashcode_Spec
* Add some polyglot tests
* Add Text.== tests for NFD normalization
* Fix NFD normalization bug in Text.java
* Improve performance of EqualsAnyNode.equalsTexts specialization
* Properly compute hash code for Atom and cache it
* Fix Text specialization in HashCodeAnyNode
* Add Hash_Map_Spec as part of all tests
* Remove HashMapTest.java
Providing all the infrastructure for all the needed Truffle nodes is no longer manageable.
* Remove rest of identityHashCode message implementations
* Replace old Map with Hash_Map
* Add some docs
* Add TruffleBoundaries
* Formatting
* Fix some tests to accept unsorted vector from Map.to_vector
* Delete Map.first and Map.last methods
* Add specialization for big integer hash
* Introduce proper HashCodeTest and EqualsTest.
- Use jUnit theories.
- Call nodes directly
* Fix some specializations for primitives in HashCodeAnyNode
* Fix host object specialization
* Remove Any.hash_code
* Fix import in Map.enso
* Update changelog
* Reformat
* Add truffle boundary to BigInteger.hashCode
* Fix performance of HashCodeTest - initialize DataPoints just once
* Fix MetaIsATest
* Fix ValuesGenerator.textual - Java's char is not Text
* Fix indent in Map_Spec.enso
* Add maps to datapoints in HashCodeTest
* Add specialization for maps in HashCodeAnyNode
* Add multiLevelAtoms to ValuesGenerator
* Provide a workaround for non-linear key inserts
* Fix specializations for double and BigInteger
* Cosmetics
* Add truffle boundaries
* Add allowInlining=true to some truffle boundaries.
Increases performance a lot.
* Increase the size of vectors, and warmup time for Vector.Distinct benchmark
* Various small performance fixes.
* Fix Geo_Spec tests to accept unsorted Map.to_vector
* Implement Map.remove
* FIx Visualization tests to accept unsorted Map.to_vector
* Treat java.util.Properties as Map
* Add truffle boundaries
* Invoke polyglot methods on java.util.Properties
* Ignore python tests if python lang is missing
The fix consists of two parts:
1. All the "review-apply" and "store temporary md" actions in the searcher controller are now guarded by an ignored transaction.
2. Because some of the temporary state may reach the UR frames assigned to other actions, added a bunch of code for removing all temporary expressions from the code and use it after restoring a frame. We may consider using it after project load as well.
### Important Notes
Added a useful method "log_err" to ResultOps (so every Result will have those).
Logging: Replace tracing with an efficient logging implementation, with 0-runtime cost for disabled log levels. (https://www.pivotaltracker.com/story/show/183755412)
Profiling: Support submitting `profiler` events to the User Timing Web API, so that measurements can be viewed directly in the browser. (https://www.pivotaltracker.com/story/show/184003550)
# Important Notes
Logging interface:
- The macros (`warn!`, etc.) now take standard `format_args!` arguments (the tracing implementations accepted a broader syntax).
- Compile-time log levels can now be set through the CLI, like so:
`./run ide start --log-level=trace --uncollapsed-log-level=info`
Profiling:
- The hotkey Ctrl+Alt+Shift+P submits all `profiler` events logged since the application was loaded to the Web API, so that they can then be viewed with the browser's developer tools. Note that standard tools are not able to represent async task lifetimes or metadata; this is a convenient interface to a subset of `profiler` data.
- As an alternative interface, a runtime flag enables continuous measurement submission. In the browser it can be set through a URL parameter, like http://localhost:8080/?emit_user_timing_measurements=true. Note that this mode significantly impacts performance.
Fixes an error when the logger processes NPE:
```
[internal-logger-error] One of the printers failed to write a message: java.lang.NullPointerException
```
Fixes https://www.pivotaltracker.com/story/show/184216698
Reduced impact of node dropdown widgets on load times by deferring creation of grid views until each widget is opened. This also improves node editing time, as the dropdowns are not recreated immediately.
This approach of lazy initialization now caused a significant lag when opening the dropdown. Two major causes of the lag spike is glyph generation (msdfgen, `new_glyph`) and shader compilation (happened every time, because each dropdown has unique layer stack). To reduce the impact of that, the shader compiler now caches the shaders based on generated shader source. Glyph creation hasn't been changed and is still slow. The startup performance is now roughly where it was before introducing widgets.
[Task link](https://www.pivotaltracker.com/story/show/184012434)
This PR implements Intermediate Representation for our documentation. Later these data structures would be used to generate HTML and CSS for the documentation panel. For now, we display it in the debug scene.
https://user-images.githubusercontent.com/6566674/210674850-480a3e6e-76c3-4f34-a235-15c44dc9ec01.mp4
# Important Notes
- `suggestion-database` now lives in a separate crate
- also, two utility crates were introduced for the `notification` and `executor` modules of enso-gui
- documentation debug scene is moved to a separate crate
- All refactorings are done in the last two commits
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/184023445
Added a dropdown widget to graph node for all span tree nodes that have tag values present. When an option is selected, the controller receives a partial expression update, which targets specific crumbs of the expression (similar to how edge endpoint updates work).
https://user-images.githubusercontent.com/919491/210219931-8ae418fd-3ac4-44a5-abea-9e670f15cdf9.mp4
# Important Notes
Right now the dropdown widget is recreated every time the node is edited, including a dropdown option being selected. This causes it to close every time. I wanted to get around that by diffing span trees, but I wasn't able to do it in useful way. Additionally, current implementation of node input expression view heavily relies on being reinitialized from scratch every time. This led to more necessary changes than I was comfortable with for this task. I believe it will be easier to implement it as part of more complete widget support, especially after dynamic data support, as we will have proper widget type information.
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/184023380
Dropdown component. Planned to be used in nodes as a single and multiple selection widget, both for static and dynamically loaded values. Initial support is focused on static data, with limited support for dynamic sources. Notably, loading states are not supported yet. Full support for that is planned to be added later with widget lazy-loading.
- Supports single and multiple selections.
- Dedicated API for providing a static list of all entries.
- Range-based query API for dynamically loading data as it is scrolled (only basic support - will need more work for proper async lazy-loading).
- Internal entry cache and query batching to avoid querying data one by one (the batching for now is very basic, will have to be improved for proper lazy-loading).
- Automatic dropdown width adjustment based on the entry label lengths, up to a set max allowed value.
- Open and close animation.
- Keyboard support for focusing and selecting entries.
# Important Notes
Implementing the dropdown on top of grid-view have uncovered some assumptions around grid-view layers. It was assumed to always be a part of the component browser. Removing that assumption required a mechanism for propagating camera update information through layer tree. This is now implemented using a `camera_parent` layer field. Ideally each layer should simply have at most a single parent, and camera inheritance would follow that. That refactor turned out to be quite involved, so right now the simpler temporary solution is introduced in order to not delay this PR further.
Implements `getMetaObject` and related messages from Truffle interop for Enso values and types. Turns `Meta.is_a` into builtin and re-uses the same functionality.
# Important Notes
Adds `ValueGenerator` testing infrastructure to provide unified access to special Enso values and builtin types that can be reused by other tests, not just `MetaIsATest` and `MetaObjectTest`.
This PR provides a visual indication of whether the project's current state differs from the most recent snapshot saved in the VCS. The project name displayed in the IDE changes to a darker text to indicate that the VCS snapshot is outdated, and back to a lighter text when the current project state corresponds to the last saved VCS snapshot.
https://user-images.githubusercontent.com/117099775/208088438-20dfc2aa-2a7d-47bf-bc12-3d3dff7a4974.mp4
The outdated project snapshot indicator is set when:
* A node is moved.
* A node is added or removed.
* The text editor is used to edit the text.
* The project is auto-saved, and the auto-saved project state does not correspond to the last saved snapshot in the VCS.
The outdated project snapshot indicator is cleared when:
* A new project snapshot is successfully saved using `ctrl+s`.
* The project is auto-saved, and the auto-saved project state is confirmed to correspond to the last saved snapshot in the VCS. This occurs, for example, when a project change is undone and the project is reverted to the last saved snapshot state.
The auto-save events do not occur immediately after a project change but have a short delay, thus the VCS status update is affected by the same delay when triggered by an auto-save event.
This removes the special handling of polyglot exceptions and allows matching on Java exceptions in the same way as for any other types.
`Polyglot_Error`, `Panic.catch_java` and `Panic.catch_primitive` are gone
The change mostly deals with the backslash of removing `Polyglot_Error` and two `Panic` methods.
`Panic.catch` was implemented as a builtin instead of delegating to `Panic.catch_primitive` builtin that is now gone.
This fixes https://www.pivotaltracker.com/story/show/182844611
This is an enhancement of the `Slider` component implemented in #3852. It adds the following features:
* Tooltips and precision change hints
* Selectable slider limit behaviors
* Textual slider value editing
* Vertical slider layout
#### Tooltips
An information tooltip can now be added to a slider, it is shown when the mouse hovers over the component. Additionally, a pop-up indicating the slider's precision appears when the slider's precision has been adjusted.
https://user-images.githubusercontent.com/117099775/206148098-3b4dc059-18aa-4200-9ee0-5d4382363810.mp4
#### Slider limits
The previous slider implementation clamped the adjusted value to the slider's minimum/maximum limits. Now the following behaviors are available:
* Hard limits: Clamp the value to a range within the slider's limits.
* Soft limits: The value can extend beyond the slider's limits. When this occurs, an overflow indicator will be displayed on the side of the limit that is exceeded.
* Adaptive limits: The value can extend beyond the slider's limits. When this occurs, the exceeded limit will temporarily be adjusted to double the slider's range. This will be performed iteratively until the value falls within the extended limits. When a limit is extended and the value is adjusted to fit a smaller range, the extended limit will be iteratively halved until only the necessary range is covered. The slider's extended limits will never shrink to a range smaller than the original range.
These behaviors can be set to the lower and upper limits of a slider independently.
https://user-images.githubusercontent.com/117099775/206148139-6149c91d-ef49-4e2d-97f6-71084f52591c.mp4
#### Textual editing
The slider's value can now be entered through a text input field. Double-click to edit the slider's current value. To confirm the edit press `enter`, or press `escape` to cancel the edit. If an invalid value is entered on confirmation the slider will revert to its value before the edit. The slider's precision will be adjusted based on the number of decimal places of the value entered.
https://user-images.githubusercontent.com/117099775/206148170-d3fa4c82-6e73-4b1c-9be9-cb99979f7b70.mp4
#### Vertical layout
The slider component now supports a vertical layout. In this case value adjustment is performed by a vertical mouse movement, and a horizontal movement adjusts the slider's precision. The slider's track now fills the component in a vertical direction, and the slider's label is displayed near the top end of the component.
https://user-images.githubusercontent.com/117099775/206148211-0f176aaf-bc1b-45e2-afd7-0d28391aafcb.mp4
#### Scroll bar mode
The slider component supports two indicator modes:
* `Track`: The component is filled with a colored bar from the lower limit (empty) to the upper limit (full) dependent on the slider's value.
* `Thumb`: The component contains a rounded indicator that moves along the slider from one end to the other, indicating the slider's value proportionally to the slider's limits. The width of the indicator is configurable.
In addition, the value text, text entry, and precision adjustment can be turned off to provide a scroll bar appearance when used with the `Thumb` indicator.
https://user-images.githubusercontent.com/117099775/206148261-ae291073-85e9-4082-9f91-39b65fecdc0f.mp4
#### Example scene shortcuts
The example scene contains two shortcuts in order to evaluate the dynamic addition and removal of the slider components:
* `CTRL+D` drops all the slider components that are added to the scene.
* `CTRL+A` adds a new set of example slider components to the scene.
Fixes regression https://www.pivotaltracker.com/n/projects/2539304/stories/183895961
# Important Notes
The root cause of the issue was a missing deref before `flavor()` call. The autoderef doesn't work there, because the `ShapeSystemFlavorProvider` trait is implemented for all types, including `Ref`. We should reconsider if blanket default impl is a good idea, as this is a very scary footgun.
Apart from that, the cleanup of layer elements was not handling systems with same ID but different flavors correctly. That could cause text to disappear when given layer had text of multiple fonts, and all shapes of one of them got removed. Now the element is removed from layer only when all systems sharing given system ID have no more instances to render.
* Sequence literal (Vector) should preserve warnings
When Vector was created via a sequence literal, we simply dropped any
associated any warnings associated with it.
This change propagates Warnings during the creation of the Vector.
Ideally, it would be sufficient to propagate warnings from the
individual elements to the underlying storage but doesn't go well with
`Vector.fromArray`.
* update changelog
* Array-like structures preserver warnings
Added a WarningsLibrary that exposes `hasWarnings` and `getWarnings`
messages. That way we can have a single storage that defines how to
extract warnings from an Array and the others just delegate to it.
This simplifies logic added to sequence literals to handle warnings.
* Ensure polyglot method calls are warning-free
Since warnings are no longer automatically extracted from Array-like
structures, we delay the operation until an actual polyglot method call
is performed.
Discovered a bug in `Warning.detach_selected_warnings` which was missing
any usage or tests.
* nits
* Support multi-dimensional Vectors with warnings
* Propagate warnings from case branches
* nit
* Propagate all vector warnings when reading element
Previously, accessing an element of an Array-like structure would only
return warnings of that element or of the structure itself.
Now, accessing an element also returns warnings from all its elements as
well.
This PR is a draft PR while I learn EnsoGL. The eventual goal is to implement the projects list portion of the cloud dashboard in this PR. This PR will implement part of https://www.pivotaltracker.com/n/projects/2539513/stories/183557950
### Important Notes
This PR is still really rough and contains a lot of hacks & hard-coded values. The FRP usage is also likely to be suboptimal and need fixing.
Fix issues noted here: https://github.com/enso-org/enso/pull/3678#issuecomment-1273623924
- Time complexity of an operation during line-redrawing scaled quadratically with number of lines in a change; now linear.
- Time complexity of adding `n` selections to a group was `O(n^2)`. Now it is `O(n log n)`, even if the selections are added one by one.
Also fix a subtle bug I found in `Group::newest_mut`: It returned a mutable reference that allowed breaking the *sorted* invariant of the selection group. The new implementation moves the element to invalidated space before returning a reference (internally to `LazyInvariantVec`), so that if it is mutated it will be moved to its correct location.
### Important Notes
New APIs:
- `NonEmptyVec::extend_at` supports inserting a sequence of elements at a location, with asymptotically-better performance than a series of `insert`s. (This is a subset of the functionality of `Vec::splice`, a function which we can't safely offer for `NonEmptyVec`).
- `LazyInvariantVec` supports lazily-restoring an invariant on a vector. For an invariant such as *sorted* (or in this case, *sorted and merged*), this allows asymptotically-better performance than maintaining the invariant with each mutation.
This PR fixes the `code_to_insert` method of entry to insert valid code according to the newest language version. Also created a separate method for getting imports required by given entry.
Now, method entries do not add imports (except when they are extensions), and are insterted with place for this type `_.method`. Static methods and constructors are inserted with the type name, and proper import for type is added.
There are some additional work done:
* The ReferentName and NormalizedName were removed, as we are now case-sensitive.
* All QualifiedName structures were replaced with new one in `name` module, as there is no longer functional difference between type qualified name and module qualified name.
* The QualifiedName structure removes "Main" module segment where it is not necessary, thus simplifying our code base and avoiding potential issues.
* Added macro `mock_suggestion_database` which should make creating consistent mocks of SuggestionDatabase much simpler.
* Fixed bug where the visualization preview show no value for some time.
https://user-images.githubusercontent.com/3919101/202750275-0d378d5f-1482-4637-bdcd-c428a9eac0d4.mp4
# Important Notes
The tests in controller/searcher.rs file are not of the best quality, but those will be overhauled anyway when implementing my next task.
This `Slider` component allows adjusting a numeric value with the mouse. The value is increased or decreased by clicking on the component and dragging it to the left or right.
The `Slider` has a configurable default value. `Ctrl`+clicking on the component resets its value to that default. When the value is moved away from the default, the value is printed in **bold**.
The `Slider` precision is increased or decreased by clicking the component and dragging upward or downward. This precision influences how quickly the value changes when the mouse moves horizontally, the steps in which the value is incremented or decremented, and the number of digits used to display the value. There is a margin around the component within which the precision is not changed. Beyond this margin, the precision is increased or decreased in powers of 10 (e.g. `0.1` -> `0.01` -> `0.001` when moving the mouse downwards, or `0.1` -> `1.0` -> `10.0` when moving the mouse upwards). The margin and distance between consecutive steps along the vertical axis are configurable.
The value of the `Slider` is limited to a configurable range, and cannot be adjusted beyond that range. A colored bar fills the component to indicate the current value within the range.
#### Video demonstration
https://user-images.githubusercontent.com/117099775/202244982-2f6f419d-7281-41f6-8607-7e492ad25b46.mp4
#### Future additions
This is the first iteration of the `Slider` component. Additional features are planned for the future:
* Textual editing of the value.
* Improved visual feedback on precision changes.
* Additional out-of-range behaviors.
It appears that we were always adding builtin methods to the scope of the module and the builtin type that shared the same name.
This resulted in some methods being accidentally available even though they shouldn't.
This change treats differently builtins of types and modules and introduces auto-registration feature for builtins.
By default all builtin methods are registered with a type, unless explicitly defined in the annotation property.
Builtin methods that are auto-registered do not have to be explicitly defined and are registered with the underlying type.
Registration correctly infers the right type, depending whether we deal with static or instance methods.
Builtin methods that are not auto-registered have to be explicitly defined **always**. Modules' builtin methods are the prime example.
# Important Notes
Builtins now carry information whether they are static or not (inferred from the lack of `self` parameter).
They also carry a `autoRegister` property to determine if a builtin method should be automatically registered with the type.
Libraries: Revert changes that were necessitated by a new rule we have decided not to introduce.
Parser:
- Support mixed constructors/bindings in types.
- Disallow zero-length hex sequences in character escapes: `\x`, `\u`, `\u{}`, `\U`, `\U{}` are no longer legal synonyms for `\0` (matches old parser behavior).
This change adds support for Version Controlled projects in language server.
Version Control supports operations:
- `init` - initialize VCS for a project
- `save` - commit all changes to the project in VCS
- `restore` - ability to restore project to some past `save`
- `status` - show the status of the project from VCS' perspective
- `list` - show a list of requested saves
# Important Notes
Behind the scenes, Enso's VCS uses git (or rather [jGit](https://www.eclipse.org/jgit/)) but nothing stops us from using a different implementation as long as it conforms to the establish API.
Computing length of a text takes time. Let's cache it after first computation.
# Important Notes
Wrote `StringBenchmarks` that sums lengths of (the same) `Text` present many time in a `Vector`. Initially it took `383.673 ms` per operation. Then it took `0.031 ms/op`. Looks like the `length` calls are returning instantly as they get cached.
Ensure all tokens from the input are represented in trees resulting from invalid inputs--tests now cover every reachable code line that creates an `Invalid` node. (Also implemented stricter validation, mainly of `import`/`export` statements.)
See: https://www.pivotaltracker.com/story/show/183405907
Another set of improvements extracted from #3611. This time it includes a fix to the Rust part of the parser.
# Important Notes
After digging into metadata parsing I realized the positions used to query the BTree data structure are wrong. This PR tries to address that by re-arranging the order of serialized fields and passing `startCode` and `endCode` locations in.
Originally I though I need changes on the Rust side to support `in` operator. Turned out I can do that just with changes on the Java side.
Qualified names in imports were missing UUIDs. Fixed now.
Split `HasOutputTypeLabel::output_type_label` method implementation non-generic part into a separate function. This significantly reduces compile time without risking any performance regressions. That function is currently only used for debug network visualization using graphviz, but still contributed a significant compilation time.
I've made a single attempt to profile the compiler itself, and it turned out that the compiler spent a significant amount of time trying to resolve `Pattern` implementation for closures in that method. Each of those also had to separately go through codegen and optimization. That happened for each node type in the codebase, per crate. Moving that into separate non-inline function removed all those unnecessary duplicates from. I also took this opportunity to rewrite that small piece of parsing to make it a bit cleaner.
The method of measurement is explained here: https://blog.rust-lang.org/inside-rust/2020/02/25/intro-rustc-self-profile.html
In this specific case, I've used `self-profile` to generate a profile for all crates, then used `crox` and [perfetto](https://ui.perfetto.dev/) to analyze the output.
This also shows a pattern to be aware of - using closures in generic context forces the compiler to make a separate closure type per each instantiation, even if that closure doesn't close on anything and could be a static function. This in effect forces even more instantiations or unnecessary type resolutions for all code paths that touch that closure. Using static functions or separating the non-generic part away in those cases would likely continue to help with compile times and file size.
## Comparison
The measurement was done on same machine under same environment, cleaning the build artifacts inbetween runs.
| | before | after |
|-|-|-|
|total build time|5m 5.5s|3m 59.0s|
| `ide-view-graph-editor` crate build time| 85.68s | 49.73s |
| `ide-view-component-list-panel-grid` crate build time | 49.88s | 32.07s |
| `ide-view` crate build time | 29.05s | 17.5s |
| `enso_gui.wasm` file size before wasm-opt | 83.6 MB | 80.9 MB |
| `enso_gui.wasm` file size after wasm-opt | 67.3 MB | 65.3 MB |
Make sure `libenso_parser.so`, `.dll` or `.dylib` are packaged and included when `sbt buildEngineDistribution`.
# Important Notes
There was [a discussion](https://discord.com/channels/401396655599124480/1036562819644141598) about proper location of the library. It was concluded that _"there's no functional difference between a dylib and a jar."_ and as such the library is placed in `component` folder.
Currently the old parser is still used for parsing. This PR just integrates the build system changes and makes us ready for smooth flipping of the parser in the future as part of #3611.
Wrap big and commonly copied `Application` struct into `Rc` to avoid large structure sizes for all widgets.
The `Application` struct by itself takes 704 bytes, measured by `size_of::<Application>()`. This structure is very often directly copied into components and widgets. Most notably, the generic `Widget` struct contains it. There are also cases, where structs (especially various `Model`s) contain multiple other child widgets directly, and end up indirectly copying the application multiple times.
All of `Application` clones are logical references (via `CloneRef`), so wrapping it into extra `Rc` doesn't change semantics in any way, but makes all structs that clone it way smaller. This reduces the amount of `memcpy`s and overall volume of allocated memory.
Measurement of a few example structs:
Before change:
```
size_of Application: 704 B
size_of Scrollbar: 712 B
size_of Scrollbar Model: 1576 B
```
After change:
```
size_of Application: 4 B
size_of ApplicationData: 704 B
size_of Scrollbar: 12 B
size_of Scrollbar Model: 176 B
```
Ideally we would not need to clone application reference into each component, but that's out of scope of this PR since it requires a lot more effort.
Fix bugs in `TreeToIr` (rewrite) and parser. Implement more undocumented features in parser. Emulate some old parser bugs and quirks for compatibility.
Changes in libs:
- Fix some bugs.
- Clean up some odd syntaxes that the old parser translates idiosyncratically.
- Constructors are now required to precede methods.
# Important Notes
Out of 221 files:
- 215 match the old parser
- 6 contain complex types the old parser is known not to handle correctly
So, compared to the old parser, the new parser parses 103% of files correctly.
1. Changes how we do monadic state – rather than a haskelly solution, we now have an implicit env with mutable data inside. It's better for the JVM. It also opens the possibility to have state ratained on exceptions (previously not possible) – both can now be implemented.
2. Introduces permission check system for IO actions.
Fixes https://www.pivotaltracker.com/story/show/182926584
[Task link](https://www.pivotaltracker.com/story/show/183426449)
This PR fixes an IDE freeze introduced by https://github.com/enso-org/enso/pull/3732 and reimplements reverting edited nodes to their previous state.
The cause of the IDE freeze is quite interesting. A detailed investigation is available [here](https://gist.github.com/vitvakatu/785e34881368b8cfda61715d7543cbd0).
The graph editor needs to update the Presenter state only if the user is editing the node. Before this PR, the graph editor notified the Presenter with a visual representation of the node content instead of code expression. It caused inconsistency between the states of the controller and Presenter and caused severe performance issues.
https://user-images.githubusercontent.com/6566674/195831224-6d6e8258-e347-48b4-890a-d89c7300bc39.mp4
# Important Notes
- ~~There is a more complex alternative solution – it requires refactoring of the `component::node::input::area` module. The Presenter can be notified with `expression.code` changes, not `expression.viz_code`. I found a simpler solution (`.gate(&edit_mode)`), which has the same effect but does not require additional refactoring.~~ Said solution is implemented in a separate commit
We've had an old attempt at integrating a Rust parser with our Scala/Java projects. It seems to have been abandoned and is not used anywhere - it is also superseded by the new integration of the Rust parser. I think it was used as an experiment to see how to approach such an integration.
Since it is not used anymore - it make sense to remove it, because it only adds some (slight, but non-zero) maintenance effort. We can always bring it back from git history if necessary.
When hovering the mouse pointer over the Marketplace button on the left bar of the Component Browser, show a caption informing that the Marketplace will be available soon.
https://www.pivotaltracker.com/story/show/182613789
#### Visuals
The video below demonstrates the caption shown when hovering the Marketplace button on the left bar of the Component Browser. It shows the caption disappearing after a hardcoded time, or when the mouse pointer is moved away from the button.
https://user-images.githubusercontent.com/273837/196195809-45a712e1-ad86-47d8-99ff-1475a0b74c6e.mov
# Important Notes
- The "Label" visual component was fixed. Previously, the width calculation of the background was not synchronized correctly with the text width. As a result, a zero-width background was displayed when a Label was shown for the first time.
There is one place in code, where we can potentially panic on double borrow, because the drop routine is done while having data borrowed.
# Important Notes
The issue was easily reproducible on @Frizi machine.
Another part of #3611 ready for integration into `develop` branch.
# Important Notes
Test `org.enso.compiler.EnsoCompilerTest.testTestGroup` is ignored as it has problems with source offsets - identifiers don't have the appropriate names due to `Tree.codeRepr()` being _off_.
- Special precedence rules for case-of so that `:` operator works without parens or nospace-grouping.
- Support an old-lambda syntax: `x->x-> x`. According to the usual rules, the first nospace group would be parsed as an operator section. The expression now parses as a lambda that contains a lambda.
- Match old parser treatment of # in doc comments.
- Tweak precedence so (a : B = c) works.
- Documented constructors.
- Reimplement the `Duration` type to a built-in type.
- `Duration` is an interop type.
- Allow Enso method dispatch on `Duration` interop coming from different languages.
# Important Notes
- The older `Duration` type should now be split into new `Duration` builtin type and a `Period` type.
- This PR does not implement `Period` type, so all the `Period`-related functionality is currently not working, e.g., `Date - Period`.
- This PR removes `Integer.milliseconds`, `Integer.seconds`, ..., `Integer.years` extension methods.
This PR introduced an overhauled Component List Panel implementation, making use of the efficient EnsoGL grid view component. Also, it delivers a couple of new features:
* A part of the new design: there are no more section headers in grid, instead groups are "glued" together. The local scope section is under "popular" (old "favorites").
* The keyboard management inside grid works.
* there is a mouse hover highlight
* selecting the lowest entry in section when jumping with navigation bar.
* accepting input as-is with cmd/ctrl + Enter.
https://user-images.githubusercontent.com/3919101/194561890-fffb9b41-2f0d-4357-8d9a-5038a6bcb023.mp4
### Important Notes
**What is not implemented:**
* [Focus management between panels.](https://www.pivotaltracker.com/story/show/180872763) The grid is always focused. To accept the current input, use ctrl+Enter shortcut.
* [Proper handling of selection when having empty space on the right and pressing right arrow.](https://www.pivotaltracker.com/story/show/183487880)
* When entering a module, its name is not added to the input as described in the design doc. Will be a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
**Known issues**
* [the selection, especially in the local scope section, has sometimes an undesirable offset](https://www.pivotaltracker.com/story/show/183487730). The cause is known, but not so easy to fix.
* The inserted nodes are often producing errors. The Browser's inherits the outdated understanding of the language from old Node Searcher, and it does not include new form of imports, static methods etc. Those all will be fixed as a part of [this User Story](https://www.pivotaltracker.com/story/show/181058321).
* The performance is improved, but still not ideal, due to problems in [text areas](https://www.pivotaltracker.com/story/show/183406745).
* To scroll the documentation panel, you must first click on it.
- Implement macro-contexts-lite (`from` is now only a keyword at the beginning of a line)
- Support special nospace-group handling for old lambdas (so expressions like this work: `x-> y-> x + y`)
- Fix a text-escape incompatibility
# Important Notes
- There is now an `OperatorFunction`, which is like a `Function` but has an operator for a name, and likewise an `OperatorTypeSignature`.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/182675703)
This PR implements the actual integration of the breadcrumbs panel with the component list panel. A special breadcrumbs controller (`controller::searcher::breadcrumbs`) is tracking the currently opened module with a list of its parents. The searcher presenter uses the API of the controller to sync the displayed list of breadcrumbs with the controller state.
https://user-images.githubusercontent.com/6566674/193064122-7d3fc4d6-9148-4ded-a73e-767ac9ac83f8.mp4
# Important Notes
- There is an `All` breadcrumb displayed at all times at the beginning of the list. It will be replaced with a section name as part of [Section Title on Component Browser's Breadcrumbs Panel](https://www.pivotaltracker.com/story/show/182610561) task.
- I changed the implementation of `project::main_module_id`, `project::QualifiedName::main_module`, and `API::main_module` so that they are logically connected which each other.
- I adjusted the Breadcrumbs View to avoid "appearance" animation glitches when opening new modules. `set_entries` was replaced with the `set_entries_from` endpoint.
When running the profiling run-graph and flamegraph demo scenes, if a profile file is not found in the directory served over http, fall back to generating demo data.
This PR disables the wasm-opt optimization in the crates that can be used as WASM entry points. Unfortunately, wasm-pack does not allow for disabling wasm-opt through a command line flag, so we have to disable it by setting an appropriate flag in each Cargo.toml.
Use an `ArraySlice` to slice `Vector`.
Avoids memory copying for the slice function.
# Important Notes
| Test | Ref | New |
| --- | --- | --- |
| New Vector | 71.9 | 71.0 |
| Append Single | 26.0 | 27.7 |
| Append Large | 15.1 | 14.9 |
| Sum | 156.4 | 165.8 |
| Drop First 20 and Sum | 171.2 | 165.3 |
| Drop Last 20 and Sum | 170.7 | 163.0 |
| Filter | 76.9 | 76.9 |
| Filter With Index | 166.3 | 168.3 |
| Partition | 278.5 | 273.8 |
| Partition With Index | 392.0 | 393.7 |
| Each | 101.9 | 102.7 |
- Note: the performance of New and Append has got slower from previous tests.
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720
real 0m4.503s
user 0m9.248s
sys 0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720
real 0m0.176s
user 0m0.042s
sys 0m0.038s
```
# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.
Will need to add the above command to CI.
When a GridView is navigated using the keyboard, scroll it to display the newly selected entry.
https://www.pivotaltracker.com/story/show/182593635
#### Visuals
See below for a video demonstrating automatic scrolling of the GridView when arrow keys are pressed on the keyboard. The first video below shows the default scrolling behavior in a GridView without headers.
Note:
- When the Grid View is scrolled, the mouse hover highlight moves away from the mouse position. This is not a new regression, the behavior is the same in the `develop` branch (e.g. when scrolling using the mouse).
https://user-images.githubusercontent.com/273837/190183984-91f7808c-3606-43f8-bcda-ac4d5f84e00f.mov
The video below shows the behavior in a GridView with headers when the GridView is first scrolled to its top-left corner. The following guidance from the Design Doc is enabled and showcased:
> Users can change the selected component by pressing the arrow keys. The Focus does not move up if it does not have to (in most cases, the focus is located in the second row from the bottom). Instead, the component list scrolls down if there are enough entries.
https://user-images.githubusercontent.com/273837/189151546-e50aaf22-6f4d-41cb-809f-60038305745f.mov
The next video shows the behavior in the same GridView as in the previous example when the GridView is first scrolled away from any of its boundary entries. Notably, scrolling happens only when the selection is moved using the keyboard arrow keys, not when changing the selection using the mouse. This behavior is based on a discussion with @wdanilo on Discord.
https://user-images.githubusercontent.com/273837/189151974-d992be93-f61f-4e9f-9f4c-dfe260bbec5b.mov
This PR contains an entry definition for Grid View to be used inside Component List Panel View. The Example grid view with the entry definition may be seen on new_component_list_panel_view debug scene.
https://user-images.githubusercontent.com/3919101/190663278-23c35ab0-f426-4001-8128-df7147aafb9e.mp4
# Important Notes
* The styling is not detailed yet due to time constraints (I want to move to integration this grid view to Component Panel List ASAP) and the fact that I could not get new mplus1 font working with text Area.
* Implementing this required adding a "contour offset" feature to the Grid View.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181445628).
This PR implements a Breadcrumbs panel for the new component browser.
The Breadcrumbs is a horizontal list of text labels separated by a special icon and has an optional ellipsis icon at the end.
It is implemented using the new GridView component.
Video:
Demo of adding new breadcrumbs, scrolling behavior, and selecting breadcrumbs with the mouse.
https://user-images.githubusercontent.com/6566674/189199432-77807cef-00dc-4abe-b95c-b17a536f59f6.mp4
Demo of selecting breadcrumbs with keyboard shortcuts:
https://user-images.githubusercontent.com/6566674/189199603-53e55335-73ba-4ed7-8291-4455144c06aa.mp4
# Important Notes
- This PR implements an old interaction of the design of the component browser. The new design of the breadcrumbs can not be easily integrated into the current look of the component browser, so we would need to update styles later. It should be a relatively simple task. *The implementation uses color from the new design though. (but not fonts and sizes)*
- I found a bug in the grid view implementation that causes panics at runtime in some conditions. The reason is triggering FRP endpoints while constructing new entries. This issue is fixed in the PR.
Add support for moving the selection in a Grid View using the keyboard.
https://www.pivotaltracker.com/story/show/182585789
#### Visuals
See below for videos showcasing GridView selection keyboard navigation in the `grid_view` debug scene. In the videos, messages in the Developer Console can be observed. When a keypress would result in the selection being moved out of the GridView, the selection is not moved and a message is emitted in the Developer Console instead, showcasing an FRP output signal emitted on such event. Please note that the videos are recorded with the tracing level changed to `DEBUG`. In a default build, the tracing level is set to `WARN`, and the messages visible in the videos are not displayed in the Developer Console.
https://user-images.githubusercontent.com/273837/188483972-89d79f7b-1303-457b-869f-282e0809a755.movhttps://user-images.githubusercontent.com/273837/188484294-e9b6461c-a84f-4817-9447-d792f2ebdbb5.mov
The following video shows moving the selection between "regular" entries and header entries. It also shows a current usability limitation of the selection keyboard navigation feature, such that the Grid is not scrolled when the selection leaves the visible part of the Grid, and the selection may thus disappear from view.
https://user-images.githubusercontent.com/273837/188485238-29a82b27-de2f-4cf8-a2e7-ff8c3f41478d.mov
# Important Notes
- Keyboard navigation only works when a GridView has focus.
- Selection keyboard navigation only works if the selection was already set to some entry beforehand.
- If keyboard navigation would move selection outside of the grid, the selection movement is canceled and an FRP event is emitted.
`Vector` type is now a builtin type. This requires a bunch of additional builtin methods for its creation:
- Use `Vector.from_array` to convert any array-like structure into a `Vector` [by copy](f628b28f5f)
- Use (already existing) `Vector.from_polyglot_array` to convert any array-like structure into a `Vector` **without** copying
- Use (already existing) `Vector.fill 1 item` to create a singleton `Vector`
Additional, for pattern matching purposes, we had to implement a `VectorBranchNode`. Use following to match on `x` being an instance of `Vector` type:
```
import Standard.Base.Data.Vector
size = case x of
Vector.Vector -> x.length
_ -> 0
```
Finally, `VectorLiterals` pass that transforms `[1,2,3]` to (roughly)
```
a1 = 1
a2 = 2
a3 = 3
Vector (Array (a1,a2, a3))
```
had to be modified to generate
```
a1 = 1
a2 = 2
a3 = 3
Vector.from_array (Array (a1, a2, a3))
```
instead to accomodate to the API changes. As of 025acaa676 all the known CI checks passes. Let's start the review.
# Important Notes
Matching in `case` statement is currently done via `Vector_Data`. Use:
```
case x of
Vector.Vector_Data -> True
```
until a better alternative is found.
Parse text literals. See: https://www.pivotaltracker.com/story/show/182496940
# Important Notes
- The left-trimming algorithm (https://github.com/enso-org/design/blob/wip/wd/enso-spec/epics/enso-spec-1.0/04.%20Expressions.md#inline-and-block-text-literals) requires two passes over the sequence of text segments. This implementation performs one pass while parsing (identifying the correct amount of trim). The other pass (applying the trim) can be done when building the value of the quoted string: Trim the amount of whitespace identified by the `trim` field off of the whitespace of each `TextSection` (the value will not exceed the amount of whitespace found in the tokens' offsets, except for tokens with 0 offset, in which case no trimming is necessary/possible).
Implements:
- UUIDs: https://www.pivotaltracker.com/story/show/182931137
- Comments: https://www.pivotaltracker.com/story/show/182981779
- Type annotations and signatures: https://www.pivotaltracker.com/story/show/182497454
- Fix getter names (https://github.com/enso-org/enso/pull/3627#discussion_r940887460).
# Important Notes
- I can't fully test UUIDs; I have tested that the data obtained in Rust matches my understanding of how the format is supposed to work. What remains to be tested is that the data in Java matches the way the old parser handles the format. So @JaroslavTulach, let me know if you see any cases where I'm not returning the same values.
- This implementation of type annotations and signatures accepts any expression in type context. It would probably be nice to narrow this down at some point, but for now I have no design info on what specifically should be allowed in type expressions; this implementation should be at least an incremental improvement.
This is a step towards the new language spec. The `type` keyword now means something. So we now have
```
type Maybe a
Some (from_some : a)
None
```
as a thing one may write. Also `Some` and `None` are not standalone types now – only `Maybe` is.
This halfway to static methods – we still allow for things like `Number + Number` for backwards compatibility. It will disappear in the next PR.
The concept of a type is now used for method dispatch – with great impact on interpreter code density.
Some APIs in the STDLIB may require re-thinking. I take this is going to be up to the libraries team – some choices are not as good with a semantically different language. I've strived to update stdlib with minimal changes – to make sure it still works as it did.
It is worth mentioning the conflicting constructor name convention I've used: if `Foo` only has one constructor, previously named `Foo`, we now have:
```
type Foo
Foo_Data f1 f2 f3
```
This is now necessary, because we still don't have proper statics. When they arrive, this can be changed (quite easily, with SED) to use them, and figure out the actual convention then.
I have also reworked large parts of the builtins system, because it did not work at all with the new concepts.
It also exposes the type variants in SuggestionBuilder, that was the original tiny PR this was based on.
PS I'm so sorry for the size of this. No idea how this could have been smaller. It's a breaking language change after all.
- Added `Zone`, `Date_Time` and `Time_Of_Day` to `Standard.Base`.
- Renamed `Zone` to `Time_Zone`.
- Added `century`.
- Added `is_leap_year`.
- Added `length_of_year`.
- Added `length_of_month`.
- Added `quarter`.
- Added `day_of_year`.
- Added `Day_Of_Week` type and `day_of_week` function.
- Updated `week_of_year` to support ISO.
# Important Notes
- Had to pass locale to formatter for date/time tests to work on my PC.
- Changed default of `week_of_year` to use ISO.
* Builtin Date_Time, Time_Of_Day, Zone
Improved polyglot support for Date_Time (formerly Time), Time_Of_Day and
Zone. This follows the pattern introduced for Enso Date.
Minor caveat - in tests for Date, had to bend a lot for JS Date to pass.
This is because JS Date is not really only a Date, but also a Time and
Timezone, previously we just didn't consider the latter.
Also, JS Date does not deal well with setting timezones so the trick I
used is to first call foreign function returning a polyglot JS Date,
which is converted to ZonedDateTime and only then set the correct
timezone. That way none of the existing tests had to be changes or
special cased.
Additionally, JS deals with milliseconds rather than nanoseconds so
there is loss in precision, as noted in Time_Spec.
* Add tests for Java's LocalTime
* changelog
* Make date formatters in table happy
* PR review, add more tests for zone
* More tests and fixed a bug in column reader
Column reader didn't take into account timezone but that was a mistake
since then it wouldn't map to Enso's Date_Time.
Added tests that check it now.
* remove redundant conversion
* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Time.enso
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
* First round of addressing PR review
* don't leak java exceptions in Zone
* Move Date_Time to top-level module
* PR review
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/182955595)
This PR implements variable column widths in the new Grid View component. We need this feature to quickly implement various parts of the UI, including the breadcrumbs panel of the component browser.
There are two ways to change the width of the specific column:
1. "From the outside", using the `set_column_width` endpoint of the Grid View
2. "From the inside", using the `override_column_width` endpoint of the EntryFrp.
Both ways work similarly, but the latter is helpful for our breadcrumbs implementation, as it allows for entry to decide on the width of the column by its content.
See the screencast with three grid views. The top-left one has every even column shrunk by GridView API. Every grid view has a second column extended by EntryFrp API.
https://user-images.githubusercontent.com/6566674/185060985-7b7df076-c659-41fa-977a-22875493f8d4.mp4
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181870555)
This PR changes the relative position of the edited node in such a way that it is left-aligned to the component browser window. This change reflects the most recent version of the [design doc](https://github.com/enso-org/design/blob/main/epics/component-browser/design.md#overview)
<img width="1157" alt="Screenshot 2022-08-08 at 19 15 47" src="https://user-images.githubusercontent.com/6566674/183454192-81960e0a-ab69-43a4-b7df-d13320a9d16d.png">
As an additional change, the FRP implementation of the `Camera2d` was extended with a new output (screen dimensions) and fixed. With the old implementation, there was a possibility of panic at runtime because of non-exclusive borrows of `RefCell`. The FRP event for camera position was emited inside the scope with a mutable `RefCell` borrow. Any attempt to borrow the camera one more time (e.g., by calling one of the getters, such as `zoom()`) caused panic at runtime.
Importing individual methods didn't work as advertised because parser
would allow them but later drop that information. This slipped by because we never had mixed atoms and methods in stdlib.
# Important Notes
Added some basic tests but we need to ensure that the new parser allows for this.
@jdunkerley will be adding some changes to stdlib that will be testing this functionality as well.
Based on usage; I believe this handles every case in current `.enso` files.
# Important Notes
- `import` is a built-in macro, so an import statement parses as a `MultiSegmentApp`.
- Every `import` syntax will have a segment whose leading keyword is `import`; however `import` macros can be identified more efficiently by looking at only the first keyword. A `MultiSegmentApp` is an import if and only if its first keyword is in the set { "polyglot", "from", "import" }.
Show custom icons in Component Browser for entries that have a non-empty `Icon` section in their docs with the section's body containing a name of a predefined icon.
https://www.pivotaltracker.com/story/show/182584336
#### Visuals
A screenshot of a couple custom icons in the Component Browser:
<img width="346" alt="Screenshot 2022-07-27 at 15 55 33" src="https://user-images.githubusercontent.com/273837/181265249-d57f861f-8095-4933-9ef6-e62644e11da3.png">
# Important Notes
- The PR assigns icon names to four items in the standard library, but only three of them are shown in the Component Browser because of [a parsing bug in the Engine](https://www.pivotaltracker.com/story/show/182781673).
- Icon names are assigned only to four items in the standard library because only two currently predefined icons match entries in the currently defined Virtual Component Groups. Adjusting the definitions of icons and Virtual Component Groups is covered by [a different task](https://www.pivotaltracker.com/story/show/182584311).
- A bug in the documentation of the Enso protocol message `DocSection` is fixed. A `text` field in the `Tag` interface is renamed to `body` (this is the field name used in Engine).
This PR adds a new variant of selection, where the mouse-hovered entry is highlighted and may be selected by clicking.
In the video below, we have three grid views with slightly different settings:
* In the left-top corner, both hover and selection highlight is just a shape under the label. Such a grid view does not require additional layers (when compared to non-selectable grid view).
* In the left-bottom corner the hover is normal shape, but selection is a _masked layer_ which allows us to have different text color. This setting requires three more layers to render.
* In the right-top corner, both hover and selection are displayed in the masked layer, creating 6 additional layers.
https://user-images.githubusercontent.com/3919101/181514178-f243bfeb-f2dd-4507-adc3-5344ae0579b7.mp4
This change allows for importing modules using a qualified name and deals with any conflicts on the way.
Given a module C defined at `A/B/C.enso` with
```
type C
type C a
```
it is now possible to import it as
```
import project.A
...
val x = A.B.C 10
```
Given a module located at `A/B/C/D.enso`, we will generate
intermediate, synthetic, modules that only import and export the successor module along the path.
For example, the contents of a synthetic module B will look like
```
import <namespace>.<pkg-name>.A.B.C
export <namespace>.<pkg-name>.A.B.C
```
If module B is defined already by the developer, the compiler will _inject_ the above statements to the IR.
Also removed the last elements of some lowercase name resolution that managed to survive recent
changes (`Meta.Enso_Project` would now be ambiguous with `enso_project` method).
Finally, added a pass that detects shadowing of the synthetic module by the type defined along the path.
We print a warning in such a situation.
Related to https://www.pivotaltracker.com/n/projects/2539304
# Important Notes
There was an additional request to fix the annoying problem with `from` imports that would always bring
the module into the scope. The changes in stdlib demonstrate how it is now possible to avoid the workaround of
```
from X.Y.Z as Z_Module import A, B
```
(i.e. `as Z_Module` part is almost always unnecessary).
Fixes random timeout failures on CI.
```
INFO ide_ci::program::command: sbtℹ️ up to date, audited 98 packages in 3s
```
`npm install` takes 3s of test time doing unnecessary package auditing. On CI the command is executed once before running the tests and redundant `npm install` calls can be omitted.
This change modifies the current language by requiring explicit `self` parameter declaration
for methods. Methods without `self` parameter in the first position should be treated as statics
although that is not yet part of this PR. We add an implicit self to all methods
This obviously required updating the whole stdlib and its components, tests etc but the change
is pretty straightforward in the diff.
Notice that this change **does not** change method dispatch, which was removed in the last changes.
This was done on purpose to simplify the implementation for now. We will likely still remove all
those implicit selfs to bring true statics.
Minor caveat - since `main` doesn't actually need self, already removed that which simplified
a lot of code.
Provide a JNI dynamic-library interface to `enso_parser`.
# Important Notes
- The library can be built with: `cargo build -p enso-parser-jni`.
- A new `org.enso.syntax2.Parser` API is implemented on top of the JNI interface provided by `enso-parser-jni`.
- We are using the `jni` crate, since apparently Java cannot just call C-ABI functions. The crate is not well-maintained. I came across an obviously-unsound `safe` function, and found it was reported over a year ago, with a PR to fix: jni-rs/jni-rs#303. However our needs are simple. We can't trust any safety guarantees they imply, but I think we are unlikely to encounter any logic bugs using the basic bindings.
Significantly improves the polyglot Date support (as introduced by #3374). It enhances the `Date_Spec` to run it in four flavors:
- with Enso Date (as of now)
- with JavaScript Date
- with JavaScript Date wrapped in (JavaScript) array
- with Java LocalDate allocated directly
The code is then improved by necessary modifications to make the `Date_Spec` pass.
# Important Notes
James has requested in [#181755990](https://www.pivotaltracker.com/n/projects/2539304/stories/181755990) - e.g. _Review and improve InMemory Table support for Dates, Times, DateTimes, BigIntegers_ the following program to work:
```
foreign js dateArr = """
return [1, new Date(), 7]
main =
IO.println <| (dateArr.at 1).week_of_year
```
the program works with here in provided changes and prints `27` as of today.
@jdunkerley has provided tests for proper behavior of date in `Table` and `Column`. Those tests are working as of [f16d07e](f16d07e640). One just needs to accept `List<Value>` and then query `Value` for `isDate()` when needed.
Last round of changes is related to **exception handling**. 8b686b12bd makes sure `makePolyglotError` accepts only polyglot values. Then it wraps plain Java exceptions into `WrapPlainException` with `has_type` method - 60da5e70ed - the remaining changes in the PR are only trying to get all tests working in the new setup.
The support for `Time` isn't part of this PR yet.
**Note**: This PR also contains content of previous Grid View PR. We decided to discard the previous, because this one did some refactoring of old one, and it's not a big addition.
Added a scrollable::GridView component, which just embeds the GridView in ScrollArea. Also, re-worked the idea of text layers.
https://user-images.githubusercontent.com/3919101/179020359-512ee127-c333-4f86-bff5-f1cb4154e03c.mp4
This PR contains all work for finishing integration of first Component List Panel in the IDE:
* It adds a stub for the whole Component Browser View. The documentation panel is re-used from the old searcher.
* It has the presenter implementation, integrating the view with Hierarchical Component List from the controller.
* It extends the View API, so the integration is possible, making use of Component Group Set wrapper.
* The selection integration was also merged into this PR, because it depended on the API extension mentioned above. However, we should avoid such practice in the future.
https://user-images.githubusercontent.com/3919101/177816427-8c4285b4-8941-4048-a400-52f4acf77a9f.mp4
# Important Notes
There are some known issues, to-be-fixed in the future.
* The performance is bad. It should be improved with new text::Area, and the decent one shall come with [GridView inside component browser](https://www.pivotaltracker.com/story/show/182561072)
* There is no keyboard navigation. It should also be delivered with [GridView](https://www.pivotaltracker.com/story/show/182561072).
* The Favorites section is not [filtered out by node source type](https://www.pivotaltracker.com/story/show/182661634).
implement simple variable assignments and function definitions.
This implements:
- https://www.pivotaltracker.com/story/show/182497122
- https://www.pivotaltracker.com/story/show/182497144 (the code blocks are not created yet, but the function declaration is recognized.)
# Important Notes
- Introduced S-expression-based tests, and pretty-printing-roundtrip testing.
- Started writing tests for TypeDef based on the examples in the issue. None of them parse successfully.
- Fixed Number tokenizing.
- Moved most contents of parser's `main.rs` to `lib.rs` (fixes a warning).
Modified UppercaseNames to now resolve methods without an explicit `here` to point to the current module.
`here` was also often used instead of `self` which was allowed by the compiler.
Therefore UppercaseNames pass is now GlobalNames and does some extra work -
it translated method calls without an explicit target into proper applications.
# Important Notes
There was a long-standing bug in scopes usage when compiling standalone expressions.
This resulted in AliasAnalysis generating incorrect graphs and manifested itself only in unit tests
and when running `eval`, thus being a bit hard to locate.
See `runExpression` for details.
Additionally, method name resolution is now case-sensitive.
Obsolete passes like UndefinedVariables and ModuleThisToHere were removed. All tests have been adapted.
Implement generation of Java AST types from the Rust AST type definitions, with support for deserializing in Java syntax trees created in Rust.
### New Libraries
#### `enso-reflect`
Implements a `#[derive(Reflect)]` macro to enable runtime analysis of datatypes. Macro interface includes helper attributes; **the Rust types and the `reflect` attributes applied to them fully determine the Java types** ultimately produced (by `enso-metamodel`). This is the most important API, as it is used in the subject crates (`enso-parser`, and dependencies with types used in the AST). [Module docs](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/reflect/macros/src/lib.rs).
#### `enso-metamodel`
Provides data models for data models in Rust/Java/Meta (a highly-abstracted language-independent model--I have referred to it before as the "generic representation", but that was an overloaded term).
The high-level interface consists of operations on data models, and between them. For example, the only operations needed by [the binary that drives datatype transpilation](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/parser/generate-java/src/main.rs) are: `rust::to_meta`, `java::from_meta`, `java::transform::optional_to_null`, `java::to_syntax`.
The low-level interface consists of direct usage of the datatypes; this is used by [the module that implements some serialization overrides](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/parser/generate-java/src/serialization.rs) (so that the Java interface to `Code` references can produce `String`s on demand based on serialized offset/length pairs). The serialization override mechanism is based on customizing, not replacing, the generated deserialization methods, so as to be as robust as possible to changes in the Rust source or in the transpilation process.
### Important Notes
- Rust/Java serialization is exhaustively tested for structural compatibility. A function [`metamodel::meta::serialization::testcases`](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/metamodel/src/meta/serialization.rs) uses `reflect`-derived data to generate serialized representations of ASTs to use as test cases. Its should-accept cases cover every type a tree can contain; it also produces a representative set of should-reject cases. A Rust `#[test]` confirms that these cases are accepted/rejected as expected, and generated Java tests (see Binaries below) check the generated Java deserialization code against the same test cases.
- Deserializing `Code` is untested. The mechanism is in place (in Rust, we serialize only the offset/length of the `Cow`; in Java, during deserialization we obtain a context object holding a buffer for all string data; the accessor generated in Java uses the buffer and the offset/length to return `String`s), but it will be easier to test once we have implemented actually parsing something and instantiating the `Cow`s with source code.
- `#[tagged_enum]` [now supports](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/shapely/macros/src/tagged_enum.rs#L36-L51) control over what is done with container-level attributes; they can be applied to the container and variants (default), only to the container, or only to variants.
- Generation of `sealed` classes is supported, but currently disabled by `TARGET_VERSION` in `metamodel::java::syntax` so that tests don't require Java 15 to run. (The same logic is run either way; there is a shallow difference in output.)
### Binaries
The `enso-parser-generate-java` crate defines several binaries:
- `enso-parser-generate-java`: Performs the transpilation; after integration, this will be invoked by the build script.
- `java-tests`: Generates the Java code that tests format deserialization; after integration this command will be invoked by the build script, and its Java output compiled and run during testing.
- `graph-rust`/`graph-meta`/`graph-java`: Produce GraphViz representations of data models in different typesystems; these are for developing and understanding model transformations.
Until integration, a **script regenerates the Java and runs the format tests: `./tools/parser_generate_java.sh`**. The generated code can be browsed in `target/generated_java`.
If a node created by the user gets placed off-screen, the screen's camera is panned to make the node visible.
https://www.pivotaltracker.com/story/show/181188687
#### Visuals
A screencast showing a number of node creation scenarios when the camera is panned to the newly created node, including when zoomed out.
https://user-images.githubusercontent.com/273837/177169716-50a12b0a-c742-4b01-9766-56206e7938b9.mov
# Important Notes
- Camera is panned also if the node is only partially visible, or if there's not enough free space visible around the node. The specific amount of free space that needs to be visible around a newly created node is configured in the theme.
- If the screen area is so small that the node cannot be fully fit in it (either horizontally or vertically), showing the left and top boundaries of the node's area takes priority over showing the corresponding opposite edges.
Implement simple placeholder icons for all entry kinds supported in the Suggestion Database. The icons are planned to be used in Component Browser as default icons for entries. This is intended to allow visually distinguishing different entry kinds.
The following additional fixes and tweaks are applied:
- Icons previously using only 1 color from the theme now use the color provided through shape parameters instead.
- The `data_science` and `network` icons now use only the 2 colors provided through shape parameters.
- The `join` icon has its shape and colors modified and uses only the 2 colors provided through shape parameters.
- The demo scene now parametrizes icon shapes using colors from the Component Browser Design Doc.
https://www.pivotaltracker.com/story/show/182584322
#### Visuals
Original contents of the demo scene before the PR:
<img width="2197" alt="x-orig" src="https://user-images.githubusercontent.com/273837/176669422-ee2e14c7-9ef4-42fd-acb7-ae3be6b68587.png">
Final contents of the demo scene after the PR:
<img width="2201" alt="x2-final" src="https://user-images.githubusercontent.com/273837/176668720-6f1685fd-f7e6-44d7-85f5-f6a6d6789644.png">
A semi-manual s/this/self appied to the whole standard library.
Related to https://www.pivotaltracker.com/story/show/182328601
In the compiler promoted to use constants instead of hardcoded
`this`/`self` whenever possible.
# Important Notes
The PR **does not** require explicit `self` parameter declaration for methods as this part
of the design is still under consideration.
[Task link](https://www.pivotaltracker.com/story/show/182194574).
[ci no changelog needed]
This PR implements a new selection box that will replace an old (not really working) one in the component browser. The old selection box wasn't working well with the headers of the component groups, so we were forced to make a much harder implementation.
The new implementation duplicates some visual components and places them in a separate layer. Then, a rectangular mask cuts off everything that is not "selected". This way:
- We have more control over what the selected entries should look like.
- We can easily support the multi-layer structure of the component groups with headers.
- We avoid problems with nested masks that our renderer doesn't support at the moment.
To be more precise, we duplicate the following:
- Background of the component group becomes the "fill" of the selection.
- Entries text and icons - we can alter them easily.
- Header background and header text. By placing them in separate scene layers we ensure correct rendering order.
https://user-images.githubusercontent.com/6566674/173657899-1067f538-4329-44f9-9dc2-78c8a4708b5a.mp4
# Important Notes
- This PR implements the base of our future selection mechanism, selecting entries with a mouse and keyboard still has several issues that will be fixed in the future tasks.
- The scrolling behavior will also be improved in future tasks. Right we only restrict the selection box position so that it never leaves the borders of the component group.
- I added a new function to `add` shapes to new layers in a non-exclusive way (we had only `add_exclusive`) before. I have no idea how we didn't use this feature before even though we mention it a lot in the docs.
- The demo scene restricts the position of the selection box for one-column component groups but does not for the wide component group.
Remove a `Symbol`from the `SymbolRegistry` when its `SpriteSystem` is dropped.
This fixes the remaining buffer leak (after #3504) in https://www.pivotaltracker.com/story/show/181943457
# Important Notes
- The `SymbolRegistry` now assigns unique `SymbolId`s, so that we can tell if a `SymbolId` refers to a `Symbol` that has already been unregistered (this shouldn't happen, but it's not statically-obvious that it doesn't, so if it occurs we shouldn't misbehave).
- Also fix a bug in how `buffer_count` was tracked (we were decrementing more than incrementing!).
Auto-generate all builtin methods for builtin `File` type from method signatures.
Similarly, for `ManagedResource` and `Warning`.
Additionally, support for specializations for overloaded and non-overloaded methods is added.
Coverage can be tracked by the number of hard-coded builtin classes that are now deleted.
## Important notes
Notice how `type File` now lacks `prim_file` field and we were able to get rid off all of those
propagating method calls without writing a single builtin node class.
Similarly `ManagedResource` and `Warning` are now builtins and `Prim_Warnings` stub is now gone.
[ci no changelog needed]
This PR implements a new helper for the future Component Browser - `component_group::multi::Wrapper`. It propagates FRP events from multiple component groups and ensures that only a single component group is focused at all times.
See the updated component group demo scene (console logs shows propagated FRP events from all component groups):
https://user-images.githubusercontent.com/6566674/172359141-8ea6f1ba-e357-4c1b-852a-adb4d5207e03.mp4
- Fixed a `define_endpoints_2!` macro. FRP endpoints for `focus` events weren't connected properly.
- List View now uses an overlay shape to catch mouse events, it allows much easier implementation of `is_header_selected` in the component group.
Drop `Core` implementation (replacement for IR) as it (sadly) looks increasingly
unlikely this effort will be continued. Also, it heavily relies
on implicits which increases some compilation time (~1sec from `clean`)
Related to https://www.pivotaltracker.com/story/show/182359029
This PR introduces a new structure delivered by Searcher Controller: The Component List.
The Component list is hierarchical, and its structure reflects how the components are displayed in Component Browser - only "submodule" section at this point, other sections will be covered in next tasks.
This does not introduce anything visual; the structures are tested in unit tests.
- Change dev profile settings. Improves build performance; will not affect anything else. Details below.
- Introduce script for benchmarking various incremental builds. Usage is explained in the script comments.
- Add a line to `intervals` showing total main-thread CPU work logged in a profile; this can be used to compare the results of optimizations (I'll be starting a discussion informed by that data separately; this change just enables the tooling to report it).
Define some workflows for batch-mode profiling.
Implemented:
- collapse nodes
- create node
- enter collapsed node
- new project
- open visualization
They can currently be built and run with a command like:
`./run.sh ide build --profiling-level=debug && dist/ide/linux-unpacked/enso --entry-point profile --workflow create_node --save-profile out.json`
And the data can be displayed with:
`dist/ide/linux-unpacked/enso --entry-point profiling_run_graph --load-profile out.json`
Demo of recording and viewing a profile with a command-line one-liner:
https://user-images.githubusercontent.com/1047859/169954795-2d9520ca-84f9-45d2-b83a-5063ebe6f718.mp4
See: https://www.pivotaltracker.com/story/show/182195399.
# Important Notes
- When defining workflows, two helpers are enough to allow us to tell when the action is really done: `Fixture::compile_new_shaders`, and `Fixture::backend_execution`. Often, it is appropriate to await both, but it depends on the task.
- The shader compiler is now driven by a `Controller`; while the `Compiler` is reset if context is lost, the `Controller`'s state survives context loss.
- A new `--load-profile` option supports specifying a profile by path when running `profiling_run_graph`.
- Drop the `with_same_start` profiler interface; we ended up preferring a child profiler convention, and this interface was not implemented compatibly with the stricter data model we've had since the introduction of `profiler::data`.
- Fix the noisy `rustfmt` output.
This change introduces a custom LogManager for console that allows for
excluding certain log messages. The primarily reason for introducing
such LogManager/Appender is to stop issuing hundreds of pointless
warnings coming from the analyzing compiler (wrapper around javac) for
classes that are being generated by annotation processors.
The output looks like this:
```
[info] Cannot install GraalVM MBean due to Failed to load org.graalvm.nativebridge.jni.JNIExceptionWrapperEntryPoints
[info] compiling 129 Scala sources and 395 Java sources to /home/hubert/work/repos/enso/enso/engine/runtime/target/scala-2.13/classes ...
[warn] Unexpected javac output: warning: File for type 'org.enso.interpreter.runtime.type.ConstantsGen' created in the last round will not be subject to annotation processing.
[warn] 1 warning.
[info] [Use -Dgraal.LogFile=<path> to redirect Graal log output to a file.]
[info] Cannot install GraalVM MBean due to Failed to load org.graalvm.nativebridge.jni.JNIExceptionWrapperEntryPoints
[info] foojavac Filer
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.decimal.CeilMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.resource.TakeNodeGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.error.ThrowErrorMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.smallInteger.MultiplyMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.warning.GetWarningsNodeGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.smallInteger.BitAndMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.error.ErrorToTextNodeGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.warning.GetValueMethodGen
[warn] Could not determine source for class org.enso.interpreter.runtime.callable.atom.AtomGen$MethodDispatchLibraryExports$Cached
....
```
The output now has over 500 of those and there will be more. Much more
(generated by our and Truffle processors).
There is no way to tell SBT that those are OK. One could potentially
think of splitting compilation into 3 stages (Java processors, Java and
Scala) but that will already complicate the non-trivial build definition
and we may still end up with the initial problem.
This is a fix to make it possible to get reasonable feedback from
compilation without scrolling mutliple screens *every single time*.
Also fixed a spurious warning in javac processor complaining about
creating files in the last round.
Related to https://www.pivotaltracker.com/story/show/182138198
There are two dirty flags in layers: depth_order_dirty and element_depth_order_dirty - one marking changed in Layer, second marking change in one of sublayers. The depth_order_dirty has a proper callback for setting element_depth_order_dirty of its parent. However, the latter did not propagate up.
I fixed it by adding callback for element_depth_order_dirty which sets the depth_order_dirty of the parent.
# Important Notes
* The question to @wdanilo : is it possible, that I can propagate dirty directly to element_depth_order_dirty, without setting depth_order_dirty? As far as I understand the code, it would also work (and we would omit some unnecessary updates).
* I tried to leave some logs, but I don't feel how to do that: the tooling I used was very specific, only the concrete ids of symbols and layers were logged, and I don't know how to generalize it.
Add a method in `SuggestionDatabase` allowing to find a suggestion entry by a fully qualified path, working faster than a linear search through all the entries in the `SuggestionDatabase`.
https://www.pivotaltracker.com/story/show/181852566
# Important Notes
- **Testing:** when testing the PR with the newest currently available nightly version of the Engine, you may observe the following warnings in the Chrome JS Developer Console (the numerical values may differ):
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Nothing.Nothing.is_nothing was overwritten with 768.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Numbers.Integer.up_to was overwritten with 936.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Numbers.Integer.down_to was overwritten with 937.
index.ts?ab16:289 WARN app/gui/src/model/suggestion_database.rs:61 An existing suggestion entry id at Standard.Base.Data.Text.Text.== was overwritten with 971.
This is a result of bugs in the Standard Library. Those are planned to be addressed by:
- #3480 short-term (to fix the specific bugs currently present in the Standard Library),
- https://www.pivotaltracker.com/story/show/182283983 long-term (to improve the Engine such that it disallows introducing this category of bugs in the Standard Library in the future).
(For more details, see also: https://discord.com/channels/401396655599124480/978929754138877962.)
As a result of the bugs mentioned above, the Engine is responding with some invalid replies. In case of such invalid replies, warnings are emitted in the JS Dev Console. Other than the warnings, the code is expected to work correctly with the Engine for the cases when the Engine returns correct data.
- A `HashMapTree` was used for storing the map. A quick back-of-the-envelope estimation of memory usage of a simpler alternative (a one-level `HashMap`) would put the alternative at ~1MB (~10k entities × averaged ~100 bytes per entity path), which was considered too much in a discussion with @farmaazon.
- The `HashMapTree::remove` method deletes a whole subtree of a `HashMapTree`. An alternative removal method was implemented for use in `suggestion_database::QualifiedNameToIdMap` to better match `HashMap` entry removal semantics.
- In case of path collisions, a warning is emitted.
- Paths are treated case-sensitively.
- The new method of the `SuggestionDatabase` type is currently only used in unit tests. This matches the explicit requirement in the Task's description.
- JS Developer Console logging was enabled for the `tracing` package. The `WARN` level was picked to match the default level enabled in the "old" logging infrastructure.
[ci no changelog needed]
@radeusgd discovered that no formatting was being applied to std-bits projects.
This was caused by the fact that `enso` project didn't aggregate them. Compilation and
packaging still worked because one relied on the output of some tasks but
```
sbt> javafmtAll
```
didn't apply it to `std-bits`.
# Important Notes
Apart from `build.sbt` no manual changes were made.
This PR extends the Component Group Entry with icon and option to highlight the text. Here the convert has highlighted "con".
https://user-images.githubusercontent.com/3919101/169046537-4f8b823c-322e-40dc-8abb-24d1d7092341.mp4
### Important Notes
Although this PR includes effort for adjusting Component Group style to better reflect the design, it is not entirely finished: the selection still works badly and will be fixed in another PR.
This is the 2nd part of DSL improvements that allow us to generate a lot of
builtins-related boilerplate code.
- [x] generate multiple method nodes for methods/constructors with varargs
- [x] expanded processing to allow for @Builtin to be added to classes and
and generate @BuiltinType classes
- [x] generate code that wraps exceptions to panic via `wrapException`
annotation element (see @Builtin.WrapException`
Also rewrote @Builtin annotations to be more structured and introduced some nesting, such as
@Builtin.Method or @Builtin.WrapException.
This is part of incremental work and a follow up on https://github.com/enso-org/enso/pull/3444.
# Important Notes
Notice the number of boilerplate classes removed to see the impact.
For now only applied to `Array` but should be applicable to other types.
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/181725003)
This PR implements a fully visible component group header while scrolling the group (using the ScrollArea).
The header moves in sync with scrolling movements (using new `set_header_pos` FRP input), so it looks like the component group is scrolled. ScrollArea masks the "scrolled" entries above the header. This design allows a fully visible header even though our renderer doesn't support nested layers masking yet.
The screencast:
https://user-images.githubusercontent.com/6566674/168320360-2c2017b2-0ef5-42ce-9c79-82b9641c1d73.mp4
The most recent one, with the updated demo scene from develop:
https://user-images.githubusercontent.com/6566674/168555268-8552c4b0-f887-4388-89a1-e65ddf668be6.mp4
# Important Notes
- I fixed the API of the list view so now it supports non-hardcoded scene layers (previously it did not). I also believe it was implemented incorrectly.
- I've found a [pretty weird bug](https://www.pivotaltracker.com/story/show/182193824): the component group inside the ScrollArea is invisible unless I add some arbitrary shape to the scroll area content. I use a `transparent_circle` for this purpose in the demo scene. The bug is probably related to masking the sublayers, though I wasn't able to reproduce it properly on a simpler example.
- The selection box is removed from the demo scene as agreed with @farmaazon . The correct implementation has proven to be much harder than I expected, and we will implement another approach in a separate PR.
- I also modified the `shadow::Parameters` so that it uses `Var`s instead of plain values.
* Extends the instrumentation of the code base and upgrades some FRPs to the newer API macro.
* Extends the run-graph demo scene to specify a profile via URL without recompilation.
* Fixes labels in the flame graph demo scene.
* Fixes an issue with loading profiles that contains escaped characters.
# Important Notes
* no longer contains the upgrade of the `text::View` to `define_endpoints_2`. This should be fixed as part of the text rendering rewrite.
[ci no changelog needed]
Parametrize the colors used in a Component Group view based on a single color passed to an FRP input.
Customizing the colors of a Component Group will be needed for the larger Component Group List panel. This customization will work as a visual hint for the User, helping them to distinguish different Component Groups in the panel. A single input color will be configured for every Component Group in the `package.yaml` file (see the Design Doc). Therefore, all shades of the color required by the Component Group view must be calculated from this single input color.
https://www.pivotaltracker.com/story/show/181725039
#### Visuals
The following screencast of the `component_group` debug scene shows how all required shades of color are calculated from a single input color. It also shows a new "dimmed" display mode of the Component Group. The debug scene does not support selecting entries in a "dimmed" Component Group, as this is not required by the Design Doc.
https://user-images.githubusercontent.com/273837/168074651-bf3d5ea5-99b0-4b69-9934-ad8565ffc54e.mov
The following is a screenshot of the Node Searcher, to demonstrate that it still works correctly:
<img width="623" alt="Screenshot 2022-05-09 at 17 13 01" src="https://user-images.githubusercontent.com/273837/167441109-e9a47b5a-45a2-4172-85ed-c593e43e02d6.png">
# Important Notes
- A new type `Params` was added in the `list_view::entry::Entry` trait. This was needed to allow passing FRP information to entries separately for every ListView instance.
- Note: `style_prefix` and `max_width_px` parameters of the `list_view::entry::Entry::new` function may get moved into the new `Params` type in the future. To save time, this was not attempted in this PR, as agreed with @farmaazon.
[ci no changelog needed]
* Extends the instrumentation of the code base and upgrades some FRPs to the newer API macro.
* Extends the run-graph demo scene to specify a profile via URL without recompilation.
* Fixes labels in the flame graph demo scene.
* Fixes an issue with loading profiles that contains escaped characters.
# Important Notes
[ci no changelog needed]
A low-hanging fruit where we can automate the generation of many
@BuiltinMethod nodes simply from the runtime's methods signatures.
This change introduces another annotation, @Builtin, to distinguish from
@BuiltinType and @BuiltinMethod processing. @Builtin processing will
always be the first stage of processing and its output will be fed to
the latter.
Note that the return type of Array.length() is changed from `int` to
`long` because we probably don't want to add a ton of specializations
for the former (see comparator nodes for details) and it is fine to cast
it in a small number of places.
Progress is visible in the number of deleted hardcoded classes.
This is an incremental step towards #181499077.
# Important Notes
This process does not attempt to cover all cases. Not yet, at least.
We only handle simple methods and constructors (see removed `Array` boilerplate methods).
