Enables `engine.TruffleCompilation` in `std-benchmarks`, collects the logs and dumps compilation into to `System.err` when a benchmark is influenced by dynamic compilation.
Single-phase whitespace-aware precedence resolution.
#### Performance
Since this is a major refactor of the core of the parser, I benchmarked it; it's about 3% faster.
# Important Notes
- Move operator-identifier recognition to lexer.
- Move compound-token assembly out of precedence resolver
- This PR only re-arranges code, splitting the **huge** `processModule` function into a few smaller ones.
- I decided to do it, because when I was working with `processModule` on #9812 I was constantly getting lost in this huge method (this **one** method had 570 lines!) - there is too much happening at once there. Now it's been split into smaller methods, each dealing with one thing.
Reducing the number of log events that we spam regular users. Not the last PR in that area, but already a progress.
Also replaced `fileWalk` with a stream approach hoping that maybe it is a bit more stable on Windows.
* Fix context lock removal
Removal of context lock assumed that one still holds a lock on it. This
is no longer the case when using a `withContextLock` block that
correctly manages the resource. This change fixes the
`IllegalMonitorStateException`.
Closes#10354.
* address review
* Add missing file
* Hiding ContextLock internals from other than ReentrantLocking classes
---------
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
* Mitiigate LS DDOS scenario on startup
For a medium-size project with a lot of visualizations, Language Server
will be flooded with visualization requests on startup. Due to an
existing limit for the job execution engine, some of those requests
might have been silently dropped.
This change lifts that limit until a better fix can be invented.
Additionally, a slow startup might lead to a timeout when serving open
file request. This change adds some retries as a fallback
mechanism/progress monitoring.
* add runtime-fat-jar to a list of aggregates
Fix#10503 by creating a benchmark and then speeding it up by making sure usage of `InteropLibrary` reminds in partially evaluated code and isn't hidden behind `@TruffleBoundary`.
Addresses one of two concerns of #5298 - adds support for `--jvm` argument to allow us to switch from _native image_ built Enso binary (as developed by #10126) to regular JVM based Enso execution. This change _doesn't affect production builds_. The _native executable_ continues to be only built by `engine-runner/buildNativeImage` which is tested on CI, but not in the production jobs.
- Remove publishing the constructors.
- Fix any missed use in libs.
- Alter tests to generally use auto-scoped calls.
- `on_incomparable` to `on_problems`.
Building Engine, Context, ApplicationConfig and Ydoc was a adding a rather large delay during the initial startup step as all of those were blocking operations.
Moving all of those to the resource initialization step hopes to amortize some of that cost since it can be done in parallel. Had to add a `ComponentSupervisor` (open for a different name suggestion) to ensure that such delayed components are properly closed on shutdown.
# Important Notes
Adding Ydoc has added a visible delay during startup. I'm hoping that we can amortize some of that with this PR:
Now:
* Eliminating circe-yaml
This change adds our very-own YAML parser on top of SnakeYAML. Compared
to Circe parser on top of SnakeYAML. The advantage? In some not-so-distant
future we might actually get rid of circe and the related performance
issues.
The logic is similar to what circe does i.e. analyzing SnakeYAML to
build our own structure.
This change is not complete, as there are still some tests failing, but
most common Configs are already parseable.
We _could_ auto-generate some of the code but still some of the logic
would have to be tweaked by hand; the current logic has a number of
special cases, as I found out the hard way.
* wip: more tests passing
* Fix remaining tests in ConfigSpec
* Fixing YAML decoder for editions
Dropping circe as a decoder for Editions revealed some problems. Turns
out the current implementation had even more special cases to deal with.
* nit
* Allow for empty exports
* Mostly complete encodin part
Replaced almost all `toYAML` locations with SnakeYAML equivalent.
The encoding has to use Java collections for which there exists a
built-in support. If we were to use Scala collections we would have to
deal with tagging, at the very least.
* Remove the last remaining Circe's YAML parser
* Bug fix + further loop optimization
* removal of some dependencies
* Remove circe-yaml
Added a custom SnakeYAML Node updater to mimick the JSON -> YAML -> JSON
conversion needed for updating fields. The algorithm recursively follows
the key-path and inserts the desired Node. This is not a performance
oriented code on purpose.
* Fix compilation issues
`circe-core` was marked as `provided` but no one eventually included it
in the final jar, hence `NoClassFoundException`.
* fix licensing
* Removing obsolete circe definitions
* fmt
* nits
* s/SnakeYamlDecoder/YamlDecoder
* fmt
* Partial revert, PM needs JSON decoders/encoders
* style
* incremental compilation gone wrong
- Rename `Location.Start` to `Location.Left`.
- Rename `Location.End` to `Location.Right`.
- Use auto-scoping for `Location`.
- Tune widgets for `Text.trim`.
- Correct signature of `Text.split`.
- Adjist `generateLocallyUniqueIdent` to not fail on bad signature.
- Part of #9486
- Fixing our tests to not rely on deterministic ordering of created Tables in Database backends
- Before, SQLite and Postgres used to mostly return rows in the order they were inserted in, but Snowflake does not.
- Fixing various parts of Snowflake dialect.
Ultimately, we want to forbid the `from ... export all` syntax. This PR starts by providing a way to explicitly export extension and conversion methods by name.
Stdlib code will be modified in upcoming PR.
# Important Notes
A single name can refer to multiple extension or conversion methods. Exports are not qualified. For example,
```
type My_Type
type Other_Type
My_Type.ext_method x = x
Other_Type.ext_method x = x
```
```
from project.Mod export ext_method
```
will export both `My_Type.ext_method` and `Other_Type.ext_method`.
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.
As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.
Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.
In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
close#10182
Changelog:
- add: IdMap parameter to the `text/applyEdit` request
- add: IdMap to the runtime module
- update: set IdMap during the interactive compilation
- update: set the IR identifiers in the `TreeToIR` parsing step
Outline view and completions for Enso code in VSCode.
# Important Notes
This PR provides the necessary infrastructure for building VSCode extension that includes `enso_parser` library compiled for all supported platforms.
VSCode extension can now use libraries from `sbt` that are `publishM2`-ready. To make that possible a documentation must have been provided and fixed for those modules - hence so many changes in `.scala` classes.
<img width="862" alt="image" src="https://github.com/enso-org/enso/assets/26887752/7374bf41-bdc6-4322-b562-85a2e761de2a">
Last, but not least. The outline view and completions display something.
A quick and dirty workaround for slow processing in a similar spirit to PR #9858.
Long compilation of stdlib holds a write compilation lock, while opening the file needs to set module sources and requires read compilation lock and file lock. The expectation was that setting module sources is instantaneous except not, because of locks.
This PR adds soft retries.
Partially closes#10231. There are still problems related to https://github.com/enso-org/enso/issues/9993 once this PR is merged.
# Important Notes
We need to figure out a more fine-grained lock system or, ideally, make it lock free to avoid such hacks.
Reducing the number of dependencies. Explicit `cats` are almost gone (present in `cli`). `enumeration` is completely gone. `cats` is also still included implicitly via `io.circe` but that's a different kind of beast.
Also, really removed `jackson` from dependencies by fixing the dependency on `http-test-helper`.
# Important Notes
In a number of places importing all cats implicits could be simply replaced with a single or two method calls. Not to mention that this will reduce compilation times due to reduced implicit search space.
One example of how the changes affect performance (not only startup):
Before:
Now:
(frequently executed)
Also appears to be gone.
This PR is by no means finished. The purge will continue in follow up PRs.
- Fixes#9980
- Adds some tests to ensure types like `|` or `&` (in addition to `!` from the ticket) correctly work in return type check.
- Fixes a weird behaviour where we used to avoid processing type related IR transformations inside of type ascriptions.
- Adds parentheses to type representations if they are more complex: `A | B & C` is unclear as it can either mean `A | (B & C)` or `(A | B) & C` which have different meanings. If we now have an operation with such nesting, the sub expressions are wrapped in parentheses to disambiguate.
Fixes `Standard.Base.Meta.Enso_Project.enso_project` to return a project descriptor for the *main* project, i.e., the one configured as a *root* for the engine.
# Important Notes
`enso_project` builtin no longer iterates the stack frames to infer the project descriptor. It derives it from the default package repository.
- Follow-up to #9361
- Enables assertions and fixes `count` check
- Tests and fixes null references
- Tests and fixes serializing a deserialized structure - by saving the id of the `Persistance` corresponding to the entry
- After the change to how we determine which `Persistance` instance to use, the most specific one is now used (based on the saved id). This has an unfortunate consequence that `Seq` which is most of the time represented by a subtype of `List`, is now using `PersistScalaList` which is not lazy.
- To alleviate that, we no longer use `Seq` to store some field lazily and instead use a dedicated type for that purpose: `InlineReference`.
- Remove remnants of deprecated Scala parser
- The following projects are now JPMS modules provided on system module-path (in components directory):
- `ydoc-server`
- `profiling-utils`
- `syntax-rust-definition`
- The contents of the aforementioned modules are excluded from both `runner.jar` and `runtime.jar` fat jars.
- Suggestions are serialized and deserialized with our Persistance framework, rather than via the default Java OutputObjectWriter.
Refactored mutable parts of `ModuleScope` into builder to make it easier to reduce unnecessary locks.
# Important Notes
Elements of ModuleScope (types, imports etc) are used while _building_ of it may still be in progress. In order to make static typing happy, every `ModuleScope.Builder` can be exposed as (unmodifiable) `ModuleScope`.
While working on #10056 I realized the names of method and closure nodes are incomprehensible to anyone. This PR replaces the infamous `<anonymous>` with a name hinting where the method actually is.
# Important Notes
I assume this change will be visible not only in IGV, but also in _stacktraces_ and we may need to adjust few tests.
