Fix#10503 by creating a benchmark and then speeding it up by making sure usage of `InteropLibrary` reminds in partially evaluated code and isn't hidden behind `@TruffleBoundary`.
Addresses one of two concerns of #5298 - adds support for `--jvm` argument to allow us to switch from _native image_ built Enso binary (as developed by #10126) to regular JVM based Enso execution. This change _doesn't affect production builds_. The _native executable_ continues to be only built by `engine-runner/buildNativeImage` which is tested on CI, but not in the production jobs.
- Remove publishing the constructors.
- Fix any missed use in libs.
- Alter tests to generally use auto-scoped calls.
- `on_incomparable` to `on_problems`.
Building Engine, Context, ApplicationConfig and Ydoc was a adding a rather large delay during the initial startup step as all of those were blocking operations.
Moving all of those to the resource initialization step hopes to amortize some of that cost since it can be done in parallel. Had to add a `ComponentSupervisor` (open for a different name suggestion) to ensure that such delayed components are properly closed on shutdown.
# Important Notes
Adding Ydoc has added a visible delay during startup. I'm hoping that we can amortize some of that with this PR:
Now:
* Eliminating circe-yaml
This change adds our very-own YAML parser on top of SnakeYAML. Compared
to Circe parser on top of SnakeYAML. The advantage? In some not-so-distant
future we might actually get rid of circe and the related performance
issues.
The logic is similar to what circe does i.e. analyzing SnakeYAML to
build our own structure.
This change is not complete, as there are still some tests failing, but
most common Configs are already parseable.
We _could_ auto-generate some of the code but still some of the logic
would have to be tweaked by hand; the current logic has a number of
special cases, as I found out the hard way.
* wip: more tests passing
* Fix remaining tests in ConfigSpec
* Fixing YAML decoder for editions
Dropping circe as a decoder for Editions revealed some problems. Turns
out the current implementation had even more special cases to deal with.
* nit
* Allow for empty exports
* Mostly complete encodin part
Replaced almost all `toYAML` locations with SnakeYAML equivalent.
The encoding has to use Java collections for which there exists a
built-in support. If we were to use Scala collections we would have to
deal with tagging, at the very least.
* Remove the last remaining Circe's YAML parser
* Bug fix + further loop optimization
* removal of some dependencies
* Remove circe-yaml
Added a custom SnakeYAML Node updater to mimick the JSON -> YAML -> JSON
conversion needed for updating fields. The algorithm recursively follows
the key-path and inserts the desired Node. This is not a performance
oriented code on purpose.
* Fix compilation issues
`circe-core` was marked as `provided` but no one eventually included it
in the final jar, hence `NoClassFoundException`.
* fix licensing
* Removing obsolete circe definitions
* fmt
* nits
* s/SnakeYamlDecoder/YamlDecoder
* fmt
* Partial revert, PM needs JSON decoders/encoders
* style
* incremental compilation gone wrong
- Rename `Location.Start` to `Location.Left`.
- Rename `Location.End` to `Location.Right`.
- Use auto-scoping for `Location`.
- Tune widgets for `Text.trim`.
- Correct signature of `Text.split`.
- Adjist `generateLocallyUniqueIdent` to not fail on bad signature.
- Part of #9486
- Fixing our tests to not rely on deterministic ordering of created Tables in Database backends
- Before, SQLite and Postgres used to mostly return rows in the order they were inserted in, but Snowflake does not.
- Fixing various parts of Snowflake dialect.
Ultimately, we want to forbid the `from ... export all` syntax. This PR starts by providing a way to explicitly export extension and conversion methods by name.
Stdlib code will be modified in upcoming PR.
# Important Notes
A single name can refer to multiple extension or conversion methods. Exports are not qualified. For example,
```
type My_Type
type Other_Type
My_Type.ext_method x = x
Other_Type.ext_method x = x
```
```
from project.Mod export ext_method
```
will export both `My_Type.ext_method` and `Other_Type.ext_method`.
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.
As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.
Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.
In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
close#10182
Changelog:
- add: IdMap parameter to the `text/applyEdit` request
- add: IdMap to the runtime module
- update: set IdMap during the interactive compilation
- update: set the IR identifiers in the `TreeToIR` parsing step
Outline view and completions for Enso code in VSCode.
# Important Notes
This PR provides the necessary infrastructure for building VSCode extension that includes `enso_parser` library compiled for all supported platforms.
VSCode extension can now use libraries from `sbt` that are `publishM2`-ready. To make that possible a documentation must have been provided and fixed for those modules - hence so many changes in `.scala` classes.
<img width="862" alt="image" src="https://github.com/enso-org/enso/assets/26887752/7374bf41-bdc6-4322-b562-85a2e761de2a">
Last, but not least. The outline view and completions display something.
A quick and dirty workaround for slow processing in a similar spirit to PR #9858.
Long compilation of stdlib holds a write compilation lock, while opening the file needs to set module sources and requires read compilation lock and file lock. The expectation was that setting module sources is instantaneous except not, because of locks.
This PR adds soft retries.
Partially closes#10231. There are still problems related to https://github.com/enso-org/enso/issues/9993 once this PR is merged.
# Important Notes
We need to figure out a more fine-grained lock system or, ideally, make it lock free to avoid such hacks.
Reducing the number of dependencies. Explicit `cats` are almost gone (present in `cli`). `enumeration` is completely gone. `cats` is also still included implicitly via `io.circe` but that's a different kind of beast.
Also, really removed `jackson` from dependencies by fixing the dependency on `http-test-helper`.
# Important Notes
In a number of places importing all cats implicits could be simply replaced with a single or two method calls. Not to mention that this will reduce compilation times due to reduced implicit search space.
One example of how the changes affect performance (not only startup):
Before:
Now:
(frequently executed)
Also appears to be gone.
This PR is by no means finished. The purge will continue in follow up PRs.
- Fixes#9980
- Adds some tests to ensure types like `|` or `&` (in addition to `!` from the ticket) correctly work in return type check.
- Fixes a weird behaviour where we used to avoid processing type related IR transformations inside of type ascriptions.
- Adds parentheses to type representations if they are more complex: `A | B & C` is unclear as it can either mean `A | (B & C)` or `(A | B) & C` which have different meanings. If we now have an operation with such nesting, the sub expressions are wrapped in parentheses to disambiguate.
Fixes `Standard.Base.Meta.Enso_Project.enso_project` to return a project descriptor for the *main* project, i.e., the one configured as a *root* for the engine.
# Important Notes
`enso_project` builtin no longer iterates the stack frames to infer the project descriptor. It derives it from the default package repository.
- Follow-up to #9361
- Enables assertions and fixes `count` check
- Tests and fixes null references
- Tests and fixes serializing a deserialized structure - by saving the id of the `Persistance` corresponding to the entry
- After the change to how we determine which `Persistance` instance to use, the most specific one is now used (based on the saved id). This has an unfortunate consequence that `Seq` which is most of the time represented by a subtype of `List`, is now using `PersistScalaList` which is not lazy.
- To alleviate that, we no longer use `Seq` to store some field lazily and instead use a dedicated type for that purpose: `InlineReference`.
- Remove remnants of deprecated Scala parser
- The following projects are now JPMS modules provided on system module-path (in components directory):
- `ydoc-server`
- `profiling-utils`
- `syntax-rust-definition`
- The contents of the aforementioned modules are excluded from both `runner.jar` and `runtime.jar` fat jars.
- Suggestions are serialized and deserialized with our Persistance framework, rather than via the default Java OutputObjectWriter.
Refactored mutable parts of `ModuleScope` into builder to make it easier to reduce unnecessary locks.
# Important Notes
Elements of ModuleScope (types, imports etc) are used while _building_ of it may still be in progress. In order to make static typing happy, every `ModuleScope.Builder` can be exposed as (unmodifiable) `ModuleScope`.
While working on #10056 I realized the names of method and closure nodes are incomprehensible to anyone. This PR replaces the infamous `<anonymous>` with a name hinting where the method actually is.
# Important Notes
I assume this change will be visible not only in IGV, but also in _stacktraces_ and we may need to adjust few tests.
Add support for private methods. Most of the changes are in parser and compiler. The runtime checking of private functions was already present since #9692
# Important Notes
- Only top-level methods can be declared `private`.
- private method cannot be called from different project
- private method cannot be accessed from polyglot code (private method does not exist for polyglot code)
Introduce a new `test-utils` project, and moves the `TestBase` there. Moreover, `TestBase` is renamed to `TestUtils` and is no longer an abstract class.
# Important Notes
`test-utils` project does not depend on junit, so it can be used, for example, by any benchmarks as well.
- related #7954
Changelog:
- update: Ydoc starts with the language server on the `localhost:1234` by default. The hostname and ports can be configured by setting environment variables `LANGUAGE_SERVER_YDOC_HOSTNAME` and `LANGUAGE_SERVER_YDOC_PORT`
- update: by default `npm dev run` uses the node Ydoc server. You can control it with `POLYGLOT_YDOC_SERVER` env variable. For example,
```
env POLYGLOT_YDOC_SERVER='true' npm --workspace=enso-gui2 run dev
```
To connect to the Ydoc server running on the 1234 port (the one started with the language server)
⠀
```
env POLYGLOT_YDOC_SERVER='ws://127.0.0.1:1235' npm --workspace=enso-gui2 run dev
```
To connect to the provided URL. Can be useful for debugging when you start a separate Ydoc process.
- update: run `npm install` before the engine build. It is required to create the Ydoc JS bundle.
This change is a re-do of old PR https://github.com/enso-org/enso/pull/8270 that has been abandoned a while ago.
The change attempts to address to case of random `InterruptedExceptions` that can result when a task is interrupted while waiting on a lock. In comparison to the original PR, this change introduces `withXYZLock` methods with a `Callable` parameter. As a result we ensure correct acquisition/release semantics, logging and handling of interruptions.
# Important Notes
https://github.com/enso-org/enso/pull/8270 was too much hassle to rebase against latest develop.
