We don't seem to run `abortJobs` under a lock, and especially not under the write compilation lock, in other scenarios. This is causing some major slowdown when there is a long running execution or compilation, as currently experienced in the cloud.
This should reduce chances of a timeout.
Also added an option to override the global executor. Currently it would always default to the runtime number of available process which may be suboptimal.
# Important Notes
Pending testing on the impact it will have.
This PR introduces a new installer and uninstaller for the Windows platform.
Both are written in Rust and compiled to a single executable. The executable has no dependencies (other than what is included in the Windows), links the C++ runtime statically if needed.
The change is motivated by numerous issues with with the `electron-builder`-generated installers. The new installer should behave better, not have issues with long paths and unblock the `electron-builder` upgrade (which will significantly simplify the workflow definitions).
To build an installer, one needs to provide the unpacked application (generated by `electron-builder`) and the `electron-builder` configuration (with a few minor extensions). Code signing is also supported.
When `PROFILING_FILENAME` and `PROFILING_TIME` are set, language server will collect profiling data on startup and place it under `/opt/enso/profiling/$PROFILING_NAME` where it can be fetched from.
Needed to better analyze #9789.
part of #7954
# Important Notes
The workflow is:
- `$ npm install` -- just in case
- `$ npm --workspace=enso-gui2 run build-ydoc-server-polyglot` -- build the `ydocServer.js` bundle
- `$ sbt ydoc-server/assembly` -- build the ydoc server jar
- `env POLYGLOT_YDOC_SERVER=true npm --workspace=enso-gui2 run dev` -- run the dev server with the polyglot ydoc server. Providing `POLYGLOT_YDOC_SERVER_DEBUG=true` env variable enables the chrome debugger
Closes#8836.
Atom constructors can be declared as private (project-private). project-private constructors can be called only from the same project. See the encapsulation.md docs for more info.
---------
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
This change replaces an sqllite-backed suggestions' repo with a simple, in-memory, one.
As `completion` functionality has been implemented completely in GUI, there is no need to support it in backend, which simplifies a lot of functionality.
Closes#9650 and #9471.
# Important Notes
Loading suggestions and sending them to GUI on startup is almost instantaneous. Previously it would take ~10s just for `Standard.Base`.
* Text literals: Accept unpaired-surrogate escape codes.
Unpaired surrogates are not allowed by Unicode, but they occur in practice
because many systems accept them; for example, they may be present in filenames
on Windows (which are otherwise constrained to UTF-16).
Programs written in Enso should be able to work with them, if only because they
represent edge cases that should be tested when converting encodings and at
other system boundaries.
- Generalize the representation of interpreted-text-escapes in the lexer, so
that we are not tied to the strict Unicode of Rust's `str`.
- Move some doc-comment code from the parser to test utilities.
- Simplify token serialization.
The change fixes the problem with suggestions loading by
1. Making sure that bundle is always the first on the search path for editions and engine
2. Passing language home to language server
Verified by building/running AppImage locally (previously would fail).
Closes#9728. Regressed in #9647.
* Reduce parser dependencies
- `enso-parser-syntax-tree-visitor` is now only used when building tests and debug tools.
- Remove `enso-logging` crate and its macros.
- The main bin for `enso-parser` has been moved to a `check_syntax` tool in `enso-parser-debug`.
- Close#8610
# Important Notes
QA notes:
- The GUI2 warning screen should not show up - the arguments that GUI2 do not understand have been removed.
- However, it should be tested that the warnings screen should correctly work when invalid arguments really *are* passed in:
- Via URL query parameters (electron, might need to open the electron app then the browser, *or* do `location.href = ` in DevTools in Electron.)
- By editing `Editor.tsx` to inject invalid args to the big configuration object we pass to the GUI entrypoint.
`42 == (Error.throw "foo")` now correctly returns an `Error` rather than False
# Important Notes
The error was in the wrong usage of the `org.enso.interpreter.dsl.AcceptsError` DSL annotation.
Removes a bulk of rust crates that we no longer need, but that added significant install, build and testing time to the Rust parser.
Most significantly, removed `enso-web` and `enso-shapely`, and got rid of many no longer necessary `#![feature]`s. Moved two still used proc-macros from shapely to prelude. The last remaining usage of `web-sys` is within the logger (`console.log`), but we may actually want to keep that one.
close#9351
Changelog:
- update: deprecate the `reexport` suggestion field
- add: `reexports` suggestion field containing the list of modules re-exporting this symbol
- update: exports logic to gather all the symbols exported from a given module
This PR updates the Rust toolchain to recent nightly.
Most of the changes are related to fixing newly added warnings and adjusting the feature flags. Also the formatter changed its behavior slightly, causing some whitespace changes.
Other points:
* Changed debug level of the `buildscript` profile to `lint-tables-only` — this should improve the build times and space usage somewhat.
* Moved lint configuration to the worksppace `Cargo.toml` definition. Adjusted the formatter appropriately.
* Removed auto-generated IntelliJ run configurations, as they are not useful anymore.
* Added a few trivial stdlib nightly functions that were removed to our codebase.
* Bumped many dependencies but still not all:
* `clap` bump encountered https://github.com/clap-rs/clap/issues/5407 — for now the warnings were silenced by the lint config.
* `octocrab` — our forked diverged to far with the original, needs more refactoring.
* `derivative` — is unmaintained and has no updated version, despite introducing warnings in the generated code. There is no direct replacement.
This PR removes enso-pack (ensogl-pack) crate.
It still keeps the `enso-runner` JS package, as it is used for CLI argument parser and logger. The runner should be probably refactored (and possible removed altogether).
# Important Notes
I've temporarily extracted the `enso-runner` to `lib/js` directory, as I wanted to avoid keeping pure JS library under `lib/rust`. Attempts at integrating this with `app/ide-desktop` and family caused too much trouble for this PR. The expectation is that the package will be removed or moved elsewhere soon anyway.
If some benchmark fails in dry-run (compileOnly) mode, the whole process exits with non-zero return code. Also fixes failing engine compiler benchmarks.
# Important Notes
Manually added failure:
```diff
diff --git a/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java b/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java
index c8d86cecc..f9f4d7cbc 100644
--- a/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java
+++ b/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java
@@ -95,7 +95,8 @@ public class ArrayProxyBenchmarks {
@Benchmark
public void sumOverComputingProxy(Blackhole matter) {
- performBenchmark(matter);
+ //performBenchmark(matter);
+ throw new AssertionError("My error");
}
@Benchmark
```
Run with `sbt "-Dbench.compileOnly=true runtime-benchmarks/benchOnly org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy"` fails with:
```
[info] Running benchmarks [org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy] in compileOnly mode
[info] # JMH version: 1.36
[info] # VM version: JDK 21.0.2, Java HotSpot(TM) 64-Bit Server VM, 21.0.2+13-LTS-jvmci-23.1-b30
[info] # VM invoker: /home/pavel/.sdkman/candidates/java/21.0.2-graal/bin/java
[info] # VM options: -XX:ThreadPriorityPolicy=1 -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCIProduct -XX:-UnlockExperimentalVMOptions -Dslf4j.provider=org.slf4j.nop.NOPServiceProvider -Dbench.compileOnly=true --module-path=/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/nativeimage/23.1.2/nativeimage-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/word/23.1.2/word-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/jniutils/23.1.2/jniutils-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/collections/23.1.2/collections-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/polyglot/polyglot/23.1.2/polyglot-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-api/23.1.2/truffle-api-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-runtime/23.1.2/truffle-runtime-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-compiler/23.1.2/truffle-compiler-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/js/js-language/23.1.2/js-language-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/regex/regex/23.1.2/regex-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/shadowed/icu4j/23.1.2/icu4j-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/python/python-language/23.1.2/python-language-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/python/python-resources/23.1.2/python-resources-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcutil-jdk18on/1.76/bcutil-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcpkix-jdk18on/1.76/bcpkix-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcprov-jdk18on/1.76/bcprov-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/llvm/llvm-api/23.1.2/llvm-api-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-nfi/23.1.2/truffle-nfi-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-nfi-libffi/23.1.2/truffle-nfi-libffi-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/tools/profiler-tool/23.1.2/profiler-tool-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/shadowed/json/23.1.2/json-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/2.0.9/slf4j-api-2.0.9.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-nop/2.0.9/slf4j-nop-2.0.9.jar:/home/pavel/dev/enso/runtime.jar --add-modules=org.enso.runtime --add-exports=org.slf4j.nop/org.slf4j.nop=org.slf4j
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: <none>
[info] # Measurement: 1 iterations, 1 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Average time, time/op
[info] # Benchmark: org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy
[info] # Run progress: 0.00% complete, ETA 00:00:01
[info] # Fork: N/A, test runs in the host VM
[info] # *** WARNING: Non-forked runs may silently omit JVM options, mess up profilers, disable compiler hints, etc. ***
[info] # *** WARNING: Use non-forked runs only for debugging purposes, not for actual performance runs. ***
[error] SLF4J: Attempting to load provider "org.slf4j.nop.NOPServiceProvider" specified via "slf4j.provider" system property
[info] Iteration 1: <failure>
[info] java.lang.AssertionError: My error
[info] at org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy(ArrayProxyBenchmarks.java:99)
[info] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_avgt_jmhStub(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:232)
[info] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_AverageTime(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:173)
[info] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[info] at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[info] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:475)
[info] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:458)
[info] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[info] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
[info] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[info] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[info] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] Benchmark run failed: Benchmark caught the exception
[info] at java.base/java.lang.Thread.run(Thread.java:1583)
[error] org.openjdk.jmh.runner.RunnerException: Benchmark caught the exception
[error] at org.openjdk.jmh.runner.Runner.runBenchmarks(Runner.java:575)
[error] at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:310)
[error] at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
[error] at org.enso.interpreter.bench.BenchmarksRunner.runCompileOnly(BenchmarksRunner.java:93)
[error] at org.enso.interpreter.bench.BenchmarksRunner.run(BenchmarksRunner.java:36)
[error] at org.enso.interpreter.bench.benchmarks.RuntimeBenchmarksRunner.main(RuntimeBenchmarksRunner.java:8)
[error] Caused by: org.openjdk.jmh.runner.BenchmarkException: Benchmark error during the run
[error] at org.openjdk.jmh.runner.BenchmarkHandler.runIteration(BenchmarkHandler.java:424)
[error] at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:281)
[error] at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:233)
[error] at org.openjdk.jmh.runner.BaseRunner.doSingle(BaseRunner.java:138)
[error] at org.openjdk.jmh.runner.BaseRunner.runBenchmarksEmbedded(BaseRunner.java:110)
[error] at org.openjdk.jmh.runner.Runner.runBenchmarks(Runner.java:555)
[error] ... 5 more
[error] Suppressed: java.lang.AssertionError: My error
[error] at org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy(ArrayProxyBenchmarks.java:99)
[error] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_avgt_jmhStub(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:232)
[error] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_AverageTime(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:173)
[error] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[error] at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[error] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:475)
[error] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:458)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] at java.base/java.lang.Thread.run(Thread.java:1583)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 5 s, completed Mar 13, 2024, 12:49:59 PM
```
Removed `enso-types` crate which had only one reference in unused part of the code. Removed some unused dependencies from `Cargo.toml` files.
# Important Notes
CI has a similar hiccup as before. Please disregard this for now in the review.
Fixes#9331
Fixed issues with wrong initial size and missing edit caret in text widgets.
<img width="311" alt="image" src="https://github.com/enso-org/enso/assets/919491/44f257cc-18a1-4a9f-9ae0-c1dd9b86674e">
# Important Notes
Automated tests for font loading/initial size will follow shortly. The text caret is not really testable, since it is a hosted object visual issue.
Including arrow language in the distribution by default. Added a basic example for creating an Arrow array.
Making sure that memory layout agrees with Arrow specification (padding, continuous allocation of memory chunks).
Related to #9118.
This should unblock work on allowing serialization/deserialization to/from Parquet but I'd like to delay it to a follow up ticket as it is going to be a significant amount of specialized work.
Removes the old GUI1 code base and reduces the Rust code footprint by removing unused code.
# Important Notes
Updates build scripts and reformats part of the codebase with the autoformatter.
- Improved performance by batching simulatenous node edits, including metadata updates when dragging many selected nodes together.
- Updated Vue to new version, allowing us to use `defineModel`.
- Fixed#9161
- Unified all handling of auto-blur by making `useAutoBlur` cheap to register - all logic goes through a single window event handler.
- Combined all `ResizeObserver`s into one.
- Fixed the behaviour of repeated toast messages. Now only the latest compilation status is visible at any given time, and the errors disappear once compilation passes.
- Actually fixed broken interaction of node and visualization widths. There no longer is a style feedback loop and the visible node backdrop width no longer jumps or randomly fails to update.
Fixes downloading of files. It used to fail because it returned almost immediately. Also fixes progress reporting when fetching a String.
# Important Notes
Tested by removing appropriate engine and runtime from `$HOME/.local/share/enso` and with manually running:
```
java -jar launcher.jar --launcher-log-level trace install engine 2024.1.1-nightly.2024.2.29
```
In PR #8953, in commit ba0a69de6e, I have introduced argument files to the `native-image`. In this PR, let's try to upload these argfiles as artifacts on GH, so that we can inspect them later.
* Test describing the current behavior of chained if then else application
* Chained block should behave just like Group around if_then_else
* Finishing line on BlockStart fixes if_then_else_chained_block
* Only finish the line when there was not start of a macro segment
* Fix tests
* Refine else-body with macro patterns.
* Update test syntax to maintain original semantics
* Few additional tests
---------
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
`Bump` library uses parser combinators behind the scenes which are known to be good at expressing grammars but are not performance-oriented.
This change ditches the dependency in favour of an existing Java implementation. `jsemver` implements the full specification, which is probably an overkill in our case, but proved to be an almost drop-in replacement for the previous library.
Closes#8692
# Important Notes
Peformance improvements:
- roughly 50ms compared to the previous approach (from 80ms to 20-40ms)
I don't see any time spent in the new implementation during startup so it could be potentially aggressively inlined.
Further more, we could use a facade and offer our own strip down version of semver.
For our users "Nodes" are called "Components". Which the users selects using the component browser. This MR just sets the text in the default workflow.
There are two projects transitively required by `runtime`, that have akka dependencies:
- `downloader`
- `connected-lock-manager`
This PR replaces the `akka-http` dependency in `downloader` by HttpClient from JDK, and splits `connected-lock-manager` into two projects such that there are no akka classes in `runtime.jar`.
# Important Notes
- Simplify the `downloader` project - remove akka.
- Add HTTP tests to the `downloader` project that uses our `http-test-helper` that is normally used for stdlib tests.
- It required few tweaks so that we can embed that server in a unit test.
- Split `connected-lock-manager` project into two projects - remove akka from `runtime`.
- **Native image build fixes and quality of life improvements:**
- Output of `native-image` is captured 743e167aa4
- The output will no longer be intertwined with the output from other commands on the CI.
- Arguments to the `native-image` are passed via an argument file, not via command line - ba0a69de6e
- This resolves an issue on Windows with "Command line too long", for example in https://github.com/enso-org/enso/actions/runs/7934447148/job/21665456738?pr=8953#step:8:2269
related #8689
Fixes a race between the language server SQL updating logic and the engine `DeserializeLibrarySuggestionsJob`s when the library suggestions may start loading before the database is properly cleaned up after the reconnect.
close#8650
Changes in project manager required to work with projects located in custom directories.
Changelog:
- add: `--projects-directory /path/to/projects` `--project-list` command line parameters returning the projects located in custom directory
- update: `project/create`, `project/open`, `project/delete`, `project/rename` commands to work with projects located in custom directories
Random.Seed doesn't work in the GUI and the TEXT_ONLY tag doesn't do anything so it was incorrectly showing up in the component browser.
This MR makes Random.Seed private to hide it from the GUI and completely removes the TEXT_ONLY tag which is unused and unimplemented.
Fixes#8710 by making sure suspended atom fields support works also for "normal" `Atom` instances without any special `Layout`. Moves all _atom related_ classes into single package and hides as much of classes as possible by making them _package private_.
Implements `Warnings.get_all wrap_errors=True` which wraps warnings attached to values inside vectors with `Map_Error`, which includes the position of the value within the vector. See [the documentation](https://github.com/enso-org/enso/blob/develop/docs/semantics/wrapped-errors.md) for more details.
`get_all wrap_errors=True` does not change the warnings that are attached to values -- it wraps them before returning them to the caller, but does not change the original warnings attached to the values.
Wrapped warnings only appear attached to the vector itself. The values inside the vector do not have their warnings wrapped.
Warning propagation is not changed at all; `Warnings.get_all` (with default `wrap_errors=False`) behaves as before. `get_all wrap_errors=True` is meant to be used primarily by the IDE, although it can be used anywhere this wrapping is desired.
The change adds a convenient trait `ReportLogsOnFailure` that, when merged with the test class, will keep logs in memory and only delegate to the underlying appender on failure. For now we only support forwarding to the console which is sufficient.
A corresponding entry in `application-test.conf` has to point to the new `memory` appender. The additional complexity in the implementation ensures that if someone forgets to mixin `ReportLogsOnFailure` logs appear as before i.e. they respect the log level.
As a bonus fixed arguments passed to ScalaTest in build.sbt so that we are now, again, showing timings of individual tests.
Closes#8603.
# Important Notes
Before:
```
[info] VcsManagerTest:
[info] Initializing project
[ERROR] [2024-01-04 17:27:03,366] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a repository (3 seconds, 538 milliseconds)
[info] - must fail to create a repository for an already existing project (141 milliseconds)
[info] Save project
[ERROR] [2024-01-04 17:27:08,346] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a commit with a timestamp (198 milliseconds)
[ERROR] [2024-01-04 17:27:08,570] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a commit with a name (148 milliseconds)
[ERROR] [2024-01-04 17:27:08,741] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must force all pending saves (149 milliseconds)
[info] Status project
[ERROR] [2024-01-04 17:27:08,910] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must report changed files since last commit (148 milliseconds)
[info] Restore project
[ERROR] [2024-01-04 17:27:09,076] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to the last state with committed changes (236 milliseconds)
[ERROR] [2024-01-04 17:27:09,328] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to a named save (pending)
[ERROR] [2024-01-04 17:27:09,520] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to a named save and notify about removed files *** FAILED *** (185 milliseconds)
[info] Right({
[info] "jsonrpc" : "2.0",
[info] "method" : "file/event",
[info] "params" : {
[info] "path" : {
[info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }
[info] }) did not equal Right({
[info] "jsonrpc" : "1.0",
[info] "method" : "file/event",
[info] "params" : {
[info] "path" : {
[info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }
[info] }) (VcsManagerTest.scala:1343)
[info] Analysis:
[info] Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> {
[info] "path" : {
[info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> {
[info] "path" : {
[info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }]))
[ERROR] [2024-01-04 17:27:09,734] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] List project saves
[info] - must return all explicit commits (146 milliseconds)
[info] Run completed in 9 seconds, 270 milliseconds.
[info] Total number of tests run: 9
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1
[info] *** 1 TEST FAILED ***
```
After:
```
[info] VcsManagerTest:
[info] Initializing project
[info] - must create a repository (3 seconds, 554 milliseconds)
[info] - must fail to create a repository for an already existing project (164 milliseconds)
[info] Save project
[info] - must create a commit with a timestamp (212 milliseconds)
[info] - must create a commit with a name (142 milliseconds)
[info] - must force all pending saves (185 milliseconds)
[info] Status project
[info] - must report changed files since last commit (142 milliseconds)
[info] Restore project
[info] - must reset to the last state with committed changes (202 milliseconds)
[info] - must reset to a named save (pending)
[ERROR] [2024-01-04 17:24:55,738] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/8456553964637757156].
[info] - must reset to a named save and notify about removed files *** FAILED *** (186 milliseconds)
[info] Right({
[info] "jsonrpc" : "2.0",
[info] "method" : "file/event",
[info] "params" : {
[info] "path" : {
[info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }
[info] }) did not equal Right({
[info] "jsonrpc" : "1.0",
[info] "method" : "file/event",
[info] "params" : {
[info] "path" : {
[info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }
[info] }) (VcsManagerTest.scala:1343)
[info] Analysis:
[info] Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> {
[info] "path" : {
[info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> {
[info] "path" : {
[info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info] "segments" : [
[info] "src",
[info] "Bar.enso"
[info] ]
[info] },
[info] "kind" : "Removed"
[info] }]))
[info] List project saves
[info] - must return all explicit commits (131 milliseconds)
[info] Run completed in 9 seconds, 400 milliseconds.
[info] Total number of tests run: 9
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1
[info] *** 1 TEST FAILED ***
```
Removing an explicit import of `cats` implicits which triggers rather slow classloading, just for a single method that transforms `List[Either[A, B]]` to `Either[A, List[B]]`.
Addresses one of the problems mentioned in #8692.
Make sure that the correct test logging provider is loaded in `project-manager/Test`, so that only WARN and ERROR log messages are displayed. Also, make sure that the test log provider parses the correct configuration file - Rename all the `application.conf` files in the test resources to `application-test.conf`.
The problem was introduced in #8467
It is currently not possible to understand the nature of random timeouts for launcher tests on MacOS. Hopefully the additional info will bring us closer to understanding where is the problem.
Related to #8546.
close#7555
Compiler passes after `GenerateMethodBodies` expect the method body to be a function.
After fixing the pass, the compilation returns a proper compiler error:
```
built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Table/0.0.0-dev/src/Data/Column.enso:869:22: error: Methods must have only one definition of the `this` argument, and it must be the first.
869 | round self round self (decimal_places:Integer = 0) (use_bankers:Boolean = False) = Value_Type.expect_numeric self <|
| ^~~~
Aborting due to 1 errors and 0 warnings.
```
Add a local clone of javaFormatter plugin. The upstream is not maintained anymore. And we need to update it to use the newest Google java formatter because the old one, that we use, cannot format sources with Java 8+ syntax.
# Important Notes
Update to Google java formatter 1.18.1 - https://github.com/google/google-java-format/releases/tag/v1.18.1
Implements #6166.
# Important Notes
- More consistent handling of `default` arguments. `default` is a valid identifier, and only has special meaning when it isn't bound in scope. Since distinguishing the builtin `default` from an identifier called `default` cannot be done until alias analysis has been performed, `default` is now represented in the AST as a regular identifier.
- `TreeToIr`: Remove `insideTypeAscription`. It was only used for bug-for-bug compatibility with the old parser during the transition.
We've been experiencing consistently failures on MacOS due to timeouts.
Doubling the timeout, hoping this will be sufficient to eliminate such
false failures. Will seek alternative solutions if that does not rememdy
the problem on CI.
* Test illustrating problems with FQNs
Inline execution fails with `Compile error: The name `Standard` could
not be found.`.
* Ensure InlineContext carries Package Repos info
Previously, there was no requirement that inline execution should allow
for FQNs. This meant that the omission of Package Repository info went
unnoticed.
In order to be able to refer to `Standard.Visualization.Preprocessor` it
has to be exported as well.
* Build distribution for amd64 and aarch64 MacOS
Possible after the GraalVM upgrade.
* Another attempt at building on MacOS M1
* One less hardcoded architecture
* Eliminate one more hardcoded architecture
* add more debug info
* nit
This is a follow-up of #7991. #7991 broken `runtime-version-manager`. This is mostly reverts.
### Important Notes
Launcher now correctly recognizes that the newest engine needs some runtime:
```sh
> java -jar launcher.jar list
Enso 2023.2.1-nightly.2023.10.31 -> GraalVM 23.0.0-java17.0.7
Enso 0.0.0-dev -> GraalVM 23.1.0-java21.0.1
```
(this has not worked before)
Changelog:
- update: always create an event log next to the profiling file when the engine is started with the `--profiling-path` flag
- remove: `--profiling-events-log-path` flag
close#8249
Changelog:
- add: `profiling/snapshot` request that takes a heap dump of the language server and puts it in the `ENSO_DATA_DIRECTORY/profiling` direcotry
Attaching or modifying a visualizations returns early on, to avoid a situation when a background job is stalled (by other jobs) and eventually the request timeouts.
This has an unfortunate consequence that any error reported in the `UpsertVisualizationJob` cannot be reported as a directly reply to a request because the sender has already been removed from the list.
Added more logs to discover why we get errors in the first place.
Modified the API a bit so that we carry `VisualizationContext` instead of three parameters all over the place.
Bonus:
Modified `JsonRpcServerTestKit` to implicitly require a position so that we get better error reporting on failures.
Upgrade to GraalVM JDK 21.
```
> java -version
openjdk version "21" 2023-09-19
OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15)
OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing)
```
With SDKMan, download with `sdk install java 21-graalce`.
# Important Notes
- After this PR, one can theoretically run enso with any JRE with version at least 21.
- Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution.
- `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded.
- sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368
- Which means that we can have `module-info.java` files only for Java-only projects.
- Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar.
- `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent.
- Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory.
- The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module.
- The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725)
- [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
close#8248
Changelog:
- add: `profiling/start` request starts the sampler and starts collecting runtime events to the log file
- add: `profiling/stop` request stop the sampler and write the profiling data to the `$ENSO_DATA_DIR/profiling` directory
- refactor: rewrite the profiling logic into Java
Fixes a random crash (*) during instrumentation. Notice how `onTailCallReturn` calls `onReturnValue` with `null` frame.
Bonus: noticed that for some reason we weren't getting logs for `ExecutionService`. This turned out to be the problem with the logger name which by default was `[enso]` not
`[enso.org.enso.interpreter.service.ExecutionService`] and there is some logic there that normalizes the name and assumed a dot after `enso`. This change fixes the logic.
(*)
```
[enso.org.enso.interpreter.service.ExecutionService] Execution of function main failed (Cannot invoke "com.oracle.truffle.api.frame.VirtualFrame.materialize()" because "frame" is null).
java.lang.NullPointerException: Cannot invoke "com.oracle.truffle.api.frame.VirtualFrame.materialize()" because "frame" is null
at org.enso.interpreter.instrument.IdExecutionInstrument$IdEventNodeFactory$IdExecutionEventNode.onReturnValue(IdExecutionInstrument.java:246)
at org.enso.interpreter.instrument.IdExecutionInstrument$IdEventNodeFactory$IdExecutionEventNode.onTailCallReturn(IdExecutionInstrument.java:274)
at org.enso.interpreter.instrument.IdExecutionInstrument$IdEventNodeFactory$IdExecutionEventNode.onReturnExceptional(IdExecutionInstrument.java:258)
at org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode$EventProviderChainNode.innerOnReturnExceptional(ProbeNode.java:1395)
at org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode$EventChainNode.onReturnExceptional(ProbeNode.java:1031)
at org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode.onReturnExceptionalOrUnwind(ProbeNode.java:296)
at org.enso.interpreter.node.ExpressionNodeWrapper.executeGeneric(ExpressionNodeWrapper.java:119)
at org.enso.interpreter.node.ClosureRootNode.execute(ClosureRootNode.java:85)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:718)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:641)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:574)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:558)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:504)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:69)
at org.enso.interpreter.node.callable.thunk.ThunkExecutorNode.doCached(ThunkExecutorNode.java:69)
at org.enso.interpreter.node.callable.thunk.ThunkExecutorNodeGen.executeAndSpecialize(ThunkExecutorNodeGen.java:207)
at org.enso.interpreter.node.callable.thunk.ThunkExecutorNodeGen.executeThunk(ThunkExecutorNodeGen.java:167)
...
```
# Important Notes
Fixes regressions introduced in #8148 and #8162
This change fixes a regression introduced in #7918, which prevented the execution from setting the right log level either via env var or parameter.
Now passing either of the options returns logs of the expected level in the log file:
- `ENSO_LOG_TO_FILE_LOG_LEVEL = trace`
- ... `-vv` ...
Fixes#8274
Previously custom log levels applied only to non-Truffle loggers. To allow it, filtering has to be applied appropriately at two places - first at Java's Handler and then essentially re-confirmed at SLF4J's logger to which the former forwards to.
Filters compose in an `AND` condition, therefore default log level check had to be merged into our custom filters.
`TruffleLogger` has a builtin functionality to perform the filtering when context is configured appropriately. This should be much more efficient than adding a `Filter` to the JUL Handler explicitly.
# Important Notes
```
JAVA_OPTS="-org.enso.compiler.SerializationManager.Logger.level=debug" ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run
```
will now assign a custom log level to `SerializationManager` Logger.
Add `line:column` information to source code references produced by the parser. This information will be used by GUI2 as part of the solution to #8134.
# Important Notes
- `parse_all_enso_files.sh` has been used to ensure this doesn't affect tree structures.
- `parse_all_enso_files.sh` now checks emitted locations for consistency, and has been used to verify that all line:col references match the values found by an independent scan of the source up to the given UTF8 position.
The change upgrades `directory-watcher` library, hoping that it will fix the problem reported in #7695 (there has been a number of bug fixes in MacOS listener since then).
Once upgraded, tests in `WatcherAdapterSpec` because the logic that attempted to ensure the proper initialization order in the test using semaphore was wrong. Now starting the watcher using `watchAsync` which only returns the future when the watcher successfully registers for paths. Ideally authors of the library would make the registration bit public
(3218d68a84/core/src/main/java/io/methvin/watcher/DirectoryWatcher.java (L229C7-L229C20)) but it is the best we can do so far.
Had to adapt to the new API in PathWatcher as well, ensuring the right order of initialization.
Should fix#7695.
Using a `TruffleLogger` in `SerializationManager` that is bound to the engine rather than the context prevents reaching an illegal state when using thread pools.
Also cleaned up some tests for consistency.
To verify the fix
```diff
--- a/engine/runtime/src/main/scala/org/enso/compiler/SerializationManager.scala
+++ b/engine/runtime/src/main/scala/org/enso/compiler/SerializationManager.scala
@@ -31,7 +31,7 @@ final class SerializationManager(compiler: Compiler) {
import SerializationManager._
/** The debug logging level. */
- private val debugLogLevel = Level.FINE
+ private val debugLogLevel = Level.INFO
```
and run
`sbt:enso> runtime/test`
Closes#8147.
Towards reduced reliance on Scala semantics.
Translated IR.scala to IR.java and extracted implicits that now need to be imported explicitly.
# Important Notes
1:1 translation. For now `@Identifier` and `@ExternalID` represent the old type aliases but are not verified at compile time.
This is because in a mixed Scala/Java world this seems impossible to employ such frameworks as Checker.
In order to execute `runtime-parser` in browser, we need to slightly simplify dependencies of our code.
# Important Notes
The actual PR explaining why these changes are desirable is #8094
- Fixes#8049
- Adds tests for handling of Date_Time upload/download in Postgres.
- Adds tests for edge cases of handling of Decimal and Binary types in Postgres.
close#8033
Changelog:
- update: run language server initialization once
- fix: issues with async `getSuggestionDatabase` message handling in new IDE
- update: implement unique background jobs
- refactor: initialization logic to Java
- refactor: `UniqueJob` to a marker interface
- Validate spans during existing lexer and parser unit tests, and in `enso_parser_debug`.
- Fix lost span info causing failures of updated tests.
# Important Notes
- [x] Output of `parse_all_enso_files.sh` is unchanged since before #7881 (modulo libs changes since then).
- When the parser encounters an input with the first line indented, it now creates a sub-block for lines at than indent level, and emits a syntax error (every indented block must have a parent).
- When the parser encounters a number with a base but no digits (e.g. `0x`), it now emits a `Number` with `None` in the digits field rather than a 0-length digits token.
* Reduce extra output in compilation and tests
I couldn't stand the amount of extra output that we got when compiling
a clean project and when executing regular tests. We should strive to
keep output clean and not print anything additional to stdout/stderr.
* Getting rid of explicit setup by service loading
In order for SL4J to use service loading correctly had to upgrade to
latest slf4j. Unfortunately `TestLogProvider` which essentially
delegates to `logback` provider will lead to spurious ambiguous warnings
on multiple providers. In order to dictate which one to use and
therefore eliminate the warnings we can use the `slf4j.provider` env
var, which is only available in slf4j 2.x.
Now, there is no need to explicitly call `LoggerSetup.get().setup()` as
that is being called during service setup.
* legal review
* linter
* Ensure ConsoleHandler uses the default level
ConsoleHandler's constructor uses `Level.INFO` which is unnecessary for
tests.
* report warnings
It seems that Runtime Connector wasn't respecting the protocol it defined itself. The connector should be waiting on the `Api.InitializedNotification` message and only then start forwarding messages. So far it seems this hasn't been a problem, or at least wasn't reported as such, because initialization was fast enough.
Modified `Handler` so that we are certain that its fields hold initialized values when being accessed by different threads.
Should fix problems mentioned in #7898.
* Add support for https and wss
Preliminary support for https and wss. During language server startup we
will read the application config and search for the `https` config with
necessary env vars set.
The configuration supports two modes of creating ssl-context - via
PKCS12 format and certificat+private key.
Fixes#7839.
* Added tests, improved documentation
Generic improvements along with actual tests.
* lint
* more docs + wss support
* changelog
* Apply suggestions from code review
Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
* PR comment
* typo
* lint
* make windows line endings happy
---------
Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
Generate TS bindings and lazy deserialization for the parser types.
# Important Notes
- The new API is imported into `ffi.ts`, but not yet used.
- I have tested the generated code in isolation, but cannot commit tests as we are not currently able to load WASM modules when running in `vitest`.
Fixes#7474
Again, a quite complicated issue. The code editor was broken by https://github.com/enso-org/enso/pull/7167 for a few reasons:
1. The text component's shortcuts depend on the `focused` FRP endpoint, which is controlled by `deprecated_(de)focus` inputs. `focused` output was not changed by the code editor `model.focus()` call. (we still use `deprecated_focus` for all other components)
2. `focus()` FRP input was added to the text component but never triggered. It's too easy to confuse it with the `focus()` method of the `object::Instance` (Deref is evil!)
3. The graph editor handles clicks on the background by `blur_tree()`, effectively blurring any focused element in the application. But clicking on code editor (to focus it) also causes a click on the background! So we focus the code editor, and then handle the background click, defocusing it from the graph editor's point of view. It allows graph editor to open CB on <kbd>Enter</kbd> while the code editor is still active.
I solved (3) by disallowing CB opening while code editor is visible. Not ideal, but I guess it solves the issue.
Do you think the other solution is possible – to add some overlay for the code editor and avoid triggering `blur_tree` when clicking on it? That won't work because you still can't defocus the code editor using the new API, so it will always be focused.
So we have a situation when half of the app uses the new focus system, another half uses the old one, and none works properly.
https://github.com/enso-org/enso/assets/6566674/3a0d36ed-4d50-4606-9486-272fbd5d9cb7
Adds the ability to declare a module as *private*. Modifies the parser to add the `private` keyword as a reserved keyword. All the checks for private modules are implemented as an independent *Compiler pass*. No checks are done at runtime.
# Important Notes
- Introduces new keyword - `private` - a reserved keyword.
- Modules that have `private` keyword as the first statement are declared as *private* (Project private)
- Public module cannot have private submodules and vice versa.
- This would require runtime access checks
- See #7088 for the specification.
This PR adds support for the new Vue-based GUI (aka `gui2`).
The user-facing changes are primarily:
* support for `./run gui2` and `./run ide2` commands (that build just the new GUI and the whole IDE package with new GUI embedded — respectively);
* the top-level `test` and `lint` commands will now invoke the relevant commands on the new GUI
---------
Co-authored-by: Paweł Grabarz <frizi09@gmail.com>
* Enable log-to-file configuration
PR #7825 enabled parallel logging to a file with a much more
fine-grained log level by default.
However, logging at `TRACE` level on Windows appears to be still
problematic.
This PR reduced the default log level to file from `DEBUG` to `TRACE`
and allows to control it via an environment variable if one wishes to
change the verbosity without making code changes.
* PR comments
close#7750close#7834
Changelog:
- update: project manager uses the packaged language server to open projects
- fix: remove stack traces from connection errors on initial ping handler request (when the language server is booting)
- update: add engine and edition versions to the `initProtocolConnection` response for easier debug
- update: do not resolve project ensoVersion in the `project/list` to eliminate unnecessary network calls
* Always log verbose to a file
The change adds an option by default to always log to a file with
verbose log level.
The implementation is a bit tricky because in the most common use-case
we have to always log in verbose mode to a socket and only later apply
the desired log levels. Previously socket appender would respect the
desired log level already before forwarding the log.
If by default we log to a file, verbose mode is simply ignored and does
not override user settings.
To test run `project-manager` with `ENSO_LOGSERVER_APPENDER=console` env
variable. That will output to the console with the default `INFO` level
and `TRACE` log level for the file.
* add docs
* changelog
* Address some PR requests
1. Log INFO level to CONSOLE by default
2. Change runner's default log level from ERROR to WARN
Took a while to figure out why the correct log level wasn't being passed
to the language server, therefore ignoring the (desired) verbose logs
from the log file.
* linter
* 3rd party uses log4j for logging
Getting rid of the warning by adding a log4j over slf4j bridge:
```
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
```
* legal review update
* Make sure tests use test resources
Having `application.conf` in `src/main/resources` and `test/resources`
does not guarantee that in Tests we will pick up the latter. Instead, by
default it seems to do some kind of merge of different configurations,
which is far from desired.
* Ensure native launcher test log to console only
Logging to console and (temporary) files is problematic for Windows.
The CI also revealed a problem with the native configuration because it
was not possible to modify the launcher via env variables as everything
was initialized during build time.
* Adapt to method changes
* Potentially deal with Windows failures
- Closes#7461 by introducing a `Date_Time_Formatter` type and making parsing date time formats more robust and safer.
- The default ('simple') set of patterns is slightly simplified and made case insensitive (except for `M/m` and `H/h`) to avoid the `YYYY` vs `yyyy` issues and make it less error prone.
- The `YYYY` now has the same meaning as `yyyy` in simple mode. The old meaning (week-based year) is moved to a _separate mode_, triggered by `Date_Time_Formatter.from_iso_week_date_pattern`.
- Full Java syntax, as well as custom-built Java `DateTimeFormatter` can also be used by `Date_Time_Formatter.from_java`.
- Text-based constants (e.g. `ISO_ZONED_DATE_TIME`) have now become methods on `Date_Time_Formatter`, e.g. `Date_Time_Formatter.iso_zoned_date_time`).
* Improve shutdown logic of language server
This PR addresses problems mentioned in #7470 and #7729:
- shutting a language server explicitly will not lead to a soft shutdown
- `project/status` endpoint returns the state of the language server
`LanguageServerController` now also signed up for `ClientConnect`
messages. For it to be unambiguous, we need to carry around the port
number of the language server as a way of identifying the right one.
One can now use `project/status` to additionally determine the state of
the language server.
Also relies on a proper fix for #7765.
* changelog
* PR comments
# Important Notes
- Binary LS endpoint is not yet handled.
- The parsing of provided source is not entirely correct, as each line (including imports) is treated as node. The usage of actual enso AST for nodes is not yet implemented.
- Modifications to the graph state are not yet synchronized back to the language server.
Resolve macros eagerly. Improves performance; allows parser to handle arbitrarily-long lines (fixes#7691).
# Important Notes
- A new utility, `lib/rust/parser/debug/tools/parse_all_enso_files.sh`, supports comparing ASTs parsed with different versions of the parser. This tool has been used to verify that this refactor doesn't change the result of parsing any standard library or test file.
Add support for recovering from GL context loss. When the context is restored, the loading spinner is shown until shaders finish recompiling.
[vokoscreenNG-2023-08-25_09-39-11.webm](https://github.com/enso-org/enso/assets/1047859/cfa90ec5-72a1-41e6-bafa-177fa5e85fb2)
*While the context is missing, the loading spinner is rendered in the 0% state. (This condition will not normally be observed, except momentarily, as the browser should restore the context immediately if it is lost while the page is visible.) When we receive a new context, the spinner switches to the 90% state until restoration completes. Restoration is fast, as we don't need to do much work except recompiling shaders.*
# Important Notes
- A new debug hotkey, Ctrl+Alt+Shift+X, causes context loss for testing. Pressing it a second time causes context restoration.
- `Texture` is still a CPU-bound texture. It now uses the "immutable" `texStorage/texSubImage` API, which is a ["preferred alternative"](https://registry.khronos.org/webgl/specs/latest/2.0/#3.7.6) to the `texImage` API because it can be more efficient.
- The type for texture uniforms is now `Uniform<Option<Texture>>`. Texture uniforms are decoupled from the context.
- A new `ContextLost` error type can be returned by functions that cannot complete if the context is lost.
- Fix some crashes that could occur when context was lost.
- Clarify ownership of some rendering-related types: Externalize, and where possible eliminate, `Rc/RefCell`s.
Closes: #7309
The changed origin of the visualization container caused the issue. Now, the origin is at the top left corner of the visualization, and bounding box abstraction expects the origin at the bottom left. Despite the comments in the code, the bounding box is designed to work with bottom-left origin only.
https://github.com/enso-org/enso/assets/6566674/273abfaa-45b6-4374-8d4c-3b8c4e2c1fc2
Fixes#7468
The fix is pretty simple: we reuse the existing functionality for importing stuff and generating expressions. It fixes issues with `Nothing` or `Report_Unmatched` types.
https://github.com/enso-org/enso/assets/6566674/4e7addf9-2175-4f2a-a571-4ef823de5cb0
While debugging, I found it easier to work with a suggestion database when exported to some external format. Hence, I implemented serde serialization support for database entries and also a new debug shortcut <kbd>ctrl</kbd>+<kbd>shift</kbd>+<kbd>u</kbd> to dump all entries to the console.
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.
Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.
Most important differences have been outlined in `docs/infrastructure/logging.md`.
# Important Notes
Addresses:
- #7253
- #6739
- Closes#7633
- Moves `Round_Spec.enso` from published `Standard.Test` into our `test/Tests` project; the `Table_Tests` that depend on it, simply `import enso_dev.Tests`.
- Changes the layout of the local libraries directory:
- It used to be `root/<namespace>/<name>`.
- Now it is `root/<dir>` - the namespace and name are now read from `package.yaml` instead.
- Adds the parent directory of the current project to the default `ENSO_LIBRARY_PATH`.
- It is treated as a secondary path, so the default `ENSO_HOME/lib` still takes precedence.
- This allows projects to reference and load 'sibling' projects easily - the only requirement is for the project to enable `prefer-local-libraries: true` or add the other local project to its edition. The edition resolution logic is **not changed**.
Closes#6261
- Adds support for copy-pasting nodes with `cmd + C` and `cmd + V` shortcuts.
- Only a single, currently selected node will be copied. Adding support for multiple node copies seems easy, though (but was out of the scope of the task).
- We use a custom data format for clipboard content. Node's metadata is also copied, so opened visualizations are preserved. However, the visualization's size is not preserved, as we do not store this info in metadata.
- For custom format to work, we use a pretty new feature called [Clipboard pickling](https://github.com/w3c/editing/blob/gh-pages/docs/clipboard-pickling/explainer.md), but it is available in Electron and in most browsers already.
- Pasting plain text from other applications (or from Enso, if the code is copied in edit mode) is supported and is currently enabled. There are some security concerns related to this, though. I will create a separate issue/discussion for that.
- Undo/redo works as you expect.
- New node is pasted at the cursor position.
https://github.com/enso-org/enso/assets/6566674/7a04d941-19f7-4a39-9bce-0e554af50ba3
When building the dev IDE version with `./run ide build`, it creates the engine distribution with version `2023.2.1-dev`. This causes the version check to fail preventing opening the projects.
- Closes#6730
- Changes config to allow environment variables to override server host and port
- Adds port scanning to Electron app to ensure the PM is started at a free port
# Important Notes
- `SERVER_PORT=abcd enso.AppImage` does NOT work. It would not be difficult to implement, but it probably needs discussion on how exactly it should be implemented - for example, `SERVER_PORT` is quite a generic name, should the Electron app pass though something like `ENSO_PM_SERVER_PORT` to the PM as `SERVER_PORT` instead?
⚠️ Port scanning is *only* implemented in the JS frontend. It is not implemented:
- In Scala, because the JS/Rust code calling it needs to know the port as well. There shouldn't be any problems with adding port scanning though, if that's desired
- In Rust, because I'm not sure parsing the host and port from a string is a good idea.
- (This also applies to JS, but it *must* work in JS, and port scanning is already a dependency there so it's quite a bit easier)
- QA *will* need a new PM (`sbt buildProjectManagerDistribution` or `./run backend sbt` -> `buildProjectManagerDistribution`), and the path must be supplied as: `-engine.project-manager-path=path/to/new/pm/here`
This commit introduces a new cursor shape, a double arrow, to be displayed during the hover over the resize grip in the graph editor's visualization component. The rotation of the double arrow depends on whether the cursor is to the left or right of the dividing line from the top left to bottom-right corner of the grip. This gives a more intuitive visual cue to users about the direction of resizing.
Looks like that
```
[info] [2023-08-23T13:12:58.119Z] [org.enso.projectmanager.boot.ProjectManager$] Starting Enso Project Manager
Version: 0.0.0-dev
Built with: scala-2.13.11 for GraalVM 17.0.7
Built from: wip/db/fix-dev-version-check* @ 52bc6b8fcf
Built on: Linux (amd64)
```
