Commit Graph

428 Commits

Author SHA1 Message Date
Pavel Marek
53c1b3f48b
PathProgressBodyHandler does not return immediately (#9222)
Fixes downloading of files. It used to fail because it returned almost immediately. Also fixes progress reporting when fetching a String.

# Important Notes
Tested by removing appropriate engine and runtime from `$HOME/.local/share/enso` and with manually running:
```
java -jar launcher.jar --launcher-log-level trace install engine 2024.1.1-nightly.2024.2.29
```
2024-02-29 16:42:33 +00:00
Pavel Marek
f48caac586
Upload native-image argfiles (#9094)
In PR #8953, in commit ba0a69de6e, I have introduced argument files to the `native-image`. In this PR, let's try to upload these argfiles as artifacts on GH, so that we can inspect them later.
2024-02-26 19:25:37 +00:00
Dmitry Bushev
7ff90aa3f7
Synchronize suggestions loading after the reconnect 2 (#9142)
related #8689, #9072

Fixes a race between the language server SQL updating logic and the engine `DeserializeLibrarySuggestionsJob`s when the library suggestions may start loading before the database is properly cleaned up after the reconnect.

# Important Notes
As a side effect, arguments are showing slightly (~1 second) faster due to the lower contention between the engine jobs.

#### Before
https://github.com/enso-org/enso/assets/357683/cbda2da4-9080-4b9b-b836-81e54694d468

#### After
https://github.com/enso-org/enso/assets/357683/bf442284-47be-456d-b1dd-2413b6ad8244
2024-02-23 11:18:01 +00:00
Hubert Plociniczak
0133e9ef4f
Drop inefficient SemVer implementation (#9089)
`Bump` library uses parser combinators behind the scenes which are known to be good at expressing grammars but are not performance-oriented.

This change ditches the dependency in favour of an existing Java implementation. `jsemver` implements the full specification, which is probably an overkill in our case, but proved to be an almost drop-in replacement for the previous library.

Closes #8692

# Important Notes
Peformance improvements:
- roughly 50ms compared to the previous approach (from 80ms to 20-40ms)

I don't see any time spent in the new implementation during startup so it could be potentially aggressively inlined.
Further more, we could use a facade and offer our own strip down version of semver.
2024-02-22 09:59:09 +00:00
AdRiley
6eb3601e49
Rename node to component (#9102)
For our users "Nodes" are called "Components". Which the users selects using the component browser. This MR just sets the text in the default workflow.
2024-02-20 10:19:59 +00:00
Pavel Marek
96082c3bae
Remove akka from runtime (#8953)
There are two projects transitively required by `runtime`, that have akka dependencies:
- `downloader`
- `connected-lock-manager`

This PR replaces the `akka-http` dependency in `downloader` by HttpClient from JDK, and splits `connected-lock-manager` into two projects such that there are no akka classes in `runtime.jar`.

# Important Notes
- Simplify the `downloader` project - remove akka.
- Add HTTP tests to the `downloader` project that uses our `http-test-helper` that is normally used for stdlib tests.
- It required few tweaks so that we can embed that server in a unit test.
- Split `connected-lock-manager` project into two projects - remove akka from `runtime`.
- **Native image build fixes and quality of life improvements:**
- Output of `native-image` is captured 743e167aa4
- The output will no longer be intertwined with the output from other commands on the CI.
- Arguments to the `native-image` are passed via an argument file, not via command line - ba0a69de6e
- This resolves an issue on Windows with "Command line too long", for example in https://github.com/enso-org/enso/actions/runs/7934447148/job/21665456738?pr=8953#step:8:2269
2024-02-19 16:39:05 +00:00
Hubert Plociniczak
5c624d21d7
Revert "Synchronize suggestions loading after the reconnect (#9043)" (#9072) 2024-02-15 13:39:56 +00:00
Dmitry Bushev
9c982e07b0
Synchronize suggestions loading after the reconnect (#9043)
related #8689

Fixes a race between the language server SQL updating logic and the engine `DeserializeLibrarySuggestionsJob`s when the library suggestions may start loading before the database is properly cleaned up after the reconnect.
2024-02-13 17:52:15 +00:00
Pavel Marek
5919eda753
Fix incremental compilation of runtime/test (#8975) 2024-02-13 10:05:31 +01:00
Dmitry Bushev
eb1f52984e
Local project manager work against any folder (#8985)
close #8650

Changes in project manager required to work with projects located in custom directories.

Changelog:
- add: `--projects-directory /path/to/projects` `--project-list` command line parameters returning the projects located in custom directory
- update: `project/create`, `project/open`, `project/delete`, `project/rename` commands to work with projects located in custom directories
2024-02-12 09:46:40 +00:00
Hubert Plociniczak
b8612344ca
Disable flaky test (#8967) 2024-02-05 13:41:43 +00:00
Hubert Plociniczak
34e1bacdf1
Further optimizations to test resources (#8936)
TestRuntime should be deprecated as it creates a number of threads and doesn't allow to easily modify ZIO's runtime.
But the biggest drop stems from fixing leaking `FileSystemService` that weren't being closed for every `TextOperationsTest` test.
The change is a follow up on #8892 but this time focused on ZIO usage.

Hopefully fixes #8806 for good.

# Important Notes
Running `language-server/test`.
Before:
![Screenshot from 2024-02-02 09-48-32](https://github.com/enso-org/enso/assets/292128/fb414c74-7d7a-4e7b-8b0c-d25dc3721bbf)

After:
![Screenshot from 2024-02-02 09-46-02](https://github.com/enso-org/enso/assets/292128/db9429df-d861-4f48-818f-888d5bbbb089)
2024-02-02 22:49:48 +00:00
Hubert Plociniczak
68ce4ae1b7
Reduced resources for various Akka dispatchers (#8892)
The defaults picked up by Akka tend to make us of all resources which is unnecessary and overwhelming for tests.

Improves #8806, potentially.

Before
![Screenshot from 2024-01-28 22-34-42](https://github.com/enso-org/enso/assets/292128/f80eb66a-2f37-44d5-bcdb-f00a78fe72fd)
After
![Screenshot from 2024-01-31 00-12-10](https://github.com/enso-org/enso/assets/292128/c5223912-5f6e-413c-a0a4-050afa3ed463)

when running the problematic `LibrariesTest`.

Full `language-server` test suite.
Before
![Screenshot from 2024-01-31 00-20-50](https://github.com/enso-org/enso/assets/292128/f1c94a66-6905-4f57-8a7d-7df049714353)
After
![Screenshot from 2024-01-31 00-18-40](https://github.com/enso-org/enso/assets/292128/3a11125e-d593-43df-8d35-1a8915812b2b)

# Important Notes
Note that Executors assigned to Zio and initializers should also be improved. Unfortunately due to various blocking threadpools  it is easy to get timeouts when running the whole suite.
2024-01-31 10:25:17 +00:00
Radosław Waśko
368e4867b4
Allow secrets in AWS_Credential (#8774)
- Closes #8722
2024-01-19 19:00:56 +00:00
Hubert Plociniczak
7d56cdc234
Thread dump on test timeout (#8785) 2024-01-17 11:20:38 +00:00
Jaroslav Tulach
09f484f00d
Make suspended atom fields work for boxed atoms (#8712)
Fixes #8710 by making sure suspended atom fields support works also for "normal" `Atom` instances without any special `Layout`. Moves all _atom related_ classes into single package and hides as much of classes as possible by making them _package private_.
2024-01-16 20:20:38 +00:00
GregoryTravis
f2cb1f097e
Support on_problems=Problem_Behavior.Report_Warning and Map_Error wrapping in Vector.map (#8595)
Implements `Warnings.get_all wrap_errors=True` which wraps warnings attached to values inside vectors with `Map_Error`, which includes the position of the value within the vector. See [the documentation](https://github.com/enso-org/enso/blob/develop/docs/semantics/wrapped-errors.md) for more details.

`get_all wrap_errors=True` does not change the warnings that are attached to values -- it wraps them before returning them to the caller, but does not change the original warnings attached to the values.

Wrapped warnings only appear attached to the vector itself. The values inside the vector do not have their warnings wrapped.

Warning propagation is not changed at all; `Warnings.get_all` (with default `wrap_errors=False`) behaves as before. `get_all wrap_errors=True` is meant to be used primarily by the IDE, although it can be used anywhere this wrapping is desired.
2024-01-16 09:36:22 +00:00
AdRiley
b8e93b3cba
Add new text_left and text_right functions (#8691)
Added text_left and text_right functions for in-memory and databases
2024-01-15 23:43:23 +00:00
Hubert Plociniczak
31a0dcef65
Suppress logs for test cases unless a failure is reported (#8694)
The change adds a convenient trait `ReportLogsOnFailure` that, when merged with the test class, will keep logs in memory and only delegate to the underlying appender on failure. For now we only support forwarding to the console which is sufficient.
A corresponding entry in `application-test.conf` has to point to the new `memory` appender. The additional complexity in the implementation ensures that if someone forgets to mixin `ReportLogsOnFailure` logs appear as before i.e. they respect the log level.

As a bonus fixed arguments passed to ScalaTest in build.sbt so that we are now, again, showing timings of individual tests.

Closes #8603.

# Important Notes
Before:
```
[info] VcsManagerTest:
[info] Initializing project
[ERROR] [2024-01-04 17:27:03,366] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a repository (3 seconds, 538 milliseconds)
[info] - must fail to create a repository for an already existing project (141 milliseconds)
[info] Save project
[ERROR] [2024-01-04 17:27:08,346] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a commit with a timestamp (198 milliseconds)
[ERROR] [2024-01-04 17:27:08,570] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a commit with a name (148 milliseconds)
[ERROR] [2024-01-04 17:27:08,741] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must force all pending saves (149 milliseconds)
[info] Status project
[ERROR] [2024-01-04 17:27:08,910] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must report changed files since last commit (148 milliseconds)
[info] Restore project
[ERROR] [2024-01-04 17:27:09,076] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to the last state with committed changes (236 milliseconds)
[ERROR] [2024-01-04 17:27:09,328] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to a named save (pending)
[ERROR] [2024-01-04 17:27:09,520] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to a named save and notify about removed files *** FAILED *** (185 milliseconds)
[info]   Right({
[info]     "jsonrpc" : "2.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) did not equal Right({
[info]     "jsonrpc" : "1.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) (VcsManagerTest.scala:1343)
[info]   Analysis:
[info]   Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }]))
[ERROR] [2024-01-04 17:27:09,734] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] List project saves
[info] - must return all explicit commits (146 milliseconds)
[info] Run completed in 9 seconds, 270 milliseconds.
[info] Total number of tests run: 9
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1
[info] *** 1 TEST FAILED ***
```

After:
```
[info] VcsManagerTest:
[info] Initializing project
[info] - must create a repository (3 seconds, 554 milliseconds)
[info] - must fail to create a repository for an already existing project (164 milliseconds)
[info] Save project
[info] - must create a commit with a timestamp (212 milliseconds)
[info] - must create a commit with a name (142 milliseconds)
[info] - must force all pending saves (185 milliseconds)
[info] Status project
[info] - must report changed files since last commit (142 milliseconds)
[info] Restore project
[info] - must reset to the last state with committed changes (202 milliseconds)
[info] - must reset to a named save (pending)
[ERROR] [2024-01-04 17:24:55,738] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/8456553964637757156].
[info] - must reset to a named save and notify about removed files *** FAILED *** (186 milliseconds)
[info]   Right({
[info]     "jsonrpc" : "2.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) did not equal Right({
[info]     "jsonrpc" : "1.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) (VcsManagerTest.scala:1343)
[info]   Analysis:
[info]   Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }]))
[info] List project saves
[info] - must return all explicit commits (131 milliseconds)
[info] Run completed in 9 seconds, 400 milliseconds.
[info] Total number of tests run: 9
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1
[info] *** 1 TEST FAILED ***
```
2024-01-09 09:59:10 +00:00
Dmitry Bushev
5e0a7337d0
Do not change project module name to Upper_Snake_Case (#8697)
close #8649

Do not mangle the project module name. Only strip the unsupported characters and make it a valid Enso module identifier.
2024-01-08 16:26:35 +00:00
Hubert Plociniczak
b30f558585
Cats are not welcome (#8696)
Removing an explicit import of `cats` implicits which triggers rather slow classloading, just for a single method that transforms `List[Either[A, B]]` to `Either[A, List[B]]`.

Addresses one of the problems mentioned in #8692.
2024-01-07 21:03:14 +00:00
Pavel Marek
428e83de36
Remove org.bouncycastle dependency (#8664)
Remove `org.bouncycastle` dependency from `org.enso.runtime`.
2024-01-04 17:16:41 +01:00
Dmitry Bushev
8c70ba7041
Kill lingering FanOutReceiver actors when the connection is closed (#8642)
close #8426

It is safe to kill the receiver actor when closing the connection. Otherwise, it will be cleaned up only on the actor system shutdown.
2024-01-02 13:57:49 +00:00
Dmitry Bushev
5f070febcc
Lock critical section in projects/list command (#8647)
close #8516

`repo.getAll()` can update metadata of projects. On startup, gui sends two `project/list` commands resulting in a race.
2024-01-02 11:59:50 +00:00
Pavel Marek
edc98593b9
All unit tests use the same testing logging provider (#8593)
Make sure that the correct test logging provider is loaded in `project-manager/Test`, so that only WARN and ERROR log messages are displayed. Also, make sure that the test log provider parses the correct configuration file - Rename all the `application.conf` files in the test resources to `application-test.conf`.

The problem was introduced in #8467
2023-12-21 13:45:33 +00:00
Hubert Plociniczak
aa05389f4a
Add thread dump info to investigate MacOS timeouts (#8577)
It is currently not possible to understand the nature of random timeouts for launcher tests on MacOS. Hopefully the additional info will bring us closer to understanding where is the problem.

Related to #8546.
2023-12-19 16:11:44 +00:00
Hubert Plociniczak
fce6d5dce6
Carry position for project management operations (#8589)
It's quite annoying to debug problems when position is inaccurate.
Example: https://github.com/enso-org/enso/actions/runs/7260847544/job/19780840613?pr=8577#step:10:2238
2023-12-19 15:37:15 +00:00
Pavel Marek
c1098865f2
Update java formatter sbt plugin (#8543)
Add a local clone of javaFormatter plugin. The upstream is not maintained anymore. And we need to update it to use the newest Google java formatter because the old one, that we use, cannot format sources with Java 8+ syntax.

# Important Notes
Update to Google java formatter 1.18.1 - https://github.com/google/google-java-format/releases/tag/v1.18.1
2023-12-15 14:45:23 +00:00
Pavel Marek
4b65e44ef3
EpbLanguage re-uses other TruffleContext support to run tests with assertions enabled (#7882) 2023-12-15 13:31:32 +01:00
Adam Obuchowicz
af50d32553
Small GUI improvements (#8535)
Minor improvements requested by @jdunkerley recently:
* Removed system cursor
* Changed shortcut for Code Editor
* Added `import Standard.Visualization` as a workaround allowing visualization previews. See https://github.com/orgs/enso-org/discussions/6832#discussioncomment-7789804

https://github.com/enso-org/enso/assets/3919101/38d822ac-28df-4a9c-a2dc-dee7adbd58ea
2023-12-13 16:26:13 +00:00
Jaroslav Tulach
e4b2b56a40
Turning Sieve benchmarks into Enso benchmarks (#8475) 2023-12-08 10:27:52 +01:00
Hubert Plociniczak
a14ebd6259
Increase timeout for running launcher command (#8486)
We've been experiencing consistently failures on MacOS due to timeouts.
Doubling the timeout, hoping this will be sufficient to eliminate such
false failures. Will seek alternative solutions if that does not rememdy
the problem on CI.
2023-12-07 16:34:55 +01:00
Hubert Plociniczak
e7fd90aa0c
Add missing resources for aarch64 (#8471)
* Add missing resources for aarch64

* one more
2023-12-06 14:51:09 +01:00
Hubert Plociniczak
021ff2b40e
Mark some tests as flaky on Windows (#8446) 2023-12-06 11:17:34 +00:00
Hubert Plociniczak
a11bddcb74
Inline execution should support FQNs (#8454)
* Test illustrating problems with FQNs

Inline execution fails with `Compile error: The name `Standard` could
not be found.`.

* Ensure InlineContext carries Package Repos info

Previously, there was no requirement that inline execution should allow
for FQNs. This meant that the omission of Package Repository info went
unnoticed.

In order to be able to refer to `Standard.Visualization.Preprocessor` it
has to be exported as well.
2023-12-06 10:03:06 +01:00
Jaroslav Tulach
7f0cb88fa1
Consistent simple and qualified type name (#8448)
Fixes #8255 by unifying `get_qualified_type_name` and `get_simple_type_name` implementations.
2023-12-06 04:30:24 +00:00
Hubert Plociniczak
49e78adb88
Build distribution for amd64 and aarch64 MacOS (#8407)
* Build distribution for amd64 and aarch64 MacOS

Possible after the GraalVM upgrade.

* Another attempt at building on MacOS M1

* One less hardcoded architecture

* Eliminate one more hardcoded architecture

* add more debug info

* nit
2023-12-05 11:24:02 +01:00
Pavel Marek
534bece541
project-manager can start without prior GraalVM installation (#8410)
This is a follow-up of #7991. #7991 broken `runtime-version-manager`. This is mostly reverts.


### Important Notes

Launcher now correctly recognizes that the newest engine needs some runtime:
```sh
> java -jar launcher.jar list
Enso 2023.2.1-nightly.2023.10.31 -> GraalVM 23.0.0-java17.0.7
Enso 0.0.0-dev -> GraalVM 23.1.0-java21.0.1
```
(this has not worked before)
2023-11-29 20:36:21 +01:00
Hubert Plociniczak
b6bdf901a9
Cleanup after each test run (#8418)
Reducing leaks when running our test suite.

Potentially fixes #8408.

# Important Notes
Managed to keep thread count and memory in between runs relatively stable.
Initially:
![Screenshot from 2023-11-29 11-06-04](https://github.com/enso-org/enso/assets/292128/af437d8a-9111-4bd6-9033-a59030c7ebed)
Now:
![Screenshot from 2023-11-29 15-57-34](https://github.com/enso-org/enso/assets/292128/3c1f8aef-fe7e-4f5b-a236-12c86ea8b906)

The screenshot illustrates for `RuntimeVisualizationsTest` only. Will need to be applied in other places as well.

Applying the same style to `runtime-with-instruments`.
Before:
![Screenshot from 2023-11-29 16-55-11](https://github.com/enso-org/enso/assets/292128/385e11be-0265-431d-b0d7-c5096df11c50)
After:
![Screenshot from 2023-11-29 16-50-07](https://github.com/enso-org/enso/assets/292128/5e8c28ea-e921-484a-a82c-9f2d3e827e8b)
2023-11-29 19:02:59 +00:00
Pavel Marek
5244003be8
Fix Benchmark processor when run from a root directory with different name than "enso" (#8382)
This PR fixes commands like `std-benchmarks/benchOnly Startup` for users that have `enso` clonned in multiple directories with different names.
2023-11-24 17:58:58 +00:00
Dmitry Bushev
53d1f727da
Always create events log when profiling (#8337)
Changelog:
- update: always create an event log next to the profiling file when the engine is started with the `--profiling-path` flag
- remove: `--profiling-events-log-path` flag
2023-11-20 16:01:25 +00:00
Dmitry Bushev
b224f95639
On-deman backend heap and thread dump (#8320)
close #8249

Changelog:
- add: `profiling/snapshot` request that takes a heap dump of the language server and puts it in the `ENSO_DATA_DIRECTORY/profiling` direcotry
2023-11-20 11:41:01 +00:00
Hubert Plociniczak
b8ebed69c3
Report failures when upserting visualizations (#8306)
Attaching or modifying a visualizations returns early on, to avoid a situation when a background job is stalled (by other jobs) and eventually the request timeouts.

This has an unfortunate consequence that any error reported in the `UpsertVisualizationJob` cannot be reported as a directly reply to a request because the sender has already been removed from the list.

Added more logs to discover why we get errors in the first place.

Modified the API a bit so that we carry `VisualizationContext` instead of three parameters all over the place.

Bonus:
Modified `JsonRpcServerTestKit` to implicitly require a position so that we get better error reporting on failures.
2023-11-20 11:38:51 +00:00
Jaroslav Tulach
ba19813511
Speeding up "hello world" example by 16% 2023-11-19 16:38:31 +01:00
Pavel Marek
5a7ad6bfe4
Upgrade enso to GraalVM for jdk 21 (#7991)
Upgrade to GraalVM JDK 21.
```
> java -version
openjdk version "21" 2023-09-19
OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15)
OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing)
```

With SDKMan, download with `sdk install java 21-graalce`.

# Important Notes
- After this PR, one can theoretically run enso with any JRE with version at least 21.
- Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution.
- `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded.
- sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368
- Which means that we can have `module-info.java` files only for Java-only projects.
- Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar.
- `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent.
- Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory.
- The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module.
- The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725)
- [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 18:02:36 +00:00
Dmitry Bushev
a286ab7daa
Backend self sampling (#8309)
close #8248

Changelog:
- add: `profiling/start` request starts the sampler and starts collecting runtime events to the log file
- add: `profiling/stop` request stop the sampler and write the profiling data to the `$ENSO_DATA_DIR/profiling` directory
- refactor: rewrite the profiling logic into Java
2023-11-17 15:04:10 +00:00
Hubert Plociniczak
348f5170ab
Avoid NPE during instrumentation (#8317)
Fixes a random crash (*) during instrumentation. Notice how `onTailCallReturn` calls `onReturnValue` with `null` frame.

Bonus: noticed that for some reason we weren't getting logs for `ExecutionService`. This turned out to be the problem with the logger name which by default was `[enso]` not
`[enso.org.enso.interpreter.service.ExecutionService`] and there is some logic there that normalizes the name and assumed a dot after `enso`. This change fixes the logic.

(*)
```
[enso.org.enso.interpreter.service.ExecutionService] Execution of function main failed (Cannot invoke "com.oracle.truffle.api.frame.VirtualFrame.materialize()" because "frame" is null).
java.lang.NullPointerException: Cannot invoke "com.oracle.truffle.api.frame.VirtualFrame.materialize()" because "frame" is null
at org.enso.interpreter.instrument.IdExecutionInstrument$IdEventNodeFactory$IdExecutionEventNode.onReturnValue(IdExecutionInstrument.java:246)
at org.enso.interpreter.instrument.IdExecutionInstrument$IdEventNodeFactory$IdExecutionEventNode.onTailCallReturn(IdExecutionInstrument.java:274)
at org.enso.interpreter.instrument.IdExecutionInstrument$IdEventNodeFactory$IdExecutionEventNode.onReturnExceptional(IdExecutionInstrument.java:258)
at org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode$EventProviderChainNode.innerOnReturnExceptional(ProbeNode.java:1395)
at org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode$EventChainNode.onReturnExceptional(ProbeNode.java:1031)
at org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode.onReturnExceptionalOrUnwind(ProbeNode.java:296)
at org.enso.interpreter.node.ExpressionNodeWrapper.executeGeneric(ExpressionNodeWrapper.java:119)
at org.enso.interpreter.node.ClosureRootNode.execute(ClosureRootNode.java:85)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:718)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:641)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:574)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.doInvoke(OptimizedCallTarget.java:558)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:504)
at jdk.internal.vm.compiler/org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:69)
at org.enso.interpreter.node.callable.thunk.ThunkExecutorNode.doCached(ThunkExecutorNode.java:69)
at org.enso.interpreter.node.callable.thunk.ThunkExecutorNodeGen.executeAndSpecialize(ThunkExecutorNodeGen.java:207)
at org.enso.interpreter.node.callable.thunk.ThunkExecutorNodeGen.executeThunk(ThunkExecutorNodeGen.java:167)
...
```

# Important Notes
Fixes regressions introduced in #8148 and #8162
2023-11-17 14:38:27 +00:00
Hubert Plociniczak
c649ed87af
Let logging to file respect log level (#8264)
This change fixes a regression introduced in #7918, which prevented the execution from setting the right log level either via env var or parameter.

Now passing either of the options returns logs of the expected level in the log file:
- `ENSO_LOG_TO_FILE_LOG_LEVEL = trace`
- ... `-vv` ...

Fixes #8274
2023-11-14 11:59:27 +00:00
Hubert Plociniczak
51abb3e1b0
Apply custom log levels to Truffle logger (#8162)
Previously custom log levels applied only to non-Truffle loggers. To allow it, filtering has to be applied appropriately at two places - first at Java's Handler and then essentially re-confirmed at SLF4J's logger to which the former forwards to.
Filters compose in an `AND` condition, therefore default log level check had to be merged into our custom filters.

`TruffleLogger` has a builtin functionality to perform the filtering when context is configured appropriately. This should be much more efficient than adding a `Filter` to the JUL Handler explicitly.

# Important Notes
```
JAVA_OPTS="-org.enso.compiler.SerializationManager.Logger.level=debug" ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso  --run
```
will now assign a custom log level to `SerializationManager` Logger.
2023-11-10 16:34:04 +00:00
Dmitry Bushev
5b4716e65a
Enable profiling for all commands of runner executable (#8254)
close #8156
2023-11-08 19:30:28 +00:00