Commit Graph

122 Commits

Author SHA1 Message Date
Hubert Plociniczak
0133e9ef4f
Drop inefficient SemVer implementation (#9089)
`Bump` library uses parser combinators behind the scenes which are known to be good at expressing grammars but are not performance-oriented.

This change ditches the dependency in favour of an existing Java implementation. `jsemver` implements the full specification, which is probably an overkill in our case, but proved to be an almost drop-in replacement for the previous library.

Closes #8692

# Important Notes
Peformance improvements:
- roughly 50ms compared to the previous approach (from 80ms to 20-40ms)

I don't see any time spent in the new implementation during startup so it could be potentially aggressively inlined.
Further more, we could use a facade and offer our own strip down version of semver.
2024-02-22 09:59:09 +00:00
Dmitry Bushev
eb1f52984e
Local project manager work against any folder (#8985)
close #8650

Changes in project manager required to work with projects located in custom directories.

Changelog:
- add: `--projects-directory /path/to/projects` `--project-list` command line parameters returning the projects located in custom directory
- update: `project/create`, `project/open`, `project/delete`, `project/rename` commands to work with projects located in custom directories
2024-02-12 09:46:40 +00:00
Hubert Plociniczak
34e1bacdf1
Further optimizations to test resources (#8936)
TestRuntime should be deprecated as it creates a number of threads and doesn't allow to easily modify ZIO's runtime.
But the biggest drop stems from fixing leaking `FileSystemService` that weren't being closed for every `TextOperationsTest` test.
The change is a follow up on #8892 but this time focused on ZIO usage.

Hopefully fixes #8806 for good.

# Important Notes
Running `language-server/test`.
Before:
![Screenshot from 2024-02-02 09-48-32](https://github.com/enso-org/enso/assets/292128/fb414c74-7d7a-4e7b-8b0c-d25dc3721bbf)

After:
![Screenshot from 2024-02-02 09-46-02](https://github.com/enso-org/enso/assets/292128/db9429df-d861-4f48-818f-888d5bbbb089)
2024-02-02 22:49:48 +00:00
Hubert Plociniczak
68ce4ae1b7
Reduced resources for various Akka dispatchers (#8892)
The defaults picked up by Akka tend to make us of all resources which is unnecessary and overwhelming for tests.

Improves #8806, potentially.

Before
![Screenshot from 2024-01-28 22-34-42](https://github.com/enso-org/enso/assets/292128/f80eb66a-2f37-44d5-bcdb-f00a78fe72fd)
After
![Screenshot from 2024-01-31 00-12-10](https://github.com/enso-org/enso/assets/292128/c5223912-5f6e-413c-a0a4-050afa3ed463)

when running the problematic `LibrariesTest`.

Full `language-server` test suite.
Before
![Screenshot from 2024-01-31 00-20-50](https://github.com/enso-org/enso/assets/292128/f1c94a66-6905-4f57-8a7d-7df049714353)
After
![Screenshot from 2024-01-31 00-18-40](https://github.com/enso-org/enso/assets/292128/3a11125e-d593-43df-8d35-1a8915812b2b)

# Important Notes
Note that Executors assigned to Zio and initializers should also be improved. Unfortunately due to various blocking threadpools  it is easy to get timeouts when running the whole suite.
2024-01-31 10:25:17 +00:00
Hubert Plociniczak
31a0dcef65
Suppress logs for test cases unless a failure is reported (#8694)
The change adds a convenient trait `ReportLogsOnFailure` that, when merged with the test class, will keep logs in memory and only delegate to the underlying appender on failure. For now we only support forwarding to the console which is sufficient.
A corresponding entry in `application-test.conf` has to point to the new `memory` appender. The additional complexity in the implementation ensures that if someone forgets to mixin `ReportLogsOnFailure` logs appear as before i.e. they respect the log level.

As a bonus fixed arguments passed to ScalaTest in build.sbt so that we are now, again, showing timings of individual tests.

Closes #8603.

# Important Notes
Before:
```
[info] VcsManagerTest:
[info] Initializing project
[ERROR] [2024-01-04 17:27:03,366] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a repository (3 seconds, 538 milliseconds)
[info] - must fail to create a repository for an already existing project (141 milliseconds)
[info] Save project
[ERROR] [2024-01-04 17:27:08,346] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a commit with a timestamp (198 milliseconds)
[ERROR] [2024-01-04 17:27:08,570] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must create a commit with a name (148 milliseconds)
[ERROR] [2024-01-04 17:27:08,741] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must force all pending saves (149 milliseconds)
[info] Status project
[ERROR] [2024-01-04 17:27:08,910] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must report changed files since last commit (148 milliseconds)
[info] Restore project
[ERROR] [2024-01-04 17:27:09,076] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to the last state with committed changes (236 milliseconds)
[ERROR] [2024-01-04 17:27:09,328] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to a named save (pending)
[ERROR] [2024-01-04 17:27:09,520] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] - must reset to a named save and notify about removed files *** FAILED *** (185 milliseconds)
[info]   Right({
[info]     "jsonrpc" : "2.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) did not equal Right({
[info]     "jsonrpc" : "1.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) (VcsManagerTest.scala:1343)
[info]   Analysis:
[info]   Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }]))
[ERROR] [2024-01-04 17:27:09,734] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318].
[info] List project saves
[info] - must return all explicit commits (146 milliseconds)
[info] Run completed in 9 seconds, 270 milliseconds.
[info] Total number of tests run: 9
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1
[info] *** 1 TEST FAILED ***
```

After:
```
[info] VcsManagerTest:
[info] Initializing project
[info] - must create a repository (3 seconds, 554 milliseconds)
[info] - must fail to create a repository for an already existing project (164 milliseconds)
[info] Save project
[info] - must create a commit with a timestamp (212 milliseconds)
[info] - must create a commit with a name (142 milliseconds)
[info] - must force all pending saves (185 milliseconds)
[info] Status project
[info] - must report changed files since last commit (142 milliseconds)
[info] Restore project
[info] - must reset to the last state with committed changes (202 milliseconds)
[info] - must reset to a named save (pending)
[ERROR] [2024-01-04 17:24:55,738] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/8456553964637757156].
[info] - must reset to a named save and notify about removed files *** FAILED *** (186 milliseconds)
[info]   Right({
[info]     "jsonrpc" : "2.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) did not equal Right({
[info]     "jsonrpc" : "1.0",
[info]     "method" : "file/event",
[info]     "params" : {
[info]       "path" : {
[info]         "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]         "segments" : [
[info]           "src",
[info]           "Bar.enso"
[info]         ]
[info]       },
[info]       "kind" : "Removed"
[info]     }
[info]   }) (VcsManagerTest.scala:1343)
[info]   Analysis:
[info]   Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> {
[info]   "path" : {
[info]     "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d",
[info]     "segments" : [
[info]       "src",
[info]       "Bar.enso"
[info]     ]
[info]   },
[info]   "kind" : "Removed"
[info] }]))
[info] List project saves
[info] - must return all explicit commits (131 milliseconds)
[info] Run completed in 9 seconds, 400 milliseconds.
[info] Total number of tests run: 9
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1
[info] *** 1 TEST FAILED ***
```
2024-01-09 09:59:10 +00:00
Dmitry Bushev
8c70ba7041
Kill lingering FanOutReceiver actors when the connection is closed (#8642)
close #8426

It is safe to kill the receiver actor when closing the connection. Otherwise, it will be cleaned up only on the actor system shutdown.
2024-01-02 13:57:49 +00:00
Dmitry Bushev
5f070febcc
Lock critical section in projects/list command (#8647)
close #8516

`repo.getAll()` can update metadata of projects. On startup, gui sends two `project/list` commands resulting in a race.
2024-01-02 11:59:50 +00:00
Pavel Marek
edc98593b9
All unit tests use the same testing logging provider (#8593)
Make sure that the correct test logging provider is loaded in `project-manager/Test`, so that only WARN and ERROR log messages are displayed. Also, make sure that the test log provider parses the correct configuration file - Rename all the `application.conf` files in the test resources to `application-test.conf`.

The problem was introduced in #8467
2023-12-21 13:45:33 +00:00
Hubert Plociniczak
fce6d5dce6
Carry position for project management operations (#8589)
It's quite annoying to debug problems when position is inaccurate.
Example: https://github.com/enso-org/enso/actions/runs/7260847544/job/19780840613?pr=8577#step:10:2238
2023-12-19 15:37:15 +00:00
Pavel Marek
534bece541
project-manager can start without prior GraalVM installation (#8410)
This is a follow-up of #7991. #7991 broken `runtime-version-manager`. This is mostly reverts.


### Important Notes

Launcher now correctly recognizes that the newest engine needs some runtime:
```sh
> java -jar launcher.jar list
Enso 2023.2.1-nightly.2023.10.31 -> GraalVM 23.0.0-java17.0.7
Enso 0.0.0-dev -> GraalVM 23.1.0-java21.0.1
```
(this has not worked before)
2023-11-29 20:36:21 +01:00
Dmitry Bushev
53d1f727da
Always create events log when profiling (#8337)
Changelog:
- update: always create an event log next to the profiling file when the engine is started with the `--profiling-path` flag
- remove: `--profiling-events-log-path` flag
2023-11-20 16:01:25 +00:00
Pavel Marek
5a7ad6bfe4
Upgrade enso to GraalVM for jdk 21 (#7991)
Upgrade to GraalVM JDK 21.
```
> java -version
openjdk version "21" 2023-09-19
OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15)
OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing)
```

With SDKMan, download with `sdk install java 21-graalce`.

# Important Notes
- After this PR, one can theoretically run enso with any JRE with version at least 21.
- Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution.
- `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded.
- sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368
- Which means that we can have `module-info.java` files only for Java-only projects.
- Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar.
- `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent.
- Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory.
- The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module.
- The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725)
- [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 18:02:36 +00:00
Hubert Plociniczak
c649ed87af
Let logging to file respect log level (#8264)
This change fixes a regression introduced in #7918, which prevented the execution from setting the right log level either via env var or parameter.

Now passing either of the options returns logs of the expected level in the log file:
- `ENSO_LOG_TO_FILE_LOG_LEVEL = trace`
- ... `-vv` ...

Fixes #8274
2023-11-14 11:59:27 +00:00
Dmitry Bushev
5b4716e65a
Enable profiling for all commands of runner executable (#8254)
close #8156
2023-11-08 19:30:28 +00:00
Hubert Plociniczak
352ad06d2f
Reduce extra output in compilation and tests (#7809)
* Reduce extra output in compilation and tests

I couldn't stand the amount of extra output that we got when compiling
a clean project and when executing regular tests. We should strive to
keep output clean and not print anything additional to stdout/stderr.

* Getting rid of explicit setup by service loading

In order for SL4J to use service loading correctly had to upgrade to
latest slf4j. Unfortunately `TestLogProvider` which essentially
delegates to `logback` provider will lead to spurious ambiguous warnings
on multiple providers. In order to dictate which one to use and
therefore eliminate the warnings we can use the `slf4j.provider` env
var, which is only available in slf4j 2.x.

Now, there is no need to explicitly call `LoggerSetup.get().setup()` as
that is being called during service setup.

* legal review

* linter

* Ensure ConsoleHandler uses the default level

ConsoleHandler's constructor uses `Level.INFO` which is unnecessary for
tests.

* report warnings
2023-10-16 10:57:44 +02:00
Hubert Plociniczak
cfba3c6887
Add support for https and wss (#7937)
* Add support for https and wss

Preliminary support for https and wss. During language server startup we
will read the application config and search for the `https` config with
necessary env vars set.

The configuration supports two modes of creating ssl-context - via
PKCS12 format and certificat+private key.

Fixes #7839.

* Added tests, improved documentation

Generic improvements along with actual tests.

* lint

* more docs + wss support

* changelog

* Apply suggestions from code review

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>

* PR comment

* typo

* lint

* make windows line endings happy

---------

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
2023-10-12 00:03:34 +02:00
Hubert Plociniczak
63802544f7
Enable log-to-file configuration (#7918)
* Enable log-to-file configuration

PR #7825 enabled parallel logging to a file with a much more
fine-grained log level by default.
However, logging at `TRACE` level on Windows appears to be still
problematic.

This PR reduced the default log level to file from `DEBUG` to `TRACE`
and allows to control it via an environment variable if one wishes to
change the verbosity without making code changes.

* PR comments
2023-10-02 14:25:09 +02:00
Dmitry Bushev
71442fe32c
Open projects using the packaged language server (#7868)
close #7750
close #7834

Changelog:
- update: project manager uses the packaged language server to open projects
- fix: remove stack traces from connection errors on initial ping handler request (when the language server is booting)
- update: add engine and edition versions to the `initProtocolConnection` response for easier debug
- update: do not resolve project ensoVersion in the `project/list` to eliminate unnecessary network calls
2023-09-28 14:13:14 +00:00
Hubert Plociniczak
18b2491a41
Always log to console and file (#7825)
* Always log verbose to a file

The change adds an option by default to always log to a file with
verbose log level.
The implementation is a bit tricky because in the most common use-case
we have to always log in verbose mode to a socket and only later apply
the desired log levels. Previously socket appender would respect the
desired log level already before forwarding the log.

If by default we log to a file, verbose mode is simply ignored and does
not override user settings.

To test run `project-manager` with `ENSO_LOGSERVER_APPENDER=console` env
variable. That will output to the console with the default `INFO` level
and `TRACE` log level for the file.

* add docs

* changelog

* Address some PR requests

1. Log INFO level to CONSOLE by default
2. Change runner's default log level from ERROR to WARN

Took a while to figure out why the correct log level wasn't being passed
to the language server, therefore ignoring the (desired) verbose logs
from the log file.

* linter

* 3rd party uses log4j for logging

Getting rid of the warning by adding a log4j over slf4j bridge:
```
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
```

* legal review update

* Make sure tests use test resources

Having `application.conf` in `src/main/resources` and `test/resources`
does not guarantee that in Tests we will pick up the latter. Instead, by
default it seems to do some kind of merge of different configurations,
which is far from desired.

* Ensure native launcher test log to console only

Logging to console and (temporary) files is problematic for Windows.
The CI also revealed a problem with the native configuration because it
was not possible to modify the launcher via env variables as everything
was initialized during build time.

* Adapt to method changes

* Potentially deal with Windows failures
2023-09-26 11:32:04 +02:00
Hubert Plociniczak
3e615f385b
Improve shutdown logic of language server (#7801)
* Improve shutdown logic of language server

This PR addresses problems mentioned in #7470 and #7729:
- shutting a language server explicitly will not lead to a soft shutdown
- `project/status` endpoint returns the state of the language server

`LanguageServerController` now also signed up for `ClientConnect`
messages. For it to be unambiguous, we need to carry around the port
number of the language server as a way of identifying the right one.

One can now use `project/status` to additionally determine the state of
the language server.

Also relies on a proper fix for #7765.

* changelog

* PR comments
2023-09-22 12:05:44 +02:00
Hubert Plociniczak
ddbccd8764
Fix console output for native image (#7744)
* Add serialization config to fix native image

Discovered when switching to console output.

* Add more documentation
2023-09-06 12:12:52 +02:00
Hubert Plociniczak
8a60bc6dcd
Replace a custom logger with off the shelf implementation (#7559)
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.

Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.

Most important differences have been outlined in `docs/infrastructure/logging.md`.

# Important Notes
Addresses:
- #7253
- #6739
2023-09-04 09:40:16 +00:00
Dmitry Bushev
8860ed71fe
Fix dev version check (#7640)
When building the dev IDE version with `./run ide build`, it creates the engine distribution with version `2023.2.1-dev`. This causes the version check to fail preventing opening the projects.
2023-08-25 10:48:24 +00:00
somebody1234
aa2f72cbd0
Allow configuring Project Manager server host and port (#6774)
- Closes #6730
- Changes config to allow environment variables to override server host and port
- Adds port scanning to Electron app to ensure the PM is started at a free port

# Important Notes
- `SERVER_PORT=abcd enso.AppImage` does NOT work. It would not be difficult to implement, but it probably needs discussion on how exactly it should be implemented - for example, `SERVER_PORT` is quite a generic name, should the Electron app pass though something like `ENSO_PM_SERVER_PORT` to the PM as `SERVER_PORT` instead?

⚠️ Port scanning is *only* implemented in the JS frontend. It is not implemented:
- In Scala, because the JS/Rust code calling it needs to know the port as well. There shouldn't be any problems with adding port scanning though, if that's desired
- In Rust, because I'm not sure parsing the host and port from a string is a good idea.
- (This also applies to JS, but it *must* work in JS, and port scanning is already a dependency there so it's quite a bit easier)

- QA *will* need a new PM (`sbt buildProjectManagerDistribution` or `./run backend sbt` -> `buildProjectManagerDistribution`), and the path must be supplied as: `-engine.project-manager-path=path/to/new/pm/here`
2023-08-25 07:33:31 +00:00
Dmitry Bushev
c80b8624a8
Log project manager version on startup (#7646)
Looks like that
```
[info] [2023-08-23T13:12:58.119Z] [org.enso.projectmanager.boot.ProjectManager$] Starting Enso Project Manager
Version:    0.0.0-dev
Built with: scala-2.13.11 for GraalVM 17.0.7
Built from: wip/db/fix-dev-version-check* @ 52bc6b8fcf
Built on:   Linux (amd64)
```
2023-08-24 16:45:22 +00:00
Dmitry Bushev
5dc2c4c5fd
Notification about the project rename action (#7613)
close #7604

After moving the rename action to the dashboard, IDE is unaware of the new project name. PR implements a new `refactoring/projectRenamed` notification that is sent from the server to clients and informs them about the changed project name.

# Important Notes
https://github.com/enso-org/enso/assets/357683/7c62726d-217e-4e69-8e48-568e0b7b8c34
2023-08-22 11:32:46 +00:00
Dmitry Bushev
8416e19eef
Refactor project normalized name (#7444)
Followup to #7397. When started this refactoring I only changed the config types and forgot to change the API.

Changelog:
- refactor: rename `projectModule` to `projectNormalizedName` in the API

# Important Notes
https://github.com/enso-org/enso/assets/357683/a840dc90-15f8-4b6d-bbfb-571c53cd2f7d
2023-07-31 17:47:28 +00:00
Dmitry Bushev
80c4b1ca06
Allow users to give a project other than Upper_Snake_Case name (#7397)
close #6356

Allow arbitrary names for user projects.

# Important Notes
https://github.com/enso-org/enso/assets/357683/55a3b660-af23-4b09-959b-eac515766788
2023-07-28 13:44:39 +00:00
Dmitry Bushev
980657df08
StopServer timing out on project stopping (#7416)
close #7345

#7254 introduced a delayed shutdown timeout. The `LanguageServerGateway` timeout should include the delayed shutdown time to prevent false timeouts.
2023-07-28 08:00:51 +00:00
Pavel Marek
cab6968340
Update GraalVM to 17.0.7 (23.0.0 JDK17) (#7176)
- Previous GraalVM update: https://github.com/enso-org/enso/pull/6750

Removed warnings:
- Remove deprecated `ConditionProfile.createCountingProfile()`.
- Add `@Shared` to some `@Cached` parameters (Truffle now emits warnings about potential `@Share` usage).
- Specialization method names should not start with execute
- Add limit attribute to some specialization methods
- Add `@NeverDefault` for some cached initializer expressions
- Add `@Idempotent` or `@NonIdempotent` where appropriate

BigInteger and potential Node inlining are tracked in follow-up issues.

# Important Notes
For `SDKMan` users:
```
sdk install java 17.0.7-graalce
sdk use java 17.0.7-graalce
```

For other users - download link can be found at https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-17.0.7

Release notes: https://www.graalvm.org/release-notes/JDK_17/

R component was dropped from the release 23.0.0, only `python` is available to install via `gu install python`.
2023-07-20 15:11:30 +00:00
Hubert Plociniczak
9252eb5f3a
Delay LS shutdown when last client disconnects (#7254)
* Delay LS shutdown when last client disconnects

Rather than closing Language Server immediately, we delay the shutdown
until some timeout hits. This gives a chance for new clients to connect
without paying the price of the initialization again.

More importantly, during hibernation/restart, the connection between
client (IDE) and LS is severed so it appears as if client disconnect. In
fact a few moments later IDE would attempt to re-establish the
connection on the same port. Without this change, LS shutsdown and
further attempts to connect on that particular port will fail.

There are still problems on the IDE-side after waking up from
hibernation but it is not related to Language Server.

* Introduce a separate timeout for delayed shutdown

Can't/shouldn't use the same timeout value as for shutdown timeout for
delaying shutdowns initiated by lack of clients.

* Add test demonstrating the new functionality
2023-07-18 08:37:11 +02:00
Jaroslav Tulach
6a6d7dbff3
Simplify onboarding instructions for engine & project-manager developers (#7181)
The current instructions to _build, use and debug_ `project-manager` and its engine/ls process are complicated and require a lot of symlinks to properly point to each other. This pull requests simplifies all of that by introduction of `ENSO_ENGINE_PATH` and `ENSO_JVM_PATH` environment variables. Then it hides all the complexity behind a simple _sbt command_: `runProjectManagerDistribution --debug`.

# Important Notes
I decided to tackle this problem as I have three repositories with different branches of Enso and switching between them requires me to mangle the symlinks. I hope I will not need to do that anymore with the introduction of the `runProjectManagerDistribution` command.
2023-07-17 17:38:54 +00:00
Dmitry Bushev
f691713077
Add endpoint for downloading a project (#7291)
close #7194

Changelog:

- add: `/projects/{project_id}/enso_project` HTTP endpoint returning an `.enso-project` archive structure
- update: archive enso project to a `.enso-project` `.tar.gz` archive
- update: make project `path` a required field
2023-07-14 13:18:08 +00:00
Hubert Plociniczak
dd06c59380
Initialize Language Server's JSON RPC protocol asynchronously (#7232)
As discovered in #7224, Json RPC protocol was added to the asynchronous resource initialization stage, as part of #6306, but was not in fact initialized at that point.
Instead it was initialized when the server was started to be able to serve correctly the initialization messages. A classic Catch-22. It was really hard to discover this just by looking at the code, but the profiling clearly showed where the time was spent.

This change splits Language Server's protocol into two:
- the first one accepts `heartbeat/init` and `session/initProtocolConnection`
- the second one enriches it with the full set of supported messages

This shifts the initialization from blocking for 0.5 sec to only ~30ms, and performing the second stage asynchronously.

Closes #7224.

# Important Notes
Before the change (blocking server startup):
![Screenshot from 2023-07-05 18-53-24](https://github.com/enso-org/enso/assets/292128/bcfa9043-d00a-4b36-a44c-782a388a16b9)
![Screenshot from 2023-07-05 18-53-10](https://github.com/enso-org/enso/assets/292128/54927787-4c95-46db-bd68-f3a3b82367d5)

After the change (1st stage):
![Screenshot from 2023-07-06 14-02-34](https://github.com/enso-org/enso/assets/292128/d7a7bc34-39dc-46f1-9e64-6d350697c30b)

After the change (2nd, asynchronous initialization, stage):
![Screenshot from 2023-07-06 14-21-17](https://github.com/enso-org/enso/assets/292128/def8c0a1-f211-4fc0-9df0-7c1634312166)
2023-07-07 11:49:23 +00:00
Hubert Plociniczak
72a6c54dfc
Update context on timeout retry (#7207)
Minor follow up on #7173. Noticed while testing nightly on Windows machine.
2023-07-05 12:22:55 +00:00
Hubert Plociniczak
414b124123
Add timeout retries for project-related operations (#7173)
It is relatively easy to reach timeouts on weak systems for startup
operations on project manager. Once a timeout is reached, startup will
not proceed any further.

This PR is a bit of an experiment. It adds adds timeout retries to give a bit of a leeway to
under-powered machines and to log some progress on the way, so that we
know that certain actions are still in progress.
2023-07-03 12:44:41 +00:00
Hubert Plociniczak
3c93c25a5a
Reload project's config on rename (#7179)
Package's config information, once loaded, never changed. While there is typically no need for it, this was problematic when the config became out-of-sync with the filesystem, like in the case of project rename action.
In rename, the config's properties would be updated in the FS, but that would never be reflected in module's package. Therefore further compilations would continue to ask for the old namespace.

Most of the changes are cosmetic (s/`.config`/`.getConfig()`) except for the new `reloadConfig` method on `Package` that is being called in `RenameProjectCmd` handler.

Closes #7062.

# Important Notes
The reported `ExecutionFailed` error should have been mostly fixed already via #7143. This change makes sure that all the related warnings are gone as well and the compiler uses the updated namespace.
2023-07-03 09:36:19 +00:00
Hubert Plociniczak
26bd95cf3d
Don't report EOFException on shutdown (#7157)
It's part of the normal shutdown workflow and confused users are reporting it as a failure.
2023-06-29 13:42:13 +00:00
Dmitry Bushev
48f0c6f5e8
Scala 2.13.11 and libraries update (#7010)
Update Scala and libraries.
2023-06-14 13:15:57 +00:00
Hubert Plociniczak
86432b5ca5
Detect and override hooks of the same kind (#6842)
This change ensures that we can have at most one hook of the same action during shutdown.
Verified the change on a real project.

Closes #6767.
2023-05-29 07:25:30 +00:00
somebody1234
6693bdb5cd
Fallback to opened date when ordering projects (#6814)
Fixes #6787

# Important Notes
I can't get Project Manager compilation to work locally so I guess I'll be relying on CI to verify that it's working correctly?

Of course, QA should be able to catch any problems too - the websocket API hasn't been changed so it should work out of the box with the current dashboard.
2023-05-25 15:44:15 +00:00
Dmitry Bushev
0318880990
Add project creation time to project metadata (#6780)
close #6686

Changelog:
- add: `created` field to the project metadata response
- update: project manager tests
2023-05-22 13:40:10 +00:00
Hubert Plociniczak
06624f34a8
Ensure slow shutdown of LS always kicks off hooks (#6665)
This change fixes the rather elusive bug where shutdown hooks could not be fired when shutdown was taking too long and termination was forced.

Under the circumstances described in detail in ticket #6515 there was a small chance that we could have a shutdown race condition. Essentially the messages received when client was disconnected and language server forced the termination could lead to language server not sending the public `ProjectClosed` message which triggers shutdown hook. Now we always do.

Also made sure that multiple `ProjectClosed` messages don't lead to firing multiple shutdown hooks, which was another possibility.

No tests as one would have to be able to introduce different delays in various message handlers to simulate the problem.
Having ability to do such chaos testing would be nice but it is beyond the scope of this ticket.
I was able to reproduce the problem 100% with my specially crafted setup so I'm fairly confident about the change.

Closes #6515.
2023-05-15 12:15:37 +00:00
Dmitry Bushev
706791779b
SuggestionBuilder needs to send ascribedType of constructor parameters (#6655)
close #6611

Changelog:
- update: run compiler passes on the `ascribedType` field of the constructor arguments
- update: suggestion builder uses the type information attached to `ascribedType`
- feat: resolve qualified names in type signatures
2023-05-13 18:33:03 +00:00
Hubert Plociniczak
b93edb050b
Force pending saves if client closes abruptly (#6514)
Force pending saves when JsonSession is terminated for the client.

Potentially closes #6395.
2023-05-03 23:30:20 +00:00
Dmitry Bushev
38212786a3
Fix application config (#6513)
Related #6482
2023-05-02 17:59:44 +00:00
Hubert Plociniczak
ce4ecc6706
Limit Dead Letter logging (#6482)
Dead Letter logging is occasionally flooding our logs which is confusing to users reporting bugs. Left the possibility of a single report so that we know that something is happening.
2023-05-02 14:00:50 +00:00
Dmitry Bushev
2d168308af
Better cleanup of project management test suite (#6392)
Address some arbitrary failures of the `ProjectManagementApiSpec` test suite.
2023-04-24 13:11:27 +00:00
Dmitry Bushev
74c0718604
Project folder not renamed after project name change (#6369)
close #6254

Changelog:
- fix: race when the actor system may be stopped before the shutdown hooks are executed
- fix: project management spec
- fix: recover from `readLine` failure during the shutdown
2023-04-20 16:57:37 +00:00
Dmitry Bushev
aa97787603
Update ZIO library (#6072)
close #6069

Changelog:
- update: ZIO major version bump
2023-03-28 07:58:59 +00:00