Commit Graph

27 Commits

Author SHA1 Message Date
Hubert Plociniczak
65010dffa7
Simplify shutdown logic on client disconnect in project-manager (#11712)
* Drop soft-shutdown on last client disconnect

Suspend on Windows confuses the reconnection logic and triggers a full
shutdown. This change simply drop shutdown on last client disconnect and
expects and explicit command.

* Various cherry-picks

Minor cherry-picks from the debugging branch. Should reduce  the amount
of non-critical warnings.
2024-12-02 21:19:55 +01:00
Dmitry Bushev
e5a65a26cb
Update message handler supervisor to watch children (#11630)
close #11248

Changelog:
- update: bind message handler supervisor lifetime to the message handler lifetime
2024-11-22 16:48:05 +00:00
Pavel Marek
9182f91e35
engine-runner and language-server are separate JPMS modules (#10823)
* Use moduleDependencies instead of modulePath

* Fix compilation of editions

* Fix compilation of distribution-manager

* polyglot-api needs to explicitly compile module-info

* Fix compilationOrder in library-manager and edition-updater

* engine-runner-common is module

* JPMSPlugin provides default implementation of compileModuleInfo

* Remove unused setting key from JPMSUtils.compileModuleInfo

* JPMSPlugin has internalModuleDependencies and exportedModule tasks.

* Use BuildVersion instead of buildInfo

* Manual compilation of module-info.java is reported as warning

* Define org.enso.scalalibs.wrapper meta project.

* Fix module check in JPMSPlugin.

This is a fix for projects that declare `Compile /exportJars := true`

* version-output is a module

* ydoc-server uses internalModuleDependencies

* persistance is module

* engine-common uses internalModuleDependencies

* polyglot-api does not override compileModuleInfo task

* runtime-parser uses internalModuleDependencies

* edition-updater is module

* Update moduleDependencies for distribution-manager

* editions is module

* Fix some dependencies of modules

* scala-yaml is a module

* Add scala-compiler to scala-libs-wrapper

* cli depends on scala-library module

* Add dependencies for distribution-manager module

* Add some scala-library dependencies in some modules

* engine-runner uses internalModuleDependencies

* Fix module dependencies of library-manager

* Rename org.enso.scalalibs.wrapper to org.enso.scala.wrapper

* Add jsoniter-scala-macros to org.enso.scala.wrapper fat module

* Fix dependencies of some projects

* polyglot-api does not depend on truffle-api

* Fix dependencies of some projects

* runtime does not use com.google.common

* runtime is a module

* text-buffer is a module

* refactoring-utils is a module

* runtime-compiler is a module

* runtime-instrument-common is a module

* connected-lock-manager is a module

* JPMSUtils reports project name in some error messages

* Modularize some instruments

* module-info compilation is cached

* runtime-instrument-runtime-server is module

* runtime-language-epb is module

* Remove runtime-fat-jar

* engine-runner is not a fat jar

* JPMSPlugin defines exportedModuleBin task

* Redefine componentModulesPaths task

* interpreter-dsl is module

* Redefine componentModulesPaths task

* fmt sbt

* scala-libs-wrapper is a modular fat jar

* Add some module deps to org.enso.runtime

* engine-runner is not a fat jar

* Rename package in logging-config

* Rename package in logging-service

* Rename package in logging-service-logback

* Fix dependencies of exportedModuleBin task

* Mixed projects have own compileJava task

this task does not compile only module-info.java but all the java sources. So that we can see errors more easily.

When only module-info.java is compiled, the only errors that we can see are that we did not include some modules on module-path.

* Fix definition of exportedModule task.

* Remove usages of non-existing buildInfo and replace it with BuildVersion

* Fix some dependencies of org.enso.runtime module

* module-info compilation is handled directly by FrgaalCompiler

* module-info compilation is forced for projects that has only Scala sources with single module-info.java

* Fix compilation of org.enso.runtime

* manual module-info compilation is not a warning

* Rename packages in logging-utils-akka

* Create org.enso.language.server.deps.wrapper module

* language-server is module

* Creat akka-wrapper modular fat jar

* fmt

* Define common settings for modularFatJarWrapper

* Fix compilation of json-rpc-server

* Use akka and zio wrappers

* language-server depends on org.eclipse.jgit

* Fix some dependencies - update library manifests works now!

* update library manifests invokes runner directly

* buildEngineDistribution does not copy runner.jar

* Remove EngineRunnerBootLoader

* Fix compilation of std libs

* --patch-module and --add-exports are also passed to javac

* Rename package in runtime-integration-tests.

The package name org.enso.compiler clashes with the package from the module

* Remove usage of buildInfo

* FrgaalJavaCompiler can deal with non-existing module-info.java when shouldCompileModuleInfo is true.

It just generates a warning in such case as it suggests that there is something wrong with the project configuration.

* Revert AliasAnalysisTest.scala

* Fix dependencies and java cmdline options for runtime-integration-tests

* Rename test package

* runtime-integration-test depends on logging-service-logback/Test/compile

* Rename package in logging-service-logback/Test

* Fix FrgaalJavaCompiler creation for projects

* Sanitize Test/javaOptions arguments

* Sanitize Test/javaOptions arguments

* All the JPMSPlugin settings are scoped

* Remove unused sbt tasks

* modularFatJarWrapperSettings do not override javacOptions

* Resolve issue "Cannot find TestLoggerProvider" in runtime-integration-tests

* org.enso.runtime module is open

* Test that test classes are unconditionally opened and exported

* polyglot-api-macros is a module

* JPMSPlugin handles --add-opens cmdline option

* RuntimeServerTest ensures instruments are initialized

* Add some exports to org.enso.runtime.compiler

* Add instruments on module-path to runtime-integration-tests

* Replace TestLogProviderOnClassPath with TestLogProviderOnModulePath

* Replace buildInfo with BuildVersion

* Add jpms-wrapper-scalatest

* ReportLogsOnFailure is in non-modular testkit project

* Add necessary dependencies to testkit project

* Revert "Add jpms-wrapper-scalatest"

This reverts commit 732b3427a2.

* modularize filewatcher and wrap its dependencies

* Initial fix for language-server/test

* frgaal compiler setting are scoped for Compile and Test

* Rename package in language-server/test

* Exclude com.sun.jna from wrapper jars

* Rename package in library-manager-test

* testkit is an automatic module

* process-utils is module

* akka-wrapper contains akka-http

* Some fixes for library-manager-test

* Fix dependencies for akka-wrapper

* scala-libs-wrapper exports shapeless

* lang server deps wrapper exports pureconfig

* json-rpc-server requires org.slf4j

* Add some dependencies

* lang server deps wrapper exports pureconfig.generic

* language server test requires bouncycastle provider

* language server depends on cli

* directory-watcher wrapper requires org.slf4j

* WatcherAdapter logs unsuccessful initialization errors

* Fix error reporting in WatcherAdapter

* Fix rest of the language-server tests

* language-server-deps-wrapper depends on scala-libs-wrapper

* Fix rest of the language-server tests

* Missing module-info.class in an internal project is a warning, not an error

* Rename jpms-methvin-directory-watcher-wrapper to a simpler name

* compileOrder has to be specified before libraryDependencies

* exclude module-info.java from polyglot-api-macros

* Remove temporary logging in customFrgaalCompilerSettings

* Fix compilation of logging-service-logback

* Fix compilation of runtime-benchmarks

* Fix runtime-benchmarks/run

* HostClassLoader delegates to org.graalvm.polyglot class loader if org.enso.runtime is not on boot layer

* org.enso.runtime.lnaguage.epb module must be opened to allow it to be used by annnotation processor

* fmt

* Fix afetr merge

* Add module deps after merge

* Print stack trace of the uncaught exception from the annotation processor

* Remove akka-actor-typed from akka-wrapper

* runtime-instrument-common depends on slf4j

* Fix module-path for runtime-instrument-repl-debugger

* runtime-benchmarks depends on runtime-language-arrow

* --module-path is passed directly to frgaal

* Fix some module-related cmd line options for std-benchmarks

* Revert "--module-path is passed directly to frgaal"

This reverts commit da63f66a0e.

* Avoid closing of System.err when closing Context

* Avoid processing altogether when requested annotations are empty

* Pass shouldNotLimitModules opt to frgaal

* Pass module-path and add-modules options with -J prefix to frgaal

* BenchProcessor annotation processor creates its own truffle module layer

* bench-processor and benchmarks-common are modules

* fmt

* Fix after mege

* Enable JMH annotation processor

* Fix compileOrder in some projects

* Insert TruffleBoundary to QualifiedName.

This is a revert

* Fix building of engine-runner native image

* Add more deps to the native image

* Force module-info compilation in instruments.

This fixes some weird sbt bug

* Don't run engine-runner/assembly from Rust build script

* Update docs of JPMSPlugin

* fmt

* runtime-benchmarks depends on benchmarks-common module

* Fix benchmark report writing

* std-benchmarks annot processing does not take settings from runtime-benchmarks

* Suppress interpreter only warning in annotation processor

* Runtime version manager does not expect runtime.jar fat jar

* fmt

* Fix module entry point

* Move some polyglot tests to runtime-integration-tests.

Also make their output silent

* pkg has no dependency on org.graalvm.truffle

* Fix compiler dependencies test

* Rename all runtime.jar in fake releases

* Add language-server with dependencies to component dir

* No module-info.class in target dir is warning not error

* language-server does not depend on netbeans lookup uitl

* Declare LanguageServerApi service provider in module-info

* connected-lock-manager-server is JPMS module

* task-progress-notifications is module

* Add fansi-wrapper module

* Fix compilation of connected-lock-manager-server

* Define correct Test/internalModuleDependencies for project-manager

* fmt

* Fix LauncherRunnerSpec - no runtime.jar

* Add fansi-wrapper to runtime-integration-tests and runtime-benchmarks

* Fix engine-runner native image build

* Use newer JNA version - fixes running of hyperd

* DRY

* scala-compiler DRY

* fmt

* More build.sbt refactoring

* Include runtime-instrument-id-execution in engine-runner native image

* TruffleBoundary for QualifiedName.toString

* Finding a needle in a haystack

🤦

* More scala-library DRY

* more mixed-java/scala goodies

* Fix compilation of syntax-rust-definition

* Test that engine-runner does not depend on language-server

* Append rather than assign `moduleDependencies`

`++=` is less error prone than `:=`. Also discovered some unnecessary
dependencies.

* Replace : with File.pathSeparator

* [WIP] Make logging in ProjectService more verbose

* language-server/test didn't start because of missing lookup and fansi modules

* Formatting

* org.enso.cli.task.notifications needs Akka and Circe to link

* project-manager/test depends on buildEngineDistribution

* [WIP] Even more verbose logging for creating projects

* [WIP] Even more verbose logging for creating projects

* Revert "[WIP] Even more verbose logging for creating projects"

This reverts commit a7067c8472.

* Revert "[WIP] Even more verbose logging for creating projects"

This reverts commit fc6f53d4f1.

* Revert "[WIP] Make logging in ProjectService more verbose"

This reverts commit 427428e142.

* All the project with JPMSPlugin has stripped artifact names

* Revert all placeholder fake release components to runtime.jar without version

* Eliminate a cross version hack

We shouldn't be specifying Scala dependencies with a Scala cross version
in the suffix.

* Address SBT lint warnings

* Revert "Eliminate a cross version hack"

This reverts commit 8861dab288.

* logging-service-logback  is mixedJavaScalaProject

* fmt

* Stripped artifact name contains classifier.

This fixes tests as those were named like `artifact-tests.jar`.

* Don't use LocalProject unless really needed

* Add more logging when BenchProcessor fails

* logging-service-logback is not mixed project

* Work with java.io.File.getPath to avoid mixing slash and backslash on Windows

* Reapply "Eliminate a cross version hack"

This reverts commit edaa436ee8.

* Pass scalaBinaryVersion correctly

* Remove scala-compiler from the distribution

* Fix IllegalAccessErrors from serde

* typos

* License review

* fmt

* Move testLogProviderOnModulePath to TestJPMSConfiguration

* logging-service-logback is not a mixed project

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
2024-09-25 21:33:13 +02:00
Hubert Plociniczak
1e6e69b539
Akka-related startup improvements (#10905)
* Akka-related startup improvements

Realized that Akka Typed and Spray were not used **at all** yet there
were in the list of dependencies.
Additionally, the former would also show up in the list of Akka's
library-extensions, initialized on startup.

* Hardcode list of library extensions

Hardcoding a list of library extensions is not recommended but I don't
see any other way of disabling expensive Serialization extension, which
we don't seem to use anyway.

* fmt

* licensing update
2024-08-29 20:43:37 +02:00
Dmitry Bushev
d08cb704b0
Record interaction between GUI and LS (#10107)
close #8328

Changelog:
- add: message callbacks to JsonRpc and Binary servers
- update: use events log to collect the RPC messages
2024-06-03 06:50:59 +00:00
Dmitry Bushev
5995a00958
Run ydoc-server with GraalVM (#9528)
part of #7954

# Important Notes
The workflow is:
- `$ npm install` -- just in case
- `$ npm --workspace=enso-gui2 run build-ydoc-server-polyglot` -- build the `ydocServer.js` bundle
- `$ sbt ydoc-server/assembly` -- build the ydoc server jar
- `env POLYGLOT_YDOC_SERVER=true npm --workspace=enso-gui2 run dev` -- run the dev server with the polyglot ydoc server. Providing `POLYGLOT_YDOC_SERVER_DEBUG=true` env variable enables the chrome debugger
2024-05-02 06:28:57 +00:00
Hubert Plociniczak
34e1bacdf1
Further optimizations to test resources (#8936)
TestRuntime should be deprecated as it creates a number of threads and doesn't allow to easily modify ZIO's runtime.
But the biggest drop stems from fixing leaking `FileSystemService` that weren't being closed for every `TextOperationsTest` test.
The change is a follow up on #8892 but this time focused on ZIO usage.

Hopefully fixes #8806 for good.

# Important Notes
Running `language-server/test`.
Before:
![Screenshot from 2024-02-02 09-48-32](https://github.com/enso-org/enso/assets/292128/fb414c74-7d7a-4e7b-8b0c-d25dc3721bbf)

After:
![Screenshot from 2024-02-02 09-46-02](https://github.com/enso-org/enso/assets/292128/db9429df-d861-4f48-818f-888d5bbbb089)
2024-02-02 22:49:48 +00:00
Hubert Plociniczak
68ce4ae1b7
Reduced resources for various Akka dispatchers (#8892)
The defaults picked up by Akka tend to make us of all resources which is unnecessary and overwhelming for tests.

Improves #8806, potentially.

Before
![Screenshot from 2024-01-28 22-34-42](https://github.com/enso-org/enso/assets/292128/f80eb66a-2f37-44d5-bcdb-f00a78fe72fd)
After
![Screenshot from 2024-01-31 00-12-10](https://github.com/enso-org/enso/assets/292128/c5223912-5f6e-413c-a0a4-050afa3ed463)

when running the problematic `LibrariesTest`.

Full `language-server` test suite.
Before
![Screenshot from 2024-01-31 00-20-50](https://github.com/enso-org/enso/assets/292128/f1c94a66-6905-4f57-8a7d-7df049714353)
After
![Screenshot from 2024-01-31 00-18-40](https://github.com/enso-org/enso/assets/292128/3a11125e-d593-43df-8d35-1a8915812b2b)

# Important Notes
Note that Executors assigned to Zio and initializers should also be improved. Unfortunately due to various blocking threadpools  it is easy to get timeouts when running the whole suite.
2024-01-31 10:25:17 +00:00
Hubert Plociniczak
cfba3c6887
Add support for https and wss (#7937)
* Add support for https and wss

Preliminary support for https and wss. During language server startup we
will read the application config and search for the `https` config with
necessary env vars set.

The configuration supports two modes of creating ssl-context - via
PKCS12 format and certificat+private key.

Fixes #7839.

* Added tests, improved documentation

Generic improvements along with actual tests.

* lint

* more docs + wss support

* changelog

* Apply suggestions from code review

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>

* PR comment

* typo

* lint

* make windows line endings happy

---------

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
2023-10-12 00:03:34 +02:00
Hubert Plociniczak
3e615f385b
Improve shutdown logic of language server (#7801)
* Improve shutdown logic of language server

This PR addresses problems mentioned in #7470 and #7729:
- shutting a language server explicitly will not lead to a soft shutdown
- `project/status` endpoint returns the state of the language server

`LanguageServerController` now also signed up for `ClientConnect`
messages. For it to be unambiguous, we need to carry around the port
number of the language server as a way of identifying the right one.

One can now use `project/status` to additionally determine the state of
the language server.

Also relies on a proper fix for #7765.

* changelog

* PR comments
2023-09-22 12:05:44 +02:00
Hubert Plociniczak
8a60bc6dcd
Replace a custom logger with off the shelf implementation (#7559)
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.

Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.

Most important differences have been outlined in `docs/infrastructure/logging.md`.

# Important Notes
Addresses:
- #7253
- #6739
2023-09-04 09:40:16 +00:00
Dmitry Bushev
eefe74ed93
When opening the project for a second time suggestion database is not sent (#7699)
close #7413

Changelog:
- update: the language server listens for the client disconnection event and invalidates the suggestions index

# Important Notes
The component browser contains suggestion entries after the refresh


https://github.com/enso-org/enso/assets/357683/bcebb8bf-e09f-4fb0-86cf-95ced58413f3
2023-08-31 14:06:58 +00:00
Hubert Plociniczak
dd06c59380
Initialize Language Server's JSON RPC protocol asynchronously (#7232)
As discovered in #7224, Json RPC protocol was added to the asynchronous resource initialization stage, as part of #6306, but was not in fact initialized at that point.
Instead it was initialized when the server was started to be able to serve correctly the initialization messages. A classic Catch-22. It was really hard to discover this just by looking at the code, but the profiling clearly showed where the time was spent.

This change splits Language Server's protocol into two:
- the first one accepts `heartbeat/init` and `session/initProtocolConnection`
- the second one enriches it with the full set of supported messages

This shifts the initialization from blocking for 0.5 sec to only ~30ms, and performing the second stage asynchronously.

Closes #7224.

# Important Notes
Before the change (blocking server startup):
![Screenshot from 2023-07-05 18-53-24](https://github.com/enso-org/enso/assets/292128/bcfa9043-d00a-4b36-a44c-782a388a16b9)
![Screenshot from 2023-07-05 18-53-10](https://github.com/enso-org/enso/assets/292128/54927787-4c95-46db-bd68-f3a3b82367d5)

After the change (1st stage):
![Screenshot from 2023-07-06 14-02-34](https://github.com/enso-org/enso/assets/292128/d7a7bc34-39dc-46f1-9e64-6d350697c30b)

After the change (2nd, asynchronous initialization, stage):
![Screenshot from 2023-07-06 14-21-17](https://github.com/enso-org/enso/assets/292128/def8c0a1-f211-4fc0-9df0-7c1634312166)
2023-07-07 11:49:23 +00:00
Dmitry Bushev
48f0c6f5e8
Scala 2.13.11 and libraries update (#7010)
Update Scala and libraries.
2023-06-14 13:15:57 +00:00
Dmitry Bushev
74c0718604
Project folder not renamed after project name change (#6369)
close #6254

Changelog:
- fix: race when the actor system may be stopped before the shutdown hooks are executed
- fix: project management spec
- fix: recover from `readLine` failure during the shutdown
2023-04-20 16:57:37 +00:00
Dmitry Bushev
8125d89676
Delay initialization of JSON-RPC protocol (#5997)
close #5881

Changelog
- add: ProtocolFactory object that initializes and returns the protocol object
- update: Add protocol initialization to the initialization component
2023-03-20 11:04:16 +00:00
Dmitry Bushev
52a8c72303
Message handler supervisor (#3881)
Changelog:
- Fix a potential `null` value in diagnostic messages
- Add a supervising strategy to the message handler
2022-11-17 19:02:04 +00:00
Dmitry Bushev
f9d2964e83
Update profiling CLI arguments (#3461) 2022-05-24 16:01:26 +03:00
Dmitry Bushev
d3846578cc
Fix Error Payload Serialization (#3315) 2022-03-04 11:57:49 +03:00
Dmitry Bushev
ca0a6f0bf6
Update Scala to 2.13.7 (#3214) 2021-12-31 17:50:32 +03:00
Dmitry Bushev
d9e1a47460
Report Visualization Errors (#1671)
Add `executionContext/visualisationEvaluationFailed`
notification
2021-04-21 16:32:22 +03:00
Łukasz Olczak
4c1e7a4319
Readiness and liveness probes for Language Server (#1359) 2020-12-16 17:05:37 +01:00
Dmitry Bushev
a6b2415188
Add JSON-RPC Timeout Error (#1332)
Add a separate timeout error to distinguish between
the request timeouts and other service errors.
2020-12-09 15:53:00 +03:00
Radosław Waśko
5701db6d89
Update Project Manager Protocol (#1240) 2020-11-02 09:58:33 +01:00
Ara Adkins
746521f8b2
Bump SBT and Scalafmt (#1203)
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
2020-10-22 16:12:28 +02:00
Ara Adkins
f5ffbe8fa7
Add a markdown style guide (#1022) 2020-07-21 13:59:40 +01:00
Ara Adkins
919ffbdfac
Refactor the crate structure (#967) 2020-07-03 14:42:45 +01:00