* Rename packages in logging-utils-akka
* Migrate buildInfo to Java
* Rename packages in logging
* Rename package in scala-yaml
* No usage of CompilerDirectives inside pkg
* log errors of initialization of directory watcher
* HashCodeNode does not use com.google.common.base.Objects
* Rename rest of the packages
* fmt
* Fix dependencies on version-output
* Add necessary dependencies to testkit
* Rename instruments in runtime-fat-jar module-info
* Fix compilation errors because of BuildVersion
* Fix logger renames
* Use java.util.List directly
* Fixes after merge
* Improve error message in NativeLauncherSpec
* Fix logger renames
* Fix json version formatting
* Revert "No usage of CompilerDirectives inside pkg"
This reverts commit cc7e078416.
* fmt
Scheduling of jobs is asynchronous and nothing prevents it from scheduling execution before compilation leading to stale nodes in GUI. The scenario is easily reproducible by adding a `Thread.sleep` before `EnsureCompiledJob` requests compilation locks.
This change ensures that execution is only scheduled after compilation is finished, when an edit request is being processed.
# Important Notes
This fix was prompted by me getting random stale nodes after edits:
(it would never recover unless another edit was made)
- Closes#10716 by un-marking the `Snowflake_Connection` as `private`.
- Does the same to `SQLServer_Connection`.
- Makes sure that `Snowflake_Connection` has correct visualization and enables links for it.
- Adds a fallback in `to_default_visualization_data` and in Table's `prepare_visualization` so that if `to_js_object` of a given type is malformed, we still get some kind of visualization + debug info about the underlying error.
Majority of warnings handling is now done via newly introduced nodes. Moreover, the underlying representation of warnings storage in `WithWarnings` was changed from `Warning[]` to `EnsoHashMap`.
# Important Notes
- Remove `ArrayRope`.
The `System.exit 42` component is treated the same way as any other Panic error - it does not interfere with other component evaluation:
After removing the `System.exit 42` component, the workflow works as expected. I have also tried opening the project with the component and then removing it.
Reducing the number of log events that we spam regular users. Not the last PR in that area, but already a progress.
Also replaced `fileWalk` with a stream approach hoping that maybe it is a bit more stable on Windows.
* Fix context lock removal
Removal of context lock assumed that one still holds a lock on it. This
is no longer the case when using a `withContextLock` block that
correctly manages the resource. This change fixes the
`IllegalMonitorStateException`.
Closes#10354.
* address review
* Add missing file
* Hiding ContextLock internals from other than ReentrantLocking classes
---------
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
* Mitiigate LS DDOS scenario on startup
For a medium-size project with a lot of visualizations, Language Server
will be flooded with visualization requests on startup. Due to an
existing limit for the job execution engine, some of those requests
might have been silently dropped.
This change lifts that limit until a better fix can be invented.
Additionally, a slow startup might lead to a timeout when serving open
file request. This change adds some retries as a fallback
mechanism/progress monitoring.
* add runtime-fat-jar to a list of aggregates
close#10182
Changelog:
- add: IdMap parameter to the `text/applyEdit` request
- add: IdMap to the runtime module
- update: set IdMap during the interactive compilation
- update: set the IR identifiers in the `TreeToIR` parsing step
Reducing the number of dependencies. Explicit `cats` are almost gone (present in `cli`). `enumeration` is completely gone. `cats` is also still included implicitly via `io.circe` but that's a different kind of beast.
Also, really removed `jackson` from dependencies by fixing the dependency on `http-test-helper`.
# Important Notes
In a number of places importing all cats implicits could be simply replaced with a single or two method calls. Not to mention that this will reduce compilation times due to reduced implicit search space.
One example of how the changes affect performance (not only startup):
Before:
Now:
(frequently executed)
Also appears to be gone.
This PR is by no means finished. The purge will continue in follow up PRs.
Refactored mutable parts of `ModuleScope` into builder to make it easier to reduce unnecessary locks.
# Important Notes
Elements of ModuleScope (types, imports etc) are used while _building_ of it may still be in progress. In order to make static typing happy, every `ModuleScope.Builder` can be exposed as (unmodifiable) `ModuleScope`.
This change is a re-do of old PR https://github.com/enso-org/enso/pull/8270 that has been abandoned a while ago.
The change attempts to address to case of random `InterruptedExceptions` that can result when a task is interrupted while waiting on a lock. In comparison to the original PR, this change introduces `withXYZLock` methods with a `Callable` parameter. As a result we ensure correct acquisition/release semantics, logging and handling of interruptions.
# Important Notes
https://github.com/enso-org/enso/pull/8270 was too much hassle to rebase against latest develop.
Ensure strict errors are off to prevent unrecoverable errors in IDE.
Also partially reverted a change that prevented application of changes on errors (errors are still logged, as intended).
Closes#10014.
Setting execution environment to the existing one should have no effect.
Should (positively) affect startup in #9789.
# Important Notes
Cancelling jobs and triggering a fresh execute job is expensive and unnecessary, especially on startup, when the result should be the same as before.
Using a fully qualified name for type identifier does not play well when dealing with different classloaders (classes are not recognized as equal/subtypes).
Closes#9876. Likely also affecting #9306.
# Important Notes
There appears to be a number of cases loosely related issues on that subject. Gave up on adding a test cases to simulate the problem but I could no longer reproduce it with LS/runtime on a live project.
```
[org.enso.languageserver.runtime.RuntimeConnector$Endpoint] Failed to deserialize runtime API envelope
com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve type id 'org.enso.polyglot.data.Tree$Node' as a subtype of `org.enso.polyglot.data.Tree$Node<org.enso.polyglot.runtime.Runtime$Api$SuggestionUpdate>`: Not a subtype
at [Source: (byte[])[6718 bytes]; byte offset: #394] (through reference chain: org.enso.polyglot.runtime.Runtime$Api$Response["payload"]->org.enso.polyglot.runtime.Runtime$Api$SuggestionsDatabaseModuleUpdateNotification["updates"]->org.enso.polyglot.data.Tree$Root["children"]->com.fasterxml.jackson.module.scala.deser.GenericFactoryDeserializerResolver$BuilderWrapper[0])
at org.enso.IsolatedClassLoader//com.fasterxml.jackson.databind.exc.InvalidTypeIdException.from(InvalidTypeIdException.java:43)
```
appears to be gone.
We don't seem to run `abortJobs` under a lock, and especially not under the write compilation lock, in other scenarios. This is causing some major slowdown when there is a long running execution or compilation, as currently experienced in the cloud.
This should reduce chances of a timeout.
Also added an option to override the global executor. Currently it would always default to the runtime number of available process which may be suboptimal.
# Important Notes
Pending testing on the impact it will have.
1. Experimenting with invalidating modules' indexes without requiring full write-context locks. That should significantly improve the execution.
2. Improving performance by making background job executor run in a larger threadpool than 1.
`ExecuteJob` can now be interrupted.
We now have a separate threadpool for visualization-related jobs.
# Important Notes
In a lock step situation, a job or command could have been interrupted while waiting for one of the locks. As locks ensured only that they were released once all of them have been acquired this could leave engine in a broken state.
Once `ExecuteJob` could be interrupted this became a blocker as it prevented project startup almost in every case.
The change also makes it careful to avoid constant `ExecuteJob` restarts.
Addresses #9278. There will be follow up work.
The `null` check creates a new Array but always assumed a non-empty one which may lead to
```
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.enso.runtime/org.enso.interpreter.service.ExecutionService$FunctionPointer.collectNotAppliedArguments(ExecutionService.java:778)
at org.enso.runtime/org.enso.interpreter.instrument.job.ProgramExecutionSupport$.sendExpressionUpdate(ProgramExecutionSupport.scala:430)
at org.enso.runtime/org.enso.interpreter.instrument.job.ProgramExecutionSupport$.$anonfun$executeProgram$3(ProgramExecutionSupport.scala:81)
at org.enso.runtime/org.enso.interpreter.service.ExecutionCallbacks.callOnComputedCallback(ExecutionCallbacks.java:146)
at
org.enso.runtime/org.enso.interpreter.service.ExecutionCallbacks.updateCachedResult(ExecutionCallbacks.java:117
...
```
Added a guard to prevent the exception. The flag will be useless anyway as we won't enter the for-loop in this case.
Appears to be introduced via #8743. Discovered while debugging #9389.
close#8965
Changelog:
- update: keep a single `ExecuteExpressionJob` in the queue
- update: make `ExecuteExpressionCommand` synchronous to preserve the order of commands
- refactor: separate data structures for `Visualization` and `OneshotExpression` to simplify the logic
related #8689
Fixes a race between the language server SQL updating logic and the engine `DeserializeLibrarySuggestionsJob`s when the library suggestions may start loading before the database is properly cleaned up after the reconnect.
related #8689
Fixes the NPE during the serialization of update messages.
```
java.lang.NullPointerException: Cannot invoke "java.util.UUID.toString()" because "a" is null
```
Uniqueness check of `UpsertVisualizationJob` only involved expressionId. Apparently now GUI sends mutliple visualizations for the same expressions and expects all of them to exist. Since previously we would cancel duplicate jobs, this was problematic.
This change makes sure that uniqueness also takes into account visualization id. Fixed a few logs that were not passing arguments properly.
Closes#8801
# Important Notes
I have not noticed any more problems with loading visualizations so the issue appears to be resolved with this change.
Added a unit test case that would previously fail due to cancellation of a job that upserts visualization.
This is a quick fix to a long standing problem of
`org.enso.interpreter.service.error.FailedToApplyEditsException` which would prevent backend from processing any more changes, rendering GUI (and backend) virtually useless.
Edits are submitted for (background) processing in the order they are handled. However the order of execution of such tasks is not guaranteed. Most of the time edits are processed in the same order as their requests but when they don't, files get quickly out of sync.
Related to #8770.
# Important Notes
I'm not a fan of this change because it essentially blocks all open/file requests until all edits are processed and we already have logic to deal with that appropriately. Moreover those tasks can and should be processed independently. Since we already had the single thread executor present to ensure correct synchronization of open/file/push commands, we are simply adding edit commands to the list.
Ideally we want to have a specialized executor that executes tasks within the same group sequentially but groups of tasks can be executed in parallel, thus ensuring sufficient throughput. The latter will take much longer and will require significant rewrite of the command execution.
Added tests that would previously fail due to non-deterministic execution.
Fixes#8710 by making sure suspended atom fields support works also for "normal" `Atom` instances without any special `Layout`. Moves all _atom related_ classes into single package and hides as much of classes as possible by making them _package private_.
Implements `Warnings.get_all wrap_errors=True` which wraps warnings attached to values inside vectors with `Map_Error`, which includes the position of the value within the vector. See [the documentation](https://github.com/enso-org/enso/blob/develop/docs/semantics/wrapped-errors.md) for more details.
`get_all wrap_errors=True` does not change the warnings that are attached to values -- it wraps them before returning them to the caller, but does not change the original warnings attached to the values.
Wrapped warnings only appear attached to the vector itself. The values inside the vector do not have their warnings wrapped.
Warning propagation is not changed at all; `Warnings.get_all` (with default `wrap_errors=False`) behaves as before. `get_all wrap_errors=True` is meant to be used primarily by the IDE, although it can be used anywhere this wrapping is desired.
close#8663
Changelog:
- update: use `MethodRootNode` for the atom constructor function to preserve the call info in runtime
- fix: return function schema for atom constructors
close#7184
The constructor value was not accessible because during the re-compilation a new instance of the type was registered in runtime. Then during the execution, an old cached instance of the type was used in method resolution.
Changelog:
- update: the registration of types in runtime
- update: invalidate cached nodes that became a resolution error after applying the edit
