close#6306
Changelog:
- add: `AsyncResourceInitialization` component
- update: initialize language server resources asynchronously
# Important Notes
Speeds up `session/InitProtocolConnection` request by ~300ms (~10%) 3051ms (before) vs 2740ms (after) on my machine.
close#6232
Changelog:
- remove: `SqlVersionsRepo`
- update: `SuggestionsDatabaseModuleUpdateNotification` message removing the version
- update: cleanup versions repo usages in the language server
close#6080
Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table
# Important Notes
Speeds up libraries loading by ~1 second.
close#6139close#6137
When the project is renamed, the engine cleans up affected modules and initiates modules re-indexing to fill the suggestions database with new records. This way it reduces the amount of information stored in the suggestions database and helps implement #6080 optimization.
Changelog:
- remove: rename features from the suggestions database
- update: rename command to initiate modules cleanup and project re-execution
- fix: #6137
It is sometimes impossible to figure out the real reason for invalid text edit request. Added a bit of context to failures to narrow down the cause of the failure.
# Important Notes
Should help with diagnosing issues like #6099.
close#5874
Changelog:
- add: `isStatic` parameter to `search/completion` request to search by the `static` suggestion attribute
- update: search non-static suggestions when opening component browser
# Important Notes
Component browser doesn't show `Table.new` and `Table.from_rows` suggestions when a `Table` node is selected.
close#5881
Changelog
- add: ProtocolFactory object that initializes and returns the protocol object
- update: Add protocol initialization to the initialization component
close#5911
In interactive mode, perform writing IR caches in the background jobs queue. Background jobs execution is delayed until the first execution is complete.
The `logAvailableComponentsForDebugging` will check and install all necessary components of GraalVM for every mentioned version. While not harmful, it adds up to startup time.
Additionally added an option in language server startup to skip installation of GraalVM components. The latter is already performed by project-manager when opening the project and it is unnecessary to do it twice. Due to LS' architecture this configuration has to be passed around via multiple configs.
Finally, skipped the attempt to install Python component on Windows - this is not supported by GraalVM atm.
Closes#5749.
# Important Notes
The impact of this problem could be really felt the more versions of Enso and GraalVM one had since it would go through all of them.
Implement new Enso documentation parser; remove old Scala Enso parser.
Performance: Total time parsing documentation is now ~2ms.
# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
close#5892
Changelog:
add: feature to delay background jobs execution
add: start background jobs when program finishes
add: start background jobs on `search/completion` request
close#5070
Changelog:
- Include the original exception to log expressions
- Enable logging of Akka Actors' lifecycle events on debug logging level
- Decrease the severity of interruption log messages because interruptions are part of the workflow. The computation can be interrupted at any time, and still be recomputed after. Warnings are just misleading in this case.
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.
When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.
The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.
Part of https://github.com/enso-org/enso/issues/5568 work.
Implements the #5643 idea. As soon as `MainModule` creates `Context` for GraalVM execution, it schedules a background task to initialize JavaScript. The initialization finishes sooner than Enso compiler is ready to work, saving time when it is actually needed.
# Important Notes
Only modifies boot sequence of `MainModule` (used in the IDE) and `VerifyJavaScriptIsAvailableTest` (to verify the _"context passing logic"_ works OK between threads). Regular CLI execution remains unchanged for now assuming batch execution may not need JavaScript in all the cases and if it does the initialization speed isn't that critical.
When the `project.yaml` specifies an unknown edition, the project is started with the fallback edition (current engine version). It works this way:
- IDE sends project/open request to project manager (specifying only which project to open)
- project manager resolves the project edition (based on `project.yaml`) or selects the fallback if the requested edition cannot be loaded
- project manager starts the language server specifying the edition with the `--server-edition` command line parameter
- language server specifies the correct edition when creates the runtime context
Closes#5036
Move the logic that looks up method pointers from the language server to IDE. This way we can keep the suggestion updates and expression updates async, otherwise it will hurt the initial startup time of LS.
Fixes the issue when some expression updates does not contain the method pointer.
Follow up on
16ba57d465 which marked the tests as flaky.
This test goes back to the original implementation where JGit did all its book-keeping synchronously. When we switched to asynchronous operation, cleaning up the test directory would sometimes not succeed because there would be a race-condition with the ongoing work of JGit.
# Important Notes
In the current setup tagging a test as flaky is only valid for Windows. The other platforms ignore the flag. So we were still getting random failures.
Start `project-manager` with following options to provide first 20s of the startup sequence:
```
$ project-manager --profiling-events-log-path=start.log --profiling-path=start.npss --profiling-time=20
```
once the `start.log` and `start.npss` files are generated (next to each other), open them in GraalVM's VisualVM:
```
$ graalvm/bin/jvisualvm --openfile start.npss
```
analyze.
Implementation of https://www.pivotaltracker.com/story/show/184012743https://user-images.githubusercontent.com/919491/214082311-cf49e43c-1d1f-4654-903c-a4224cd954d8.mp4
This is also a step towards more general widget support. The widget metadata is queried using `Meta.get_annotation` method through a dedicated visualization. For now only `Single_Choice` case is handled, and always all suggestions are is returned.
# Important Notes
There are limitations as to which node segments receive a widget. Only chain method calls are supported now (`thing.method` syntax), and only outside of lambda scope. Widgets in lambdas will require support for visualisations of lambda subexpressions, which is currently missing in the engine. The IDE technically tries to place the widgets there, but the data never arrives. It should work once the engine support is added.
This PR includes a mock for `Meta.get_annotation` call that only supports `Table.at` method. Real implementation is a separate task that is already in progress.
JGit ops generally run fast (as in a few milliseconds) except for the first commit. The initialization + first commit was taking at least 3.5 seconds constistenly, but only in the first test case. Now, this led to frequent timeouts down the chain when the request was expected to finish fast.
The bookkeeping involved some timestamping and other expensive calls in order to calculate clock drift. The default appears to be to run it in a blocking mode, hence adding at least 3 seconds to the first command call.
Setting the job to run in the background makes the cost of the repo initialization acceptable (~300 milliseconds on a cold JVM). The other commands are unaffected and take < 10 milliseconds.
# Important Notes
Added a test to ensure that we don't introduce the regression. Marked it as potentially flaky because it uses timestamps and it is therefore prone to random system hiccups.
There is no need for JGit to try to save file attributes to its config in `<home>/jgit/config`. This change ensures that JGit operates on a stub file, without polluting users' configs, similarly to user's git config.
Additionally, noticed that all FS operations during `init` are rather slow and saw at least on one occassion when the handler timed out because of that (https://www.pivotaltracker.com/story/show/184359934). This change provides an init-specific timeout for `vcs/init`.
# Important Notes
This change should reduce chances of failures during VCS initialization.
When a large long would be passed to a host call expecting a double, it would crash with a
```
Cannot convert '<some long>'(language: Java, type: java.lang.Long) to Java type 'double': Invalid or lossy primitive coercion
```
That is unlikely to be expected by users. It also came up in the Statistics examples during Sum. One could workaround it by forcing the conversion manually with `.to_decimal` but it is not a permanent solution.
Instead this change adds a custom type mapping from Long to Double that will do it behind the scenes with no user interaction. The mapping kicks in only for really large longs.
# Important Notes
Note that the _safe_ range is hardcoded in Truffle and it is not accessible in enso packages. Therefore a simple c&p for that max safe long value was necessary.
In order to investigate `engine/language-server` project, I need to be able to open its sources in IGV and NetBeans.
# Important Notes
By adding same Java source (this time `package-info.java`) and compiling with our Frgaal compiler the necessary `.enso-sources*` files are generated for `engine/language-server` and then the `enso4igv` plugin can open them and properly understand their compile settings.
In addition to that this PR enhances the _"logical view"_ presentation of the project by including all source roots found under `src/*/*`.
In the event when the action of the underlying actor crashes really bad, the result of the future will be `Failure`. All such results will be then wrapped in `Status.Failure` via `pipeTo` (unlike `Success` which just forwards the response).
In some cases we don't handle `Status.Failure` messages, meaning we are rather left in the dark to the reason of the failure with a non-informative message `Received unknown message: class akka.actor.Status$Failure`.
I'm keeping this change small on purpose to keep this change self-contained. I think there are more cases but it needs careful investigation into how messages are being sent.
# Important Notes
This PR does not attempt to fix the underlying problem of the ticket, yet. We should however have a better overview where/why things go wrong.
LS needs to notify runtime that modules' sources need to be reloaded from FS, once its own buffer has been reloaded as well.
# Important Notes
Discovered during integration of https://github.com/enso-org/enso/pull/4050.
The test illustrates the problem if we don't reload module's sources - the sources essentially become stale even though they have changed.
Unsetting literal sources during the `text/fileClose` operation would also unlink sources from the actual file. That would be never set back during the reopening of the file, resulting in an NPE during application of changes at a later point.
# Important Notes
Revealed during https://github.com/enso-org/enso/pull/4050 which was the first(?) to do close/open of the file.
The exception showing up in GUI and preventing further actions is now gone.
Potential workaround to line endings problems after `vcs/restore` operation is executed.
# Important Notes
Not really able to reproduce the problem myself so this PR has a lot of _leap of faith_ in it.
VCS restore operation was correctly restoring the state of projects to the requested commit. Unfortunately, after the operation file system was becoming out-of-sync with language server's buffers (and IDE's content versions).
A few important changes are introduced here that complicate the interaction between components:
1) `vcs restore` returns an actual diff between the current state and the
requested commit
2) the response is forwarded to buffer registry first rather than to the client
3) the diff is used to identify appropriate collaborative editors and
notify them about the need to reload buffers from file system
4) all clients of affected open buffers are notified of the change via
`text/didChange` notification. If a file was removed and there were open buffers for it, clients will be notified via `file/event` and editor will be stopped
5) only then the client is notified about a successful restore operation
This PR addresses one of the two problems reported in https://www.pivotaltracker.com/story/show/184097084.
# Important Notes
We need to make sure that IDE correctly responds to `text/didChange` notifications.
Implements `getMetaObject` and related messages from Truffle interop for Enso values and types. Turns `Meta.is_a` into builtin and re-uses the same functionality.
# Important Notes
Adds `ValueGenerator` testing infrastructure to provide unified access to special Enso values and builtin types that can be reused by other tests, not just `MetaIsATest` and `MetaObjectTest`.
Apparently
```
git --git-dir .enso/.vcs log refs/heads/master
fatal: ambiguous argument 'refs/heads/master': unknown revision or path not in the working tree.
```
but
```
git --git-dir .enso/.vcs log HEAD
fatal: ambiguous argument 'refs/heads/master': unknown revision or path not in the working tree.
```
works just fine on Windows.
Added some safeguards to avoid propagating weird errors because of retrieving element from an
empty Option.
The presence of `.git` in Enso's data directory (`.enso`), which should be under VCS as well, turned out to be really problematic. Git (and JGit) treat `.enso` as a submodule which prevents as from manual inspection of the repo. Additionally, every commit changed the submodule version, which in turn was reporting invalid/changed status always.
This change renames `.enso/.git` to `.enso/.vcs` which eliminates a lot of problems. Additionally, rather than doing `git add .` we selectively add files to the index, thus preventing most of the issues reported in the ticket.
This change also means that JGit's status needs to filter `.enso/.vcs` as those will be marked as untracked always.
# Important Notes
To manually test the status of the project's repo do
```
git --git-dir .enso/.vcs status
```
- Moved `Any`, `Error` and `Panic` to `Standard.Base`.
- Separated `Json` and `Range` extensions into own modules.
- Tidied `Case`, `Case_Sensitivity`, `Encoding`, `Matching`, `Regex_Matcher`, `Span`, `Text_Matcher`, `Text_Ordering` and `Text_Sub_Range` in `Standard.Base.Data.Text`.
- Tidied `Standard.Base.Data.Text.Extensions` and stopped it re-exporting anything.
- Tidied `Regex_Mode`. Renamed `Option` to `Regex_Option` and added type to export.
- Tidied up `Regex` space.
- Tidied up `Meta` space.
- Remove `Matching` from export.
- Moved `Standard.Base.Data.Boolean` to `Standard.Base.Boolean`.
# Important Notes
- Moved `to_json` and `to_default_visualization_data` from base types to extension methods.
Rather than hard-coding `.git` in the root of the project, VCS should save data into Enso's data directory (i.e. `.enso`).
This change reshuffles initialization and configuration to store Git VCS metadata by default at `.enso/.git`.
This is follow up to https://github.com/enso-org/enso/pull/3851
# Important Notes
Apparently a custom Git directory in JGit means that it always creates a `.git` **file** with `gitdir` pointing to the custom location.
This is not necessary in our case since all our commands provide that explicitly.
That is why `init` operation removes `.git` file, which may seem a bit counter-intuitive.
Upgrading to GraalVM 22.3.0.
# Important Notes
- Removed all deprecated `FrameSlot`, and replaced them with frame indexes - integers.
- Add more information to `AliasAnalysis` so that it also gathers these indexes.
- Add quick build mode option to `native-image` as default for non-release builds
- `graaljs` and `native-image` should now be downloaded via `gu` automatically, as dependencies.
- Remove `engine-runner-native` project - native image is now build straight from `engine-runner`.
- We used to have `engine-runner-native` without `sqldf` in classpath as a workaround for an internal native image bug.
- Fixed chrome inspector integration, such that it shows values of local variables both for current stack frame and caller stack frames.
- There are still many issues with the debugging in general, for example, when there is a polyglot value among local variables, a `NullPointerException` is thrown and no values are displayed.
- Removed some deprecated `native-image` options
- Remove some deprecated Truffle API method calls.
