The presence of `.git` in Enso's data directory (`.enso`), which should be under VCS as well, turned out to be really problematic. Git (and JGit) treat `.enso` as a submodule which prevents as from manual inspection of the repo. Additionally, every commit changed the submodule version, which in turn was reporting invalid/changed status always.
This change renames `.enso/.git` to `.enso/.vcs` which eliminates a lot of problems. Additionally, rather than doing `git add .` we selectively add files to the index, thus preventing most of the issues reported in the ticket.
This change also means that JGit's status needs to filter `.enso/.vcs` as those will be marked as untracked always.
# Important Notes
To manually test the status of the project's repo do
```
git --git-dir .enso/.vcs status
```
- Moved `Any`, `Error` and `Panic` to `Standard.Base`.
- Separated `Json` and `Range` extensions into own modules.
- Tidied `Case`, `Case_Sensitivity`, `Encoding`, `Matching`, `Regex_Matcher`, `Span`, `Text_Matcher`, `Text_Ordering` and `Text_Sub_Range` in `Standard.Base.Data.Text`.
- Tidied `Standard.Base.Data.Text.Extensions` and stopped it re-exporting anything.
- Tidied `Regex_Mode`. Renamed `Option` to `Regex_Option` and added type to export.
- Tidied up `Regex` space.
- Tidied up `Meta` space.
- Remove `Matching` from export.
- Moved `Standard.Base.Data.Boolean` to `Standard.Base.Boolean`.
# Important Notes
- Moved `to_json` and `to_default_visualization_data` from base types to extension methods.
Rather than hard-coding `.git` in the root of the project, VCS should save data into Enso's data directory (i.e. `.enso`).
This change reshuffles initialization and configuration to store Git VCS metadata by default at `.enso/.git`.
This is follow up to https://github.com/enso-org/enso/pull/3851
# Important Notes
Apparently a custom Git directory in JGit means that it always creates a `.git` **file** with `gitdir` pointing to the custom location.
This is not necessary in our case since all our commands provide that explicitly.
That is why `init` operation removes `.git` file, which may seem a bit counter-intuitive.
Upgrading to GraalVM 22.3.0.
# Important Notes
- Removed all deprecated `FrameSlot`, and replaced them with frame indexes - integers.
- Add more information to `AliasAnalysis` so that it also gathers these indexes.
- Add quick build mode option to `native-image` as default for non-release builds
- `graaljs` and `native-image` should now be downloaded via `gu` automatically, as dependencies.
- Remove `engine-runner-native` project - native image is now build straight from `engine-runner`.
- We used to have `engine-runner-native` without `sqldf` in classpath as a workaround for an internal native image bug.
- Fixed chrome inspector integration, such that it shows values of local variables both for current stack frame and caller stack frames.
- There are still many issues with the debugging in general, for example, when there is a polyglot value among local variables, a `NullPointerException` is thrown and no values are displayed.
- Removed some deprecated `native-image` options
- Remove some deprecated Truffle API method calls.
`LocalDate.now()` returns time in system time zone, and git uses UTC. Sometimes it can cause test to fail
```
INFO ide_ci::program::command: sbtℹ️ [info] - must create a commit with a timestamp *** FAILED ***
INFO ide_ci::program::command: sbtℹ️ [info] "2022-11-18T23:05:01.768499800Z" did not start with substring "2022-11-19" (VcsManagerTest.scala:201)
```
Ignore gitconfig located in the user's home directory when creating Git instance.
For example, I setup git to automatically sign commits. And all git related tests are failing because of that:
```
[info] - should return last X commits *** FAILED ***
[info] org.eclipse.jgit.api.errors.ServiceUnavailableException: Signing service is not available
[info] at org.eclipse.jgit.api.CommitCommand.sign(CommitCommand.java:328)
[info] at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:283)
[info] at org.enso.languageserver.vcsmanager.GitSpec$InitialRepoSetup.setup(GitSpec.scala:360)
[info] at org.enso.languageserver.vcsmanager.GitSpec$InitialRepoSetup.setup$(GitSpec.scala:348)
[info] at org.enso.languageserver.vcsmanager.GitSpec$$anon$16.setup(GitSpec.scala:275)
[info] at org.enso.languageserver.vcsmanager.GitSpec$InitialRepoSetup.$init$(GitSpec.scala:346)
[info] at org.enso.languageserver.vcsmanager.GitSpec$$anon$16.<init>(GitSpec.scala:275)
[info] at org.enso.languageserver.vcsmanager.GitSpec.$anonfun$new$35(GitSpec.scala:275)
[info] at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info] at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info] ...
```
This change adds support for Version Controlled projects in language server.
Version Control supports operations:
- `init` - initialize VCS for a project
- `save` - commit all changes to the project in VCS
- `restore` - ability to restore project to some past `save`
- `status` - show the status of the project from VCS' perspective
- `list` - show a list of requested saves
# Important Notes
Behind the scenes, Enso's VCS uses git (or rather [jGit](https://www.eclipse.org/jgit/)) but nothing stops us from using a different implementation as long as it conforms to the establish API.
When nodes get invalidated in the cache, they have to be recomputed. Let the IDE know which of the nodes are pending by sending `Api.ExpressionUpdate.Payload.Pending` message.
# Important Notes
This PR introduces new `Api.ExpressionUpdate.Payload.Pending` message. This message is delivered before re-computation of nodes. Later `Api.ExpressionUpdate.Payload.Value` or other is sent to notify the IDE that a value for given node is available.
Trivial implementation of of the `Api.ExpressionUpdate.Payload.Pending` message in the IDE is provided by this PR to (improperly) visualize pending node status - further improvements needed in follow up PRs.
This change adds Autosave action for open buffers. The action is scheduled
after every edit request and is cancelled by every explicit save file request, if
necessary. Successful autosave also notifies any active clients of the buffer.
Related to https://www.pivotaltracker.com/story/show/182721656
# Important Notes
WIP
PR allows to attach metod pointers as a visualization expressions. This way it allows to attach a runtime instrument that enables caching of intermediate expressions.
# Important Notes
ℹ️ API is backward compatible.
To attach the visualization with caching support, the same `executionContext/attachVisualisation` method is used, but `VisualisationConfig` message should contain the message pointer.
While `VisualisationConfiguration` message has changed, the language server accepts both new and old formats to keep visualisations working in IDE.
#### Old format
```json
{
"executionContextId": "UUID",
"visualisationModule": "local.Unnamed.Main",
"expression": "x -> x.to_text"
}
```
#### New format
```json
{
"executionContextId": "UUID",
"expression": {
"module": "local.Unnamed.Main",
"definedOnType": "local.Unnamed.Main",
"name": "encode"
}
}
```
In order to analyse why the `runner.jar` is slow to start, let's _"self sample"_ it using the [sampler library](https://bits.netbeans.org/dev/javadoc/org-netbeans-modules-sampler/org/netbeans/modules/sampler/Sampler.html). As soon as the `Main.main` is launched, the sampling starts and once the server is up, it writes its data into `/tmp/language-server.npss`.
Open the `/tmp/language-server.npss` with [VisualVM](https://visualvm.github.io) - you should have one copy in your
GraalVM `bin/jvisualvm` directory and there has to be a GraalVM to run Enso.
#### Changelog
- add: the `MethodsSampler` that gathers information in `.npss` format
- add: `--profiling` flag that enables the sampler
- add: language server processes the updates in batches
This PR replaces hard-coded `@Builtin_Method` and `@Builtin_Type` nodes in Builtins with an automated solution
that a) collects metadata from such annotations b) generates `BuiltinTypes` c) registers builtin methods with corresponding
constructors.
The main differences are:
1) The owner of the builtin method does not necessarily have to be a builtin type
2) You can now mix regular methods and builtin ones in stdlib
3) No need to keep track of builtin methods and types in various places and register them by hand (a source of many typos or omissions as it found during the process of this PR)
Related to #181497846
Benchmarks also execute within the margin of error.
### Important Notes
The PR got a bit large over time as I was moving various builtin types and finding various corner cases.
Most of the changes however are rather simple c&p from Builtins.enso to the corresponding stdlib module.
Here is the list of the most crucial updates:
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/builtin/Builtins.java` - the core of the changes. We no longer register individual builtin constructors and their methods by hand. Instead, the information about those is read from 2 metadata files generated by annotation processors. When the builtin method is encountered in stdlib, we do not ignore the method. Instead we lookup it up in the list of registered functions (see `getBuiltinFunction` and `IrToTruffle`)
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java` has now information whether it corresponds to the builtin type or not.
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/RuntimeStubsGenerator.scala` - when runtime stubs generator encounters a builtin type, based on the @Builtin_Type annotation, it looks up an existing constructor for it and registers it in the provided scope, rather than creating a new one. The scope of the constructor is also changed to the one coming from stdlib, while ensuring that synthetic methods (for fields) also get assigned correctly
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/IrToTruffle.scala` - when a builtin method is encountered in stdlib we don't generate a new function node for it, instead we look it up in the list of registered builtin methods. Note that Integer and Number present a bit of a challenge because they list a whole bunch of methods that don't have a corresponding method (instead delegating to small/big integer implementations).
During the translation new atom constructors get initialized but we don't want to do it for builtins which have gone through the process earlier, hence the exception
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor not only generates the actual code fpr nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor no longer generates only (root) nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/TypeProcessor.java` - Similar to MethodProcessor but handles @Builtin_Type annotations. It doesn't, **yet**, generate any builtin objects. It also collects the names, as present in stdlib, if any, so that we can generate the names automatically (see generated `types/ConstantsGen.java`)
- `engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin` - various classes annotated with @BuiltinType to ensure that the atom constructor is always properly registered for the builitn. Note that in order to support types fields in those, annotation takes optional `params` parameter (comma separated).
- `engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala` - drop manual creation of test list which seemed to be a relict of the old design
A draft of simple changes to the compiler to expose sum type information. Doesn't break the stdlib & at the same time allows for dropdowns. This is still broken, for example it doesn't handle exporting/importing types, only ones defined in the same module as the signature. Still, seems like a step in the right direction – please provide feedback.
# Important Notes
I've decided to make the variant info part of the type, not the argument – it is a property of the type logically.
Also, I've pushed it as far as I'm comfortable – i.e. to the `SuggestionHandler` – I have no idea if this is enough to show in IDE? cc @4e6
Changelog:
- fix: `search/completion` request with the position parameter.
- fix: `refactoring/renameProject` request. Previously it did not take into account the library namespace (e.g. `local.`)
Result of automatic formatting with `scalafmtAll` and `javafmtAll`.
Prerequisite for https://github.com/enso-org/enso/pull/3394
### Important Notes
This touches a lot of files and might conflict with existing PRs that are in progress. If that's the case, just run
`scalafmtAll` and `javafmtAll` after merge and everything should be in order since formatters should be deterministic.
Changelog:
- add: component groups to package descriptions
- add: `executionContext/getComponentGroups` method that returns component groups of libraries that are currently loaded
- doc: cleanup unimplemented undo/redo commands
- refactor: internal component groups datatype
PR adds a monitor that handles messages between the language server and the runtime and dumps them as a CSV file `/tmp/enso-api-events-*********.csv`
```
UTC timestamp,Direction,Request Id,Message class
```
# Important Notes
⚠️ Monitor is enabled when the log level is set to trace. You should pass `-vv` (very verbose) option to the backend when starting IDE
```
enso -- -vv
```
This changes intends to cleanup some directories that are being left
behind after running `sbt test`:
- a random `foobar` directory was being created in the `engine` project
directory
- every run of a test suite would add more temporary directories in `/tmp`
The change does not make use of `deleteOnExit` which can pretty
unreliable. Instead we recursively delete files in directories and
directories to make sure nothing is left behind.
Changelog:
- feat: during the boot, prune outdated modules
from the suggestions database
- feat: when renaming the project, send updates
about changed records in the database
- refactor: remove deprecated
executionContext/expressionValuesComputed
notification
PR adds the new executionContext/expressionUpdates
API that replaces executionContext/expressionValueUpdates
notification, and in the future will be extended to support
the dataflow errors.
add: missing to_json conversions
fix: NPE in instrumentation
fix: EditFileCmd scheduling
fix: send visualization errors to the text endpoint
fix: preserve original location in the VectorLiterals pass
A bunch of improvements to the suggestions
system. Suggestions are extracted to the tree data
structure. The tree allows producing better diffs
between the file versions. And better diffs reduce
the number of updates that are sent to the IDE
after a file change, and consequently fix the
issue when the runtime type got overwritten with
the compile-time type.
Add new `executionContext/executionStatus`
notification returning a list of diagnostic
messages containing localized (linked to the
location in the code) information about
compilation errors and warnings, as well as
runtime errors with stack traces.
In the current workflow, at first, the default Unnamed project is
created, and the Suggestions database is populated with entries from the
Unnamed.* modules. When the user changes the name of the project, we
should update all modules in the Suggestion Database with the new
project name.
This PR implements module renaming in the Suggestions database and fixes
the initialization issues.
- add: search/invalidateSuggestionsDatabase JSON-RPC command that resets
the corrupted Suggestions database
- update: SuggestionsHandler to rename the modules in the
SuggestionsDatabase when the project is renamed
- fix: MainModule initialization
ExpressionValueUpdate notification contains information about the
executed object. To have the full information about this object, IDE
needs the id of the corresponding suggestion. PR updates the
notification adding the suggestion id of the executed object.
- update: public API for ExpressionValueUpdate notification
- update: ContextEventsListener groups ExpressionValueUpdates and sends
them in a batch
- update: ContextRegistry listens to the notifications from runtime and
routes them to the corresponding listener.
- test: add ContextEventsListenerSpec
1. This commit adds an extensive library for parsing commandline
arguments and formatting output.
2. It implements the expected CLI for the launcher.
3. It turns off parallel execution on the CI in hopes of making it
more stable.
4. It implements plugin support in the launcher.
