Commit Graph

281 Commits

Author SHA1 Message Date
somebody1234
87e076720e
Fix links in Markdown (#6235) 2023-04-11 15:55:06 +02:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Dmitry Bushev
4c62dc9061
Search suggestions by static attribute (#6036)
close #5874

Changelog:
- add: `isStatic` parameter to `search/completion` request to search by the `static` suggestion attribute
- update: search non-static suggestions when opening component browser

# Important Notes
Component browser doesn't show `Table.new` and `Table.from_rows` suggestions when a `Table` node is selected.

![2023-03-21-151117_1301x877_scrot](https://user-images.githubusercontent.com/357683/226874291-1ff99994-1bb6-41df-96b4-dc5c5178ba41.png)
2023-03-23 15:02:25 +00:00
Stijn ("stain") Seghers
0bdf44cba8
Use .node-version for pinning Node.js version (#6057)
Closes #6008: allowing Node.js to be used through Node version managers.
2023-03-23 12:00:08 +00:00
Wojciech Daniło
abb0b447d5
Improving Performance Monitor (#5895) 2023-03-21 09:17:54 +01:00
Kaz Wesley
c9806496ee
Rendering improvement/debugging support (#6019)
Some small improvements relating to rendering:

- Add a debug option: `-debug.pixel-read-period`. This can be used to measure the performance impact of checking the pointer location on different hardware. [On my development box, it makes no difference to performance.] (Closes #5490).
- Unbind pixel pack buffers after each use. This is recommended practice. It has no performance impact on my machine, and allows SpectorJS to run (`-debug.enable-spector`). (Closes #5941).

Also, simplify the profiling CLI: the `profile.load-profile` and `profile.save-profile` options have been renamed to `profile.load`/`profile.save`; `profile.save` now has a default filename, so you can capture a profile at any time in Electron with Ctrl+Alt+P and it will be written to `profile.json`.
2023-03-21 06:34:24 +00:00
Kaz Wesley
e171fba301
New documentation parser (#5917)
Implement new Enso documentation parser; remove old Scala Enso parser.

Performance: Total time parsing documentation is now ~2ms.

# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2023-03-15 15:43:51 +00:00
Hubert Plociniczak
6769ab0ee7
Downgrade hashing to SHA-1 and other optimizations (#5791)
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.

Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.

Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.

Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.

Finally fix an SBT warning when generating library cache.

Closes https://github.com/enso-org/enso/issues/5763

# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
2023-03-09 07:36:59 +00:00
Hubert Plociniczak
941512e0ba
Optimize import/export resolution (#5700)
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.

When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.

The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.

Part of https://github.com/enso-org/enso/issues/5568 work.
2023-03-01 08:53:29 +00:00
Nctdt
6dfbde5afd
Update system requirements of CONTRIBUTING documents (#5682)
Older versions of `Powershell (5.x)` do not support running the `run` command, so you need to replace it with a higher version of `Powershell (7.x)` or use `cmd`
2023-02-25 05:49:34 +00:00
Jaroslav Tulach
f53696eda4
sbt runEngineDistribution ...args... to build and execute the engine (#5609)
Automating the assembly of the engine and its execution into a single task. If you are modifying standard libraries, engine sources or Enso tests, you can launch `sbt` and then just:
```
sbt:enso> runEngineDistribution --run test/Tests/src/Data/Maybe_Spec.enso
[info] Engine package created at built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev
[info] Executing built-distribution/enso-engine-...-dev/bin/enso --run test/Tests/src/Data/Maybe_Spec.enso
Maybe:  [5/5, 30ms]
- should have a None variant [14ms]
- should have a Some variant [5ms]
- should provide the `maybe` function [4ms]
- should provide `is_some` [2ms]
- should provide `is_none` [3ms]
5 tests succeeded.
0 tests failed.unEngineDistribution 4s
0 tests skipped.
```
the [runEngineDistribution](3a581f29ee/docs/CONTRIBUTING.md (running-enso)) `sbt` input task makes sure all your sources are properly compiled and only then executes your enso source. Everything ready at a single press of Enter.

# Important Notes
To debug in chrome dev tools, just add `--inspect`:
```
sbt:enso> runEngineDistribution --inspect --run test/Tests/src/Data/Maybe_Spec.enso
E.g. in Chrome open: devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/7JsgjXlntK8
```
everything gets build and one can just attach the Enso debugger.
2023-02-16 21:37:12 +00:00
Pavel Marek
3e8467c204
Update GraalVM to 22.3.1 (#5602)
Updates the engine to GraalVM 22.3.1 version, which contains fixes for:
- Chrome inspector issues - 38eb3b5932

# Important Notes
- Update to GraalVM 22.3.1
- Remove host object wrapping workaround
2023-02-14 15:51:17 +00:00
Wojciech Daniło
ac90418200
Update CODE_OF_CONDUCT.md 2023-02-13 23:10:51 +01:00
Dmitry Bushev
53d5487f47
Lookup method pointers in IDE (#5578)
Closes #5036

Move the logic that looks up method pointers from the language server to IDE. This way we can keep the suggestion updates and expression updates async, otherwise it will hurt the initial startup time of LS.
Fixes the issue when some expression updates does not contain the method pointer.
2023-02-08 00:57:32 +00:00
Jaroslav Tulach
04415a2b5e
Write the log in XML format suitable for VisualVM 'UI Actions' (#4110)
Start `project-manager` with following options to provide first 20s of the startup sequence:
```
$ project-manager --profiling-events-log-path=start.log --profiling-path=start.npss --profiling-time=20
```
once the `start.log` and `start.npss` files are generated (next to each other), open them in GraalVM's VisualVM:
```
$ graalvm/bin/jvisualvm --openfile start.npss
```
analyze.
2023-02-05 06:36:16 +00:00
Dmitry Bushev
b65dd1f3fe
Warnings API (#4093)
Add information about warngins associated with the value to the expression update payload.
2023-01-28 03:13:38 +00:00
Dmitry Bushev
ed859c2682
Allow attaching visualizations to non-existent expressions (#4064)
Requested by @Frizi. IDE does not synchronize requests creating a node and attaching the visualization, and they can be sent in arbitrary order.
2023-01-18 18:56:32 +00:00
Dmitry Bushev
0a6e6237cd
Attach visualizations to sub-expressions (#4048)
Add ability to attach visualizations to sub-expressions.
2023-01-16 10:19:19 +00:00
Dmitry Bushev
2cd880f43d
Documentation for functions and locals (#4029)
Add documentation for functions and locals to suggestions database.
2023-01-10 16:59:53 +00:00
Hubert Plociniczak
3980c48d61
Sync file system and language server after restore (#4020)
VCS restore operation was correctly restoring the state of projects to the requested commit. Unfortunately, after the operation file system was becoming out-of-sync with language server's buffers (and IDE's content versions).

A few important changes are introduced here that complicate the interaction between components:
1) `vcs restore` returns an actual diff between the current state and the
requested commit
2) the response is forwarded to buffer registry first rather than to the client
3) the diff is used to identify appropriate collaborative editors and
notify them about the need to reload buffers from file system
4) all clients of affected open buffers are notified of the change via
`text/didChange` notification. If a file was removed and there were open buffers for it, clients will be notified via `file/event` and editor will be stopped
5) only then the client is notified about a successful restore operation

This PR addresses one of the two problems reported in https://www.pivotaltracker.com/story/show/184097084.

# Important Notes
We need to make sure that IDE correctly responds to `text/didChange` notifications.
2023-01-05 14:00:00 +00:00
Jaroslav Tulach
0041b649eb
IGV can jump to JMH sources & more (#4008)
Improvements to behavior and visual appearance of IGV Enso integration.
2022-12-30 05:30:32 +00:00
Pavel Marek
e6838bc90d
Convert Any.== to a builtin (#3956)
`Any.==` is a builtin method. The semantics is the same as it used to be, except that we no longer assume `x == y` iff `Meta.is_same_object x y`, which used to be the case and caused failures in table tests.

# Important Notes
Measurements from `EqualsBenchmarks` shows that the performance of `Any.==` for recursive atoms increased by roughly 20%, and the performance for primitive types stays roughly the same.
2022-12-29 21:20:00 +00:00
Dmitry Bushev
cf67ca60d9
Add globs to logger configuration (#4004)
An artifact left from debugging some SQL last week. Just a nice feature to have.
2022-12-22 19:09:21 +00:00
Dmitry Bushev
3e74afca51
misc: bump wasm-pack (#3983) 2022-12-14 18:45:39 +01:00
Hubert Plociniczak
88effc41f8
Windows doesn't like resolving last commit with refs/heads/master (#3974)
Apparently
```
git --git-dir .enso/.vcs log refs/heads/master
fatal: ambiguous argument 'refs/heads/master': unknown revision or path not in the working tree.
```
but
```
git --git-dir .enso/.vcs log HEAD
fatal: ambiguous argument 'refs/heads/master': unknown revision or path not in the working tree.
```
works just fine on Windows.

Added some safeguards to avoid propagating weird errors because of retrieving element from an
empty Option.
2022-12-12 22:13:03 +00:00
Dmitry Bushev
43167c1617
Add executionContext/interrupt API command (#3952)
Implement the `executionContext/interrupt` API command that forcibly stops the program execution.
2022-12-08 00:04:46 +00:00
Pavel Marek
4641426ce9
Allow arbitrary expression evaluation in chromeinspector (#3941)
Allow arbitrary expression evaluation in the chromeinspector console. Moreover, allow modifications of any variable in any stack frame.

# Important Notes
- Implement inline parsing in `EnsoLanguage.parse(InlineParsingRequest)`.
- Debugging experience is affected by this [bug in Truffle](https://github.com/oracle/graal/issues/5513), which causes NPEs when a host object gets into chromeinspector. I tried to implement a workaround, but it does not work all the time. Nevertheless, it should not matter that much - if there is a NPE in the debugger, you can just ignore it, as it should be concealed in the debugger and should not be propagted outside. See comments in the `docs/debugger`.
2022-12-07 23:02:42 +00:00
Dmitry Bushev
768747a55e
Update suggestion argument doc (#3948)
Updated the doc to match the implemented API.
2022-12-05 13:29:18 +00:00
Michał Wawrzyniec Urbańczyk
4de8e44871
Release process (#3909) 2022-12-02 02:56:22 +01:00
Kaz Wesley
0e37cb1957
Add profiling manual from design repo (#3930) 2022-11-30 13:33:50 +01:00
Galin Bajlekov
99485d92ad
Change spacing rules to match the autoformatter (#3888) 2022-11-28 12:47:29 +01:00
Hubert Plociniczak
5b434d19c3
Update 'naming' docs (#3858) 2022-11-24 12:55:42 +00:00
Jaroslav Tulach
402ebb2f8e
Upgrade to GraalVM 22.3.0 (#3663)
Upgrading to GraalVM 22.3.0.

# Important Notes
- Removed all deprecated `FrameSlot`, and replaced them with frame indexes - integers.
- Add more information to `AliasAnalysis` so that it also gathers these indexes.
- Add quick build mode option to `native-image` as default for non-release builds
- `graaljs` and `native-image` should now be downloaded via `gu` automatically, as dependencies.
- Remove `engine-runner-native` project - native image is now build straight from `engine-runner`.
- We used to have `engine-runner-native` without `sqldf` in classpath as a workaround for an internal native image bug.
- Fixed chrome inspector integration, such that it shows values of local variables both for current stack frame and caller stack frames.
- There are still many issues with the debugging in general, for example, when there is a polyglot value among local variables, a `NullPointerException` is thrown and no values are displayed.
- Removed some deprecated `native-image` options
- Remove some deprecated Truffle API method calls.
2022-11-23 14:30:48 +00:00
Hubert Plociniczak
85d4337f26
Project save backed by git (#3851)
This change adds support for Version Controlled projects in language server.
Version Control supports operations:
- `init` - initialize VCS for a project
- `save` - commit all changes to the project in VCS
- `restore` - ability to restore project to some past `save`
- `status` - show the status of the project from VCS' perspective
- `list` - show a list of requested saves

# Important Notes
Behind the scenes, Enso's VCS uses git (or rather [jGit](https://www.eclipse.org/jgit/)) but nothing stops us from using a different implementation as long as it conforms to the establish API.
2022-11-14 17:32:39 +00:00
Dmitry Bushev
14012a751f
Add parent type field to suggestion (#3846)
PR adds `parentType` field to the `Type` suggestion. All Enso types have parent type except `Any`.
2022-11-07 13:21:04 +00:00
Dmitry Bushev
a6ce49e8a5
Split Atom suggestion entry to Type and Constructor (#3835)
Changelog:
- update: split `Atom` suggestion to `Type` and `Constructor`
- update: gui API
- update: JSONRPC doc
2022-11-02 09:53:40 +00:00
Dmitry Bushev
46441ca7a8
Add isStatic method field (#3829)
PR adds `isStatic` field to suggestion. The field is required for Component Browser.
2022-10-26 09:12:45 +00:00
Hubert Plociniczak
6c440beecc
Move logic calculating the index in Vector.at to a builtin method to make the performance of Vector to be on par with Array (#3811)
The main culprit of a Vector slowdown (when compared to Array) was the normalization of the index when accessing the elements. Turns out that the Graal was very persistent on **not** inlining that particular fragment and that was degrading the results in benchmarks.

Being unable to force it to do it (looks like a combination of thunk execution and another layer of indirection) we resorted to just moving the normalization to the builtin method. That makes Array and Vector perform roughly the same.

Moved all handling of invalid index into the builtin as well, simplifying the Enso implementation. This also meant that `Vector.unsafe_at` is now obsolete.
Additionally, added support for negative indices in Array, to behave in the same way as for Vector.

# Important Notes
Note that this workaround only addresses this particular perf issue. I'm pretty sure we will have more of such scenarios.
Before the change `averageOverVector` benchmark averaged around `0.033 ms/op` now it does consistently `0.016 ms/op`, similarly to `averageOverArray`.
2022-10-20 12:50:44 +00:00
Pavel Marek
e9260227c4
Duration type is a builtin type (#3759)
- Reimplement the `Duration` type to a built-in type.
- `Duration` is an interop type.
- Allow Enso method dispatch on `Duration` interop coming from different languages.

# Important Notes
- The older `Duration` type should now be split into new `Duration` builtin type and a `Period` type.
- This PR does not implement `Period` type, so all the `Period`-related functionality is currently not working, e.g., `Date - Period`.
- This PR removes `Integer.milliseconds`, `Integer.seconds`, ..., `Integer.years` extension methods.
2022-10-14 18:08:08 +00:00
Jaroslav Tulach
835ac05218
Engine should send notification about node status (#3729)
When nodes get invalidated in the cache, they have to be recomputed. Let the IDE know which of the nodes are pending by sending `Api.ExpressionUpdate.Payload.Pending` message.

# Important Notes
This PR introduces new `Api.ExpressionUpdate.Payload.Pending` message. This message is delivered before re-computation of nodes. Later `Api.ExpressionUpdate.Payload.Value` or other is sent to notify the IDE that a value for given node is available.

Trivial implementation of of the `Api.ExpressionUpdate.Payload.Pending` message in the IDE is provided by this PR to (improperly) visualize pending node status - further improvements needed in follow up PRs.
2022-09-28 12:35:12 +00:00
Hubert Plociniczak
096fcfee82
Generate native image for engine-runner (#3638)
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m4.503s
user	0m9.248s
sys	0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m0.176s
user	0m0.042s
sys	0m0.038s
```

# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.

Will need to add the above command to CI.
2022-09-22 14:45:10 +00:00
Michał Wawrzyniec Urbańczyk
d6bd80011f
Fixing yargs and electron-builder regressions (#3725)
This PR reverts two version bumps from #3712:
* `yargs` needs to be downgraded, because a never version can trigger https://github.com/evanw/esbuild/issues/2441
* `electron-builder` needs to be downgraded, because our workaround for https://github.com/electron-userland/electron-builder/issues/6865 proved insufficient.

As the upstream issues are resolved, we should bump these dependencies again.
2022-09-21 11:32:20 +02:00
Michał Wawrzyniec Urbańczyk
10f45d7fd1
macOS Code Signing & Notarization (#3712)
This PR reenables code signing and notarization on macOS.

[ci no changelog needed]

# Important Notes
* electron-builder has been bumped, mostly to avoid missing Python issue. A workaround for a regression with Windows installer is provided as a patch.
2022-09-19 19:02:18 +00:00
Hubert Plociniczak
a04425576a
Force recompilation if imported module has changed (#3703)
IR cache never really took into account a situation when a binding from the imported module has changed. In other words, it would continue to happily use the serialized metadata without noticing that it changed.

This change forces cache invalidation when any of the imported modules was invalidated (or rather not loaded from cache).

# Important Notes
Added simple test infrastructure that simulates file modifications that would trigger the initial cache invalidation.
If they succeed, cache invalidation is propagated thus causing an error.
2022-09-15 13:41:58 +00:00
Ilya Bogdanov
0f168abbfa
Add a note about the need of Python 2 on MacOS to CONTRIBUTING.md (#3682)
[ci no changelog needed]

Python 2 is required for our version of the `electron-builder` when building Enso IDE on modern Mac OS versions. Unfortunately, Apple removed the `/usr/bin/python` executable in the macOS Monterey release, so we are forced to give instructions on reinstalling it. I took the most popular solution from https://stackoverflow.com/questions/60298514/how-to-reinstall-python2-from-homebrew
2022-09-07 09:24:37 +00:00
Dmitry Bushev
de0a231417
IDE uses new visualization API (#3661) 2022-09-01 15:33:46 +03:00
Edward Kmett
0dbbbaad00
Add type system documentation (#3376) 2022-08-15 18:36:03 +03:00
Hubert Plociniczak
3fa78afd10
Support Autosave for open buffers (#3637)
This change adds Autosave action for open buffers. The action is scheduled
after every edit request and is cancelled by every explicit save file request, if
necessary. Successful autosave also notifies any active clients of the buffer.

Related to https://www.pivotaltracker.com/story/show/182721656

# Important Notes
WIP
2022-08-11 11:45:12 +00:00
Dmitry Bushev
98d30bccf3
Enable caching in visualization functions (#3618)
PR allows to attach metod pointers as a visualization expressions. This way it allows to attach a runtime instrument that enables caching of intermediate expressions.

# Important Notes
ℹ️ API is backward compatible.

To attach the visualization with caching support, the same `executionContext/attachVisualisation` method is used, but `VisualisationConfig` message should contain the message pointer.
While `VisualisationConfiguration` message has changed, the language server accepts both new and old formats to keep visualisations working in IDE.

#### Old format

```json
{
"executionContextId": "UUID",
"visualisationModule": "local.Unnamed.Main",
"expression": "x -> x.to_text"
}
```

#### New format

```json
{
"executionContextId": "UUID",
"expression": {
"module": "local.Unnamed.Main",
"definedOnType": "local.Unnamed.Main",
"name": "encode"
}
}
```
2022-08-10 12:01:33 +00:00
Mateusz Czapliński
c6835d2de7
Show custom icons in Component Browser (#3606)
Show custom icons in Component Browser for entries that have a non-empty `Icon` section in their docs with the section's body containing a name of a predefined icon.

https://www.pivotaltracker.com/story/show/182584336

#### Visuals

A screenshot of a couple custom icons in the Component Browser:

<img width="346" alt="Screenshot 2022-07-27 at 15 55 33" src="https://user-images.githubusercontent.com/273837/181265249-d57f861f-8095-4933-9ef6-e62644e11da3.png">

# Important Notes
- The PR assigns icon names to four items in the standard library, but only three of them are shown in the Component Browser because of [a parsing bug in the Engine](https://www.pivotaltracker.com/story/show/182781673).
- Icon names are assigned only to four items in the standard library because only two currently predefined icons match entries in the currently defined Virtual Component Groups. Adjusting the definitions of icons and Virtual Component Groups is covered by [a different task](https://www.pivotaltracker.com/story/show/182584311).
- A bug in the documentation of the Enso protocol message `DocSection` is fixed. A `text` field in the `Tag` interface is renamed to `body` (this is the field name used in Engine).
2022-08-01 13:41:04 +00:00