Commit Graph

318 Commits

Author SHA1 Message Date
Hubert Plociniczak
63802544f7
Enable log-to-file configuration (#7918)
* Enable log-to-file configuration

PR #7825 enabled parallel logging to a file with a much more
fine-grained log level by default.
However, logging at `TRACE` level on Windows appears to be still
problematic.

This PR reduced the default log level to file from `DEBUG` to `TRACE`
and allows to control it via an environment variable if one wishes to
change the verbosity without making code changes.

* PR comments
2023-10-02 14:25:09 +02:00
Pavel Marek
9f15b90caa
Implement Enso-specific assert (#7883)
Implement Enso-specific assert - `Runtime.assert` that works like asserts in any other runtime.

# Important Notes
- Enso-specific assertions are enabled when JVM assertions are enabled, or when `ENSO_ENABLE_ASSERTIONS` env var is not empty (See 72cd8361cb/engine/runtime/src/main/java/org/enso/interpreter/runtime/EnsoContext.java (L139))
2023-09-29 14:46:58 +00:00
Hubert Plociniczak
18b2491a41
Always log to console and file (#7825)
* Always log verbose to a file

The change adds an option by default to always log to a file with
verbose log level.
The implementation is a bit tricky because in the most common use-case
we have to always log in verbose mode to a socket and only later apply
the desired log levels. Previously socket appender would respect the
desired log level already before forwarding the log.

If by default we log to a file, verbose mode is simply ignored and does
not override user settings.

To test run `project-manager` with `ENSO_LOGSERVER_APPENDER=console` env
variable. That will output to the console with the default `INFO` level
and `TRACE` log level for the file.

* add docs

* changelog

* Address some PR requests

1. Log INFO level to CONSOLE by default
2. Change runner's default log level from ERROR to WARN

Took a while to figure out why the correct log level wasn't being passed
to the language server, therefore ignoring the (desired) verbose logs
from the log file.

* linter

* 3rd party uses log4j for logging

Getting rid of the warning by adding a log4j over slf4j bridge:
```
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
```

* legal review update

* Make sure tests use test resources

Having `application.conf` in `src/main/resources` and `test/resources`
does not guarantee that in Tests we will pick up the latter. Instead, by
default it seems to do some kind of merge of different configurations,
which is far from desired.

* Ensure native launcher test log to console only

Logging to console and (temporary) files is problematic for Windows.
The CI also revealed a problem with the native configuration because it
was not possible to modify the launcher via env variables as everything
was initialized during build time.

* Adapt to method changes

* Potentially deal with Windows failures
2023-09-26 11:32:04 +02:00
Hubert Plociniczak
3e615f385b
Improve shutdown logic of language server (#7801)
* Improve shutdown logic of language server

This PR addresses problems mentioned in #7470 and #7729:
- shutting a language server explicitly will not lead to a soft shutdown
- `project/status` endpoint returns the state of the language server

`LanguageServerController` now also signed up for `ClientConnect`
messages. For it to be unambiguous, we need to carry around the port
number of the language server as a way of identifying the right one.

One can now use `project/status` to additionally determine the state of
the language server.

Also relies on a proper fix for #7765.

* changelog

* PR comments
2023-09-22 12:05:44 +02:00
Paweł Grabarz
42a7cb2d23
[Gui2] Opening projects and language server connection (#7813)
# Important Notes
- Binary LS endpoint is not yet handled.
- The parsing of provided source is not entirely correct, as each line (including imports) is treated as node. The usage of actual enso AST for nodes is not yet implemented.
- Modifications to the graph state are not yet synchronized back to the language server.
2023-09-22 03:43:25 +00:00
Dmitry Bushev
5c198145a1
Static parameter in search/completion request affects only methods (#7831)
close #7805

Changelog:
update: `isStatic` parameter in the `search/completion` request only affects method suggestions
2023-09-19 15:12:23 +00:00
Jaroslav Tulach
6cbd111bad
Optional Espresso support with ENSO_JAVA=espresso env variable 2023-09-19 15:10:12 +02:00
GregoryTravis
b0c1f3b00e
New Data.post for sending a payload to a Web API (#7700) 2023-09-19 11:26:29 +00:00
Jaroslav Tulach
30a62b97bb
Runtime checking of ascribed expression types (#7796) 2023-09-14 14:09:41 +02:00
Hubert Plociniczak
ddbccd8764
Fix console output for native image (#7744)
* Add serialization config to fix native image

Discovered when switching to console output.

* Add more documentation
2023-09-06 12:12:52 +02:00
Hubert Plociniczak
8a60bc6dcd
Replace a custom logger with off the shelf implementation (#7559)
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.

Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.

Most important differences have been outlined in `docs/infrastructure/logging.md`.

# Important Notes
Addresses:
- #7253
- #6739
2023-09-04 09:40:16 +00:00
Radosław Waśko
87ce78615a
Change layout of local library search path in order to be able to move Round_Spec.enso back to Tests (#7634)
- Closes #7633
- Moves `Round_Spec.enso` from published `Standard.Test` into our `test/Tests` project; the `Table_Tests` that depend on it, simply `import enso_dev.Tests`.
- Changes the layout of the local libraries directory:
- It used to be `root/<namespace>/<name>`.
- Now it is `root/<dir>` - the namespace and name are now read from `package.yaml` instead.
- Adds the parent directory of the current project to the default `ENSO_LIBRARY_PATH`.
- It is treated as a secondary path, so the default `ENSO_HOME/lib` still takes precedence.
- This allows projects to reference and load 'sibling' projects easily - the only requirement is for the project to enable `prefer-local-libraries: true` or add the other local project to its edition. The edition resolution logic is **not changed**.
2023-09-01 20:20:04 +00:00
Hubert Plociniczak
24f263b20b
Towards shallow nesting in IR (#7681)
Refactoring deeply nested IR classes to shallow nesting.
Fixes #7017

# Important Notes
It's a big PR but fairly consistent. Split multiple hierarchy levels into subpackages.
2023-08-31 15:42:53 +00:00
Jaroslav Tulach
198ab7f487
Support for Python libraries like numpy (#7678) 2023-08-30 06:10:18 +02:00
James Dunkerley
7d83b3d7b4
Add GROUP to functions (#7622)
- Update list of groups to agreed list.
- Lower case `ALIAS` names to be consistent with function names.
- Add `GROUP` to methods.
- All constructors and functions have doc comments.
- Correct a few typos (e.g. `PRVIATE`).
- Mark some more things as `PRIVATE`.
- Use `ToDo:` and `Note:` consistently.
- Order tags in doc comment.

# Important Notes
We don't have all the doc comments on types and will want to add them in future,
2023-08-23 13:20:38 +00:00
Dmitry Bushev
5dc2c4c5fd
Notification about the project rename action (#7613)
close #7604

After moving the rename action to the dashboard, IDE is unaware of the new project name. PR implements a new `refactoring/projectRenamed` notification that is sent from the server to clients and informs them about the changed project name.

# Important Notes
https://github.com/enso-org/enso/assets/357683/7c62726d-217e-4e69-8e48-568e0b7b8c34
2023-08-22 11:32:46 +00:00
Dmitry Bushev
060323e511
Add a method for getting component groups without execution context (#7569)
close #7510

Changelog:
- add: `runtime/getComponentGroups` request returning the component groups currently available in runtime
2023-08-14 16:49:16 +00:00
Dmitry Bushev
b7ab6911ff
Renaming Variable or Function Support (#7515)
close #7389



Changelog:
- add: `refactoring/renameSymbol` request to rename locals or module methods
2023-08-10 21:16:33 +00:00
Pavel Marek
8e49255d92
Invoke all Enso benchmarks via JMH (#7101)
# Important Notes
#### The Plot

- there used to be two kinds of benchmarks: in Java and in Enso
- those in Java got quite a good treatment
- there even are results updated daily: https://enso-org.github.io/engine-benchmark-results/
- the benchmarks written in Enso used to be 2nd class citizen

#### The Revelation
This PR has the potential to fix it all!
- It designs new [Bench API](88fd6fb988) ready for non-batch execution
- It allows for _single benchmark in a dedicated JVM_ execution
- It provides a simple way to wrap such an Enso benchmark as a Java benchmark
- thus the results of Enso and Java benchmarks are [now unified](https://github.com/enso-org/enso/pull/7101#discussion_r1257504440)

Long live _single benchmarking infrastructure for Java and Enso_!
2023-08-07 12:39:01 +00:00
Dmitry Bushev
8416e19eef
Refactor project normalized name (#7444)
Followup to #7397. When started this refactoring I only changed the config types and forgot to change the API.

Changelog:
- refactor: rename `projectModule` to `projectNormalizedName` in the API

# Important Notes
https://github.com/enso-org/enso/assets/357683/a840dc90-15f8-4b6d-bbfb-571c53cd2f7d
2023-07-31 17:47:28 +00:00
Dmitry Bushev
80c4b1ca06
Allow users to give a project other than Upper_Snake_Case name (#7397)
close #6356

Allow arbitrary names for user projects.

# Important Notes
https://github.com/enso-org/enso/assets/357683/55a3b660-af23-4b09-959b-eac515766788
2023-07-28 13:44:39 +00:00
Jaroslav Tulach
6a6d7dbff3
Simplify onboarding instructions for engine & project-manager developers (#7181)
The current instructions to _build, use and debug_ `project-manager` and its engine/ls process are complicated and require a lot of symlinks to properly point to each other. This pull requests simplifies all of that by introduction of `ENSO_ENGINE_PATH` and `ENSO_JVM_PATH` environment variables. Then it hides all the complexity behind a simple _sbt command_: `runProjectManagerDistribution --debug`.

# Important Notes
I decided to tackle this problem as I have three repositories with different branches of Enso and switching between them requires me to mangle the symlinks. I hope I will not need to do that anymore with the introduction of the `runProjectManagerDistribution` command.
2023-07-17 17:38:54 +00:00
Dmitry Bushev
f691713077
Add endpoint for downloading a project (#7291)
close #7194

Changelog:

- add: `/projects/{project_id}/enso_project` HTTP endpoint returning an `.enso-project` archive structure
- update: archive enso project to a `.enso-project` `.tar.gz` archive
- update: make project `path` a required field
2023-07-14 13:18:08 +00:00
Dmitry Bushev
4864e5b51d
Add fileModifiedOnDisk notification (#7239)
part of #7178

Changelog:
- add: `text/fileModifiedOnDisk` notification
- update: during the auto-save, check if the file is modified on disk and send the notification. I.e. auto-save does not overwrite the file if it was changed on disk (but the save command does)
- update: IDE handles the file-modified-on-disk notification and reloads the module from disk

# Important Notes
Currently, the auto-save (and the check that the file is modified on disk) is triggered only after the file was edited. The proper check (using the file-watcher service) will be added in the next PR

https://github.com/enso-org/enso/assets/357683/ff91f3e6-2f7a-4c01-a745-98cb140e1964
2023-07-10 16:50:14 +00:00
Dmitry Bushev
66894bd79e
Send info about function values (#7168)
close #6957

Extend `ExpressionUpdate` message and send a function schema if the returned value is a function.
2023-06-30 13:31:48 +00:00
Hubert Plociniczak
bf5ddf3de0
Send executionComplete response only on success (#7143)
`executionFailed` instead is sent when an evaulation finishes with a a critical failure or a non-critical error.
The PR tries to miniminally modify the change in the messages exchange so as to avoid a major redesign at this point.

Closes #7002.

# Important Notes
Unblocks IDE which will need to modify to this new setup.
2023-06-29 07:35:28 +00:00
Jaroslav Tulach
91f661fd8b
Documenting how to debug project-manager project (#7153)
```
sbt:project-manager> withDebug run --debugger
```
2023-06-29 05:55:14 +00:00
Jaroslav Tulach
477dd82670
Drop-down widgets for extension functions via UnresolvedSymbol (#7115)
Fixes #6955 by:
- using `visualisationModule` to specify the module where the visualization is to be used
- referring to method in `Meta.get_annotation` with `.method_name` - e.g. unresolved symbol notation
- evaluating arguments to `Meta.get_annotation` in the context of the user module (which can access the extension functions)
2023-06-27 15:19:42 +00:00
Hubert Plociniczak
4e5cb9cca6
Consistent naming of visualization (#7124) 2023-06-26 17:04:35 +00:00
Kaz Wesley
bf42ed482d
Fix dropdown laziness; also misc fixes (#6991)
Several small changes:
- Dropdowns: Populate `GridView` lazily (fixes #6865).
- Clear disconnected edges when editing node (fixes case 1 in #7018).
- Fix regression in node selection rendering (2nd bug in #6975).
- Update profiler docs. The hotkey to *prOfile without exiting* is now `Ctrl+Alt+O` (`Ctrl+Alt+P` has been requisitioned by the CB).

Node selection:
| Pre-`Rectangle` | This PR |
| --- | --- |
| ![image](https://github.com/enso-org/enso/assets/1047859/bec341c1-dbf8-404d-9f2a-5d070c80ff15) | ![image](https://github.com/enso-org/enso/assets/1047859/8161390c-f64b-4bb3-8b7a-b87b2f9b4cd3) |

# Important Notes
- `Rectangle`: When `inset > border`, the extra space is now between the body and the border, not outside the border.
- More robust node layering logic. Now an inconsistent layer order cannot occur, even if something strange happens (like editing an expression and an edge at the same time).
- The dynamic drop down in the `drop_down` example scene doesn't show any entries before (or after) this, so I can't test the dynamic case.
2023-06-14 17:58:03 +00:00
Dmitry Bushev
4e9f02258e
Suggestion should contain a list of annotations (#6924)
close #6847

Add annotation names to suggestions.
2023-06-02 07:26:01 +00:00
Dmitry Bushev
ed3f9b306e
Consistent self types (#6867)
close #6800

Update the `executionContext/expressionUpdates` notification and send the list of not applied arguments in addition to the method pointer.

# Important Notes
IDE is updated to support the new API.
2023-05-31 15:47:48 +00:00
Dmitry Bushev
0318880990
Add project creation time to project metadata (#6780)
close #6686

Changelog:
- add: `created` field to the project metadata response
- update: project manager tests
2023-05-22 13:40:10 +00:00
Pavel Marek
f38033b037
Add verbose logs mention to CONTRIBUTING.md (#6703) 2023-05-16 11:14:19 +00:00
somebody1234
cd54e1b47d
Add TypeScript style guide (#6226)
Adds TypeScript style guide.

# Important Notes
This is more or less a straight port from Rust; I think the goal should be to make this a sort of living document which changes as the TypeScript parts of the codebase become more mature.

Several sections have been removed as they do not apply to the TypeScript codebase. Of note is the section about vertically aligning type annotations, local variable declarations and match arms.
2023-04-15 08:28:12 +00:00
Dmitry Bushev
b97fc39214
Remove SQL versions repo (#6242)
close #6232

Changelog:
- remove: `SqlVersionsRepo`
- update: `SuggestionsDatabaseModuleUpdateNotification` message removing the version
- update: cleanup versions repo usages in the language server
2023-04-11 19:22:30 +00:00
Dmitry Bushev
6663125410
Language Server support for execution environments (#6217)
close #6133

Changelog:
- add: `executionContext/setExecutionEnvironment` request
- update: `executionContext/recompute` request to have optional `executionEnvironment` parameter.
2023-04-11 15:12:27 +00:00
somebody1234
87e076720e
Fix links in Markdown (#6235) 2023-04-11 15:55:06 +02:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Dmitry Bushev
4c62dc9061
Search suggestions by static attribute (#6036)
close #5874

Changelog:
- add: `isStatic` parameter to `search/completion` request to search by the `static` suggestion attribute
- update: search non-static suggestions when opening component browser

# Important Notes
Component browser doesn't show `Table.new` and `Table.from_rows` suggestions when a `Table` node is selected.

![2023-03-21-151117_1301x877_scrot](https://user-images.githubusercontent.com/357683/226874291-1ff99994-1bb6-41df-96b4-dc5c5178ba41.png)
2023-03-23 15:02:25 +00:00
Stijn ("stain") Seghers
0bdf44cba8
Use .node-version for pinning Node.js version (#6057)
Closes #6008: allowing Node.js to be used through Node version managers.
2023-03-23 12:00:08 +00:00
Wojciech Daniło
abb0b447d5
Improving Performance Monitor (#5895) 2023-03-21 09:17:54 +01:00
Kaz Wesley
c9806496ee
Rendering improvement/debugging support (#6019)
Some small improvements relating to rendering:

- Add a debug option: `-debug.pixel-read-period`. This can be used to measure the performance impact of checking the pointer location on different hardware. [On my development box, it makes no difference to performance.] (Closes #5490).
- Unbind pixel pack buffers after each use. This is recommended practice. It has no performance impact on my machine, and allows SpectorJS to run (`-debug.enable-spector`). (Closes #5941).

Also, simplify the profiling CLI: the `profile.load-profile` and `profile.save-profile` options have been renamed to `profile.load`/`profile.save`; `profile.save` now has a default filename, so you can capture a profile at any time in Electron with Ctrl+Alt+P and it will be written to `profile.json`.
2023-03-21 06:34:24 +00:00
Kaz Wesley
e171fba301
New documentation parser (#5917)
Implement new Enso documentation parser; remove old Scala Enso parser.

Performance: Total time parsing documentation is now ~2ms.

# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2023-03-15 15:43:51 +00:00
Hubert Plociniczak
6769ab0ee7
Downgrade hashing to SHA-1 and other optimizations (#5791)
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.

Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.

Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.

Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.

Finally fix an SBT warning when generating library cache.

Closes https://github.com/enso-org/enso/issues/5763

# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
2023-03-09 07:36:59 +00:00
Hubert Plociniczak
941512e0ba
Optimize import/export resolution (#5700)
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.

When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.

The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.

Part of https://github.com/enso-org/enso/issues/5568 work.
2023-03-01 08:53:29 +00:00
Nctdt
6dfbde5afd
Update system requirements of CONTRIBUTING documents (#5682)
Older versions of `Powershell (5.x)` do not support running the `run` command, so you need to replace it with a higher version of `Powershell (7.x)` or use `cmd`
2023-02-25 05:49:34 +00:00
Jaroslav Tulach
f53696eda4
sbt runEngineDistribution ...args... to build and execute the engine (#5609)
Automating the assembly of the engine and its execution into a single task. If you are modifying standard libraries, engine sources or Enso tests, you can launch `sbt` and then just:
```
sbt:enso> runEngineDistribution --run test/Tests/src/Data/Maybe_Spec.enso
[info] Engine package created at built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev
[info] Executing built-distribution/enso-engine-...-dev/bin/enso --run test/Tests/src/Data/Maybe_Spec.enso
Maybe:  [5/5, 30ms]
- should have a None variant [14ms]
- should have a Some variant [5ms]
- should provide the `maybe` function [4ms]
- should provide `is_some` [2ms]
- should provide `is_none` [3ms]
5 tests succeeded.
0 tests failed.unEngineDistribution 4s
0 tests skipped.
```
the [runEngineDistribution](3a581f29ee/docs/CONTRIBUTING.md (running-enso)) `sbt` input task makes sure all your sources are properly compiled and only then executes your enso source. Everything ready at a single press of Enter.

# Important Notes
To debug in chrome dev tools, just add `--inspect`:
```
sbt:enso> runEngineDistribution --inspect --run test/Tests/src/Data/Maybe_Spec.enso
E.g. in Chrome open: devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/7JsgjXlntK8
```
everything gets build and one can just attach the Enso debugger.
2023-02-16 21:37:12 +00:00
Pavel Marek
3e8467c204
Update GraalVM to 22.3.1 (#5602)
Updates the engine to GraalVM 22.3.1 version, which contains fixes for:
- Chrome inspector issues - 38eb3b5932

# Important Notes
- Update to GraalVM 22.3.1
- Remove host object wrapping workaround
2023-02-14 15:51:17 +00:00
Wojciech Daniło
ac90418200
Update CODE_OF_CONDUCT.md 2023-02-13 23:10:51 +01:00