Commit Graph

506 Commits

Author SHA1 Message Date
Hubert Plociniczak
8a60bc6dcd
Replace a custom logger with off the shelf implementation (#7559)
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.

Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.

Most important differences have been outlined in `docs/infrastructure/logging.md`.

# Important Notes
Addresses:
- #7253
- #6739
2023-09-04 09:40:16 +00:00
Radosław Waśko
87ce78615a
Change layout of local library search path in order to be able to move Round_Spec.enso back to Tests (#7634)
- Closes #7633
- Moves `Round_Spec.enso` from published `Standard.Test` into our `test/Tests` project; the `Table_Tests` that depend on it, simply `import enso_dev.Tests`.
- Changes the layout of the local libraries directory:
- It used to be `root/<namespace>/<name>`.
- Now it is `root/<dir>` - the namespace and name are now read from `package.yaml` instead.
- Adds the parent directory of the current project to the default `ENSO_LIBRARY_PATH`.
- It is treated as a secondary path, so the default `ENSO_HOME/lib` still takes precedence.
- This allows projects to reference and load 'sibling' projects easily - the only requirement is for the project to enable `prefer-local-libraries: true` or add the other local project to its edition. The edition resolution logic is **not changed**.
2023-09-01 20:20:04 +00:00
Dmitry Bushev
eefe74ed93
When opening the project for a second time suggestion database is not sent (#7699)
close #7413

Changelog:
- update: the language server listens for the client disconnection event and invalidates the suggestions index

# Important Notes
The component browser contains suggestion entries after the refresh


https://github.com/enso-org/enso/assets/357683/bcebb8bf-e09f-4fb0-86cf-95ced58413f3
2023-08-31 14:06:58 +00:00
Hubert Plociniczak
96289e0fb3
Add more debugging info to a failed apply edit (#7668)
Still unable to reproduce locally. Adding more info to confirm suspicion that the failure is related to line endings.
Relates to #7624.
2023-08-28 14:45:45 +00:00
Dmitry Bushev
8860ed71fe
Fix dev version check (#7640)
When building the dev IDE version with `./run ide build`, it creates the engine distribution with version `2023.2.1-dev`. This causes the version check to fail preventing opening the projects.
2023-08-25 10:48:24 +00:00
somebody1234
aa2f72cbd0
Allow configuring Project Manager server host and port (#6774)
- Closes #6730
- Changes config to allow environment variables to override server host and port
- Adds port scanning to Electron app to ensure the PM is started at a free port

# Important Notes
- `SERVER_PORT=abcd enso.AppImage` does NOT work. It would not be difficult to implement, but it probably needs discussion on how exactly it should be implemented - for example, `SERVER_PORT` is quite a generic name, should the Electron app pass though something like `ENSO_PM_SERVER_PORT` to the PM as `SERVER_PORT` instead?

⚠️ Port scanning is *only* implemented in the JS frontend. It is not implemented:
- In Scala, because the JS/Rust code calling it needs to know the port as well. There shouldn't be any problems with adding port scanning though, if that's desired
- In Rust, because I'm not sure parsing the host and port from a string is a good idea.
- (This also applies to JS, but it *must* work in JS, and port scanning is already a dependency there so it's quite a bit easier)

- QA *will* need a new PM (`sbt buildProjectManagerDistribution` or `./run backend sbt` -> `buildProjectManagerDistribution`), and the path must be supplied as: `-engine.project-manager-path=path/to/new/pm/here`
2023-08-25 07:33:31 +00:00
Dmitry Bushev
c80b8624a8
Log project manager version on startup (#7646)
Looks like that
```
[info] [2023-08-23T13:12:58.119Z] [org.enso.projectmanager.boot.ProjectManager$] Starting Enso Project Manager
Version:    0.0.0-dev
Built with: scala-2.13.11 for GraalVM 17.0.7
Built from: wip/db/fix-dev-version-check* @ 52bc6b8fcf
Built on:   Linux (amd64)
```
2023-08-24 16:45:22 +00:00
Pavel Marek
c32bfad2b1
Add std-libs benchmarks GH workflow (#7597)
Adds "Benchmark Standard Libraries" workflow that can be run locally with `sbt std-benchmarks/bench`.
2023-08-23 12:18:36 +02:00
Dmitry Bushev
5dc2c4c5fd
Notification about the project rename action (#7613)
close #7604

After moving the rename action to the dashboard, IDE is unaware of the new project name. PR implements a new `refactoring/projectRenamed` notification that is sent from the server to clients and informs them about the changed project name.

# Important Notes
https://github.com/enso-org/enso/assets/357683/7c62726d-217e-4e69-8e48-568e0b7b8c34
2023-08-22 11:32:46 +00:00
Pavel Marek
a0086bb112
Ability to invoke all std benchmarks via jmh (#7519)
All the Enso benchmarks in `test/Benchmarks` can be invoked via JMH
2023-08-17 14:48:43 +02:00
GregoryTravis
c9d7c5cb2b
Convert in-memory Column.round to Java (#7521) 2023-08-16 14:45:23 +00:00
Jaroslav Tulach
7a272ec152
Encapsulating array-like data and operations into a single package (#7544) 2023-08-15 13:00:47 +02:00
GregoryTravis
d3436fae70
Implement Number.round as a builtin (#7460) 2023-08-14 15:43:39 +00:00
Adam Obuchowicz
fd972f8b0a
Change shortcut for opening Component Browser to enter (#7527)
This was meant to be a trivial change, but actually a dirty workaround needed to be applied. Because enter is used  to both open searcher and accept input/entry, both actions were fired at once. I fixed it by debouncing opening searcher event (so the searcher will be opened only once key event handling is over)
2023-08-11 18:05:17 +00:00
Dmitry Bushev
b7ab6911ff
Renaming Variable or Function Support (#7515)
close #7389



Changelog:
- add: `refactoring/renameSymbol` request to rename locals or module methods
2023-08-10 21:16:33 +00:00
Pavel Marek
52b1189212
Builtin methods handle some exceptions (#7494)
MethodProcessor generates code for builtin method invocation that is wrapped in `try-catch` and handles some predefined subset of `RuntimeException`. So far, only `com.oracle.truffle.dsl.api.UnsupportedSpecializationException`.
2023-08-07 16:42:15 +02:00
Pavel Marek
8e49255d92
Invoke all Enso benchmarks via JMH (#7101)
# Important Notes
#### The Plot

- there used to be two kinds of benchmarks: in Java and in Enso
- those in Java got quite a good treatment
- there even are results updated daily: https://enso-org.github.io/engine-benchmark-results/
- the benchmarks written in Enso used to be 2nd class citizen

#### The Revelation
This PR has the potential to fix it all!
- It designs new [Bench API](88fd6fb988) ready for non-batch execution
- It allows for _single benchmark in a dedicated JVM_ execution
- It provides a simple way to wrap such an Enso benchmark as a Java benchmark
- thus the results of Enso and Java benchmarks are [now unified](https://github.com/enso-org/enso/pull/7101#discussion_r1257504440)

Long live _single benchmarking infrastructure for Java and Enso_!
2023-08-07 12:39:01 +00:00
Radosław Waśko
c61c741476
Respect database backend naming limitations when generating table/column names and validate user-provided names to avoid silent name clashes; process JDBC warnings reported from backends (#7428)
- Closes #5951
- Ensures any SQL warnings reported by the database through the JDBC driver are processed and forwarded to the user.
- These warnings show issues like the implicit name truncation that this PR is also solving. It's good to make sure they are visible as they can help avoid and understand unexpected problems. They should not show up in most standard workflows.
- Adds simple history to our REPL.
2023-08-03 09:44:27 +00:00
Hubert Plociniczak
ac9d7f7d8f
Add common-polyglot-core-utils to projects' aggregates (#7483)
Lack of it can lead to really cryptic compilation errors.
2023-08-02 16:19:46 +00:00
Dmitry Bushev
8416e19eef
Refactor project normalized name (#7444)
Followup to #7397. When started this refactoring I only changed the config types and forgot to change the API.

Changelog:
- refactor: rename `projectModule` to `projectNormalizedName` in the API

# Important Notes
https://github.com/enso-org/enso/assets/357683/a840dc90-15f8-4b6d-bbfb-571c53cd2f7d
2023-07-31 17:47:28 +00:00
Dmitry Bushev
c629d6078d
Disable file watcher in VcsManagerTest suite (#7421)
We can't really control the timing of file watcher events, which sometimes leads to failures of VCS tests. The PR disables the file watcher in the `VcsManagerTest` suite to make tests more stable.

Changelog:
- add: a `Watcher` and `WatcherFactory` interfaces
- add: a `NoopWatcher` test watcher
- update: disable the file watcher in the `VcsManagerTest` suite
2023-07-31 09:26:20 +00:00
Dmitry Bushev
80c4b1ca06
Allow users to give a project other than Upper_Snake_Case name (#7397)
close #6356

Allow arbitrary names for user projects.

# Important Notes
https://github.com/enso-org/enso/assets/357683/55a3b660-af23-4b09-959b-eac515766788
2023-07-28 13:44:39 +00:00
Dmitry Bushev
980657df08
StopServer timing out on project stopping (#7416)
close #7345

#7254 introduced a delayed shutdown timeout. The `LanguageServerGateway` timeout should include the delayed shutdown time to prevent false timeouts.
2023-07-28 08:00:51 +00:00
GregoryTravis
628a51d8e2
Convert Number.round to Java (#7360) 2023-07-26 12:03:09 +00:00
Jaroslav Tulach
b9a6e35096
Simplifying builtins type system support (#7344) 2023-07-24 13:27:09 +02:00
Jaroslav Tulach
34c63e3b11
Ascribed types & atom_with_hole (#7348) 2023-07-24 07:02:58 +02:00
Pavel Marek
efb39ad91c
Fix GraalVM distribution download on MacOS (#7364)
Follow-up of recent GraalVM update #7176 that fixes downloading of GraalVM for Mac - instead of "darwin", the releases are now named "macos"

# Important Notes
Also re-enables the JDK/GraalVM version check as onLoad hook to the `sbt` process. We used to have that check a long time ago. Provides errors like this one if the `sbt` is run with a different JVM version:
```
[error] GraalVM version mismatch - you are running Oracle GraalVM 20.0.1+9.1 but GraalVM 17.0.7 is expected.
[error] GraalVM version check failed.
```
2023-07-23 09:21:21 +00:00
Pavel Marek
cab6968340
Update GraalVM to 17.0.7 (23.0.0 JDK17) (#7176)
- Previous GraalVM update: https://github.com/enso-org/enso/pull/6750

Removed warnings:
- Remove deprecated `ConditionProfile.createCountingProfile()`.
- Add `@Shared` to some `@Cached` parameters (Truffle now emits warnings about potential `@Share` usage).
- Specialization method names should not start with execute
- Add limit attribute to some specialization methods
- Add `@NeverDefault` for some cached initializer expressions
- Add `@Idempotent` or `@NonIdempotent` where appropriate

BigInteger and potential Node inlining are tracked in follow-up issues.

# Important Notes
For `SDKMan` users:
```
sdk install java 17.0.7-graalce
sdk use java 17.0.7-graalce
```

For other users - download link can be found at https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-17.0.7

Release notes: https://www.graalvm.org/release-notes/JDK_17/

R component was dropped from the release 23.0.0, only `python` is available to install via `gu install python`.
2023-07-20 15:11:30 +00:00
Dmitry Bushev
251f00ec1c
Allow external edits of project files (#7322)
close #7178

Changelog
- add: ZIO initialization event
- refactor: move file watcher logic to a separate project
- update: buffer manager listens for file events, and triggers reload
- update: text operations tests

# Important Notes
https://github.com/enso-org/enso/assets/357683/7cde18e6-4b3b-42f2-a1cb-911eec2cde6d
2023-07-19 19:22:08 +00:00
Hubert Plociniczak
9252eb5f3a
Delay LS shutdown when last client disconnects (#7254)
* Delay LS shutdown when last client disconnects

Rather than closing Language Server immediately, we delay the shutdown
until some timeout hits. This gives a chance for new clients to connect
without paying the price of the initialization again.

More importantly, during hibernation/restart, the connection between
client (IDE) and LS is severed so it appears as if client disconnect. In
fact a few moments later IDE would attempt to re-establish the
connection on the same port. Without this change, LS shutsdown and
further attempts to connect on that particular port will fail.

There are still problems on the IDE-side after waking up from
hibernation but it is not related to Language Server.

* Introduce a separate timeout for delayed shutdown

Can't/shouldn't use the same timeout value as for shutdown timeout for
delaying shutdowns initiated by lack of clients.

* Add test demonstrating the new functionality
2023-07-18 08:37:11 +02:00
Jaroslav Tulach
6a6d7dbff3
Simplify onboarding instructions for engine & project-manager developers (#7181)
The current instructions to _build, use and debug_ `project-manager` and its engine/ls process are complicated and require a lot of symlinks to properly point to each other. This pull requests simplifies all of that by introduction of `ENSO_ENGINE_PATH` and `ENSO_JVM_PATH` environment variables. Then it hides all the complexity behind a simple _sbt command_: `runProjectManagerDistribution --debug`.

# Important Notes
I decided to tackle this problem as I have three repositories with different branches of Enso and switching between them requires me to mangle the symlinks. I hope I will not need to do that anymore with the introduction of the `runProjectManagerDistribution` command.
2023-07-17 17:38:54 +00:00
Pavel Marek
7264d81f2a
Builtin methods can support array-like arguments (#7235)
This PR modifies the builtin method processor such that it forbids arrays of non-primitive and non-guest objects in builtin methods. And provides a proper implementation for the builtin methods in `EnsoFile`.

- Remove last `to_array` calls from `File.enso`
2023-07-17 09:17:39 +02:00
Dmitry Bushev
f691713077
Add endpoint for downloading a project (#7291)
close #7194

Changelog:

- add: `/projects/{project_id}/enso_project` HTTP endpoint returning an `.enso-project` archive structure
- update: archive enso project to a `.enso-project` `.tar.gz` archive
- update: make project `path` a required field
2023-07-14 13:18:08 +00:00
Dmitry Bushev
4864e5b51d
Add fileModifiedOnDisk notification (#7239)
part of #7178

Changelog:
- add: `text/fileModifiedOnDisk` notification
- update: during the auto-save, check if the file is modified on disk and send the notification. I.e. auto-save does not overwrite the file if it was changed on disk (but the save command does)
- update: IDE handles the file-modified-on-disk notification and reloads the module from disk

# Important Notes
Currently, the auto-save (and the check that the file is modified on disk) is triggered only after the file was edited. The proper check (using the file-watcher service) will be added in the next PR

https://github.com/enso-org/enso/assets/357683/ff91f3e6-2f7a-4c01-a745-98cb140e1964
2023-07-10 16:50:14 +00:00
Hubert Plociniczak
dd06c59380
Initialize Language Server's JSON RPC protocol asynchronously (#7232)
As discovered in #7224, Json RPC protocol was added to the asynchronous resource initialization stage, as part of #6306, but was not in fact initialized at that point.
Instead it was initialized when the server was started to be able to serve correctly the initialization messages. A classic Catch-22. It was really hard to discover this just by looking at the code, but the profiling clearly showed where the time was spent.

This change splits Language Server's protocol into two:
- the first one accepts `heartbeat/init` and `session/initProtocolConnection`
- the second one enriches it with the full set of supported messages

This shifts the initialization from blocking for 0.5 sec to only ~30ms, and performing the second stage asynchronously.

Closes #7224.

# Important Notes
Before the change (blocking server startup):
![Screenshot from 2023-07-05 18-53-24](https://github.com/enso-org/enso/assets/292128/bcfa9043-d00a-4b36-a44c-782a388a16b9)
![Screenshot from 2023-07-05 18-53-10](https://github.com/enso-org/enso/assets/292128/54927787-4c95-46db-bd68-f3a3b82367d5)

After the change (1st stage):
![Screenshot from 2023-07-06 14-02-34](https://github.com/enso-org/enso/assets/292128/d7a7bc34-39dc-46f1-9e64-6d350697c30b)

After the change (2nd, asynchronous initialization, stage):
![Screenshot from 2023-07-06 14-21-17](https://github.com/enso-org/enso/assets/292128/def8c0a1-f211-4fc0-9df0-7c1634312166)
2023-07-07 11:49:23 +00:00
Hubert Plociniczak
72a6c54dfc
Update context on timeout retry (#7207)
Minor follow up on #7173. Noticed while testing nightly on Windows machine.
2023-07-05 12:22:55 +00:00
Hubert Plociniczak
414b124123
Add timeout retries for project-related operations (#7173)
It is relatively easy to reach timeouts on weak systems for startup
operations on project manager. Once a timeout is reached, startup will
not proceed any further.

This PR is a bit of an experiment. It adds adds timeout retries to give a bit of a leeway to
under-powered machines and to log some progress on the way, so that we
know that certain actions are still in progress.
2023-07-03 12:44:41 +00:00
Hubert Plociniczak
3c93c25a5a
Reload project's config on rename (#7179)
Package's config information, once loaded, never changed. While there is typically no need for it, this was problematic when the config became out-of-sync with the filesystem, like in the case of project rename action.
In rename, the config's properties would be updated in the FS, but that would never be reflected in module's package. Therefore further compilations would continue to ask for the old namespace.

Most of the changes are cosmetic (s/`.config`/`.getConfig()`) except for the new `reloadConfig` method on `Package` that is being called in `RenameProjectCmd` handler.

Closes #7062.

# Important Notes
The reported `ExecutionFailed` error should have been mostly fixed already via #7143. This change makes sure that all the related warnings are gone as well and the compiler uses the updated namespace.
2023-07-03 09:36:19 +00:00
Hubert Plociniczak
b78a7e74e8
Provide human-readable error log during installation (#7159)
Windows insists on reporting cryptic error codes that are then hard to debug.
When identified, we can provide more cases.

As identified for #7150.
2023-07-03 08:58:23 +00:00
Hubert Plociniczak
26bd95cf3d
Don't report EOFException on shutdown (#7157)
It's part of the normal shutdown workflow and confused users are reporting it as a failure.
2023-06-29 13:42:13 +00:00
Dmitry Bushev
22259e696d
Add method call info for infix operators (#7090)
close #6374

In order to provide the method pointer information, the `IrToTruffle` pass sets the module name and the type name of the builtin node.
2023-06-27 13:11:12 +00:00
Jaroslav Tulach
c4f19e7d66
Vector.map & State (#7078) 2023-06-26 13:13:41 +02:00
Jaroslav Tulach
6abcbcb00c
Moving construction of IR into runtime parser project (#7096) 2023-06-24 06:34:21 +02:00
Hubert Plociniczak
7955bec129
Add retries to GraalVM updater commands (#7079)
The change adds logic that will attempt a few retries when executing `gu` (GraalVM updater) commands. Previously, if it failed, it failed. Retries should help with the most common case - occassional network hiccups.

Closes #6880.

# Important Notes
Note that I don't use an external library for retries on purpose. Didn't want to introduce a yet another dependency for this tiny functionality.
2023-06-23 18:25:06 +00:00
Jaroslav Tulach
5f572d0680
Avoid dependency on BindingsMap from IR (#7055) 2023-06-19 10:28:31 +02:00
James Dunkerley
760fb71798
First part of AWS S3 API, various small fixes. (#6973)
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
2023-06-15 16:20:13 +00:00
Dmitry Bushev
48f0c6f5e8
Scala 2.13.11 and libraries update (#7010)
Update Scala and libraries.
2023-06-14 13:15:57 +00:00
Dmitry Bushev
6249c79ffd
Update sbt-java-formatter plugin (#7011)
Update java formatter plugin. The new version can remove unused imports.
2023-06-12 14:18:48 +00:00
Dmitry Bushev
6de2700209
Do not return getters from global completion response (#7007)
close #6936

Changelog:
- add: new suggestion type Getter that is not exposed to the api
- update: do not return suggestion of type getter when doing a global search (without specifying self types)

Private suggestions and modules mentioned in the issue will be filtered out after we finish the work on the new (refined) exports algorithm.

# Important Notes
![2023-06-09-205327_1088x612_scrot](https://github.com/enso-org/enso/assets/357683/c6b16894-ada0-4ea9-abe8-5efc41949787)
2023-06-12 12:28:35 +00:00
Dmitry Bushev
4e9f02258e
Suggestion should contain a list of annotations (#6924)
close #6847

Add annotation names to suggestions.
2023-06-02 07:26:01 +00:00
Jaroslav Tulach
65958cba92
IGV Enso support recognizes lib/scala/pkg as project (#6904)
Addition to #4098 - make sure IGV Enso support can recognize `lib/scala/pkg` as a project.
2023-05-31 15:14:57 +00:00
Hubert Plociniczak
efda2cdf31
Empty editions are not parsed as NaN (#6873)
Empty edition (null value) was parsed as NaN, which was confusing. This change correctly detects the case before trying different fallback mechanisms.

Addresses invalid warning mentioned in #6806.
2023-05-30 10:12:39 +00:00
Hubert Plociniczak
86432b5ca5
Detect and override hooks of the same kind (#6842)
This change ensures that we can have at most one hook of the same action during shutdown.
Verified the change on a real project.

Closes #6767.
2023-05-29 07:25:30 +00:00
somebody1234
6693bdb5cd
Fallback to opened date when ordering projects (#6814)
Fixes #6787

# Important Notes
I can't get Project Manager compilation to work locally so I guess I'll be relying on CI to verify that it's working correctly?

Of course, QA should be able to catch any problems too - the websocket API hasn't been changed so it should work out of the box with the current dashboard.
2023-05-25 15:44:15 +00:00
Pavel Marek
bd70ed6830
Update GraalVM to 22.3.1 JDK17 (#6750)
Upgrade GraalVM to 22.3.2 based on JDK17.

---------

Co-authored-by: somebody1234 <ehern.lee@gmail.com>
2023-05-24 10:39:08 +02:00
Dmitry Bushev
1ddd36308e
Filter suggestion constructors when searching by self type (#6813)
close #6627

Changelog:
- update: search method excludes constructors when searched by the self type

# Important Notes
Component Browser doesn't show the `Between` constructor when called on the `Range` node.

![2023-05-23-172446_1203x865_scrot](https://github.com/enso-org/enso/assets/357683/a2553c46-6a0e-4842-ba91-7c2b76c56c6d)
2023-05-23 16:14:41 +00:00
Dmitry Bushev
0318880990
Add project creation time to project metadata (#6780)
close #6686

Changelog:
- add: `created` field to the project metadata response
- update: project manager tests
2023-05-22 13:40:10 +00:00
Radosław Waśko
447786a304
Implement cast for Table and Column (#6711)
Closes #6112
2023-05-19 10:00:20 +00:00
Dmitry Bushev
b7d51ed5c6
Create unique atom getter suggestions (#6694)
related #6611

Prevent creating atom getters with the same name.
2023-05-15 14:03:18 +00:00
Hubert Plociniczak
06624f34a8
Ensure slow shutdown of LS always kicks off hooks (#6665)
This change fixes the rather elusive bug where shutdown hooks could not be fired when shutdown was taking too long and termination was forced.

Under the circumstances described in detail in ticket #6515 there was a small chance that we could have a shutdown race condition. Essentially the messages received when client was disconnected and language server forced the termination could lead to language server not sending the public `ProjectClosed` message which triggers shutdown hook. Now we always do.

Also made sure that multiple `ProjectClosed` messages don't lead to firing multiple shutdown hooks, which was another possibility.

No tests as one would have to be able to introduce different delays in various message handlers to simulate the problem.
Having ability to do such chaos testing would be nice but it is beyond the scope of this ticket.
I was able to reproduce the problem 100% with my specially crafted setup so I'm fairly confident about the change.

Closes #6515.
2023-05-15 12:15:37 +00:00
Dmitry Bushev
706791779b
SuggestionBuilder needs to send ascribedType of constructor parameters (#6655)
close #6611

Changelog:
- update: run compiler passes on the `ascribedType` field of the constructor arguments
- update: suggestion builder uses the type information attached to `ascribedType`
- feat: resolve qualified names in type signatures
2023-05-13 18:33:03 +00:00
Jaroslav Tulach
1302b693d8
All Vector operations shall be applicable on java.util.ArrayList (#6642)
Fixes #6609 by
- e380e647af - running whole `Vector_Spec` on `java.util.ArrayList`
- 9b1229fe20 - introducing a node to handle interop values

# Important Notes
Contains additional DSL processor fix:
- 415623dcb9 - to not crash the compiler, but to properly report compiler error
2023-05-11 15:29:47 +00:00
Dmitry Bushev
3dd05c2359
Automatic type based dropdown does not include singleton in a union type (#6629)
close #6532

Set tag values to display `Auto` type in dropdowns correctly.

# Important Notes
![2023-05-10-141458_969x566_scrot](https://github.com/enso-org/enso/assets/357683/9a048b3c-d192-4382-bf76-9cbe6c9556d1)
![2023-05-10-141513_991x232_scrot](https://github.com/enso-org/enso/assets/357683/c50e1e73-23a6-4b32-90bf-f849a127c85d)
2023-05-10 16:55:16 +00:00
Hubert Plociniczak
cf3624c463
Limit the number of reported warnings (#6577)
Artifically limiting the number of reported warnings to 100. Also added benchmarks with random Ints to investigate perf issues when dealing with warnings (future task).
Ideally we would have a custom set-like collection that allows us internally to specify a maximal number of elements. But `EnsoHashMap` (and potentially `EnsoSet`) are still WIP when it comes to being PE-friendly.

The change also allows for checking if the limit for the number of reported warnings has been reached. It will visualize by adding an additional "Warnings limit reached." to the visualization.

The limit is configurable via `--warnings-limit` parameter to `run`.

Closes #6283.
2023-05-10 11:48:31 +00:00
Radosław Waśko
41a8257e8d
Separating Redshift connector from Database library into a new AWS library (#6550)
Related to #5777
2023-05-04 17:36:51 +00:00
Hubert Plociniczak
b93edb050b
Force pending saves if client closes abruptly (#6514)
Force pending saves when JsonSession is terminated for the client.

Potentially closes #6395.
2023-05-03 23:30:20 +00:00
Dmitry Bushev
38212786a3
Fix application config (#6513)
Related #6482
2023-05-02 17:59:44 +00:00
Pavel Marek
3a42d0ce76
Reimplement enso_project as a proper builtin (#6352)
Remove the magical code generation of `enso_project` method from codegen phase and reimplement it as a proper builtin method.

The old behavior of `enso_project` was special, and violated the language semantics (regarding the `self` argument):
- It was implicitly declared in every module, so it could be called without a self argument.
- It can be called with explicit module as self argument, e.g. `Base.enso_project`, or `Visualizations.enso_project`.

Let's avoid implicit methods on modules and let's be explicit. Let's reimplement the `enso_project` as a builtin method. To comply with the language semantics, we will have to change the signature a bit:
- `enso_project` is a static method in the `Standard.Base.Meta.Enso_Project` module.
- It takes an optional `project` argument (instead of taking it as an explicit self argument).

Having the `enso_project` defined as a (shadowed) builtin method, we will automatically have suggestions created for it.

# Important Notes
- Truffle nodes are no longer generated in codegen phase for the `enso_project` method. It is a standard builtin now.
- The minimal import to use `enso_project` is now `from Standard.Base.Meta.Enso_Project import enso_project`.
- Tested implicitly by `org.enso.compiler.ExecCompilerTest#testInvalidEnsoProjectRef`.
2023-05-02 16:40:58 +00:00
Hubert Plociniczak
ce4ecc6706
Limit Dead Letter logging (#6482)
Dead Letter logging is occasionally flooding our logs which is confusing to users reporting bugs. Left the possibility of a single report so that we know that something is happening.
2023-05-02 14:00:50 +00:00
Jaroslav Tulach
efe904cd9f
Introducing @BuiltinMethod.needsFrame and InlineableNode (#6442)
Fixes #6416 by introducing `InlineableNode`. It runs fast even on GraalVM CE, fixes ([forever broken](https://github.com/enso-org/enso/pull/6442#discussion_r1178782635)) `Debug.eval` with `<|` and [removes discouraged subclassing](https://github.com/enso-org/enso/pull/6442#discussion_r1178778968) of `DirectCallNode`. Introduces `@BuiltinMethod.needsFrame` - something that was requested by #6293. Just in this PR the attribute is optional - its implicit value continues to be derived from `VirtualFrame` presence/absence in the builtin method argument list. A lot of methods had to be modified to pass the `VirtualFrame` parameter along to propagate it where needed.
2023-04-28 15:32:13 +00:00
Hubert Plociniczak
c6790f1e9c
Report only unique warnings (#6372)
This change makes sure that reported warnings are unique, based on the value of internal clock tick and ignoring differences in reassignments.

Before:
![Screenshot from 2023-04-20 15-42-55](https://user-images.githubusercontent.com/292128/233415710-925c1045-37c7-49f5-9bc3-bfbfd30270a3.png)
After:
![Screenshot from 2023-04-20 15-27-27](https://user-images.githubusercontent.com/292128/233415807-8cb67bc2-ac37-4db7-924e-ae7619074b5b.png)

On the positive side, no further changes, like in LS, have to be done.


Closes #6257.
2023-04-28 07:16:00 +00:00
Hubert Plociniczak
376415ab17
Mark some tests as (unconditonally) flaky (#6453)
Too many spurious failures on CI now. Will need to be tackled at some
point.
2023-04-27 17:09:46 +02:00
Dmitry Bushev
2d168308af
Better cleanup of project management test suite (#6392)
Address some arbitrary failures of the `ProjectManagementApiSpec` test suite.
2023-04-24 13:11:27 +00:00
Dmitry Bushev
74c0718604
Project folder not renamed after project name change (#6369)
close #6254

Changelog:
- fix: race when the actor system may be stopped before the shutdown hooks are executed
- fix: project management spec
- fix: recover from `readLine` failure during the shutdown
2023-04-20 16:57:37 +00:00
Hubert Plociniczak
4c1e73c864
Don't propagate warnings on suspended arguments (#6345)
* Don't propagate warnings on suspended arguments

In the current implementation, application of arguments with warnings
first extracts warnings, does the application and appends the warnings
to the result.
This process was however too eager if the suspended argument was a
literal (we don't know if it will be executed after all).

The change modifies method processor to take into account the
`@Suspend` annotation and not gather warnings before the application
takes place.

* PR review
2023-04-19 11:55:23 +02:00
Jaroslav Tulach
e5a96b9782
Avoid NullPointerException in --dumpGraphs & test & docs (#6304) 2023-04-18 06:58:06 +02:00
Dmitry Bushev
93b9ca7789
Update the default project template (#6310)
close #6307

Changelog:
- update: the default project template

# Important Notes
[6307-update-the-default-project-template.webm](https://user-images.githubusercontent.com/357683/232487322-8aebc709-30aa-48f3-b86b-f868c83a6e21.webm)
2023-04-17 14:09:08 +00:00
Jaroslav Tulach
a74933d10f
Speed cascade of if statements up (#6255)
Fixes #5709. We have a test and a generic fix that improves inlining of every builtin. Everything seems to be faster.
2023-04-14 13:27:23 +00:00
Dmitry Bushev
b97fc39214
Remove SQL versions repo (#6242)
close #6232

Changelog:
- remove: `SqlVersionsRepo`
- update: `SuggestionsDatabaseModuleUpdateNotification` message removing the version
- update: cleanup versions repo usages in the language server
2023-04-11 19:22:30 +00:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Jaroslav Tulach
4805193428
Text.to_display_text is (shortened) identity (#6174)
Fixes #5971.
2023-04-05 19:53:07 +00:00
Dmitry Bushev
2338e5d8e6
When renaming the project clean old modules instead of updating (#6148)
close #6139
close #6137

When the project is renamed, the engine cleans up affected modules and initiates modules re-indexing to fill the suggestions database with new records. This way it reduces the amount of information stored in the suggestions database and helps implement #6080 optimization.

Changelog:
- remove: rename features from the suggestions database
- update: rename command to initiate modules cleanup and project re-execution
- fix: #6137
2023-03-31 09:40:21 +00:00
Hubert Plociniczak
b3db805f92
More elaborate error message for invalid text edit (#6108)
It is sometimes impossible to figure out the real reason for invalid text edit request. Added a bit of context to failures to narrow down the cause of the failure.

# Important Notes
Should help with diagnosing issues like #6099.
2023-03-28 14:21:50 +00:00
Dmitry Bushev
aa97787603
Update ZIO library (#6072)
close #6069

Changelog:
- update: ZIO major version bump
2023-03-28 07:58:59 +00:00
James Dunkerley
bf2545fa04
Use new common parse method throwing less exceptions. (#6075)
Avoiding exceptions by not using parseBest.

Time now in CLI is 1.15s for 500k rows vs 1.65s in GUI.

CLI:
![image](https://user-images.githubusercontent.com/4699705/227711266-bc005b0d-5011-450f-964b-65dd2e437c2e.png)

GUI:
![image](https://user-images.githubusercontent.com/4699705/227711259-f7ddda29-86c7-4eef-a002-4bf0bda6063f.png)

Added it as a function in the shared library so used by both engine and polyglot.
2023-03-27 11:02:10 +00:00
Dmitry Bushev
4c62dc9061
Search suggestions by static attribute (#6036)
close #5874

Changelog:
- add: `isStatic` parameter to `search/completion` request to search by the `static` suggestion attribute
- update: search non-static suggestions when opening component browser

# Important Notes
Component browser doesn't show `Table.new` and `Table.from_rows` suggestions when a `Table` node is selected.

![2023-03-21-151117_1301x877_scrot](https://user-images.githubusercontent.com/357683/226874291-1ff99994-1bb6-41df-96b4-dc5c5178ba41.png)
2023-03-23 15:02:25 +00:00
Dmitry Bushev
8125d89676
Delay initialization of JSON-RPC protocol (#5997)
close #5881

Changelog
- add: ProtocolFactory object that initializes and returns the protocol object
- update: Add protocol initialization to the initialization component
2023-03-20 11:04:16 +00:00
Dmitry Bushev
b2aa58bf20
Delay writing IR caches (#5957)
close #5911

In interactive mode, perform writing IR caches in the background jobs queue. Background jobs execution is delayed until the first execution is complete.
2023-03-17 22:47:15 +00:00
Dmitry Bushev
265fb5d505
Delay initialization of SQL database (#5970)
close #5875

Move database initialization logic to the initialization component.
2023-03-17 13:38:11 +00:00
Hubert Plociniczak
a6487e5105
Don't log installed engines and runtimes in prod (#5900)
The `logAvailableComponentsForDebugging` will check and install all necessary components of GraalVM for every mentioned version. While not harmful, it adds up to startup time.
Additionally added an option in language server startup to skip installation of GraalVM components. The latter is already performed by project-manager when opening the project and it is unnecessary to do it twice. Due to LS' architecture this configuration has to be passed around via multiple configs.

Finally, skipped the attempt to install Python component on Windows - this is not supported by GraalVM atm.

Closes #5749.

# Important Notes
The impact of this problem could be really felt the more versions of Enso and GraalVM one had since it would go through all of them.
2023-03-16 10:36:55 +00:00
Kaz Wesley
e171fba301
New documentation parser (#5917)
Implement new Enso documentation parser; remove old Scala Enso parser.

Performance: Total time parsing documentation is now ~2ms.

# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2023-03-15 15:43:51 +00:00
Radosław Waśko
263c3ad651
Add a common-polyglot-core-utils project (#5855)
Adds a common project that allows sharing code between the `runtime` and `std-bits`.

Due to classpath separation and the way it is compiled, the classes will be duplicated - we will have one copy for the `runtime` classpath and another copy as a small JAR for `Standard.Base` library.

This is still much better than having the code duplicated - now at least we have a single source of truth for the shared implementations.

Due to the copying we should not expand this project too much, but I encourage to put here any methods that would otherwise require us to copy the code itself.

This may be a good place to put parts of the hashing logic to then allow sharing the logic between the `runtime` and the `MultiValueKey` in the `Table` library (cc: @Akirathan).
2023-03-11 09:27:26 +00:00
Dmitry Bushev
725841467f
Log exceptions with stack traces (#5871)
close #5070

Changelog:
- Include the original exception to log expressions
- Enable logging of Akka Actors' lifecycle events on debug logging level
- Decrease the severity of interruption log messages because interruptions are part of the workflow. The computation can be interrupted at any time, and still be recomputed after. Warnings are just misleading in this case.
2023-03-11 08:23:31 +00:00
Jaroslav Tulach
632a303089
Use SimpleFormatter.formatMessage to replace {0} with actual parameters (#5870)
Fixes #5801 to properly format Truffle log records before sending them for further processing.
2023-03-11 00:15:58 +00:00
Hubert Plociniczak
6769ab0ee7
Downgrade hashing to SHA-1 and other optimizations (#5791)
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.

Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.

Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.

Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.

Finally fix an SBT warning when generating library cache.

Closes https://github.com/enso-org/enso/issues/5763

# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
2023-03-09 07:36:59 +00:00
Dmitry Bushev
9397a6ec2f
Pre compute suggestion db during build time (#5698)
Close #5068

Cache suggestions during the `buildEngineDistribution` command, and read them from the disk when the library is loaded. Initial graph coloring takes ~20 seconds vs ~25 seconds on the develop branch.

[peek-develop-branch.webm](https://user-images.githubusercontent.com/357683/223504462-e7d48262-4f5e-4724-b2b0-2cb97fc05140.webm)
[peek-suggestions-branch.webm](https://user-images.githubusercontent.com/357683/223504464-0fe86c04-8c4b-443c-ba96-6c5e2fb1e396.webm)
2023-03-08 12:37:48 +00:00
Hubert Plociniczak
941512e0ba
Optimize import/export resolution (#5700)
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.

When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.

The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.

Part of https://github.com/enso-org/enso/issues/5568 work.
2023-03-01 08:53:29 +00:00
James Dunkerley
ba3d45e782
Adjust generateCheckedArgumentRead to return correct expected type (#5769)
Correctly get the expected type and return an IllegalArgument if no Enso type.

![image](https://user-images.githubusercontent.com/4699705/221263862-bbc122ca-b11f-49f3-b7cf-6294ea811a22.png)
2023-02-27 17:23:45 +00:00
Jaroslav Tulach
dfea59c24d
Exception message may not be specified (#5715)
Getting ready for `null` values in `WSLogMessage`. Closes #5683.
2023-02-22 03:43:46 +00:00
Galin Bajlekov
725b3da486
Roll back last VCS snapshot (#4050)
Fixes #5001

This PR implements reverting the current project state to the last state saved into the VCS. This action is performed on `ctrl+r`.

https://user-images.githubusercontent.com/117099775/216645556-1bf34ee7-fdb4-4833-bcad-670d688a3199.mp4

# Important Notes
* Currently on `vcs/restore` all expressions are invalidated and all nodes are re-executed. This is tracked in [task](https://www.pivotaltracker.com/n/projects/2539304/stories/184368950).
2023-02-16 18:14:34 +00:00
Paweł Buchowski
f316ba0136
add missing templates to allTemplates seq (#5657)
Add recently added templates to allTemplates seq so they are accessed from the command line
2023-02-14 17:17:28 +00:00
Paweł Buchowski
95c66baa3c
extend list of allowed project templates (#5601)
In cloud we want to allow users to create new project from the template. List of templates is a bit outdated and doesn't contain all from the https://github.com/enso-org/project-templates. This PR simply adds missing ones
2023-02-13 13:03:51 +00:00
Hubert Plociniczak
9ea9fd56e6
Fix separate compilation for ConstantsGen (#5630)
A combination of commands triggered separate compilation that only recompiled part of builtins. A mechanism that workaround annotation processor's problems with separate compilation was updated to include changes from https://github.com/enso-org/enso/pull/4111. This was pretty tough to find given the rather unusual circumstances in CI.

# Important Notes
This eliminates the _random_ failures in CI related to separate compilation.
To reproduce a specific set of steps has to be executed:
```
sbt> all buildEngineDistribution engine-runner/assembly runtime/Benchmark/compile language-server/Benchmark/compile searcher/Benchmark/compile
(exit sbt)
sbt> test
```
2023-02-10 22:14:43 +00:00
Hubert Plociniczak
472580df4d
Report type of expressions returning polyglot values (#4111)
Expressions returning polyglot values were not reporting the type of the result because we have to do additional magic that infers the correct Enso type. Since this is exactly what `TypeOfNode` does, I re-used the logic.

Straightforward solution failed in tests because of assertions:
```
[enso] WARNING: Execution of function main failed (Invalid library usage. Cached library must be adopted by a RootNode before it is executed.).
java.lang.AssertionError: Invalid library usage. Cached library must be adopted by a RootNode before it is executed.
```

That is why this PR replaces `ExecutionEventListener` with `ExecutionEventNodeFactory`.

# Important Notes
Usage of `TypeOfNode` for programs that **do not** import stdlib means that we report types that do not involve stdlib e.g.
`Standard.Builtins.Main.Integer` instead of `Standard.Base.Data.Numbers.Integer`. While surprising, this is correct and I would say desirable. While reviewing the code, notice the difference in expectations in our runtime tests.
2023-02-09 01:06:27 +00:00
Jaroslav Tulach
f54464881f
enso4igv can open engine/language-server & co. projects (#4098)
In order to investigate `engine/language-server` project, I need to be able to open its sources in IGV and NetBeans.

# Important Notes
By adding same Java source (this time `package-info.java`) and compiling with our Frgaal compiler the necessary `.enso-sources*` files are generated for `engine/language-server` and then the `enso4igv` plugin can open them and properly understand their compile settings.

![Logical View of language-server project](https://user-images.githubusercontent.com/26887752/215472696-ec9801f3-4692-4bdb-be92-c4d2ab552e60.png)

In addition to that this PR enhances the _"logical view"_ presentation of the project by including all source roots found under `src/*/*`.
2023-01-31 08:40:04 +00:00
Hubert Plociniczak
be91b1e8dd
Create static wrappers for builtin types (#4077)
https://github.com/enso-org/enso/pull/3764 introduced static wrappers for instance methods. Except it had a limitation to only be allowed for types with at least a single constructor.
That excluded builtin types as well which, by default, don't have them. This limitation is problematic for Array/Vector consolidation and makes builtin types somehow second-citizens.

This change lifts the limitation for builtin types only. Note that we do want to share the implementation of the generated builtin methods. At the same time due to the additional argument we have to adjust the starting index of the arguments.
This change avoids messing with the existing dispatch logic, to avoid unnecessary complexity.

As a result it is now possible to call builtin types' instance methods, statically:
```
arr = Array.new_1 42
Array.length arr
```
That would previously lead to missing method exception in runtime.

# Important Notes
The only exception is `Nothing`. Primarily because it requires `Nothing` to have a proper eigentype (`Nothing.type`) which would messed up a lot of existing logic for no obvious benefit (no more calling of `foo=Nothing` in parameters being one example).
2023-01-30 19:54:51 +00:00
Hubert Plociniczak
c85377f0ac
Eliminate various compiler warnings (#4079)
Slipped through review of https://github.com/enso-org/enso/pull/3862
2023-01-24 22:58:35 +00:00
Marcin Kostrzewa
242bd52942
Unboxed atoms (#3862)
Introduces unboxed (and arity-specialized) storage schemes for Atoms. It results in improvements both in memory consumption and runtime.
Memory wise: instead of using an array, we now use object fields. We also enable unboxing. This cuts a good few pointers in an unboxed object. E.g. a quadruple of integers is now 64 bytes (4x8 bytes for long fields + 16 bytes for layout and constructor pointers + 16 bytes for a class header). It used to be 168 bytes  (4x24 bytes for boxed Longs + 16 bytes for array header + 32 bytes for array contents +  8 bytes for constructor ptr  + 16 bytes for class header), so we're saving 104 bytes a piece. In the least impressive scenarios (all-boxed fields) we're saving 8 bytes per object (saving 16 bytes for array header, using 8 bytes for the new layout field). In the most-benchmarked case (list of longs), we save 32 bytes per cons-cell.
Time wise:
All list-summing benchmarks observe a ~2x speedup. List generation benchmarks get ~25x speedups, probably both due to less GC activity and better allocation characteristics (only allocating one object per Cons, rather than Cons + Object[] for fields). The "map-reverse" family gets a neat 10x speedup (part of the work is reading, which is 2x faster, the other is allocating, which is now 25x faster, we end up with 10x when combined).
2023-01-24 13:03:06 +00:00
Hubert Plociniczak
86eee6199d
Fix TextEdit range (#4069)
Typo: TextEdit's range should refer to the old buffer's range rather than the new one.
2023-01-20 11:31:16 +00:00
Hubert Plociniczak
246755d29b
Avoid IndexOutOfBounds when edits go out of range (#4065)
While doing regular node manipulation in editior, noticed a number of situations when

```
java.lang.IndexOutOfBoundsException: None
at java.base/java.lang.Character.offsetByCodePoints(Character.java:8699)
at java.base/java.lang.String.offsetByCodePoints(String.java:820)
at org.enso.text.buffer.CodePointView$Ops$.drop(CodePointView.scala:98)
at org.enso.text.buffer.CodePointView$Ops$.drop(CodePointView.scala:57)
at org.enso.text.buffer.Node.drop(Tree.scala:218)
at org.enso.text.buffer.Rope.dropWith(Rope.scala:86)
at org.enso.text.buffer.CodePointView.drop(CodePointView.scala:30)
at org.enso.text.editing.RopeTextEditor$.cutOutTail(RopeTextEditor.scala:42)
...
```
would be thrown. Further text edits would simply be rejected requiring a complete restart. I doubt we should propagate
`IndexOutOfBoundsException`. Instead it is safer to just apply the edit to the end of the rope.
2023-01-19 11:43:46 +00:00
Pavel Marek
fcc2163ae3
All Enso objects are hasheable (#3878)
* Hash codes prototype

* Remove Any.hash_code

* Improve caching of hashcode in atoms

* [WIP] Add Hash_Map type

* Implement Any.hash_code builtin for primitives and vectors

* Add some values to ValuesGenerator

* Fix example docs on Time_Zone.new

* [WIP] QuickFix for HashCodeTest before PR #3956 is merged

* Fix hash code contract in HashCodeTest

* Add times and dates values to HashCodeTest

* Fix docs

* Remove hashCodeForMetaInterop specialization

* Introduce snapshoting of HashMapBuilder

* Add unit tests for EnsoHashMap

* Remove duplicate test in Map_Spec.enso

* Hash_Map.to_vector caches result

* Hash_Map_Spec is a copy of Map_Spec

* Implement some methods in Hash_Map

* Add equalsHashMaps specialization to EqualsAnyNode

* get and insert operations are able to work with polyglot values

* Implement rest of Hash_Map API

* Add test that inserts elements with keys with same hash code

* EnsoHashMap.toDisplayString use builder storage directly

* Add separate specialization for host objects in EqualsAnyNode

* Fix specialization for host objects in EqualsAnyNode

* Add polyglot hash map tests

* EconomicMap keeps reference to EqualsNode and HashCodeNode.

Rather than passing these nodes to `get` and `insert` methods.

* HashMapTest run in polyglot context

* Fix containsKey index handling in snapshots

* Remove snapshots field from EnsoHashMapBuilder

* Prepare polyglot hash map handling.

- Hash_Map builtin methods are separate nodes

* Some bug fixes

* Remove ForeignMapWrapper.

We would have to wrap foreign maps in assignments for this to be efficient.

* Improve performance of Hash_Map.get_builtin

Also, if_nothing parameter is suspended

* Remove to_flat_vector.

Interop API requires nested vector (our previous to_vector implementation). Seems that I have misunderstood the docs  the first time I read it.

- to_vector does not sort the vector by keys by default

* Fix polyglot hash maps method dispatch

* Add tests that effectively test hash code implementation.

Via hash map that behaves like a hash set.

* Remove Hashcode_Spec

* Add some polyglot tests

* Add Text.== tests for NFD normalization

* Fix NFD normalization bug in Text.java

* Improve performance of EqualsAnyNode.equalsTexts specialization

* Properly compute hash code for Atom and cache it

* Fix Text specialization in HashCodeAnyNode

* Add Hash_Map_Spec as part of all tests

* Remove HashMapTest.java

Providing all the infrastructure for all the needed Truffle nodes is no longer manageable.

* Remove rest of identityHashCode message implementations

* Replace old Map with Hash_Map

* Add some docs

* Add TruffleBoundaries

* Formatting

* Fix some tests to accept unsorted vector from Map.to_vector

* Delete Map.first and Map.last methods

* Add specialization for big integer hash

* Introduce proper HashCodeTest and EqualsTest.

- Use jUnit theories.
- Call nodes directly

* Fix some specializations for primitives in HashCodeAnyNode

* Fix host object specialization

* Remove Any.hash_code

* Fix import in Map.enso

* Update changelog

* Reformat

* Add truffle boundary to BigInteger.hashCode

* Fix performance of HashCodeTest - initialize DataPoints just once

* Fix MetaIsATest

* Fix ValuesGenerator.textual - Java's char is not Text

* Fix indent in Map_Spec.enso

* Add maps to datapoints in HashCodeTest

* Add specialization for maps in HashCodeAnyNode

* Add multiLevelAtoms to ValuesGenerator

* Provide a workaround for non-linear key inserts

* Fix specializations for double and BigInteger

* Cosmetics

* Add truffle boundaries

* Add allowInlining=true to some truffle boundaries.

Increases performance a lot.

* Increase the size of vectors, and warmup time for Vector.Distinct benchmark

* Various small performance fixes.

* Fix Geo_Spec tests to accept unsorted Map.to_vector

* Implement Map.remove

* FIx Visualization tests to accept unsorted Map.to_vector

* Treat java.util.Properties as Map

* Add truffle boundaries

* Invoke polyglot methods on java.util.Properties

* Ignore python tests if python lang is missing
2023-01-19 10:33:25 +01:00
Dmitry Bushev
b8967b96b9
Fix serialization of NPE in logger (#4055)
Fixes an error when the logger processes NPE:
```
[internal-logger-error] One of the printers failed to write a message: java.lang.NullPointerException
```
2023-01-16 18:38:28 +00:00
Dmitry Bushev
2cd880f43d
Documentation for functions and locals (#4029)
Add documentation for functions and locals to suggestions database.
2023-01-10 16:59:53 +00:00
Dmitry Bushev
cf67ca60d9
Add globs to logger configuration (#4004)
An artifact left from debugging some SQL last week. Just a nice feature to have.
2022-12-22 19:09:21 +00:00
Jaroslav Tulach
7252af6d62
Enso.getMetaObject, Type.isMetaInstance and Meta.is_a consolidation (#3949)
Implements `getMetaObject` and related messages from Truffle interop for Enso values and types. Turns `Meta.is_a` into builtin and re-uses the same functionality.

# Important Notes
Adds `ValueGenerator` testing infrastructure to provide unified access to special Enso values and builtin types that can be reused by other tests, not just `MetaIsATest` and `MetaObjectTest`.
2022-12-22 08:00:06 +00:00
Radosław Waśko
c0c0abe4fe
Add benchmarks comparing ArrayProxy with elements generated ad-hoc with a regular Vector (#3831) 2022-12-21 20:15:39 +00:00
Hubert Plociniczak
49204e92cf
Simplify exception handling for polyglot exceptions (#3981)
This removes the special handling of polyglot exceptions and allows matching on Java exceptions in the same way as for any other types.

`Polyglot_Error`, `Panic.catch_java` and `Panic.catch_primitive` are gone

The change mostly deals with the backslash of removing `Polyglot_Error` and two `Panic` methods.
`Panic.catch` was implemented as a builtin instead of delegating to `Panic.catch_primitive` builtin that is now gone.

This fixes https://www.pivotaltracker.com/story/show/182844611
2022-12-19 19:16:43 +00:00
James Dunkerley
79de5eecd9
Move Create New Project button to top. (#3972)
- Moves the Create button to the top.
- Adjust default project to also bring in Table and Database.
2022-12-12 15:30:36 +00:00
Dmitry Bushev
efb46c69dc
Fix adding constructor record to suggestions database (#3966) 2022-12-12 13:48:47 +03:00
Hubert Plociniczak
0855b74875
Vector should preserve warnings (#3938)
* Sequence literal (Vector) should preserve warnings

When Vector was created via a sequence literal, we simply dropped any
associated any warnings associated with it.
This change propagates Warnings during the creation of the Vector.
Ideally, it would be sufficient to propagate warnings from the
individual elements to the underlying storage but doesn't go well with
`Vector.fromArray`.

* update changelog

* Array-like structures preserver warnings

Added a WarningsLibrary that exposes `hasWarnings` and `getWarnings`
messages. That way we can have a single storage that defines how to
extract warnings from an Array and the others just delegate to it.

This simplifies logic added to sequence literals to handle warnings.

* Ensure polyglot method calls are warning-free

Since warnings are no longer automatically extracted from Array-like
structures, we delay the operation until an actual polyglot method call
is performed.

Discovered a bug in `Warning.detach_selected_warnings` which was missing
any usage or tests.

* nits

* Support multi-dimensional Vectors with warnings

* Propagate warnings from case branches

* nit

* Propagate all vector warnings when reading element

Previously, accessing an element of an Array-like structure would only
return warnings of that element or of the structure itself.
Now, accessing an element also returns warnings from all its elements as
well.
2022-12-07 11:10:11 +01:00
Pavel Marek
f5f5bff9a7
Improve undefined method error message on builtin types (#3907)
Improve undefined method error message for builtin types

### Important Notes
- Rename `org.enso.interpreter.runtime.Context` to `org.enso.interpreter.runtime.EnsoContext`.
- Rename `org.enso.interpreter.Language` to `or.enso.interpreter.EnsoLanguage`.
2022-11-30 13:37:17 +01:00
Jaroslav Tulach
f225a962ce
Allow conversion of EnsoBigInteger to double (#3865)
Make sure any Enso number (including `EnsoBigInteger`) can be passed to Java via Truffle interop and used on the Java side.
2022-11-24 10:00:16 +00:00
Dmitry Bushev
52a8c72303
Message handler supervisor (#3881)
Changelog:
- Fix a potential `null` value in diagnostic messages
- Add a supervising strategy to the message handler
2022-11-17 19:02:04 +00:00
Hubert Plociniczak
7b0759f8b3
Don't add module's builtins to the scope of a builtin type (#3791)
It appears that we were always adding builtin methods to the scope of the module and the builtin type that shared the same name.
This resulted in some methods being accidentally available even though they shouldn't.

This change treats differently builtins of types and modules and introduces auto-registration feature for builtins.
By default all builtin methods are registered with a type, unless explicitly defined in the annotation property.
Builtin methods that are auto-registered do not have to be explicitly defined and are registered with the underlying type.
Registration correctly infers the right type, depending whether we deal with static or instance methods.

Builtin methods that are not auto-registered have to be explicitly defined **always**. Modules' builtin methods are the prime example.

# Important Notes
Builtins now carry information whether they are static or not (inferred from the lack of `self` parameter).
They also carry a `autoRegister` property to determine if a builtin method should be automatically registered with the type.
2022-11-16 10:23:52 +00:00
Hubert Plociniczak
85d4337f26
Project save backed by git (#3851)
This change adds support for Version Controlled projects in language server.
Version Control supports operations:
- `init` - initialize VCS for a project
- `save` - commit all changes to the project in VCS
- `restore` - ability to restore project to some past `save`
- `status` - show the status of the project from VCS' perspective
- `list` - show a list of requested saves

# Important Notes
Behind the scenes, Enso's VCS uses git (or rather [jGit](https://www.eclipse.org/jgit/)) but nothing stops us from using a different implementation as long as it conforms to the establish API.
2022-11-14 17:32:39 +00:00
Jaroslav Tulach
ecd1fdc3f8
Caching the grapheme_length of a Text (#3864)
Computing length of a text takes time. Let's cache it after first computation.

# Important Notes
Wrote `StringBenchmarks` that sums lengths of (the same) `Text` present many time in a `Vector`. Initially it took `383.673 ms` per operation. Then it took `0.031 ms/op`. Looks like the `length` calls are returning instantly as they get cached.
2022-11-14 15:53:10 +00:00
Dmitry Bushev
14012a751f
Add parent type field to suggestion (#3846)
PR adds `parentType` field to the `Type` suggestion. All Enso types have parent type except `Any`.
2022-11-07 13:21:04 +00:00
Dmitry Bushev
73666df8fa
Cleanup old benchmark (#3845)
Old parser benchmark fails to compile, and it is not checked by CI. I assume it is safe to remove it.
2022-11-03 09:39:06 +00:00
Dmitry Bushev
a6ce49e8a5
Split Atom suggestion entry to Type and Constructor (#3835)
Changelog:
- update: split `Atom` suggestion to `Type` and `Constructor`
- update: gui API
- update: JSONRPC doc
2022-11-02 09:53:40 +00:00
Marcin Kostrzewa
901760816c
State rework & IO Contexts (#3828)
1. Changes how we do monadic state – rather than a haskelly solution, we now have an implicit env with mutable data inside. It's better for the JVM. It also opens the possibility to have state ratained on exceptions (previously not possible) – both can now be implemented.
2. Introduces permission check system for IO actions.
2022-10-26 16:22:08 +00:00
Dmitry Bushev
46441ca7a8
Add isStatic method field (#3829)
PR adds `isStatic` field to suggestion. The field is required for Component Browser.
2022-10-26 09:12:45 +00:00
Radosław Waśko
bc09c7b4c2
Remove obsolete Rust parser experiment, superseded by the much more mature new Rust parser integration (#3815)
We've had an old attempt at integrating a Rust parser with our Scala/Java projects. It seems to have been abandoned and is not used anywhere - it is also superseded by the new integration of the Rust parser. I think it was used as an experiment to see how to approach such an integration.

Since it is not used anymore - it make sense to remove it, because it only adds some (slight, but non-zero) maintenance effort. We can always bring it back from git history if necessary.
2022-10-24 14:56:07 +00:00
Pavel Marek
e9260227c4
Duration type is a builtin type (#3759)
- Reimplement the `Duration` type to a built-in type.
- `Duration` is an interop type.
- Allow Enso method dispatch on `Duration` interop coming from different languages.

# Important Notes
- The older `Duration` type should now be split into new `Duration` builtin type and a `Period` type.
- This PR does not implement `Period` type, so all the `Period`-related functionality is currently not working, e.g., `Date - Period`.
- This PR removes `Integer.milliseconds`, `Integer.seconds`, ..., `Integer.years` extension methods.
2022-10-14 18:08:08 +00:00
Paweł Grabarz
ce6267f098
Add replace_text method to In-Memory Table (#3793)
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/183415329
2022-10-14 17:42:29 +02:00
James Dunkerley
a3de3c6128
Use ArraySlice to slice a Vector (#3724)
Use an `ArraySlice` to slice `Vector`.
Avoids memory copying for the slice function.

# Important Notes
| Test | Ref | New |
| --- | --- | --- |
| New Vector | 71.9 | 71.0 |
| Append Single | 26.0 | 27.7 |
| Append Large | 15.1 | 14.9 |
| Sum | 156.4 | 165.8 |
| Drop First 20 and Sum | 171.2 | 165.3 |
| Drop Last 20 and Sum | 170.7 | 163.0 |
| Filter | 76.9 | 76.9 |
| Filter With Index | 166.3 | 168.3 |
| Partition | 278.5 | 273.8 |
| Partition With Index | 392.0 | 393.7 |
| Each | 101.9 | 102.7 |

- Note: the performance of New and Append has got slower from previous tests.
2022-09-23 15:13:16 +00:00
Hubert Plociniczak
096fcfee82
Generate native image for engine-runner (#3638)
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m4.503s
user	0m9.248s
sys	0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m0.176s
user	0m0.042s
sys	0m0.038s
```

# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.

Will need to add the above command to CI.
2022-09-22 14:45:10 +00:00
Hubert Plociniczak
fba5047acc
Improved Vector/Array interop (#3667)
`Vector` type is now a builtin type. This requires a bunch of additional builtin methods for its creation:
- Use `Vector.from_array` to convert any array-like structure into a `Vector` [by copy](f628b28f5f)
- Use (already existing) `Vector.from_polyglot_array` to convert any array-like structure into a `Vector` **without** copying
- Use (already existing) `Vector.fill 1 item` to create a singleton `Vector`

Additional, for pattern matching purposes, we had to implement a `VectorBranchNode`. Use following to match on `x` being an instance of `Vector` type:
```
import Standard.Base.Data.Vector

size = case x of
Vector.Vector -> x.length
_ -> 0
```

Finally, `VectorLiterals` pass that transforms `[1,2,3]` to (roughly)
```
a1 = 1
a2 = 2
a3 = 3
Vector (Array (a1,a2, a3))
```
had to be modified to generate
```
a1 = 1
a2 = 2
a3 = 3
Vector.from_array (Array (a1, a2, a3))
```
instead to accomodate to the API changes. As of 025acaa676 all the known CI checks passes. Let's start the review.

# Important Notes
Matching in `case` statement is currently done via `Vector_Data`. Use:
```
case x of
Vector.Vector_Data -> True
```
until a better alternative is found.
2022-09-13 03:07:17 +00:00
Dmitry Bushev
05a6415ec0
create inmem database (#3690) 2022-09-07 12:59:18 +03:00
Marcin Kostrzewa
4fc6dcced0
Get rid of free-floating atoms. Everything has a type now! (#3671)
This is a step towards the new language spec. The `type` keyword now means something. So we now have
```
type Maybe a
Some (from_some : a)
None
```
as a thing one may write. Also `Some` and `None` are not standalone types now – only `Maybe` is.
This halfway to static methods – we still allow for things like `Number + Number` for backwards compatibility. It will disappear in the next PR.

The concept of a type is now used for method dispatch – with great impact on interpreter code density.

Some APIs in the STDLIB may require re-thinking. I take this is going to be up to the libraries team – some choices are not as good with a semantically different language. I've strived to update stdlib with minimal changes – to make sure it still works as it did.

It is worth mentioning the conflicting constructor name convention I've used: if `Foo` only has one constructor, previously named `Foo`, we now have:
```
type Foo
Foo_Data f1 f2 f3
```

This is now necessary, because we still don't have proper statics. When they arrive, this can be changed (quite easily, with SED) to use them, and figure out the actual convention then.

I have also reworked large parts of the builtins system, because it did not work at all with the new concepts.

It also exposes the type variants in SuggestionBuilder, that was the original tiny PR this was based on.

PS I'm so sorry for the size of this. No idea how this could have been smaller. It's a breaking language change after all.
2022-08-30 22:54:53 +00:00
James Dunkerley
a20d43390e
Adding DateTime part functions (#3669)
- Added `Zone`, `Date_Time` and `Time_Of_Day` to `Standard.Base`.
- Renamed `Zone` to `Time_Zone`.
- Added `century`.
- Added `is_leap_year`.
- Added `length_of_year`.
- Added `length_of_month`.
- Added `quarter`.
- Added `day_of_year`.
- Added `Day_Of_Week` type and `day_of_week` function.
- Updated `week_of_year` to support ISO.

# Important Notes
- Had to pass locale to formatter for date/time tests to work on my PC.
- Changed default of `week_of_year` to use ISO.
2022-08-26 15:47:58 +00:00
Hubert Plociniczak
d87a32d019
Builtin Date_Time, Time_Of_Day, Zone (#3658)
* Builtin Date_Time, Time_Of_Day, Zone

Improved polyglot support for Date_Time (formerly Time), Time_Of_Day and
Zone. This follows the pattern introduced for Enso Date.

Minor caveat - in tests for Date, had to bend a lot for JS Date to pass.
This is because JS Date is not really only a Date, but also a Time and
Timezone, previously we just didn't consider the latter.
Also, JS Date does not deal well with setting timezones so the trick I
used is to first call foreign function returning a polyglot JS Date,
which is converted to ZonedDateTime and only then set the correct
timezone. That way none of the existing tests had to be changes or
special cased.

Additionally, JS deals with milliseconds rather than nanoseconds so
there is loss in precision, as noted in Time_Spec.

* Add tests for Java's LocalTime

* changelog

* Make date formatters in table happy

* PR review, add more tests for zone

* More tests and fixed a bug in column reader

Column reader didn't take into account timezone but that was a mistake
since then it wouldn't map to Enso's Date_Time.
Added tests that check it now.

* remove redundant conversion

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Time.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* First round of addressing PR review

* don't leak java exceptions in Zone

* Move Date_Time to top-level module

* PR review

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2022-08-24 12:31:29 +02:00
Dmitry Bushev
5e114acbb5
Update Scala to 2.13.8 (#3631)
Update Scala compiler and libraries.
2022-08-08 19:32:55 +00:00
Hubert Plociniczak
42dbd8bb59
Allow for importing methods (#3633)
Importing individual methods didn't work as advertised because parser
would allow them but later drop that information. This slipped by because we never had mixed atoms and methods in stdlib.

# Important Notes
Added some basic tests but we need to ensure that the new parser allows for this.
@jdunkerley will be adding some changes to stdlib that will be testing this functionality as well.
2022-08-05 16:25:51 +00:00
Hubert Plociniczak
d59714a29d
Support module imports using a qualified name (#3608)
This change allows for importing modules using a qualified name and deals with any conflicts on the way.
Given a module C defined at `A/B/C.enso` with
```
type C
type C a
```
it is now possible to import it as
```
import project.A
...
val x = A.B.C 10
```

Given a module located at `A/B/C/D.enso`, we will generate
intermediate, synthetic, modules that only import and export the successor module along the path.
For example, the contents of a synthetic module B will look like
```
import <namespace>.<pkg-name>.A.B.C
export <namespace>.<pkg-name>.A.B.C
```
If module B is defined already by the developer, the compiler will _inject_ the above statements to the IR.

Also removed the last elements of some lowercase name resolution that managed to survive recent
changes (`Meta.Enso_Project` would now be ambiguous with `enso_project` method).

Finally, added a pass that detects shadowing of the synthetic module by the type defined along the path.
We print a warning in such a situation.

Related to https://www.pivotaltracker.com/n/projects/2539304

# Important Notes
There was an additional request to fix the annoying problem with `from` imports that would always bring
the module into the scope. The changes in stdlib demonstrate how it is now possible to avoid the workaround of
```
from X.Y.Z as Z_Module import A, B
```
(i.e. `as Z_Module` part is almost always unnecessary).
2022-07-29 14:19:07 +00:00
Dmitry Bushev
693811fc32
Disable npm install in tests on CI (#3620)
Fixes random timeout failures on CI.
```
INFO ide_ci::program::command: sbtℹ️ up to date, audited 98 packages in 3s
```
`npm install` takes 3s of test time doing unnecessary package auditing. On CI the command is executed once before running the tests and redundant `npm install` calls can be omitted.
2022-07-28 10:35:25 +00:00
Hubert Plociniczak
f63e40df1b
Explicit self (#3569)
This change modifies the current language by requiring explicit `self` parameter declaration
for methods. Methods without `self` parameter in the first position should be treated as statics
although that is not yet part of this PR. We add an implicit self to all methods
This obviously required updating the whole stdlib and its components, tests etc but the change
is pretty straightforward in the diff.

Notice that this change **does not** change method dispatch, which was removed in the last changes.
This was done on purpose to simplify the implementation for now. We will likely still remove all
those implicit selfs to bring true statics.
Minor caveat - since `main` doesn't actually need self, already removed that which simplified
a lot of code.
2022-07-27 17:45:36 +00:00
Radosław Waśko
ee91656f30
Remove duplicate Line_Ending_Style and update defaults (#3597)
Implements https://www.pivotaltracker.com/story/show/182749831
2022-07-27 09:43:51 +00:00
Jaroslav Tulach
4465d63dd8
Improved polyglot Date support (#3559)
Significantly improves the polyglot Date support (as introduced by #3374). It enhances the `Date_Spec` to run it in four flavors:
- with Enso Date (as of now)
- with JavaScript Date
- with JavaScript Date wrapped in (JavaScript) array
- with Java LocalDate allocated directly

The code is then improved by necessary modifications to make the `Date_Spec` pass.

# Important Notes
James has requested in [#181755990](https://www.pivotaltracker.com/n/projects/2539304/stories/181755990) - e.g. _Review and improve InMemory Table support for Dates, Times, DateTimes, BigIntegers_ the following program to work:
```
foreign js dateArr = """
return [1, new Date(), 7]

main =
IO.println <| (dateArr.at 1).week_of_year
```
the program works with here in provided changes and prints `27` as of today.

@jdunkerley has provided tests for proper behavior of date in `Table` and `Column`. Those tests are working as of [f16d07e](f16d07e640). One just needs to accept `List<Value>` and then query `Value` for `isDate()` when needed.

Last round of changes is related to **exception handling**. 8b686b12bd makes sure `makePolyglotError` accepts only polyglot values. Then it wraps plain Java exceptions into `WrapPlainException` with `has_type` method - 60da5e70ed - the remaining changes in the PR are only trying to get all tests working in the new setup.

The support for `Time` isn't part of this PR yet.
2022-07-21 06:32:40 +00:00
Hubert Plociniczak
96e50648dd
Remove 'here' and make method name resolution case-sensitive (#3538)
Modified UppercaseNames to now resolve methods without an explicit `here` to point to the current module.
`here` was also often used instead of `self` which was allowed by the compiler.
Therefore UppercaseNames pass is now GlobalNames and does some extra work -
it translated method calls without an explicit target into proper applications.

# Important Notes
There was a long-standing bug in scopes usage when compiling standalone expressions.
This resulted in AliasAnalysis generating incorrect graphs and manifested itself only in unit tests
and when running `eval`, thus being a bit hard to locate.
See `runExpression` for details.

Additionally, method name resolution is now case-sensitive.

Obsolete passes like UndefinedVariables and ModuleThisToHere were removed. All tests have been adapted.
2022-07-07 10:31:06 +00:00
Michał Wawrzyniec Urbańczyk
43a893cae6
Bump the build script (#3535) 2022-07-01 03:58:14 +02:00
Dmitry Bushev
9d76ba9284
Fix GitHub Releases Lookup (#3550)
GitHub CI creates releases without `enso-` prefix, while enso ecosystem uses tags with the prefix.
2022-06-28 14:18:31 +00:00
Hubert Plociniczak
22a371a9c6
Substitute this with self (#3524)
A semi-manual s/this/self appied to the whole standard library.
Related to https://www.pivotaltracker.com/story/show/182328601

In the compiler promoted to use constants instead of hardcoded
`this`/`self` whenever possible.

# Important Notes
The PR **does not** require explicit `self` parameter declaration for methods as this part
of the design is still under consideration.
2022-06-21 10:53:52 +00:00
Jaroslav Tulach
c72a6582bc
Avoid whole source reparsing when the IDE performs a simple edit (#3508)
Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
2022-06-16 16:02:57 +02:00
Hubert Plociniczak
fd46e84e8d
Towards a full-blown builtins DSL (part 3) (#3471)
Auto-generate all builtin methods for builtin `File` type from method signatures.
Similarly, for `ManagedResource` and `Warning`.
Additionally, support for specializations for overloaded and non-overloaded methods is added.
Coverage can be tracked by the number of hard-coded builtin classes that are now deleted.

## Important notes

Notice how `type File` now lacks `prim_file` field and we were able to get rid off all of those
propagating method calls without writing a single builtin node class.
Similarly `ManagedResource` and `Warning` are now builtins and `Prim_Warnings` stub is now gone.
2022-06-13 11:48:34 +00:00
Hubert Plociniczak
b1db359f19
Minor compilation improvements (#3512)
Drop `Core` implementation (replacement for IR) as it (sadly) looks increasingly
unlikely this effort will be continued. Also, it heavily relies
on implicits which increases some compilation time (~1sec from `clean`)

Related to https://www.pivotaltracker.com/story/show/182359029
2022-06-07 14:48:50 +00:00
Hubert Plociniczak
31e3f39c55
Suppress pointless warnings coming from SBT (#3499)
This change introduces a custom LogManager for console that allows for
excluding certain log messages. The primarily reason for introducing
such LogManager/Appender is to stop issuing hundreds of pointless
warnings coming from the analyzing compiler (wrapper around javac) for
classes that are being generated by annotation processors.

The output looks like this:
```
[info] Cannot install GraalVM MBean due to Failed to load org.graalvm.nativebridge.jni.JNIExceptionWrapperEntryPoints
[info] compiling 129 Scala sources and 395 Java sources to /home/hubert/work/repos/enso/enso/engine/runtime/target/scala-2.13/classes ...
[warn] Unexpected javac output: warning: File for type 'org.enso.interpreter.runtime.type.ConstantsGen' created in the last round will not be subject to annotation processing.
[warn] 1 warning.
[info] [Use -Dgraal.LogFile=<path> to redirect Graal log output to a file.]
[info] Cannot install GraalVM MBean due to Failed to load org.graalvm.nativebridge.jni.JNIExceptionWrapperEntryPoints
[info] foojavac Filer
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.decimal.CeilMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.resource.TakeNodeGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.error.ThrowErrorMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.smallInteger.MultiplyMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.warning.GetWarningsNodeGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.smallInteger.BitAndMethodGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.error.ErrorToTextNodeGen
[warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.warning.GetValueMethodGen
[warn] Could not determine source for class org.enso.interpreter.runtime.callable.atom.AtomGen$MethodDispatchLibraryExports$Cached
....
```

The output now has over 500 of those and there will be more. Much more
(generated by our and Truffle processors).
There is no way to tell SBT that those are OK. One could potentially
think of splitting compilation into 3 stages (Java processors, Java and
Scala) but that will already complicate the non-trivial build definition
and we may still end up with the initial problem.
This is a fix to make it possible to get reasonable feedback from
compilation without scrolling mutliple screens *every single time*.

Also fixed a spurious warning in javac processor complaining about
creating files in the last round.

Related to https://www.pivotaltracker.com/story/show/182138198
2022-06-01 13:50:46 +00:00
Hubert Plociniczak
4918ccb5a3
Make sure formatting is applied to std-bits projects (#3477)
@radeusgd discovered that no formatting was being applied to std-bits projects.
This was caused by the fact that `enso` project didn't aggregate them. Compilation and
packaging still worked because one relied on the output of some tasks but
```
sbt> javafmtAll
```
didn't apply it to `std-bits`.

# Important Notes
Apart from `build.sbt` no manual changes were made.
2022-05-25 09:26:50 +00:00
Dmitry Bushev
f9d2964e83
Update profiling CLI arguments (#3461) 2022-05-24 16:01:26 +03:00
Michał Wawrzyniec Urbańczyk
14a01c4635
New IDE build script (#3466) 2022-05-23 04:16:04 +02:00
Radosław Waśko
0073f461d9
Fix Dataflow Error propagation for Builtins accepting primitives (#3400)
[ci no changelog needed]

Fixes https://www.pivotaltracker.com/story/show/181652841
2022-05-19 15:25:30 +00:00
Hubert Plociniczak
688df9825c
Part 2 of system for builtin objects (#3454)
This is the 2nd part of DSL improvements that allow us to generate a lot of
builtins-related boilerplate code.
- [x] generate multiple method nodes for methods/constructors with varargs
- [x] expanded processing to allow for @Builtin to be added to classes and
and generate @BuiltinType classes
- [x] generate code that wraps exceptions to panic via `wrapException`
annotation element (see @Builtin.WrapException`

Also rewrote @Builtin annotations to be more structured and introduced some nesting, such as
@Builtin.Method or @Builtin.WrapException.

This is part of incremental work and a follow up on https://github.com/enso-org/enso/pull/3444.

# Important Notes
Notice the number of boilerplate classes removed to see the impact.
For now only applied to `Array` but should be applicable to other types.
2022-05-19 10:43:47 +00:00
Dmitry Bushev
d74fffd550
Replace Rope with CharSequence when building suggestions (#3453)
PR addresses an issue when during the initial compilation the `EnsureCompiledJob` depends too much time building the `LineView` of ropes.
Replacing the `Rope` with `CharSequence` in the compilation job reduces total time building the suggestions from ~600 ms to ~200 ms.

#### Before
![2022-05-13-160549_1310x120_scrot](https://user-images.githubusercontent.com/357683/168289736-fe8983ef-9bcc-4df0-bcd2-881bc2949773.png)
#### After
![2022-05-13-160609_1303x123_scrot](https://user-images.githubusercontent.com/357683/168289741-3326603d-8183-4925-b995-435655d6c8be.png)

[context-registry-npss.zip](https://github.com/enso-org/enso/files/8687294/context-registry-npss.zip)
2022-05-16 06:47:35 +00:00
Hubert Plociniczak
a2dae60aa9
Generate BuiltinMethods from simple method and constructor signatures (#3444)
A low-hanging fruit where we can automate the generation of many
@BuiltinMethod nodes simply from the runtime's methods signatures.
This change introduces another annotation, @Builtin, to distinguish from
@BuiltinType and @BuiltinMethod processing. @Builtin processing will
always be the first stage of processing and its output will be fed to
the latter.

Note that the return type of Array.length() is changed from `int` to
`long` because we probably don't want to add a ton of specializations
for the former (see comparator nodes for details) and it is fine to cast
it in a small number of places.

Progress is visible in the number of deleted hardcoded classes.

This is an incremental step towards #181499077.

# Important Notes
This process does not attempt to cover all cases. Not yet, at least.
We only handle simple methods and constructors (see removed `Array` boilerplate methods).
2022-05-12 08:42:00 +00:00
Jaroslav Tulach
ab1ca54acd
Profile the language server (#3389)
In order to analyse why the `runner.jar` is slow to start, let's _"self sample"_ it using the [sampler library](https://bits.netbeans.org/dev/javadoc/org-netbeans-modules-sampler/org/netbeans/modules/sampler/Sampler.html). As soon as the `Main.main` is launched, the sampling starts and once the server is up, it writes its data into `/tmp/language-server.npss`.

Open the `/tmp/language-server.npss` with [VisualVM](https://visualvm.github.io) - you should have one copy in your
GraalVM `bin/jvisualvm` directory and there has to be a GraalVM to run Enso.

#### Changelog

- add: the `MethodsSampler` that gathers information in `.npss` format
- add: `--profiling` flag that enables the sampler
- add: language server processes the updates in batches
2022-05-10 12:44:05 +00:00
Hubert Plociniczak
4bbabc00be
Move Builtin Types and Methods to stdlib (#3363)
This PR replaces hard-coded `@Builtin_Method` and `@Builtin_Type` nodes in Builtins with an automated solution
that a) collects metadata from such annotations b) generates `BuiltinTypes` c) registers builtin methods with corresponding
constructors.
The main differences are:
1) The owner of the builtin method does not necessarily have to be a builtin type
2) You can now mix regular methods and builtin ones in stdlib 
3) No need to keep track of builtin methods and types in various places and register them by hand (a source of many typos or omissions as it found during the process of this PR)

Related to #181497846
Benchmarks also execute within the margin of error.

### Important Notes

The PR got a bit large over time as I was moving various builtin types and finding various corner cases.
Most of the changes however are rather simple c&p from Builtins.enso to the corresponding stdlib module.
Here is the list of the most crucial updates:
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/builtin/Builtins.java` - the core of the changes. We no longer register individual builtin constructors and their methods by hand. Instead, the information about those is read from 2 metadata files generated by annotation processors. When the builtin method is encountered in stdlib, we do not ignore the method. Instead we lookup it up in the list of registered functions (see `getBuiltinFunction` and `IrToTruffle`)
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java` has now information whether it corresponds to the builtin type or not.
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/RuntimeStubsGenerator.scala` - when runtime stubs generator encounters a builtin type, based on the @Builtin_Type annotation, it looks up an existing constructor for it and registers it in the provided scope, rather than creating a new one. The scope of the constructor is also changed to the one coming from stdlib, while ensuring that synthetic methods (for fields) also get assigned correctly
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/IrToTruffle.scala` - when a builtin method is encountered in stdlib we don't generate a new function node for it, instead we look it up in the list of registered builtin methods. Note that Integer and Number present a bit of a challenge because they list a whole bunch of methods that don't have a corresponding method (instead delegating to small/big integer implementations).
During the translation new atom constructors get initialized but we don't want to do it for builtins which have gone through the process earlier, hence the exception
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor not only  generates the actual code fpr nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization 
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor no longer generates only (root) nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/TypeProcessor.java` - Similar to MethodProcessor but handles @Builtin_Type annotations. It doesn't, **yet**, generate any builtin objects.  It also collects the names, as present in stdlib, if any, so that we can generate the names automatically (see generated `types/ConstantsGen.java`)
- `engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin` - various classes annotated with @BuiltinType to ensure that the atom constructor is always properly registered for the builitn. Note that in order to support types fields in those, annotation takes optional `params` parameter (comma separated). 
- `engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala` - drop manual creation of test list which seemed to be a relict of the old design
2022-05-05 20:18:06 +02:00
Hubert Plociniczak
79c82da21c
Frgaal integration in sbt (#3421)
* Initial integration with Frgaal in sbt

Half-working since it chokes on generated classes from annotation
processor.

* Replace AutoService with ServiceProvider

For reasons unknown AutoService would fail to initialize and fail to
generate required builtin method classes.
Hidden error message is not particularly revealing on the reason for
that:
```
[error] error: Bad service configuration file, or exception thrown while constructing Processor object: javax.annotation.processing.Processor: Provider com.google.auto.service.processor.AutoServiceProcessor could not be instantiated
```

The sample records is only to demonstrate that we can now use newer Java
features.

* Cleanup + fix benchmark compilation

Bench requires jmh classes which are not available because we obviously
had to limit `java.base` modules to get Frgaal to work nicely.
For now, we default to good ol' javac for Benchmarks.
Limiting Frgaal to runtime for now, if it plays nicely, we can expand it
to other projects.

* Update CHANGELOG

* Remove dummy record class

* Update licenses

* New line

* PR review

* Update legal review

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2022-05-04 21:18:40 +02:00
Dmitry Bushev
69b5e2ab8a
Fix search of local suggestions (#3417)
Changelog:
- fix: `search/completion` request with the position parameter.
- fix: `refactoring/renameProject` request. Previously it did not take into account the library namespace (e.g. `local.`)
2022-04-26 12:06:48 +00:00
Hubert Plociniczak
059bb8c7e9
Apply automatic formatting prior to turning on checks (#3405)
Result of automatic formatting with `scalafmtAll` and `javafmtAll`.
Prerequisite for https://github.com/enso-org/enso/pull/3394

### Important Notes

This touches a lot of files and might conflict with existing PRs that are in progress. If that's the case, just run
`scalafmtAll` and `javafmtAll` after merge and everything should be in order since formatters should be deterministic.
2022-04-19 12:34:34 +02:00
Dmitry Bushev
998d078b9a
Fill component groups of standard libraries (#3391)
Changelog:
- add: component groups to package descriptions
- add: `executionContext/getComponentGroups` method that returns component groups of libraries that are currently loaded
- doc: cleanup unimplemented undo/redo commands
- refactor: internal component groups datatype
2022-04-14 13:12:18 +00:00
Dmitry Bushev
29e3f05f27
Fix multiline code docparser (#3379)
Changelog:
- fix: docparser handles multiline code sections correctly
- feat: split paragraphs into keyed sections
2022-04-06 04:39:58 +00:00
Dmitry Bushev
23e5216922
Fix docparser (#3370)
Changelog:
- fix the docparser case when there is an unclosed formatted section before a list
- add the test checking the doc generation on the stdlib
2022-03-31 04:44:34 +00:00
Dmitry Bushev
a34c2bcfe6
Implement Documentation Sections API (#3360) 2022-03-25 10:49:37 +03:00
Dmitry Bushev
9d402bd599
Split documentation comment into sections (#3347) 2022-03-21 10:14:25 +03:00
Marcin Kostrzewa
4653bfeeab
Decorate values with arbitrary warnings (#3248) 2022-03-09 16:40:02 +01:00
Dmitry Bushev
d3846578cc
Fix Error Payload Serialization (#3315) 2022-03-04 11:57:49 +03:00
Dmitry Bushev
40f44be858
Update the Language Server API (#3308) 2022-03-03 16:28:04 +03:00
Dmitry Bushev
3858ae7517
Add API for component groups (#3286) 2022-02-24 15:41:14 +03:00
Dmitry Bushev
9010a2c3a3
doc: Range datatype (#3296) 2022-02-22 13:21:24 +03:00
Dmitry Bushev
792c7e3538
Fix dev version check (#3265) 2022-02-15 16:34:33 +01:00
Michał Wawrzyniec Urbańczyk
4baad5f146
Nightly proccess preparations: Setting Enso version through the environment (#3241)
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Radosław Waśko <wasko.radek@gmail.com>
2022-02-07 15:14:32 +01:00
Dmitry Bushev
2d4df96e10
Implement the component resolving algorithm (#3244) 2022-02-03 10:40:39 +03:00
Dmitry Bushev
e6d9b5741d
Add support for the component groups syntax (#3235) 2022-01-26 18:57:50 +03:00
Dmitry Bushev
ca0a6f0bf6
Update Scala to 2.13.7 (#3214) 2021-12-31 17:50:32 +03:00
Dmitry Bushev
86672ab811
Fix cleanup in tests (#3195) 2021-12-16 17:15:28 +03:00
Dmitry Bushev
0b9a5b5f65
Fix path in Windows tests (#3191) 2021-12-14 14:52:38 +03:00
Dmitry Bushev
607330712a
Add support for conversions in Language Server (#3175) 2021-12-03 11:31:44 +03:00
Radosław Waśko
46c31bb9a5
Preinstalling With Dependencies (#1981) 2021-11-23 11:51:17 +03:00
Michał Wawrzyniec Urbańczyk
8fc51bfe44
Apply unified prettier style to engine codebase (#3145) 2021-11-08 16:45:29 +01:00
Dmitry Bushev
83e35751f4
Disable ProjectManagementApiSpec (#3138) 2021-11-08 15:52:28 +01:00
Michał W. Urbańczyk
99053decd8 various default branch switch - related updates 2021-11-01 01:37:30 +01:00
Marcin Kostrzewa
f4823f0ad6
Auto Parallelism (#2000) 2021-09-28 00:48:17 +02:00
Ara Adkins
d6465e9e97
Implement a --compile command for the engine runner (#1998) 2021-09-24 12:24:44 +01:00
Dmitry Bushev
592b016caf
Add log.masking runtime option (#1993) 2021-09-10 10:56:23 +01:00
Radosław Waśko
45c01da490
Make Library-Related Operations Not Block the Actor System (#1975) 2021-08-30 15:51:38 +02:00
Radosław Waśko
826e28a149
Implement library/preinstall (Without Dependencies Yet) (#1972) 2021-08-27 14:01:13 +02:00
Ara Adkins
649fe33ccf
Add support for IR cache locations (#1973) 2021-08-26 11:52:35 +01:00
Ara Adkins
c18fe2d750
Provide regex support on Text (#1968) 2021-08-23 12:09:51 +01:00
Radosław Waśko
8cef409db2
Getting and Setting Library Metadata (#1967) 2021-08-20 10:23:05 +02:00
Radosław Waśko
63819526d7
Update the editions/listDefinedLibraries Endpoint (#1964) 2021-08-19 17:21:31 +02:00
Maciej Mikołajek
32261a180a
Fix a styling issue with docs titles (#1962) 2021-08-19 08:46:47 +01:00