Commit Graph

328 Commits

Author SHA1 Message Date
Dmitry Bushev
251f00ec1c
Allow external edits of project files (#7322)
close #7178

Changelog
- add: ZIO initialization event
- refactor: move file watcher logic to a separate project
- update: buffer manager listens for file events, and triggers reload
- update: text operations tests

# Important Notes
https://github.com/enso-org/enso/assets/357683/7cde18e6-4b3b-42f2-a1cb-911eec2cde6d
2023-07-19 19:22:08 +00:00
Hubert Plociniczak
9252eb5f3a
Delay LS shutdown when last client disconnects (#7254)
* Delay LS shutdown when last client disconnects

Rather than closing Language Server immediately, we delay the shutdown
until some timeout hits. This gives a chance for new clients to connect
without paying the price of the initialization again.

More importantly, during hibernation/restart, the connection between
client (IDE) and LS is severed so it appears as if client disconnect. In
fact a few moments later IDE would attempt to re-establish the
connection on the same port. Without this change, LS shutsdown and
further attempts to connect on that particular port will fail.

There are still problems on the IDE-side after waking up from
hibernation but it is not related to Language Server.

* Introduce a separate timeout for delayed shutdown

Can't/shouldn't use the same timeout value as for shutdown timeout for
delaying shutdowns initiated by lack of clients.

* Add test demonstrating the new functionality
2023-07-18 08:37:11 +02:00
Jaroslav Tulach
6a6d7dbff3
Simplify onboarding instructions for engine & project-manager developers (#7181)
The current instructions to _build, use and debug_ `project-manager` and its engine/ls process are complicated and require a lot of symlinks to properly point to each other. This pull requests simplifies all of that by introduction of `ENSO_ENGINE_PATH` and `ENSO_JVM_PATH` environment variables. Then it hides all the complexity behind a simple _sbt command_: `runProjectManagerDistribution --debug`.

# Important Notes
I decided to tackle this problem as I have three repositories with different branches of Enso and switching between them requires me to mangle the symlinks. I hope I will not need to do that anymore with the introduction of the `runProjectManagerDistribution` command.
2023-07-17 17:38:54 +00:00
Pavel Marek
7264d81f2a
Builtin methods can support array-like arguments (#7235)
This PR modifies the builtin method processor such that it forbids arrays of non-primitive and non-guest objects in builtin methods. And provides a proper implementation for the builtin methods in `EnsoFile`.

- Remove last `to_array` calls from `File.enso`
2023-07-17 09:17:39 +02:00
Dmitry Bushev
f691713077
Add endpoint for downloading a project (#7291)
close #7194

Changelog:

- add: `/projects/{project_id}/enso_project` HTTP endpoint returning an `.enso-project` archive structure
- update: archive enso project to a `.enso-project` `.tar.gz` archive
- update: make project `path` a required field
2023-07-14 13:18:08 +00:00
Dmitry Bushev
4864e5b51d
Add fileModifiedOnDisk notification (#7239)
part of #7178

Changelog:
- add: `text/fileModifiedOnDisk` notification
- update: during the auto-save, check if the file is modified on disk and send the notification. I.e. auto-save does not overwrite the file if it was changed on disk (but the save command does)
- update: IDE handles the file-modified-on-disk notification and reloads the module from disk

# Important Notes
Currently, the auto-save (and the check that the file is modified on disk) is triggered only after the file was edited. The proper check (using the file-watcher service) will be added in the next PR

https://github.com/enso-org/enso/assets/357683/ff91f3e6-2f7a-4c01-a745-98cb140e1964
2023-07-10 16:50:14 +00:00
Hubert Plociniczak
dd06c59380
Initialize Language Server's JSON RPC protocol asynchronously (#7232)
As discovered in #7224, Json RPC protocol was added to the asynchronous resource initialization stage, as part of #6306, but was not in fact initialized at that point.
Instead it was initialized when the server was started to be able to serve correctly the initialization messages. A classic Catch-22. It was really hard to discover this just by looking at the code, but the profiling clearly showed where the time was spent.

This change splits Language Server's protocol into two:
- the first one accepts `heartbeat/init` and `session/initProtocolConnection`
- the second one enriches it with the full set of supported messages

This shifts the initialization from blocking for 0.5 sec to only ~30ms, and performing the second stage asynchronously.

Closes #7224.

# Important Notes
Before the change (blocking server startup):
![Screenshot from 2023-07-05 18-53-24](https://github.com/enso-org/enso/assets/292128/bcfa9043-d00a-4b36-a44c-782a388a16b9)
![Screenshot from 2023-07-05 18-53-10](https://github.com/enso-org/enso/assets/292128/54927787-4c95-46db-bd68-f3a3b82367d5)

After the change (1st stage):
![Screenshot from 2023-07-06 14-02-34](https://github.com/enso-org/enso/assets/292128/d7a7bc34-39dc-46f1-9e64-6d350697c30b)

After the change (2nd, asynchronous initialization, stage):
![Screenshot from 2023-07-06 14-21-17](https://github.com/enso-org/enso/assets/292128/def8c0a1-f211-4fc0-9df0-7c1634312166)
2023-07-07 11:49:23 +00:00
Hubert Plociniczak
72a6c54dfc
Update context on timeout retry (#7207)
Minor follow up on #7173. Noticed while testing nightly on Windows machine.
2023-07-05 12:22:55 +00:00
Hubert Plociniczak
414b124123
Add timeout retries for project-related operations (#7173)
It is relatively easy to reach timeouts on weak systems for startup
operations on project manager. Once a timeout is reached, startup will
not proceed any further.

This PR is a bit of an experiment. It adds adds timeout retries to give a bit of a leeway to
under-powered machines and to log some progress on the way, so that we
know that certain actions are still in progress.
2023-07-03 12:44:41 +00:00
Hubert Plociniczak
3c93c25a5a
Reload project's config on rename (#7179)
Package's config information, once loaded, never changed. While there is typically no need for it, this was problematic when the config became out-of-sync with the filesystem, like in the case of project rename action.
In rename, the config's properties would be updated in the FS, but that would never be reflected in module's package. Therefore further compilations would continue to ask for the old namespace.

Most of the changes are cosmetic (s/`.config`/`.getConfig()`) except for the new `reloadConfig` method on `Package` that is being called in `RenameProjectCmd` handler.

Closes #7062.

# Important Notes
The reported `ExecutionFailed` error should have been mostly fixed already via #7143. This change makes sure that all the related warnings are gone as well and the compiler uses the updated namespace.
2023-07-03 09:36:19 +00:00
Hubert Plociniczak
b78a7e74e8
Provide human-readable error log during installation (#7159)
Windows insists on reporting cryptic error codes that are then hard to debug.
When identified, we can provide more cases.

As identified for #7150.
2023-07-03 08:58:23 +00:00
Hubert Plociniczak
26bd95cf3d
Don't report EOFException on shutdown (#7157)
It's part of the normal shutdown workflow and confused users are reporting it as a failure.
2023-06-29 13:42:13 +00:00
Dmitry Bushev
22259e696d
Add method call info for infix operators (#7090)
close #6374

In order to provide the method pointer information, the `IrToTruffle` pass sets the module name and the type name of the builtin node.
2023-06-27 13:11:12 +00:00
Jaroslav Tulach
c4f19e7d66
Vector.map & State (#7078) 2023-06-26 13:13:41 +02:00
Jaroslav Tulach
6abcbcb00c
Moving construction of IR into runtime parser project (#7096) 2023-06-24 06:34:21 +02:00
Hubert Plociniczak
7955bec129
Add retries to GraalVM updater commands (#7079)
The change adds logic that will attempt a few retries when executing `gu` (GraalVM updater) commands. Previously, if it failed, it failed. Retries should help with the most common case - occassional network hiccups.

Closes #6880.

# Important Notes
Note that I don't use an external library for retries on purpose. Didn't want to introduce a yet another dependency for this tiny functionality.
2023-06-23 18:25:06 +00:00
Jaroslav Tulach
5f572d0680
Avoid dependency on BindingsMap from IR (#7055) 2023-06-19 10:28:31 +02:00
James Dunkerley
760fb71798
First part of AWS S3 API, various small fixes. (#6973)
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
2023-06-15 16:20:13 +00:00
Dmitry Bushev
48f0c6f5e8
Scala 2.13.11 and libraries update (#7010)
Update Scala and libraries.
2023-06-14 13:15:57 +00:00
Dmitry Bushev
6249c79ffd
Update sbt-java-formatter plugin (#7011)
Update java formatter plugin. The new version can remove unused imports.
2023-06-12 14:18:48 +00:00
Dmitry Bushev
6de2700209
Do not return getters from global completion response (#7007)
close #6936

Changelog:
- add: new suggestion type Getter that is not exposed to the api
- update: do not return suggestion of type getter when doing a global search (without specifying self types)

Private suggestions and modules mentioned in the issue will be filtered out after we finish the work on the new (refined) exports algorithm.

# Important Notes
![2023-06-09-205327_1088x612_scrot](https://github.com/enso-org/enso/assets/357683/c6b16894-ada0-4ea9-abe8-5efc41949787)
2023-06-12 12:28:35 +00:00
Dmitry Bushev
4e9f02258e
Suggestion should contain a list of annotations (#6924)
close #6847

Add annotation names to suggestions.
2023-06-02 07:26:01 +00:00
Jaroslav Tulach
65958cba92
IGV Enso support recognizes lib/scala/pkg as project (#6904)
Addition to #4098 - make sure IGV Enso support can recognize `lib/scala/pkg` as a project.
2023-05-31 15:14:57 +00:00
Hubert Plociniczak
efda2cdf31
Empty editions are not parsed as NaN (#6873)
Empty edition (null value) was parsed as NaN, which was confusing. This change correctly detects the case before trying different fallback mechanisms.

Addresses invalid warning mentioned in #6806.
2023-05-30 10:12:39 +00:00
Hubert Plociniczak
86432b5ca5
Detect and override hooks of the same kind (#6842)
This change ensures that we can have at most one hook of the same action during shutdown.
Verified the change on a real project.

Closes #6767.
2023-05-29 07:25:30 +00:00
somebody1234
6693bdb5cd
Fallback to opened date when ordering projects (#6814)
Fixes #6787

# Important Notes
I can't get Project Manager compilation to work locally so I guess I'll be relying on CI to verify that it's working correctly?

Of course, QA should be able to catch any problems too - the websocket API hasn't been changed so it should work out of the box with the current dashboard.
2023-05-25 15:44:15 +00:00
Pavel Marek
bd70ed6830
Update GraalVM to 22.3.1 JDK17 (#6750)
Upgrade GraalVM to 22.3.2 based on JDK17.

---------

Co-authored-by: somebody1234 <ehern.lee@gmail.com>
2023-05-24 10:39:08 +02:00
Dmitry Bushev
1ddd36308e
Filter suggestion constructors when searching by self type (#6813)
close #6627

Changelog:
- update: search method excludes constructors when searched by the self type

# Important Notes
Component Browser doesn't show the `Between` constructor when called on the `Range` node.

![2023-05-23-172446_1203x865_scrot](https://github.com/enso-org/enso/assets/357683/a2553c46-6a0e-4842-ba91-7c2b76c56c6d)
2023-05-23 16:14:41 +00:00
Dmitry Bushev
0318880990
Add project creation time to project metadata (#6780)
close #6686

Changelog:
- add: `created` field to the project metadata response
- update: project manager tests
2023-05-22 13:40:10 +00:00
Radosław Waśko
447786a304
Implement cast for Table and Column (#6711)
Closes #6112
2023-05-19 10:00:20 +00:00
Dmitry Bushev
b7d51ed5c6
Create unique atom getter suggestions (#6694)
related #6611

Prevent creating atom getters with the same name.
2023-05-15 14:03:18 +00:00
Hubert Plociniczak
06624f34a8
Ensure slow shutdown of LS always kicks off hooks (#6665)
This change fixes the rather elusive bug where shutdown hooks could not be fired when shutdown was taking too long and termination was forced.

Under the circumstances described in detail in ticket #6515 there was a small chance that we could have a shutdown race condition. Essentially the messages received when client was disconnected and language server forced the termination could lead to language server not sending the public `ProjectClosed` message which triggers shutdown hook. Now we always do.

Also made sure that multiple `ProjectClosed` messages don't lead to firing multiple shutdown hooks, which was another possibility.

No tests as one would have to be able to introduce different delays in various message handlers to simulate the problem.
Having ability to do such chaos testing would be nice but it is beyond the scope of this ticket.
I was able to reproduce the problem 100% with my specially crafted setup so I'm fairly confident about the change.

Closes #6515.
2023-05-15 12:15:37 +00:00
Dmitry Bushev
706791779b
SuggestionBuilder needs to send ascribedType of constructor parameters (#6655)
close #6611

Changelog:
- update: run compiler passes on the `ascribedType` field of the constructor arguments
- update: suggestion builder uses the type information attached to `ascribedType`
- feat: resolve qualified names in type signatures
2023-05-13 18:33:03 +00:00
Jaroslav Tulach
1302b693d8
All Vector operations shall be applicable on java.util.ArrayList (#6642)
Fixes #6609 by
- e380e647af - running whole `Vector_Spec` on `java.util.ArrayList`
- 9b1229fe20 - introducing a node to handle interop values

# Important Notes
Contains additional DSL processor fix:
- 415623dcb9 - to not crash the compiler, but to properly report compiler error
2023-05-11 15:29:47 +00:00
Dmitry Bushev
3dd05c2359
Automatic type based dropdown does not include singleton in a union type (#6629)
close #6532

Set tag values to display `Auto` type in dropdowns correctly.

# Important Notes
![2023-05-10-141458_969x566_scrot](https://github.com/enso-org/enso/assets/357683/9a048b3c-d192-4382-bf76-9cbe6c9556d1)
![2023-05-10-141513_991x232_scrot](https://github.com/enso-org/enso/assets/357683/c50e1e73-23a6-4b32-90bf-f849a127c85d)
2023-05-10 16:55:16 +00:00
Hubert Plociniczak
cf3624c463
Limit the number of reported warnings (#6577)
Artifically limiting the number of reported warnings to 100. Also added benchmarks with random Ints to investigate perf issues when dealing with warnings (future task).
Ideally we would have a custom set-like collection that allows us internally to specify a maximal number of elements. But `EnsoHashMap` (and potentially `EnsoSet`) are still WIP when it comes to being PE-friendly.

The change also allows for checking if the limit for the number of reported warnings has been reached. It will visualize by adding an additional "Warnings limit reached." to the visualization.

The limit is configurable via `--warnings-limit` parameter to `run`.

Closes #6283.
2023-05-10 11:48:31 +00:00
Radosław Waśko
41a8257e8d
Separating Redshift connector from Database library into a new AWS library (#6550)
Related to #5777
2023-05-04 17:36:51 +00:00
Hubert Plociniczak
b93edb050b
Force pending saves if client closes abruptly (#6514)
Force pending saves when JsonSession is terminated for the client.

Potentially closes #6395.
2023-05-03 23:30:20 +00:00
Dmitry Bushev
38212786a3
Fix application config (#6513)
Related #6482
2023-05-02 17:59:44 +00:00
Pavel Marek
3a42d0ce76
Reimplement enso_project as a proper builtin (#6352)
Remove the magical code generation of `enso_project` method from codegen phase and reimplement it as a proper builtin method.

The old behavior of `enso_project` was special, and violated the language semantics (regarding the `self` argument):
- It was implicitly declared in every module, so it could be called without a self argument.
- It can be called with explicit module as self argument, e.g. `Base.enso_project`, or `Visualizations.enso_project`.

Let's avoid implicit methods on modules and let's be explicit. Let's reimplement the `enso_project` as a builtin method. To comply with the language semantics, we will have to change the signature a bit:
- `enso_project` is a static method in the `Standard.Base.Meta.Enso_Project` module.
- It takes an optional `project` argument (instead of taking it as an explicit self argument).

Having the `enso_project` defined as a (shadowed) builtin method, we will automatically have suggestions created for it.

# Important Notes
- Truffle nodes are no longer generated in codegen phase for the `enso_project` method. It is a standard builtin now.
- The minimal import to use `enso_project` is now `from Standard.Base.Meta.Enso_Project import enso_project`.
- Tested implicitly by `org.enso.compiler.ExecCompilerTest#testInvalidEnsoProjectRef`.
2023-05-02 16:40:58 +00:00
Hubert Plociniczak
ce4ecc6706
Limit Dead Letter logging (#6482)
Dead Letter logging is occasionally flooding our logs which is confusing to users reporting bugs. Left the possibility of a single report so that we know that something is happening.
2023-05-02 14:00:50 +00:00
Jaroslav Tulach
efe904cd9f
Introducing @BuiltinMethod.needsFrame and InlineableNode (#6442)
Fixes #6416 by introducing `InlineableNode`. It runs fast even on GraalVM CE, fixes ([forever broken](https://github.com/enso-org/enso/pull/6442#discussion_r1178782635)) `Debug.eval` with `<|` and [removes discouraged subclassing](https://github.com/enso-org/enso/pull/6442#discussion_r1178778968) of `DirectCallNode`. Introduces `@BuiltinMethod.needsFrame` - something that was requested by #6293. Just in this PR the attribute is optional - its implicit value continues to be derived from `VirtualFrame` presence/absence in the builtin method argument list. A lot of methods had to be modified to pass the `VirtualFrame` parameter along to propagate it where needed.
2023-04-28 15:32:13 +00:00
Hubert Plociniczak
c6790f1e9c
Report only unique warnings (#6372)
This change makes sure that reported warnings are unique, based on the value of internal clock tick and ignoring differences in reassignments.

Before:
![Screenshot from 2023-04-20 15-42-55](https://user-images.githubusercontent.com/292128/233415710-925c1045-37c7-49f5-9bc3-bfbfd30270a3.png)
After:
![Screenshot from 2023-04-20 15-27-27](https://user-images.githubusercontent.com/292128/233415807-8cb67bc2-ac37-4db7-924e-ae7619074b5b.png)

On the positive side, no further changes, like in LS, have to be done.


Closes #6257.
2023-04-28 07:16:00 +00:00
Hubert Plociniczak
376415ab17
Mark some tests as (unconditonally) flaky (#6453)
Too many spurious failures on CI now. Will need to be tackled at some
point.
2023-04-27 17:09:46 +02:00
Dmitry Bushev
2d168308af
Better cleanup of project management test suite (#6392)
Address some arbitrary failures of the `ProjectManagementApiSpec` test suite.
2023-04-24 13:11:27 +00:00
Dmitry Bushev
74c0718604
Project folder not renamed after project name change (#6369)
close #6254

Changelog:
- fix: race when the actor system may be stopped before the shutdown hooks are executed
- fix: project management spec
- fix: recover from `readLine` failure during the shutdown
2023-04-20 16:57:37 +00:00
Hubert Plociniczak
4c1e73c864
Don't propagate warnings on suspended arguments (#6345)
* Don't propagate warnings on suspended arguments

In the current implementation, application of arguments with warnings
first extracts warnings, does the application and appends the warnings
to the result.
This process was however too eager if the suspended argument was a
literal (we don't know if it will be executed after all).

The change modifies method processor to take into account the
`@Suspend` annotation and not gather warnings before the application
takes place.

* PR review
2023-04-19 11:55:23 +02:00
Jaroslav Tulach
e5a96b9782
Avoid NullPointerException in --dumpGraphs & test & docs (#6304) 2023-04-18 06:58:06 +02:00
Dmitry Bushev
93b9ca7789
Update the default project template (#6310)
close #6307

Changelog:
- update: the default project template

# Important Notes
[6307-update-the-default-project-template.webm](https://user-images.githubusercontent.com/357683/232487322-8aebc709-30aa-48f3-b86b-f868c83a6e21.webm)
2023-04-17 14:09:08 +00:00
Jaroslav Tulach
a74933d10f
Speed cascade of if statements up (#6255)
Fixes #5709. We have a test and a generic fix that improves inlining of every builtin. Everything seems to be faster.
2023-04-14 13:27:23 +00:00