Commit Graph

376 Commits

Author SHA1 Message Date
Radosław Waśko
4b35e0a99e
Propagate FS errors when creating a new project (#10708)
- Part of #10662
2024-08-05 10:58:03 +00:00
Hubert Plociniczak
d43ad7ce13
Disable linting passes for IDE and repl (#10705)
There is no need to generate unused variables warnings or other linting for IDE and repl users. By default linting is enabled during compilation and for those use-cases it is now disabled via runtime options.

Closes #9883
2024-07-31 08:49:45 +00:00
Jaroslav Tulach
cb72487cc9
ContextFactory reused to initialize language-server context (#10670) 2024-07-29 09:49:14 +02:00
Hubert Plociniczak
aae8370977
Disable Graal-based Ydoc temporarily (#10627)
Graal Ydoc implementation is currently not being used locally or in the cloud and giving an impression of a slower startup.
Plus it appears that there some issues in the local connection as well.
To limit the impact of it now, let's make it controllable by the same env variable as GUI is.
2024-07-23 12:29:26 +00:00
Adam Obuchowicz
71bae7e4b0
initProtocol endpoint returns success when already initialized (#10542)
* initProtocol endpoint returns success when already initialized

* Fix test

* Docs and return error when clientId differs

* fmt

* fix: session management test

* misc: json connection controller

---------

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>

Issue with Vector nothing to do with this.
2024-07-23 07:45:53 +01:00
Hubert Plociniczak
700a638f1e
Ignore ProcessBuilder logs (#10578) 2024-07-17 21:38:16 +00:00
Hubert Plociniczak
39898da664
Reduce the number of log INFO events (#10500)
Reducing the number of log events that we spam regular users. Not the last PR in that area, but already a progress.
Also replaced `fileWalk` with a stream approach hoping that maybe it is a bit more stable on Windows.
2024-07-17 14:29:48 +00:00
Hubert Plociniczak
e585904291
Mitigate Language Server visualizations DOS scenario on startup (#10549)
* Mitiigate LS DDOS scenario on startup

For a medium-size project with a lot of visualizations, Language Server
will be flooded with visualization requests on startup. Due to an
existing limit for the job execution engine, some of those requests
might have been silently dropped.
This change lifts that limit until a better fix can be invented.

Additionally, a slow startup might lead to a timeout when serving open
file request. This change adds some retries as a fallback
mechanism/progress monitoring.

* add runtime-fat-jar to a list of aggregates
2024-07-15 16:45:07 +02:00
Jaroslav Tulach
aaa3495546
engine-runner shall not depend on language-server (#10536) 2024-07-12 17:17:07 +02:00
Hubert Plociniczak
53eec66eda
Low-hanging perf improvements (#10462)
- avoid loading shapeless for the sole purpose of having a compile-time
type inequality
- don't use `sys.env` to avoid some Scala conversions
- lazy initialization of fields

# Important Notes
On a slow machine, so easier to spot.

![Screenshot from 2024-07-05 16-15-06](https://github.com/enso-org/enso/assets/292128/a07f42c5-9bee-492b-aad1-46fab7b6476f)
![Screenshot from 2024-07-05 18-14-53](https://github.com/enso-org/enso/assets/292128/694c4fb1-dfda-4629-8bd3-21c765612ec3)
2024-07-08 09:05:11 +00:00
Hubert Plociniczak
9449bf3ef5
Amortize the startup cost of some components (#10451)
Building Engine, Context, ApplicationConfig and Ydoc was a adding a rather large delay during the initial startup step as all of those were blocking operations.
Moving all of those to the resource initialization step hopes to amortize some of that cost since it can be done in parallel. Had to add a `ComponentSupervisor` (open for a different name suggestion) to ensure that such delayed components are properly closed on shutdown.

# Important Notes
Adding Ydoc has added a visible delay during startup. I'm hoping that we can amortize some of that with this PR:
![Screenshot from 2024-07-05 11-12-19](https://github.com/enso-org/enso/assets/292128/fd52f749-b2cb-414d-bd2a-847ea867026c)

Now:
![Screenshot from 2024-07-05 11-25-58](https://github.com/enso-org/enso/assets/292128/9e7c96c9-ee47-46c3-9bdb-8f96bbc4a68f)
2024-07-05 11:50:56 +00:00
Jaroslav Tulach
5e3a5b7751
Disable UpdatingEditionProvider when running Enso CLI (#10408) 2024-07-03 07:33:01 +02:00
Hubert Plociniczak
114b3a5c5e
Cleaning up YAML parsing in preparation for circe-yaml removal (#10309)
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.

As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.

Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.

In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
2024-06-20 13:07:54 +00:00
Dmitry Bushev
06a73606b9
Async Ydoc initialization (#10290)
close #10265

Changelog:
- add: YdocInitialization component
2024-06-17 08:10:21 +00:00
Dmitry Bushev
04a92ef765
TextEdit with custom IdMap (#10283)
close #10182

Changelog:
- add: IdMap parameter to the `text/applyEdit` request
- add: IdMap to the runtime module
- update: set IdMap during the interactive compilation
- update: set the IR identifiers in the `TreeToIR` parsing step
2024-06-15 18:03:18 +00:00
Hubert Plociniczak
dc6e3a7031
Soft retries when awaiting runtime responses (#10272)
A quick and dirty workaround for slow processing in a similar spirit to PR #9858.
Long compilation of stdlib holds a write compilation lock, while opening the file needs to set module sources and requires read compilation lock and file lock. The expectation was that setting module sources is instantaneous except not, because of locks.

This PR adds soft retries.

Partially closes #10231. There are still problems related to https://github.com/enso-org/enso/issues/9993 once this PR is merged.

# Important Notes
We need to figure out a more fine-grained lock system or, ideally, make it lock free to avoid such hacks.
2024-06-14 09:05:48 +00:00
Hubert Plociniczak
d6d370925a
Removing dependencies to speed up startup (#10249)
Reducing the number of dependencies. Explicit `cats` are almost gone (present in `cli`). `enumeration` is completely gone.  `cats` is also still included implicitly via `io.circe` but that's a different kind of beast.
Also, really removed `jackson` from dependencies by fixing the dependency on `http-test-helper`.

# Important Notes
In a number of places importing all cats implicits could be simply replaced with a single or two method calls. Not to mention that this will reduce compilation times due to reduced implicit search space.

One example of how the changes affect performance (not only startup):

Before:
![Screenshot from 2024-06-11 12-05-24](https://github.com/enso-org/enso/assets/292128/a1a772a9-635d-4a16-a543-e2fd2124a22c)
Now:
![Screenshot from 2024-06-11 14-27-47](https://github.com/enso-org/enso/assets/292128/b17c7fcc-9a6d-48b9-8200-60708354ee03)
(frequently executed)

![Screenshot from 2024-06-12 12-46-34](https://github.com/enso-org/enso/assets/292128/31bc4dfd-4edc-45c9-9c5d-13e3472089b9)
Also appears to be gone.

This PR is by no means finished. The purge will continue in follow up PRs.
2024-06-12 18:15:36 +00:00
Hubert Plociniczak
4da5e611c9
Replace Jackson serde (#10035)
JSON serialization setup between Language Server and Runtime is a major contributor to startup time. This PR experiments with an alternative implementation that remedies the problem.
The new serializer uses [jsoniter-scala](https://github.com/plokhotnyuk/jsoniter-scala) which by some accounts claims to be really fast. In our case, more importantly, we pay negligible cost of startup setup compared to Jackson which was horribly slow.

# Important Notes
Before:
![Screenshot from 2024-06-06 15-35-18](https://github.com/enso-org/enso/assets/292128/56103b82-777e-459f-966e-abdef25c2430)
After:
![Screenshot from 2024-06-06 15-35-02](https://github.com/enso-org/enso/assets/292128/00a36647-dfae-4dc8-a1b7-bf69069ef109)

Yes. About 0.8sec.
2024-06-11 15:03:12 +00:00
Pavel Marek
2ea2a57651
ydoc-server is a separate module (#10156)
- Remove remnants of deprecated Scala parser
- The following projects are now JPMS modules provided on system module-path (in components directory):
  - `ydoc-server`
  - `profiling-utils`
  - `syntax-rust-definition`
- The contents of the aforementioned modules are excluded from both `runner.jar` and `runtime.jar` fat jars.
- Suggestions are serialized and deserialized with our Persistance framework, rather than via the default Java OutputObjectWriter.
2024-06-07 12:56:42 +02:00
Dmitry Bushev
d08cb704b0
Record interaction between GUI and LS (#10107)
close #8328

Changelog:
- add: message callbacks to JsonRpc and Binary servers
- update: use events log to collect the RPC messages
2024-06-03 06:50:59 +00:00
Dmitry Bushev
f83ae1382e
Temporarily disable Ydoc startup (#10141)
related #10134
2024-05-30 16:28:16 +00:00
Dmitry Bushev
858e646328
Start Ydoc with the language server (#9862)
- related #7954

Changelog:
- update: Ydoc starts with the language server on the `localhost:1234` by default. The hostname and ports can be configured by setting environment variables `LANGUAGE_SERVER_YDOC_HOSTNAME` and `LANGUAGE_SERVER_YDOC_PORT`
- update: by default `npm dev run` uses the node Ydoc server. You can control it with `POLYGLOT_YDOC_SERVER` env variable. For example,
```
env POLYGLOT_YDOC_SERVER='true' npm --workspace=enso-gui2 run dev
```
To connect to the Ydoc server running on the 1234 port (the one started with the language server)
⠀
```
env POLYGLOT_YDOC_SERVER='ws://127.0.0.1:1235' npm --workspace=enso-gui2 run dev
```
To connect to the provided URL. Can be useful for debugging when you start a separate Ydoc process.
- update: run `npm install` before the engine build. It is required to create the Ydoc JS bundle.
2024-05-28 13:51:42 +00:00
Hubert Plociniczak
495eed45ba
Turn strict errors off for Language Server (#10061)
Ensure strict errors are off to prevent unrecoverable errors in IDE.
Also partially reverted a change that prevented application of changes on errors (errors are still logged, as intended).
Closes #10014.
2024-05-23 20:49:08 +00:00
Dmitry Bushev
643b66d0b7
Allow file/read to return the contents of a collaborative buffer (#9994)
part of #9960

Changelog:
- feat: file/read return contents of a collaborative buffer if they are available and fallback to reading the file from disk
2024-05-20 14:01:59 +00:00
Hubert Plociniczak
c67218c1ed
Abort jobs as early as possible (#9927)
We don't seem to run `abortJobs` under a lock, and especially not under the write compilation lock, in other scenarios. This is causing some major slowdown when there is a long running execution or compilation, as currently experienced in the cloud.

This should reduce chances of a timeout.

Also added an option to override the global executor. Currently it would always default to the runtime number of available process which may be suboptimal.

# Important Notes
Pending testing on the impact it will have.
2024-05-13 07:39:29 +00:00
Hubert Plociniczak
f1ddf1b863
Log timings of FS operations (#9908)
I'm seeing occasional IO timeouts, especially on startup operations, for cloud projects. Adding some logging to make an informed decision if there are some problems there.

Related to https://github.com/enso-org/enso/issues/9789

# Important Notes
Also added retries when closing the file as I saw a number of times:
```
Session release failed.
LsRpcError: Language server request 'text/closeFile' failed.
at LanguageServer.request (/tmp/.mount_enso-leMqqdS/resources/app.asar/index.cjs:58291:15)
at async Promise.all (index 0)
at async _LanguageServerSession.release (/tmp/.mount_enso-leMqqdS/resources/app.asar/index.cjs:59165:5)
at async /tmp/.mount_enso-leMqqdS/resources/app.asar/index.cjs:59670:7 {
cause: JSONRPCError2: Request timeout request took longer than 15000 ms to resolve
at new JSONRPCError2 (/tmp/.mount_enso-leMqqdS/resources/app.asar/index.cjs:26822:30)
at Timeout._onTimeout (/tmp/.mount_enso-leMqqdS/resources/app.asar/index.cjs:26985:20)
at listOnTimeout (node:internal/timers:569:17)
at process.processTimers (node:internal/timers:512:7) {
code: 7777,
data: undefined
},
request: 'text/closeFile',
params: {
path: {
rootId: '00000000-0000-0000-0000-000000000001',
segments: [Array]
}
}
}
```
2024-05-09 15:41:50 +00:00
Hubert Plociniczak
459a264ca2
Workaround slow responses with retries (#9858)
Follow up to #9558, this time to `SetExecutionContextEnvironmentHandler` that was timing out in #9789.
Added a base classes that handles the repeatable logic.

Maybe it will close #9789.
2024-05-06 19:37:01 +00:00
Dmitry Bushev
a4085346f1
Fix polyglot tree serialization (#9852)
close #9306

Changelog:
- fix: `polyglot.data.Tree` Jackson serialization
- update: report errors during the deserialization messages between the Runtime and the Language Server
2024-05-03 10:24:26 +00:00
Dmitry Bushev
a44bb2b1b1
Language Server API for AI (#9679)
close #9656

Changelog:
- add: `ai/completion_v2` method
- add: `Visualization.AI.print` method for converting the expression to text format
- update: The default system prompt was updated to tell AI to use the `Visualization.AI.print` method for printing.

# Important Notes
The project [New_Project_1.zip](https://github.com/enso-org/enso/files/15152993/New_Project_1.zip) contains the following main file:
```py
from Standard.Base import all
from Standard.Table import all
from Standard.Database import all
from Standard.AWS import all
import Standard.Visualization
import Standard.Visualization.Warnings
from Standard.Base.Errors.Common import Dry_Run_Operation

type Student
Value id region

main =
operator70395 = 226
operator47321 = 'east'
operator76980 = Student.Value operator70395 operator47321
operator31302 = operator47321.words True
operator91574 = 1
operator34358 = operator47321.take (Index_Sub_Range.By_Index [0, operator91574])



#### METADATA ####
[[{"index":{"value":0},"size":{"value":4}},"4cf8de7f-2014-4dfd-9ceb-0164fe26c8bf"],[{"index":{"value":0},"size":{"value":29}},"389fe7a5-e59b-440a-8801-e0bd86716094"],[{"index":{"value":0},"size":{"value":558}},"a8a81ce3-199e-479e-98dd-5f919425842d"],[{"index":{"value":5},"size":{"value":8}},"c6f0de99-3bd3-459d-a69b-5fc774dd1e3f"],[{"index":{"value":5},"size":{"value":13}},"05827411-7354-478b-99d8-8370dadd560b"],[{"index":{"value":13},"size":{"value":1}},"41de94cc-ccd2-4ccf-b9db-cf686443efa0"],[{"index":{"value":14},"size":{"value":4}},"8abd24ea-b1cb-4c10-ad16-5a190f7ed0c7"],[{"index":{"value":19},"size":{"value":6}},"5c57a62d-ed82-4195-8b78-2ed660556dd0"],[{"index":{"value":26},"size":{"value":3}},"23544334-1117-4dbc-ac7f-bb3f84575264"],[{"index":{"value":30},"size":{"value":4}},"6058ba89-dc1b-45e1-b8f3-d1a4d27975f7"],[{"index":{"value":30},"size":{"value":30}},"bdd4b4cc-9c1e-4554-b0fb-3f73dcd5a590"],[{"index":{"value":35},"size":{"value":8}},"7e7bb9b4-8e9b-4700-9dcd-83c353800da5"],[{"index":{"value":35},"size":{"value":14}},"524bafb6-d3ca-4ee5-b8de-84b5c47ca87b"],[{"index":{"value":43},"size":{"value":1}},"b68b3929-8b63-49ad-977c-b04e058f8ee4"],[{"index":{"value":44},"size":{"value":5}},"7173f374-5f53-462b-ab1b-f7b1845d729b"],[{"index":{"value":50},"size":{"value":6}},"911d381e-1bdb-4e9c-826d-a00495c8fd23"],[{"index":{"value":57},"size":{"value":3}},"2f8fa1ee-9d13-4a73-99b2-bc77d1656387"],[{"index":{"value":61},"size":{"value":4}},"11abd658-4888-476c-afcd-592edaaa53c1"],[{"index":{"value":61},"size":{"value":33}},"9f6ae949-410b-48e6-bb75-84d4ba8a1989"],[{"index":{"value":66},"size":{"value":8}},"f4726e40-dd5b-49a0-aa2a-6cea6df88d5d"],[{"index":{"value":66},"size":{"value":17}},"539b4dcf-4487-49b6-9c90-6ae58d51c2db"],[{"index":{"value":74},"size":{"value":1}},"a34a41a0-5dcf-4c47-a36c-a4f427c87c03"],[{"index":{"value":75},"size":{"value":8}},"8fdf4ee7-2e74-4cac-b9b9-4230183f91c5"],[{"index":{"value":84},"size":{"value":6}},"7ff7efdc-fed6-4cb6-bed6-392d110ff991"],[{"index":{"value":91},"size":{"value":3}},"eacff407-dc6d-4f17-a28b-329f7a0582e1"],[{"index":{"value":95},"size":{"value":4}},"f2feac9f-ea02-4070-879a-f1d493473993"],[{"index":{"value":95},"size":{"value":28}},"f9b37b67-fb54-4e47-8281-e922edb4c5d3"],[{"index":{"value":100},"size":{"value":8}},"6cf398f8-d0c9-42fa-a6e6-e740de11562f"],[{"index":{"value":100},"size":{"value":12}},"28e4d0e3-7da4-4ee2-94c7-d9a7c692f8de"],[{"index":{"value":108},"size":{"value":1}},"955e8788-816c-4eb7-b5ec-020b4863536a"],[{"index":{"value":109},"size":{"value":3}},"bdc65807-0438-4296-8b1f-1fab3a885c17"],[{"index":{"value":113},"size":{"value":6}},"fe3dd265-1855-4d66-8a2d-51153b21e699"],[{"index":{"value":120},"size":{"value":3}},"3b7a14ac-3224-4b9a-b474-f6136a658f8b"],[{"index":{"value":124},"size":{"value":6}},"7f7f18a4-06ef-46e0-bddd-b9fe60878855"],[{"index":{"value":124},"size":{"value":29}},"d49e0fe3-a638-4d7e-ba58-12db7abad20f"],[{"index":{"value":131},"size":{"value":8}},"c9411433-5e73-4927-9175-15d3b1ccb0ef"],[{"index":{"value":131},"size":{"value":22}},"b53ad817-ea4c-491e-aaf8-49967e22aff6"],[{"index":{"value":139},"size":{"value":1}},"837a4d65-8af6-4860-ae0e-ad9172af6ad2"],[{"index":{"value":140},"size":{"value":13}},"3b662906-1887-4dd6-a0e2-a222ace135ce"],[{"index":{"value":154},"size":{"value":6}},"548ca873-544f-48c9-b7ab-6aa1e7f03d72"],[{"index":{"value":154},"size":{"value":38}},"7c45872c-22b4-4f37-bcd7-b3791408f737"],[{"index":{"value":161},"size":{"value":8}},"21152f0d-4222-48e4-9376-a2a28f8f6be6"],[{"index":{"value":161},"size":{"value":22}},"1cb71d49-a06d-4dd9-bc1b-54603b416601"],[{"index":{"value":161},"size":{"value":31}},"faa46cf2-3e90-44ed-9d9e-77b7f874b338"],[{"index":{"value":169},"size":{"value":1}},"6d5fdcd5-0ddf-4c83-aafe-a446ac5ce461"],[{"index":{"value":170},"size":{"value":13}},"830d73c2-1898-41a9-8046-22d86f159ef0"],[{"index":{"value":183},"size":{"value":1}},"ab897380-c596-4182-a2c5-b7b23cf59893"],[{"index":{"value":184},"size":{"value":8}},"29f35cf2-d79d-44d0-96d0-79b11589726c"],[{"index":{"value":193},"size":{"value":4}},"45bf8220-52f2-49f4-bbcf-9566d1f1b1c4"],[{"index":{"value":193},"size":{"value":57}},"478005ac-f0ca-497a-b298-554bd64946a9"],[{"index":{"value":198},"size":{"value":8}},"c11511ce-252c-4b84-9f61-dfa5544d4916"],[{"index":{"value":198},"size":{"value":13}},"dbae65a1-bdca-44b0-a497-7934d1e42616"],[{"index":{"value":198},"size":{"value":20}},"355392ad-8c81-4304-a6d0-2bbf3690da0c"],[{"index":{"value":198},"size":{"value":27}},"0720ccb5-ebd3-4c29-8113-0aa1f251e873"],[{"index":{"value":206},"size":{"value":1}},"f5a8d310-1f58-456e-84a3-78490f05ff13"],[{"index":{"value":207},"size":{"value":4}},"10046e78-5164-407c-adef-2ab11ce77b4c"],[{"index":{"value":211},"size":{"value":1}},"ed23da0c-9193-4eea-ab9c-fee90ba8f924"],[{"index":{"value":212},"size":{"value":6}},"7a09de10-35e2-45e0-9eb8-f79fa054b733"],[{"index":{"value":218},"size":{"value":1}},"b85a0981-9d82-4204-a03a-cd3a4720c72d"],[{"index":{"value":219},"size":{"value":6}},"1b04c19d-cdf0-4661-8f4e-d980cf12c996"],[{"index":{"value":226},"size":{"value":6}},"87d57f2c-e009-48a7-98bb-0e1612905564"],[{"index":{"value":233},"size":{"value":17}},"60fd6a1c-7ada-4546-ae20-e1f8b432b4fd"],[{"index":{"value":252},"size":{"value":4}},"9558affa-b913-4576-8ee3-6af22985e0f6"],[{"index":{"value":252},"size":{"value":32}},"8479091c-9232-49d6-b375-32f89693c389"],[{"index":{"value":257},"size":{"value":7}},"3d1f13f5-e22a-4cb6-9e75-9c717e016922"],[{"index":{"value":264},"size":{"value":1}},"aa429f07-973a-4c69-8e3f-74aa4780f85a"],[{"index":{"value":269},"size":{"value":5}},"5b304cf6-5ec4-4b23-ad0d-20e38b41cdcd"],[{"index":{"value":269},"size":{"value":15}},"e3315ef2-d0ec-43b1-b8f6-5b436054aaee"],[{"index":{"value":275},"size":{"value":2}},"bc7205c4-68bf-461e-a1ba-31400b9337ff"],[{"index":{"value":278},"size":{"value":6}},"78870c05-9117-4e24-bf77-a72e7aef9c18"],[{"index":{"value":286},"size":{"value":4}},"517c743f-c775-4031-8f87-222d0f0a365f"],[{"index":{"value":286},"size":{"value":271}},"d0d107bc-4997-41d6-9c20-32295590eaac"],[{"index":{"value":291},"size":{"value":1}},"91174930-f18b-4322-84cc-deb88637d012"],[{"index":{"value":292},"size":{"value":265}},"f09a4372-3231-4f2e-99f4-84aa751f9b60"],[{"index":{"value":297},"size":{"value":13}},"613df8c9-0a40-4c94-886f-9668c2c360c2"],[{"index":{"value":297},"size":{"value":19}},"5b88a4d4-6840-4a25-a862-8b5d0b75303d"],[{"index":{"value":311},"size":{"value":1}},"6bfe1dea-09df-4358-9613-74d8326bc680"],[{"index":{"value":313},"size":{"value":3}},"e3b7fac7-0c08-4f20-8a0c-a58f1b118097"],[{"index":{"value":321},"size":{"value":13}},"37d007fe-9cb6-4f2e-a8b7-30842eac60b2"],[{"index":{"value":321},"size":{"value":22}},"4233204a-543d-437a-a9fc-9f79853a9540"],[{"index":{"value":335},"size":{"value":1}},"6c01783b-27ba-413c-84a4-6a94a37d714a"],[{"index":{"value":337},"size":{"value":1}},"34bc865a-58b5-4c49-bf85-a8cfedaf9b54"],[{"index":{"value":337},"size":{"value":6}},"bb556514-570d-4a87-8b2a-6e7f198a975f"],[{"index":{"value":338},"size":{"value":4}},"87c2cedc-898e-43c3-be3d-24e15689c373"],[{"index":{"value":342},"size":{"value":1}},"13387ff5-ec5e-453a-86f2-deae721b5549"],[{"index":{"value":348},"size":{"value":13}},"6280a86b-5469-43f3-94d5-1e1726028a54"],[{"index":{"value":348},"size":{"value":57}},"a17c5e55-5e99-4e4f-961d-9b4072ecd6e4"],[{"index":{"value":362},"size":{"value":1}},"c0720382-9ce4-451a-a04b-f248143d2528"],[{"index":{"value":364},"size":{"value":7}},"9cd51399-d9d2-4898-9bcb-be2f540999ff"],[{"index":{"value":364},"size":{"value":13}},"b4980bfc-13cc-429f-bc43-9edfc07d2406"],[{"index":{"value":364},"size":{"value":27}},"1a3f6a8c-ab27-43dd-a494-bf413334fd9a"],[{"index":{"value":364},"size":{"value":41}},"71a97e88-9b19-4ec9-b3c8-ffd2940c4cb8"],[{"index":{"value":371},"size":{"value":1}},"99f38c61-3ea7-4b02-be71-f293fbecb7d7"],[{"index":{"value":372},"size":{"value":5}},"08820818-ddb9-408a-b913-e9382e5b6dc7"],[{"index":{"value":378},"size":{"value":13}},"2cf06b4e-af9f-492e-ac13-5723a896a508"],[{"index":{"value":392},"size":{"value":13}},"6c887279-5998-45ce-81ab-c37fdeb03144"],[{"index":{"value":410},"size":{"value":13}},"6c126d22-c018-46cc-bdae-cf8b89104173"],[{"index":{"value":410},"size":{"value":40}},"a65c1fa8-9fce-4c30-bef7-d85ff1b428eb"],[{"index":{"value":424},"size":{"value":1}},"208c049e-e221-49ce-99d7-96142f2d1b1c"],[{"index":{"value":426},"size":{"value":13}},"0adf4ddd-1402-4b96-bfb0-1917cc275063"],[{"index":{"value":426},"size":{"value":19}},"f2158ac6-2dd5-482d-9f3d-0bf4812d6d6e"],[{"index":{"value":426},"size":{"value":24}},"c7bbc3e7-1377-429e-b60a-0c2d2cd4ea74"],[{"index":{"value":439},"size":{"value":1}},"1a17a7ee-31d7-4885-84ce-c902bff4ea4a"],[{"index":{"value":440},"size":{"value":5}},"47016cac-40a0-45f6-8057-2b81edb052a5"],[{"index":{"value":446},"size":{"value":4}},"ce4fa4cd-5064-4752-9dbc-5fcf4e92138a"],[{"index":{"value":455},"size":{"value":13}},"3300001d-13ea-45f4-acbe-959813bcf85b"],[{"index":{"value":455},"size":{"value":17}},"b8f5900b-2370-4b21-8a0c-06d3bbb45068"],[{"index":{"value":469},"size":{"value":1}},"19ea3187-83d6-47d1-994c-64cf0ce9af3b"],[{"index":{"value":471},"size":{"value":1}},"9457c3f8-c878-4f01-a5c7-39336f163a28"],[{"index":{"value":477},"size":{"value":13}},"a5139627-05a8-4838-b171-02666c62c348"],[{"index":{"value":477},"size":{"value":80}},"8b5c864c-055d-4d11-a470-2679328fc131"],[{"index":{"value":491},"size":{"value":1}},"eb01b0af-06cf-4d75-81ad-d0bf7664b2d7"],[{"index":{"value":493},"size":{"value":13}},"6c042eb7-e895-473d-b456-6e0eec23e958"],[{"index":{"value":493},"size":{"value":18}},"f40730ff-ac6c-45db-8970-534f3becda2a"],[{"index":{"value":493},"size":{"value":64}},"b2df8ba8-683c-45c2-9932-8e1c970c799a"],[{"index":{"value":506},"size":{"value":1}},"2289bf31-74dd-431c-97f1-3ac179715453"],[{"index":{"value":507},"size":{"value":4}},"6db4523a-0bcd-4ba5-b6c2-7720db57d93c"],[{"index":{"value":512},"size":{"value":1}},"b6faea17-664d-470a-9641-999b6a2be5bd"],[{"index":{"value":512},"size":{"value":45}},"98aea277-bc44-4ceb-85a6-24ee453b7f72"],[{"index":{"value":513},"size":{"value":15}},"c0ae1c05-026d-4712-a571-979ce260b310"],[{"index":{"value":513},"size":{"value":24}},"502e8e4d-5105-4abe-8dba-4d7293cbe80b"],[{"index":{"value":513},"size":{"value":43}},"71cee748-378b-4817-be0e-b87bc1d3d847"],[{"index":{"value":528},"size":{"value":1}},"fcaad3ac-e080-4170-9d63-b2c01295447d"],[{"index":{"value":529},"size":{"value":8}},"a053e05a-4862-4407-abe8-2746784bdfc3"],[{"index":{"value":538},"size":{"value":1}},"9adae78d-7667-4865-83e9-b64464b13696"],[{"index":{"value":538},"size":{"value":18}},"92f153e3-da05-439e-bb4f-596ec8cfa9c8"],[{"index":{"value":539},"size":{"value":1}},"dddedc5e-415d-4f5d-ab1b-19468aa079a9"],[{"index":{"value":540},"size":{"value":1}},"4adce2e5-1e60-4a63-9ff3-5781d967b704"],[{"index":{"value":542},"size":{"value":13}},"ef06ed1e-246f-4106-a4c0-1e648b8ad5e7"],[{"index":{"value":555},"size":{"value":1}},"7bd1cc5f-20b1-4dd2-8a34-ccc40f5f091a"],[{"index":{"value":556},"size":{"value":1}},"720adbd6-24b0-49ac-b123-61cc037a9636"]]
{"ide":{"node":{"e3b7fac7-0c08-4f20-8a0c-a58f1b118097":{"position":{"vector":[-224,13]}},"bb556514-570d-4a87-8b2a-6e7f198a975f":{"position":{"vector":[115,19]}},"71a97e88-9b19-4ec9-b3c8-ffd2940c4cb8":{"position":{"vector":[-224,-51]}},"c7bbc3e7-1377-429e-b60a-0c2d2cd4ea74":{"position":{"vector":[360,19]},"visualization":{"show":false,"fullscreen":false,"width":200}},"f2158ac6-2dd5-482d-9f3d-0bf4812d6d6e":{"position":{"vector":[297,-45]},"visualization":{"show":false,"fullscreen":false,"width":96}},"9457c3f8-c878-4f01-a5c7-39336f163a28":{"position":{"vector":[640,40]},"visualization":{"show":false,"fullscreen":false,"width":200}},"b2df8ba8-683c-45c2-9932-8e1c970c799a":{"position":{"vector":[219,-80]},"visualization":{"show":true,"fullscreen":false,"width":200}},"f40730ff-ac6c-45db-8970-534f3becda2a":{"position":{"vector":[219,-48]}}},"import":{}}}
```

To test the functionality, I asked AI to show me the result of the `operator70395` variable:

1. Init protocol connection
```json
{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}
```

2. Create execution context
```json
{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}
```

3. Push the main method
```json
{"jsonrpc":"2.0","id":0,"method":"executionContext/push","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","stackItem":{"methodPointer":{"module":"local.New_Project_1.Main","definedOnType":"local.New_Project_1.Main","name":"main"},"positionalArgumentsExpressions":[],"type":"ExplicitCall"}}}
```

4. Ask AI for the variable contents
```json
{"jsonrpc":"2.0","id":1,"method":"ai/completion_v2","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","expressionId":"f09a4372-3231-4f2e-99f4-84aa751f9b60","prompt":"There is 'operator70395' variable defined in the program. What is the result of the variable 'operator70395'?"}}
```

I got the following responses:
```json
{"jsonrpc":"2.0","method":"ai/completionProgress","params":{"code":"Visualization.AI.print(operator70395)","reason":"To provide the result of 'operator70395', I need to know its current value.","visualizationId":"edfb00a3-6ce5-41e1-bb8f-ab191809114e"}}
```
```json
{"jsonrpc":"2.0","id":1,"result":{"Success":{"fn":"def get_operator70395_result():\n    return operator70395","fnCall":"get_operator70395_result()"}}}
```
2024-05-02 16:55:06 +00:00
Dmitry Bushev
5995a00958
Run ydoc-server with GraalVM (#9528)
part of #7954

# Important Notes
The workflow is:
- `$ npm install` -- just in case
- `$ npm --workspace=enso-gui2 run build-ydoc-server-polyglot` -- build the `ydocServer.js` bundle
- `$ sbt ydoc-server/assembly` -- build the ydoc server jar
- `env POLYGLOT_YDOC_SERVER=true npm --workspace=enso-gui2 run dev` -- run the dev server with the polyglot ydoc server. Providing `POLYGLOT_YDOC_SERVER_DEBUG=true` env variable enables the chrome debugger
2024-05-02 06:28:57 +00:00
Jaroslav Tulach
931baa4276
Towards simplifying runtime-compiler dependencies (#8894) 2024-04-25 10:03:42 +02:00
Hubert Plociniczak
58009b7c04
In-memory suggestions (#9751)
This change replaces an sqllite-backed suggestions' repo with a simple, in-memory, one.
As `completion` functionality has been implemented completely in GUI, there is no need to support it in backend, which simplifies a lot of functionality.

Closes #9650 and #9471.

# Important Notes
Loading suggestions and sending them to GUI on startup is almost instantaneous. Previously it would take ~10s just for `Standard.Base`.
2024-04-22 11:02:17 +00:00
Michał Wawrzyniec Urbańczyk
cdf031f61f
[CI] Provide latest FlatBuffers to the Engine build. (#9654)
This PR bumps the FlatBuffers version used by the backend to `24.3.25` (the latest version as of now).

Since the newer FlatBuffers releases come with prebuilt binaries for all platforms we target, we can simplify the build process by simply downloading the required `flatc` binary from the official FlatBuffers GitHub release page. This allows us to remove the dependency on `conda`, which was the only reliable way to get the outdated `flatc`.

The `conda` setup has been removed from the CI steps and the relevant code has been removed from the build script.

The FlatBuffers version is no longer hard-coded in the Rust build script, it is inferred from the `build.sbt` definition (similar to GraalVM).

# Important Notes
This does not affect the GUI binary protocol implementation.

While I initially wanted to update it, it turned out farly non-trivial.

As there are multiple issues with the generated TS code, it was significantly refactored by hand and it is impossible to automatically update it. Work to address this problem is left as [a future task](https://github.com/enso-org/enso/issues/9658).

As the Flatbuffers binary protocol is guaranteed to be compatible between versions (unlike the generated sources), there should be no adverse effects from bumping `flatc` only on the backend side.
2024-04-12 10:10:44 +00:00
Dmitry Bushev
ab552ab677
GUI project does not receive suggestion database (#9634)
close #9558

the `InvalidateModulesIndexCommand` may be waiting for the execution lock, while the handler times out.
2024-04-08 08:59:03 +00:00
Dmitry Bushev
fb0559e7ab
Put all reexports of given entry in Suggestion Database (#9454)
close #9351

Changelog:
- update: deprecate the `reexport` suggestion field
- add: `reexports` suggestion field containing the list of modules re-exporting this symbol
- update: exports logic to gather all the symbols exported from a given module
2024-03-26 16:33:25 +00:00
Adam Obuchowicz
a6fc8cb932
Reimplement AI PoC from GUI1 (#9476)
Fixes #9313

[Screencast from 2024-03-22 09-09-07.webm](https://github.com/enso-org/enso/assets/3919101/6ad86145-6882-4bde-993d-b1270f1ec06c)

# Important Notes
* This is PoC, so I didn't spend time on polishing the visuals; the design will likely change.
* I modified the shortcut handler a bit, allowing making multiple actions for same binding - the action's handler will be called in unspecified order, until one of them handle the event (i.e. not return false).
* To make it working regardless of imports, I needed to export AI module in Standard.Visualization. Moreover, needed to remove build_ai_prompt for Any, because it was causing issues - expect a bug report soon.
2024-03-25 09:14:41 +00:00
Dmitry Bushev
d21cdb2197
Fix getters serialization (#9480)
close #9429

Changelog:
- fix: serialization of getter methods

# Important Notes
Returns a correct suggestion for the count getter
```json
{
"type":"Add",
"id":248,
"suggestion":{
"type":"method",
"module":"Standard.Base.Data.Index_Sub_Range",
"name":"count",
"arguments":[
{
"name":"self",
"reprType":"Standard.Base.Data.Index_Sub_Range.Index_Sub_Range",
"isSuspended":false,
"hasDefault":false,
"defaultValue":null,
"tagValues":null
}
],
"selfType":"Standard.Base.Data.Index_Sub_Range.Index_Sub_Range",
"returnType":"Standard.Base.Data.Numbers.Integer",
"isStatic":false,
"annotations":[ ]
}
}
```
2024-03-19 15:33:26 +00:00
Dmitry Bushev
7ff90aa3f7
Synchronize suggestions loading after the reconnect 2 (#9142)
related #8689, #9072

Fixes a race between the language server SQL updating logic and the engine `DeserializeLibrarySuggestionsJob`s when the library suggestions may start loading before the database is properly cleaned up after the reconnect.

# Important Notes
As a side effect, arguments are showing slightly (~1 second) faster due to the lower contention between the engine jobs.

#### Before
https://github.com/enso-org/enso/assets/357683/cbda2da4-9080-4b9b-b836-81e54694d468

#### After
https://github.com/enso-org/enso/assets/357683/bf442284-47be-456d-b1dd-2413b6ad8244
2024-02-23 11:18:01 +00:00
Hubert Plociniczak
0133e9ef4f
Drop inefficient SemVer implementation (#9089)
`Bump` library uses parser combinators behind the scenes which are known to be good at expressing grammars but are not performance-oriented.

This change ditches the dependency in favour of an existing Java implementation. `jsemver` implements the full specification, which is probably an overkill in our case, but proved to be an almost drop-in replacement for the previous library.

Closes #8692

# Important Notes
Peformance improvements:
- roughly 50ms compared to the previous approach (from 80ms to 20-40ms)

I don't see any time spent in the new implementation during startup so it could be potentially aggressively inlined.
Further more, we could use a facade and offer our own strip down version of semver.
2024-02-22 09:59:09 +00:00
Hubert Plociniczak
5c624d21d7
Revert "Synchronize suggestions loading after the reconnect (#9043)" (#9072) 2024-02-15 13:39:56 +00:00
Dmitry Bushev
6efa26b48e
Add support for positionalArgumentsExpressions of text visualizations (#9052)
close #9028

Changelog:
- add: support `positionalArgumentsExpressions` parameter for visualization expressions represented as strings
- add: language server API tests
- add: runtime visualization tests

# Important Notes
https://github.com/enso-org/enso/assets/357683/4e6f207b-ed64-41b9-b94b-96fb9db63ce4
2024-02-13 23:08:30 +00:00
Dmitry Bushev
9c982e07b0
Synchronize suggestions loading after the reconnect (#9043)
related #8689

Fixes a race between the language server SQL updating logic and the engine `DeserializeLibrarySuggestionsJob`s when the library suggestions may start loading before the database is properly cleaned up after the reconnect.
2024-02-13 17:52:15 +00:00
Hubert Plociniczak
b00dc9e9c0
Minor cleanups to docker image (#8925)
* Minor cleanups to docker image

* format

* nit
2024-02-11 21:52:47 +01:00
Dmitry Bushev
880da9fb56
Cleanup execution contexts when the client disconnects (#9004)
related #8689

Clean up the client's execution contexts when it disconnects from the language server. Dangling execution contexts may slow down the execution when the user reconnects to the language server.
2024-02-09 14:16:34 +00:00
Ikko Eltociear Ashimine
66dfefe810
Fix typo in ContextEventsListener.scala (#8958) 2024-02-09 14:01:02 +00:00
Hubert Plociniczak
b8612344ca
Disable flaky test (#8967) 2024-02-05 13:41:43 +00:00
Hubert Plociniczak
34e1bacdf1
Further optimizations to test resources (#8936)
TestRuntime should be deprecated as it creates a number of threads and doesn't allow to easily modify ZIO's runtime.
But the biggest drop stems from fixing leaking `FileSystemService` that weren't being closed for every `TextOperationsTest` test.
The change is a follow up on #8892 but this time focused on ZIO usage.

Hopefully fixes #8806 for good.

# Important Notes
Running `language-server/test`.
Before:
![Screenshot from 2024-02-02 09-48-32](https://github.com/enso-org/enso/assets/292128/fb414c74-7d7a-4e7b-8b0c-d25dc3721bbf)

After:
![Screenshot from 2024-02-02 09-46-02](https://github.com/enso-org/enso/assets/292128/db9429df-d861-4f48-818f-888d5bbbb089)
2024-02-02 22:49:48 +00:00
Hubert Plociniczak
68ce4ae1b7
Reduced resources for various Akka dispatchers (#8892)
The defaults picked up by Akka tend to make us of all resources which is unnecessary and overwhelming for tests.

Improves #8806, potentially.

Before
![Screenshot from 2024-01-28 22-34-42](https://github.com/enso-org/enso/assets/292128/f80eb66a-2f37-44d5-bcdb-f00a78fe72fd)
After
![Screenshot from 2024-01-31 00-12-10](https://github.com/enso-org/enso/assets/292128/c5223912-5f6e-413c-a0a4-050afa3ed463)

when running the problematic `LibrariesTest`.

Full `language-server` test suite.
Before
![Screenshot from 2024-01-31 00-20-50](https://github.com/enso-org/enso/assets/292128/f1c94a66-6905-4f57-8a7d-7df049714353)
After
![Screenshot from 2024-01-31 00-18-40](https://github.com/enso-org/enso/assets/292128/3a11125e-d593-43df-8d35-1a8915812b2b)

# Important Notes
Note that Executors assigned to Zio and initializers should also be improved. Unfortunately due to various blocking threadpools  it is easy to get timeouts when running the whole suite.
2024-01-31 10:25:17 +00:00
Hubert Plociniczak
081c8c889c
Don't cancel pending visualization's upserts (#8853)
Uniqueness check of `UpsertVisualizationJob` only involved expressionId. Apparently now GUI sends mutliple visualizations for the same expressions and expects all of them to exist. Since previously we would cancel duplicate jobs, this was problematic.

This change makes sure that uniqueness also takes into account visualization id. Fixed a few logs that were not passing arguments properly.

Closes #8801

# Important Notes
I have not noticed any more problems with loading visualizations so the issue appears to be resolved with this change.
Added a unit test case that would previously fail due to cancellation of a job that upserts visualization.
2024-01-30 00:13:43 +00:00
Radosław Waśko
368e4867b4
Allow secrets in AWS_Credential (#8774)
- Closes #8722
2024-01-19 19:00:56 +00:00