Commit Graph

298 Commits

Author SHA1 Message Date
Dmitry Bushev
b7d51ed5c6
Create unique atom getter suggestions (#6694)
related #6611

Prevent creating atom getters with the same name.
2023-05-15 14:03:18 +00:00
Hubert Plociniczak
06624f34a8
Ensure slow shutdown of LS always kicks off hooks (#6665)
This change fixes the rather elusive bug where shutdown hooks could not be fired when shutdown was taking too long and termination was forced.

Under the circumstances described in detail in ticket #6515 there was a small chance that we could have a shutdown race condition. Essentially the messages received when client was disconnected and language server forced the termination could lead to language server not sending the public `ProjectClosed` message which triggers shutdown hook. Now we always do.

Also made sure that multiple `ProjectClosed` messages don't lead to firing multiple shutdown hooks, which was another possibility.

No tests as one would have to be able to introduce different delays in various message handlers to simulate the problem.
Having ability to do such chaos testing would be nice but it is beyond the scope of this ticket.
I was able to reproduce the problem 100% with my specially crafted setup so I'm fairly confident about the change.

Closes #6515.
2023-05-15 12:15:37 +00:00
Dmitry Bushev
706791779b
SuggestionBuilder needs to send ascribedType of constructor parameters (#6655)
close #6611

Changelog:
- update: run compiler passes on the `ascribedType` field of the constructor arguments
- update: suggestion builder uses the type information attached to `ascribedType`
- feat: resolve qualified names in type signatures
2023-05-13 18:33:03 +00:00
Jaroslav Tulach
1302b693d8
All Vector operations shall be applicable on java.util.ArrayList (#6642)
Fixes #6609 by
- e380e647af - running whole `Vector_Spec` on `java.util.ArrayList`
- 9b1229fe20 - introducing a node to handle interop values

# Important Notes
Contains additional DSL processor fix:
- 415623dcb9 - to not crash the compiler, but to properly report compiler error
2023-05-11 15:29:47 +00:00
Dmitry Bushev
3dd05c2359
Automatic type based dropdown does not include singleton in a union type (#6629)
close #6532

Set tag values to display `Auto` type in dropdowns correctly.

# Important Notes
![2023-05-10-141458_969x566_scrot](https://github.com/enso-org/enso/assets/357683/9a048b3c-d192-4382-bf76-9cbe6c9556d1)
![2023-05-10-141513_991x232_scrot](https://github.com/enso-org/enso/assets/357683/c50e1e73-23a6-4b32-90bf-f849a127c85d)
2023-05-10 16:55:16 +00:00
Hubert Plociniczak
cf3624c463
Limit the number of reported warnings (#6577)
Artifically limiting the number of reported warnings to 100. Also added benchmarks with random Ints to investigate perf issues when dealing with warnings (future task).
Ideally we would have a custom set-like collection that allows us internally to specify a maximal number of elements. But `EnsoHashMap` (and potentially `EnsoSet`) are still WIP when it comes to being PE-friendly.

The change also allows for checking if the limit for the number of reported warnings has been reached. It will visualize by adding an additional "Warnings limit reached." to the visualization.

The limit is configurable via `--warnings-limit` parameter to `run`.

Closes #6283.
2023-05-10 11:48:31 +00:00
Radosław Waśko
41a8257e8d
Separating Redshift connector from Database library into a new AWS library (#6550)
Related to #5777
2023-05-04 17:36:51 +00:00
Hubert Plociniczak
b93edb050b
Force pending saves if client closes abruptly (#6514)
Force pending saves when JsonSession is terminated for the client.

Potentially closes #6395.
2023-05-03 23:30:20 +00:00
Dmitry Bushev
38212786a3
Fix application config (#6513)
Related #6482
2023-05-02 17:59:44 +00:00
Pavel Marek
3a42d0ce76
Reimplement enso_project as a proper builtin (#6352)
Remove the magical code generation of `enso_project` method from codegen phase and reimplement it as a proper builtin method.

The old behavior of `enso_project` was special, and violated the language semantics (regarding the `self` argument):
- It was implicitly declared in every module, so it could be called without a self argument.
- It can be called with explicit module as self argument, e.g. `Base.enso_project`, or `Visualizations.enso_project`.

Let's avoid implicit methods on modules and let's be explicit. Let's reimplement the `enso_project` as a builtin method. To comply with the language semantics, we will have to change the signature a bit:
- `enso_project` is a static method in the `Standard.Base.Meta.Enso_Project` module.
- It takes an optional `project` argument (instead of taking it as an explicit self argument).

Having the `enso_project` defined as a (shadowed) builtin method, we will automatically have suggestions created for it.

# Important Notes
- Truffle nodes are no longer generated in codegen phase for the `enso_project` method. It is a standard builtin now.
- The minimal import to use `enso_project` is now `from Standard.Base.Meta.Enso_Project import enso_project`.
- Tested implicitly by `org.enso.compiler.ExecCompilerTest#testInvalidEnsoProjectRef`.
2023-05-02 16:40:58 +00:00
Hubert Plociniczak
ce4ecc6706
Limit Dead Letter logging (#6482)
Dead Letter logging is occasionally flooding our logs which is confusing to users reporting bugs. Left the possibility of a single report so that we know that something is happening.
2023-05-02 14:00:50 +00:00
Jaroslav Tulach
efe904cd9f
Introducing @BuiltinMethod.needsFrame and InlineableNode (#6442)
Fixes #6416 by introducing `InlineableNode`. It runs fast even on GraalVM CE, fixes ([forever broken](https://github.com/enso-org/enso/pull/6442#discussion_r1178782635)) `Debug.eval` with `<|` and [removes discouraged subclassing](https://github.com/enso-org/enso/pull/6442#discussion_r1178778968) of `DirectCallNode`. Introduces `@BuiltinMethod.needsFrame` - something that was requested by #6293. Just in this PR the attribute is optional - its implicit value continues to be derived from `VirtualFrame` presence/absence in the builtin method argument list. A lot of methods had to be modified to pass the `VirtualFrame` parameter along to propagate it where needed.
2023-04-28 15:32:13 +00:00
Hubert Plociniczak
c6790f1e9c
Report only unique warnings (#6372)
This change makes sure that reported warnings are unique, based on the value of internal clock tick and ignoring differences in reassignments.

Before:
![Screenshot from 2023-04-20 15-42-55](https://user-images.githubusercontent.com/292128/233415710-925c1045-37c7-49f5-9bc3-bfbfd30270a3.png)
After:
![Screenshot from 2023-04-20 15-27-27](https://user-images.githubusercontent.com/292128/233415807-8cb67bc2-ac37-4db7-924e-ae7619074b5b.png)

On the positive side, no further changes, like in LS, have to be done.


Closes #6257.
2023-04-28 07:16:00 +00:00
Hubert Plociniczak
376415ab17
Mark some tests as (unconditonally) flaky (#6453)
Too many spurious failures on CI now. Will need to be tackled at some
point.
2023-04-27 17:09:46 +02:00
Dmitry Bushev
2d168308af
Better cleanup of project management test suite (#6392)
Address some arbitrary failures of the `ProjectManagementApiSpec` test suite.
2023-04-24 13:11:27 +00:00
Dmitry Bushev
74c0718604
Project folder not renamed after project name change (#6369)
close #6254

Changelog:
- fix: race when the actor system may be stopped before the shutdown hooks are executed
- fix: project management spec
- fix: recover from `readLine` failure during the shutdown
2023-04-20 16:57:37 +00:00
Hubert Plociniczak
4c1e73c864
Don't propagate warnings on suspended arguments (#6345)
* Don't propagate warnings on suspended arguments

In the current implementation, application of arguments with warnings
first extracts warnings, does the application and appends the warnings
to the result.
This process was however too eager if the suspended argument was a
literal (we don't know if it will be executed after all).

The change modifies method processor to take into account the
`@Suspend` annotation and not gather warnings before the application
takes place.

* PR review
2023-04-19 11:55:23 +02:00
Jaroslav Tulach
e5a96b9782
Avoid NullPointerException in --dumpGraphs & test & docs (#6304) 2023-04-18 06:58:06 +02:00
Dmitry Bushev
93b9ca7789
Update the default project template (#6310)
close #6307

Changelog:
- update: the default project template

# Important Notes
[6307-update-the-default-project-template.webm](https://user-images.githubusercontent.com/357683/232487322-8aebc709-30aa-48f3-b86b-f868c83a6e21.webm)
2023-04-17 14:09:08 +00:00
Jaroslav Tulach
a74933d10f
Speed cascade of if statements up (#6255)
Fixes #5709. We have a test and a generic fix that improves inlining of every builtin. Everything seems to be faster.
2023-04-14 13:27:23 +00:00
Dmitry Bushev
b97fc39214
Remove SQL versions repo (#6242)
close #6232

Changelog:
- remove: `SqlVersionsRepo`
- update: `SuggestionsDatabaseModuleUpdateNotification` message removing the version
- update: cleanup versions repo usages in the language server
2023-04-11 19:22:30 +00:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Jaroslav Tulach
4805193428
Text.to_display_text is (shortened) identity (#6174)
Fixes #5971.
2023-04-05 19:53:07 +00:00
Dmitry Bushev
2338e5d8e6
When renaming the project clean old modules instead of updating (#6148)
close #6139
close #6137

When the project is renamed, the engine cleans up affected modules and initiates modules re-indexing to fill the suggestions database with new records. This way it reduces the amount of information stored in the suggestions database and helps implement #6080 optimization.

Changelog:
- remove: rename features from the suggestions database
- update: rename command to initiate modules cleanup and project re-execution
- fix: #6137
2023-03-31 09:40:21 +00:00
Hubert Plociniczak
b3db805f92
More elaborate error message for invalid text edit (#6108)
It is sometimes impossible to figure out the real reason for invalid text edit request. Added a bit of context to failures to narrow down the cause of the failure.

# Important Notes
Should help with diagnosing issues like #6099.
2023-03-28 14:21:50 +00:00
Dmitry Bushev
aa97787603
Update ZIO library (#6072)
close #6069

Changelog:
- update: ZIO major version bump
2023-03-28 07:58:59 +00:00
James Dunkerley
bf2545fa04
Use new common parse method throwing less exceptions. (#6075)
Avoiding exceptions by not using parseBest.

Time now in CLI is 1.15s for 500k rows vs 1.65s in GUI.

CLI:
![image](https://user-images.githubusercontent.com/4699705/227711266-bc005b0d-5011-450f-964b-65dd2e437c2e.png)

GUI:
![image](https://user-images.githubusercontent.com/4699705/227711259-f7ddda29-86c7-4eef-a002-4bf0bda6063f.png)

Added it as a function in the shared library so used by both engine and polyglot.
2023-03-27 11:02:10 +00:00
Dmitry Bushev
4c62dc9061
Search suggestions by static attribute (#6036)
close #5874

Changelog:
- add: `isStatic` parameter to `search/completion` request to search by the `static` suggestion attribute
- update: search non-static suggestions when opening component browser

# Important Notes
Component browser doesn't show `Table.new` and `Table.from_rows` suggestions when a `Table` node is selected.

![2023-03-21-151117_1301x877_scrot](https://user-images.githubusercontent.com/357683/226874291-1ff99994-1bb6-41df-96b4-dc5c5178ba41.png)
2023-03-23 15:02:25 +00:00
Dmitry Bushev
8125d89676
Delay initialization of JSON-RPC protocol (#5997)
close #5881

Changelog
- add: ProtocolFactory object that initializes and returns the protocol object
- update: Add protocol initialization to the initialization component
2023-03-20 11:04:16 +00:00
Dmitry Bushev
b2aa58bf20
Delay writing IR caches (#5957)
close #5911

In interactive mode, perform writing IR caches in the background jobs queue. Background jobs execution is delayed until the first execution is complete.
2023-03-17 22:47:15 +00:00
Dmitry Bushev
265fb5d505
Delay initialization of SQL database (#5970)
close #5875

Move database initialization logic to the initialization component.
2023-03-17 13:38:11 +00:00
Hubert Plociniczak
a6487e5105
Don't log installed engines and runtimes in prod (#5900)
The `logAvailableComponentsForDebugging` will check and install all necessary components of GraalVM for every mentioned version. While not harmful, it adds up to startup time.
Additionally added an option in language server startup to skip installation of GraalVM components. The latter is already performed by project-manager when opening the project and it is unnecessary to do it twice. Due to LS' architecture this configuration has to be passed around via multiple configs.

Finally, skipped the attempt to install Python component on Windows - this is not supported by GraalVM atm.

Closes #5749.

# Important Notes
The impact of this problem could be really felt the more versions of Enso and GraalVM one had since it would go through all of them.
2023-03-16 10:36:55 +00:00
Kaz Wesley
e171fba301
New documentation parser (#5917)
Implement new Enso documentation parser; remove old Scala Enso parser.

Performance: Total time parsing documentation is now ~2ms.

# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2023-03-15 15:43:51 +00:00
Radosław Waśko
263c3ad651
Add a common-polyglot-core-utils project (#5855)
Adds a common project that allows sharing code between the `runtime` and `std-bits`.

Due to classpath separation and the way it is compiled, the classes will be duplicated - we will have one copy for the `runtime` classpath and another copy as a small JAR for `Standard.Base` library.

This is still much better than having the code duplicated - now at least we have a single source of truth for the shared implementations.

Due to the copying we should not expand this project too much, but I encourage to put here any methods that would otherwise require us to copy the code itself.

This may be a good place to put parts of the hashing logic to then allow sharing the logic between the `runtime` and the `MultiValueKey` in the `Table` library (cc: @Akirathan).
2023-03-11 09:27:26 +00:00
Dmitry Bushev
725841467f
Log exceptions with stack traces (#5871)
close #5070

Changelog:
- Include the original exception to log expressions
- Enable logging of Akka Actors' lifecycle events on debug logging level
- Decrease the severity of interruption log messages because interruptions are part of the workflow. The computation can be interrupted at any time, and still be recomputed after. Warnings are just misleading in this case.
2023-03-11 08:23:31 +00:00
Jaroslav Tulach
632a303089
Use SimpleFormatter.formatMessage to replace {0} with actual parameters (#5870)
Fixes #5801 to properly format Truffle log records before sending them for further processing.
2023-03-11 00:15:58 +00:00
Hubert Plociniczak
6769ab0ee7
Downgrade hashing to SHA-1 and other optimizations (#5791)
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.

Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.

Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.

Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.

Finally fix an SBT warning when generating library cache.

Closes https://github.com/enso-org/enso/issues/5763

# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
2023-03-09 07:36:59 +00:00
Dmitry Bushev
9397a6ec2f
Pre compute suggestion db during build time (#5698)
Close #5068

Cache suggestions during the `buildEngineDistribution` command, and read them from the disk when the library is loaded. Initial graph coloring takes ~20 seconds vs ~25 seconds on the develop branch.

[peek-develop-branch.webm](https://user-images.githubusercontent.com/357683/223504462-e7d48262-4f5e-4724-b2b0-2cb97fc05140.webm)
[peek-suggestions-branch.webm](https://user-images.githubusercontent.com/357683/223504464-0fe86c04-8c4b-443c-ba96-6c5e2fb1e396.webm)
2023-03-08 12:37:48 +00:00
Hubert Plociniczak
941512e0ba
Optimize import/export resolution (#5700)
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.

When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.

The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.

Part of https://github.com/enso-org/enso/issues/5568 work.
2023-03-01 08:53:29 +00:00
James Dunkerley
ba3d45e782
Adjust generateCheckedArgumentRead to return correct expected type (#5769)
Correctly get the expected type and return an IllegalArgument if no Enso type.

![image](https://user-images.githubusercontent.com/4699705/221263862-bbc122ca-b11f-49f3-b7cf-6294ea811a22.png)
2023-02-27 17:23:45 +00:00
Jaroslav Tulach
dfea59c24d
Exception message may not be specified (#5715)
Getting ready for `null` values in `WSLogMessage`. Closes #5683.
2023-02-22 03:43:46 +00:00
Galin Bajlekov
725b3da486
Roll back last VCS snapshot (#4050)
Fixes #5001

This PR implements reverting the current project state to the last state saved into the VCS. This action is performed on `ctrl+r`.

https://user-images.githubusercontent.com/117099775/216645556-1bf34ee7-fdb4-4833-bcad-670d688a3199.mp4

# Important Notes
* Currently on `vcs/restore` all expressions are invalidated and all nodes are re-executed. This is tracked in [task](https://www.pivotaltracker.com/n/projects/2539304/stories/184368950).
2023-02-16 18:14:34 +00:00
Paweł Buchowski
f316ba0136
add missing templates to allTemplates seq (#5657)
Add recently added templates to allTemplates seq so they are accessed from the command line
2023-02-14 17:17:28 +00:00
Paweł Buchowski
95c66baa3c
extend list of allowed project templates (#5601)
In cloud we want to allow users to create new project from the template. List of templates is a bit outdated and doesn't contain all from the https://github.com/enso-org/project-templates. This PR simply adds missing ones
2023-02-13 13:03:51 +00:00
Hubert Plociniczak
9ea9fd56e6
Fix separate compilation for ConstantsGen (#5630)
A combination of commands triggered separate compilation that only recompiled part of builtins. A mechanism that workaround annotation processor's problems with separate compilation was updated to include changes from https://github.com/enso-org/enso/pull/4111. This was pretty tough to find given the rather unusual circumstances in CI.

# Important Notes
This eliminates the _random_ failures in CI related to separate compilation.
To reproduce a specific set of steps has to be executed:
```
sbt> all buildEngineDistribution engine-runner/assembly runtime/Benchmark/compile language-server/Benchmark/compile searcher/Benchmark/compile
(exit sbt)
sbt> test
```
2023-02-10 22:14:43 +00:00
Hubert Plociniczak
472580df4d
Report type of expressions returning polyglot values (#4111)
Expressions returning polyglot values were not reporting the type of the result because we have to do additional magic that infers the correct Enso type. Since this is exactly what `TypeOfNode` does, I re-used the logic.

Straightforward solution failed in tests because of assertions:
```
[enso] WARNING: Execution of function main failed (Invalid library usage. Cached library must be adopted by a RootNode before it is executed.).
java.lang.AssertionError: Invalid library usage. Cached library must be adopted by a RootNode before it is executed.
```

That is why this PR replaces `ExecutionEventListener` with `ExecutionEventNodeFactory`.

# Important Notes
Usage of `TypeOfNode` for programs that **do not** import stdlib means that we report types that do not involve stdlib e.g.
`Standard.Builtins.Main.Integer` instead of `Standard.Base.Data.Numbers.Integer`. While surprising, this is correct and I would say desirable. While reviewing the code, notice the difference in expectations in our runtime tests.
2023-02-09 01:06:27 +00:00
Jaroslav Tulach
f54464881f
enso4igv can open engine/language-server & co. projects (#4098)
In order to investigate `engine/language-server` project, I need to be able to open its sources in IGV and NetBeans.

# Important Notes
By adding same Java source (this time `package-info.java`) and compiling with our Frgaal compiler the necessary `.enso-sources*` files are generated for `engine/language-server` and then the `enso4igv` plugin can open them and properly understand their compile settings.

![Logical View of language-server project](https://user-images.githubusercontent.com/26887752/215472696-ec9801f3-4692-4bdb-be92-c4d2ab552e60.png)

In addition to that this PR enhances the _"logical view"_ presentation of the project by including all source roots found under `src/*/*`.
2023-01-31 08:40:04 +00:00
Hubert Plociniczak
be91b1e8dd
Create static wrappers for builtin types (#4077)
https://github.com/enso-org/enso/pull/3764 introduced static wrappers for instance methods. Except it had a limitation to only be allowed for types with at least a single constructor.
That excluded builtin types as well which, by default, don't have them. This limitation is problematic for Array/Vector consolidation and makes builtin types somehow second-citizens.

This change lifts the limitation for builtin types only. Note that we do want to share the implementation of the generated builtin methods. At the same time due to the additional argument we have to adjust the starting index of the arguments.
This change avoids messing with the existing dispatch logic, to avoid unnecessary complexity.

As a result it is now possible to call builtin types' instance methods, statically:
```
arr = Array.new_1 42
Array.length arr
```
That would previously lead to missing method exception in runtime.

# Important Notes
The only exception is `Nothing`. Primarily because it requires `Nothing` to have a proper eigentype (`Nothing.type`) which would messed up a lot of existing logic for no obvious benefit (no more calling of `foo=Nothing` in parameters being one example).
2023-01-30 19:54:51 +00:00
Hubert Plociniczak
c85377f0ac
Eliminate various compiler warnings (#4079)
Slipped through review of https://github.com/enso-org/enso/pull/3862
2023-01-24 22:58:35 +00:00
Marcin Kostrzewa
242bd52942
Unboxed atoms (#3862)
Introduces unboxed (and arity-specialized) storage schemes for Atoms. It results in improvements both in memory consumption and runtime.
Memory wise: instead of using an array, we now use object fields. We also enable unboxing. This cuts a good few pointers in an unboxed object. E.g. a quadruple of integers is now 64 bytes (4x8 bytes for long fields + 16 bytes for layout and constructor pointers + 16 bytes for a class header). It used to be 168 bytes  (4x24 bytes for boxed Longs + 16 bytes for array header + 32 bytes for array contents +  8 bytes for constructor ptr  + 16 bytes for class header), so we're saving 104 bytes a piece. In the least impressive scenarios (all-boxed fields) we're saving 8 bytes per object (saving 16 bytes for array header, using 8 bytes for the new layout field). In the most-benchmarked case (list of longs), we save 32 bytes per cons-cell.
Time wise:
All list-summing benchmarks observe a ~2x speedup. List generation benchmarks get ~25x speedups, probably both due to less GC activity and better allocation characteristics (only allocating one object per Cons, rather than Cons + Object[] for fields). The "map-reverse" family gets a neat 10x speedup (part of the work is reading, which is 2x faster, the other is allocating, which is now 25x faster, we end up with 10x when combined).
2023-01-24 13:03:06 +00:00