Commit Graph

900 Commits

Author SHA1 Message Date
James Dunkerley
56688ec1e7
Minor fixes. (#7122)
Mostly stuff to tidy up the static methods in the CB.

- Remove default pattern from `parse_to_table` (caused IDE to freeze).
- Rename any `_` arguments to what they are.
- Merge `Date.now` into `Date.today`
- Merge the Interval constructors into a single constructor.
- Hide various methods.
2023-06-27 18:18:15 +00:00
Jaroslav Tulach
477dd82670
Drop-down widgets for extension functions via UnresolvedSymbol (#7115)
Fixes #6955 by:
- using `visualisationModule` to specify the module where the visualization is to be used
- referring to method in `Meta.get_annotation` with `.method_name` - e.g. unresolved symbol notation
- evaluating arguments to `Meta.get_annotation` in the context of the user module (which can access the extension functions)
2023-06-27 15:19:42 +00:00
Dmitry Bushev
22259e696d
Add method call info for infix operators (#7090)
close #6374

In order to provide the method pointer information, the `IrToTruffle` pass sets the module name and the type name of the builtin node.
2023-06-27 13:11:12 +00:00
Hubert Plociniczak
4e5cb9cca6
Consistent naming of visualization (#7124) 2023-06-26 17:04:35 +00:00
Hubert Plociniczak
ae4666c4d3
Improve TCO in the presence of warnings (#7116)
Partially revert https://github.com/enso-org/enso/pull/6849, which introduced a regression in TCO in the presence of warnings. Rather than modifying the tail call status, `TailCallException` now propagates the extracted warnings and appends them to the final result.

Closes #7093

# Important Notes
Compared to the previous attempt we don't pay the penalty of adding the warnings or even checking for them because it is being dealt in a separate specialization.
2023-06-26 12:38:36 +00:00
Jaroslav Tulach
c4f19e7d66
Vector.map & State (#7078) 2023-06-26 13:13:41 +02:00
Jaroslav Tulach
6abcbcb00c
Moving construction of IR into runtime parser project (#7096) 2023-06-24 06:34:21 +02:00
Hubert Plociniczak
6c777834e4
Don't instrument function bodies (#7097)
Function bodies cannot be instrumented even if the function is right
inside a binding. Consider a scenario when a function is assigned to a
variable and then applied to a `map` method of a really large vector.
The instrumentation will render execution extremely slow.

Alternatively we would still support instrumenting function bodies in
this limited case but take into account the number of times function is
actually called.
2023-06-23 19:16:25 +02:00
Pavel Marek
31aad1d9e6
Always call instance methods on Any (#7033)
Previously, static method calls on `Any` have not worked as expected. For example, `Any.to_text` returned Function instead of Text. That is because the function resolution for `Any.to_text` finds `Any.type.to_text` method on eigentype which expects two `self` arguments, but only one argument is provided.

Note that `Boolean.to_text` worked previously, and returned "Boolean" as expected. This is because the method resolution finds `Any.to_text` method that takes just one `self` argument.

This PR solves this issue by introducing special handling for static method dispatch on `Any`. Simply put, an additional `self` argument is prepended to the argument list.

# Important Notes
A new child node is introduced to `InvokeMethodNode`. This child node is a copy of the current `invokeFunctionNode` with one more `CallArgumentInfo` in its schema.
2023-06-23 14:46:42 +00:00
Pavel Marek
bbd9f437c5
Empty input in REPL does not throw NPE (#7111)
Fix for an annoying NullPointerException thrown when an empty line is fed to REPL, i.e., when one just presses Enter. Now, you can press Enter, creating as many blank prompts, as you want:
![image](https://github.com/enso-org/enso/assets/14013887/8f38e210-da5c-46a0-b5f1-1a5da9e62584)
2023-06-23 12:05:44 +00:00
Dmitry Bushev
3937d21001
Do not invalidate caches when building engine distribution (#7106)
There is an issue that after a clean build, the compiler is unable to resolve some types, i.e
```py
$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run ~/enso/projects/Unnamed/

/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:11: error: The name `Connection_Details` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
|           ^~~~~~~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:33: error: The name `Connection_Options` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
|                                 ^~~~~~~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:55: error: The name `Connection` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
|                                                       ^~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:68: error: The name `SQL_Error` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
|                                                                    ^~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:25:25: error: The name `Connection_Options` could not be found.
25 | connect details options=Connection_Options.Value =
|                         ^~~~~~~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:29:29: error: The name `Widget` could not be found.
29 | connection_details_widget : Widget
|                             ^~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:31:28: error: The name `Vector` could not be found.
31 |     default_constructors = Vector.from_polyglot_array <|
|                            ^~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:32:63: error: The name `False` could not be found.
32 |         DatabaseConnectionDetailsSPI.get_default_constructors False
|                                                               ^~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:36:9: error: The name `Option` could not be found.
36 |         Option name code
|         ^~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:37:5: error: The name `Single_Choice` could not be found.
37 |     Single_Choice display=Display.Always values=choices
|     ^~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:37:27: error: The name `Display` could not be found.
37 |     Single_Choice display=Display.Always values=choices
|                           ^~~~~~~
Aborting due to 11 errors and 0 warnings.
Execution finished with an error: Compilation aborted due to errors.
```

The compiler can't resolve those symbols because the `IR` cache for `Standard.Database.Connection.Database` is missing. What happens during the `buildEngineDistribution` is:

- Compiler processes libraries one by one
- Compiler processes (and compiles and generates caches) `Standard.Database` library
- Compiler starts processing `Standard.Visualization` library
- During the compilation of `Standard.Database.Connection.Database` module, it sees that the module was loaded from the cache. But some of the required dependencies from `Standard.Table` library were not loaded from the cache. But at this point, the `Standard.Table` library has not been processed yet and the caches for it don't exist. The compiler decides that the `Database` file was changed, and the cache is invalid and should be cleaned.
2023-06-22 14:04:40 +00:00
James Dunkerley
90df5adfb0
Fix defect where constructor fields not show for Value constructors. (#7080)
The `IdExecutionService` now uses `getName` not `getDisplayName` for atomConstructor.
`getDisplayName` adjusts the name if the name is `Value` or `Error`, which meant no parameters in IDE.

Before:
![image](https://github.com/enso-org/enso/assets/4699705/ddc564fe-3213-46ec-9684-ab718f8bb0bb)

Now:
![image](https://github.com/enso-org/enso/assets/4699705/59d3ece4-1148-44bb-8a13-7bc3db8dfca2)
2023-06-20 12:47:16 +00:00
James Dunkerley
1859ccbab5
Improving widgets and other minor tweaks. (#7052)
- Removed `module` argument from `enso_project` (new `Project_Description.new` API).
- Removed the custom option from date and time parse/format dropdowns.
- The `format` dropdown uses the value to create the dropdown. (Screenshot below)
- Removed `StorageType` coalescing rules and replaced them with simpler logic in `ObjectStorage`.
- Update signature for `add_row_number` and add aliases.
2023-06-19 19:03:36 +00:00
Dmitry Bushev
8b65509415
Do not add self arguments of static methods to suggestions database (#7074)
The `self` argument of static methods should not be visible to the user.

# Important Notes
![2023-06-19-190618_1243x459_scrot](https://github.com/enso-org/enso/assets/357683/f9b32911-bf8a-4cbc-ab7a-61beaae3c98b)
2023-06-19 17:48:48 +00:00
Jaroslav Tulach
5f572d0680
Avoid dependency on BindingsMap from IR (#7055) 2023-06-19 10:28:31 +02:00
Jaroslav Tulach
7d0461cb6e
Benchmarking Enso version of the Sieve of Eratosthenes algorithm (#7019) 2023-06-19 09:57:42 +02:00
Marcin Kostrzewa
90e413c7db
AI Completions (#5910) 2023-06-18 23:54:06 +02:00
Jaroslav Tulach
6c2ecc615e
Never return null from InteropLibrary calls (#7044)
Preventing `NullPointerException` from debugger and GraalVM Insight.
2023-06-17 04:30:03 +00:00
Hubert Plociniczak
e529f7bb3b
Improvements that significantly reduce the chances of request timeouts (#7042)
Request Timeouts started plaguing IDE due to numerous `executionContext/***Visualization` requests. While caused by a bug they revealed a bigger problem in the Language Server when serving large amounts of requests:
1) Long and short lived jobs are fighting for various locks. Lock contention leads to some jobs waiting for a longer than desired leading to unexpected request timeouts. Increasing timeout value is just delaying the problem.
2) Requests coming from IDE are served almost instantly and handled by various commands. Commands can issue further jobs that serve request. We apparently have and always had a single-thread thread pool for serving such jobs, leading to immediate thread starvation.

Both reasons increase the chances of Request Timeouts when dealing with a large number of requests. For 2) I noticed that while we used to set the `enso-runtime-server.jobParallelism` option descriptor key to some machine-dependent value (most likely > 1), the value set would **only** be available for instrumentation. `JobExecutionEngine` where it is actually used would always get the default, i.e. a single-threaded ThreadPool. This means that this option descriptor was simply misused since its introduction. Moved that option to runtime options so that it can be set and retrieved during normal operation.

Adding parallelism intensified problem 1), because now we could execute multiple jobs and they would compete for resources. It also revealed a scenario for a yet another deadlock scenario, due to invalid order of lock acquisition. See `ExecuteJob` vs `UpsertVisualisationJob` order for details.

Still, a number of requests would continue to randomly timeout due to lock contention. It became apparent that
`Attach/Modify/Detach-VisualisationCmd` should not wait until a triggered `UpsertVisualisationJob` sends a response to the client; long and short lived jobs will always compete for resources and we cannot guarantee that they will not timeout that way. That is why the response is sent immediately from the command handler and not from the job executed after it.

This brings another problematic scenario:
1. `AttachVisualisationCmd` is executed, response sent to the client, `UpsertVisualisationJob` scheduled.
2. In the meantime `ModifyVisualisationCmd` comes and fails; command cannot find the visualization that will only be added by `UpsertVisualisationJob`, which might have not yet been scheduled to run.

Remedied that by checking visualisation-related jobs that are still in progress. It also allowed for cancelling jobs which results wouldn't be used anyway (`ModifyVisualisationCmd` sends its own `UpsertVisualisationJob`). This is not a theoretical scenario, it happened frequently on IDE startup.

This change does not fully solve the rather problematic setup of numerous locks, which are requested by short and long lived jobs. A better design should still be investigated. But it significantly reduces the chances of Request Timeouts which IDE had to deal with.

With this change I haven't been able to experience Request Timeouts for relatively modest projects anymore.

I added the possibility of logging wait times for locks to better investigate further problems.

Closes #7005
2023-06-16 17:57:16 +00:00
James Dunkerley
760fb71798
First part of AWS S3 API, various small fixes. (#6973)
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
2023-06-15 16:20:13 +00:00
Dmitry Bushev
48f0c6f5e8
Scala 2.13.11 and libraries update (#7010)
Update Scala and libraries.
2023-06-14 13:15:57 +00:00
Pavel Marek
67821bf8df
Add compiler pass that discovers ambiguous imports (#6868)
Add a new compiler pass that analyses duplicated and ambiguous symbols from imports
2023-06-14 12:18:57 +02:00
Dmitry Bushev
6249c79ffd
Update sbt-java-formatter plugin (#7011)
Update java formatter plugin. The new version can remove unused imports.
2023-06-12 14:18:48 +00:00
Dmitry Bushev
6de2700209
Do not return getters from global completion response (#7007)
close #6936

Changelog:
- add: new suggestion type Getter that is not exposed to the api
- update: do not return suggestion of type getter when doing a global search (without specifying self types)

Private suggestions and modules mentioned in the issue will be filtered out after we finish the work on the new (refined) exports algorithm.

# Important Notes
![2023-06-09-205327_1088x612_scrot](https://github.com/enso-org/enso/assets/357683/c6b16894-ada0-4ea9-abe8-5efc41949787)
2023-06-12 12:28:35 +00:00
Dmitry Bushev
b2c1f97437
Test case with a function read from a variable (#6986)
Test checking that if the function is read from variable, it does not have the method pointer associated with it.
2023-06-09 21:53:02 +00:00
Hubert Plociniczak
1ce3505e5b
Fix potential NoSuchElementException (#6948)
As demonstrated in https://github.com/enso-org/enso/actions/runs/5175688022/jobs/9323585204?pr=6940

```
org.enso.interpreter.test.instrument.RuntimeAsyncCommandsTest *** ABORTED ***
org.graalvm.polyglot.PolyglotException: java.util.NoSuchElementException
at java.base/java.util.WeakHashMap$HashIterator.nextEntry(WeakHashMap.java:811)
at java.base/java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:848)
at java.base/java.util.WeakHashMap$EntryIterator.next(WeakHashMap.java:846)
at org.enso.interpreter.runtime.ThreadExecutors.shutdown(ThreadExecutors.java:46)
at org.enso.interpreter.runtime.EnsoContext.shutdown(EnsoContext.java:198)
at org.enso.interpreter.EnsoLanguage.finalizeContext(EnsoLanguage.java:179)
at org.enso.interpreter.EnsoLanguage.finalizeContext(EnsoLanguage.java:65)
at org.graalvm.truffle/com.oracle.truffle.api.LanguageAccessor$LanguageImpl.finalizeContext(LanguageAccessor.java:326)
at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotLanguageContext.finalizeContext(PolyglotLanguageContext.java:404)
at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotContextImpl.finalizeContext(PolyglotContextImpl.java:2925)
```
2023-06-09 16:28:03 +00:00
Hubert Plociniczak
5c3d6a7d6f
Eliminate race condition for edit/close/open cmds (#6998)
There was an inherent race condition between edit, close & open commands which could not be prevented solely using locks. `EditFileCmd` triggered `EnsureCompiledJob` which was applying edits collected over time. At the same `CloseFileCmd` and `OpenFileCmd` were executed asynchronously and required locks on compilation unit  and file lock.

Additionally, open file was resetting the module's runtime source irrespective of any edits that could already have been applied with the asynchronous execution in `EnsureCompiledJob`. This was visible especially during early manipulation of the project when open/close was performed due to a bug in IDE (#6843).

Now commands can be run either synchronously or asynchronously. Only that way can we ensure that `close` & `open` commands finish by the time any editions are being applied to module's sources.

Closes #6841.

# Important Notes
In the given video, `"foo"` would be greyed out because it would never be part of the module's (runtime) sources. Therefore no IR would be generated for it or instrumentation, meaning it would be present in `expressionUpdates` information necessary for IDE.

[Kazam_screencast_00014.webm](https://github.com/enso-org/enso/assets/292128/226a17b8-729a-415a-803f-003a9695b2f1)
2023-06-09 12:12:46 +00:00
Pavel Marek
372bc8f0d5
Improve compiler's diagnostic messages (#6931)
Improve and colorize compiler's messages. Heavily inspired by `gcc`.
2023-06-08 17:43:18 +02:00
Jaroslav Tulach
9f39fb30b7
Make sure Enso FrameDescriptors have non-null default value (#6977) 2023-06-08 16:55:40 +02:00
Jaroslav Tulach
e6652b8c9f
Separating instrument related files into runtime-instrument-common project (#6992) 2023-06-08 16:51:50 +02:00
Jaroslav Tulach
3731c9c798
Dates are dates and zones are zones (#6934) 2023-06-08 09:47:44 +02:00
Dmitry Bushev
0a4df4faf3
Skip internal field getters when building suggestions (#6968)
close #6900

This is a follow-up to the discussion of the imports/exports meeting.

Right now we have no control over the visibility of atom constructor arguments. One way to hide them is a convention of filtering getters by an `internal` prefix or suffix.
2023-06-07 12:32:58 +00:00
Radosław Waśko
1931e9e51f
Workaround for to_date_time type errors (#6964)
Related to #6912

It essentially solves it by removing any builtins that would take an EnsoDate/EnsoTimeOfDay/EnsoTimeZone and replacing them with Java utils that do the same operation.

This is not a proper solution - the builtin conversion is still invalid for the date/time types - but at this moment we may just no longer use the invalid conversion so it is much less of an issue. We still need to be aware of this if we want to introduce builtins taking date/time in the future.
2023-06-06 20:28:11 +00:00
Hubert Plociniczak
27feaf6bc5
Eliminate deadlock scenario during file edit/open/close requests (#6920)
At the beginning of the execution `EnsureCompiledJob` acquired write compilation lock. When compiling individual modules it would then
- acquire file lock
- acquire read compilation lock

The second one was spurious since it already kept the write lock. This sequence meant however that `CloseFileCmd` or `OpenFileCmd` can lead to  a deadlock when requests come in close succession. This is because commands:
- acquire file lock
- acquire read compilation lock

So `EnsureCompiledJob` might have the (write) compilation lock but the commands could have file lock. And the second required lock for either the job or the command could never be acquired.

Flipping the order did the trick.

Partially solves #6841.

# Important Notes
For some reason we don't get updates for the newly added node, as illustrated in the screenshot, but that could be related to the close/open action. Will need to dig more.

![Screenshot from 2023-06-01 16-45-17](https://github.com/enso-org/enso/assets/292128/900aa9b3-b2b2-4e4d-93c8-267f92b79352)
2023-06-06 08:05:24 +00:00
Hubert Plociniczak
e9a92a1fb5
Throw panic on "no currying for conversions" (#6940)
Previously, a `RuntimeException` would be thrown when an attempt would be made to curry a conversion function. That is problematic for IDE where `executionFailed` means we can't enter functions due to lack of method pointers info.

Closes #6897.

![Screenshot from 2023-06-02 20-31-03](https://github.com/enso-org/enso/assets/292128/a6c77544-2c47-425c-8ce0-982d837dda5b)

# Important Notes
A more generic solution that allows to recover from execution failures will need a follow up.
2023-06-05 13:16:12 +00:00
Dmitry Bushev
4e9f02258e
Suggestion should contain a list of annotations (#6924)
close #6847

Add annotation names to suggestions.
2023-06-02 07:26:01 +00:00
Jaroslav Tulach
d511602787
Don't swallow exceptions in RuntimeServerTest (#6914) 2023-06-01 14:26:36 +02:00
Dmitry Bushev
ed3f9b306e
Consistent self types (#6867)
close #6800

Update the `executionContext/expressionUpdates` notification and send the list of not applied arguments in addition to the method pointer.

# Important Notes
IDE is updated to support the new API.
2023-05-31 15:47:48 +00:00
Jaroslav Tulach
bc6b9bcf54
Search for IR.Function deeper in the IR tree to find instance methods (#6902) 2023-05-31 10:51:55 +02:00
Pavel Marek
e5c21713e7
Import polyglot java inner classes (#6818)
Adds the ability to import nested inner classes in polyglot java imports.
2023-05-31 09:38:59 +02:00
Jaroslav Tulach
c6cb937c0a
Runtime check of ascribed types (#6790) 2023-05-30 11:07:26 +02:00
Hubert Plociniczak
6eb4737330
Append warnings extracted before tail call execution (#6849)
Throwing `TailCallException` meant that exceptions that were extracted from the expression before the call was made could not be appended. This change catches the `TailCallException`, adds warnings to it and propagates it further, thus ensuring that we don't loose the information.

Closes #6765.

# Important Notes
Removed workarounds introduced in stdlib.
2023-05-29 12:44:15 +00:00
Jaroslav Tulach
e7ee2caa2e
Infrastructure for testing inter project imports and exports (#6840) 2023-05-26 17:33:54 +02:00
Jaroslav Tulach
b33ae479dd
Various test improvements to increase coverage and speed things up (#6820) 2023-05-25 10:18:46 +02:00
Jaroslav Tulach
792cbc448c
Clearly select single specialization with enum dispatch pattern (#6819) 2023-05-24 20:42:24 +02:00
Pavel Marek
fe0a06dcb5
Import/export syntax error have more specific messages (#6808) 2023-05-23 21:10:18 +02:00
Jaroslav Tulach
a49743f00c
Missing conversion of hash key in EqualsNode (#6803) 2023-05-23 10:10:22 +02:00
Dmitry Bushev
7471e38ff6
feat: set constructor args tag values (#6801)
related #6611

Changelog:
- update: set tag values for arguments with ascribed types

# Important Notes
![2023-05-23-002112_697x311_scrot](https://github.com/enso-org/enso/assets/357683/10d1c03b-45b7-4133-b4aa-80d1f1ea8e84)
2023-05-23 08:07:40 +00:00
Pavel Marek
2b1e5cd017
Add a compiler pass to analyze non-existing imported symbols (#6726)
Add diagnosis for unresolved symbols in `from ... import sym1, sym2, ...` statements.

- Adds a new compiler pass, `ImportSymbolAnalysis`, that checks these statements and iterates through the symbols and checks if all the symbols can be resolved.
  - Works with `BindingsMap` metadata.
- Add `ImportExportTest` that creates various modules with various imports/exports and checks their generated `BindingMap`.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2023-05-22 10:41:15 +02:00
GregoryTravis
4f71673718
Decimal/Integer .round and .int #6654 (#6743) 2023-05-19 19:23:17 +00:00