Commit Graph

880 Commits

Author SHA1 Message Date
Dmitry Bushev
38212786a3
Fix application config (#6513)
Related #6482
2023-05-02 17:59:44 +00:00
Pavel Marek
3a42d0ce76
Reimplement enso_project as a proper builtin (#6352)
Remove the magical code generation of `enso_project` method from codegen phase and reimplement it as a proper builtin method.

The old behavior of `enso_project` was special, and violated the language semantics (regarding the `self` argument):
- It was implicitly declared in every module, so it could be called without a self argument.
- It can be called with explicit module as self argument, e.g. `Base.enso_project`, or `Visualizations.enso_project`.

Let's avoid implicit methods on modules and let's be explicit. Let's reimplement the `enso_project` as a builtin method. To comply with the language semantics, we will have to change the signature a bit:
- `enso_project` is a static method in the `Standard.Base.Meta.Enso_Project` module.
- It takes an optional `project` argument (instead of taking it as an explicit self argument).

Having the `enso_project` defined as a (shadowed) builtin method, we will automatically have suggestions created for it.

# Important Notes
- Truffle nodes are no longer generated in codegen phase for the `enso_project` method. It is a standard builtin now.
- The minimal import to use `enso_project` is now `from Standard.Base.Meta.Enso_Project import enso_project`.
- Tested implicitly by `org.enso.compiler.ExecCompilerTest#testInvalidEnsoProjectRef`.
2023-05-02 16:40:58 +00:00
Dmitry Bushev
84e59f1403
DataflowAnalysis preserves dependencies order (#6493)
close #6324

Changelog
- feat: DataflowAnalysis compiler pass preserves the order of dependencies. This way when attaching the visualization to the sub-expression, the engine can find the first cached parent node, and properly invalidate it.
- update: runtime visualization test is updated to reproduce the issue

# Important Notes
The dropdown for the column `"LOCATION"` is available right after the Restaurants project startup.

![2023-05-01-171700_1386x975_scrot](https://user-images.githubusercontent.com/357683/235466166-9d25cfa5-0e39-49a3-9c41-93cda59edb81.png)
2023-05-02 14:22:06 +00:00
Hubert Plociniczak
ce4ecc6706
Limit Dead Letter logging (#6482)
Dead Letter logging is occasionally flooding our logs which is confusing to users reporting bugs. Left the possibility of a single report so that we know that something is happening.
2023-05-02 14:00:50 +00:00
Jaroslav Tulach
c626543d02
More reliable shutdown of the EnsoContext to save resources (#6468)
There is 572 threads during `RuntimeServerTest` execution create. They stay around because the context isn't closed after each test finishes. Adding `afterEach` cleanup sequence to `runtime-with-instruments` tests. Improving robustness of overall `EnsoContext` [shutdown](https://github.com/enso-org/enso/pull/6468#discussion_r1180348303).
2023-05-02 11:55:02 +00:00
Hubert Plociniczak
7e33300013
Make execution mode live default for CLI (#6496)
As requested in the design doc and in the ticket.

Closes #6495.
2023-05-02 09:58:41 +00:00
James Dunkerley
6b0c682b08
Add Execution Context control to Text.write (#6459)
- Adjusted `Context.is_enabled` to support default argument (moved built in so can have defaults).
- Made `environment` case-insensitive.
- Bug fix for play button.
- Short hand to execute within an enabled context.
- Forbid file writing if the Output context is disabled with a `Forbidden_Operation` error.
- Add temporary file support via `File.create_temporary_file` which is deleted on exit of JVM.
- Execution Context first pass in `Text.write`.
- Added dry run warning.
- Writes to a temporary file if disabled.
- Created a `DryRunFileManager` which will create and manage the temporary files.
- Added `format` dropdown to `File.read` and `Data.read`.
- Renamed `JSON_File` to `JSON_Format` to be consistent.

(still to unit test).
2023-04-29 08:39:18 +00:00
Jaroslav Tulach
efe904cd9f
Introducing @BuiltinMethod.needsFrame and InlineableNode (#6442)
Fixes #6416 by introducing `InlineableNode`. It runs fast even on GraalVM CE, fixes ([forever broken](https://github.com/enso-org/enso/pull/6442#discussion_r1178782635)) `Debug.eval` with `<|` and [removes discouraged subclassing](https://github.com/enso-org/enso/pull/6442#discussion_r1178778968) of `DirectCallNode`. Introduces `@BuiltinMethod.needsFrame` - something that was requested by #6293. Just in this PR the attribute is optional - its implicit value continues to be derived from `VirtualFrame` presence/absence in the builtin method argument list. A lot of methods had to be modified to pass the `VirtualFrame` parameter along to propagate it where needed.
2023-04-28 15:32:13 +00:00
Hubert Plociniczak
fbfdba2b4a
Don't attempt to resolve hostname for git init (#6463)
During initialization JGit may attempt to resolve hostname. On some systems this can take more than desired triggering timeouts. This change does two things:
- sets the default committer for changes, lack of which probably triggered the check
- sets the default hostname to `localhost` (we don't care), in case something else in JGit still wants to resolve hostname

Closes #6447.

# Important Notes
I wasn't able to reproduce this so relying on @mwu-tow since apparently he can repro it reliably.
2023-04-28 14:29:30 +00:00
Hubert Plociniczak
ae3f9025e3
Invoke instance methods for Any overrides (#6441)
This change modifies method dispatch for methods that override Any's definitions. When an overrided method is invoked statically we call Any's method to stay consistent.
This change primarily addresses the plethora of problems related to `to_text` invocations. It does not attempt to completely modify method dispatch logic.

Closes #6300.
2023-04-28 07:18:37 +00:00
Hubert Plociniczak
c6790f1e9c
Report only unique warnings (#6372)
This change makes sure that reported warnings are unique, based on the value of internal clock tick and ignoring differences in reassignments.

Before:
![Screenshot from 2023-04-20 15-42-55](https://user-images.githubusercontent.com/292128/233415710-925c1045-37c7-49f5-9bc3-bfbfd30270a3.png)
After:
![Screenshot from 2023-04-20 15-27-27](https://user-images.githubusercontent.com/292128/233415807-8cb67bc2-ac37-4db7-924e-ae7619074b5b.png)

On the positive side, no further changes, like in LS, have to be done.


Closes #6257.
2023-04-28 07:16:00 +00:00
Hubert Plociniczak
376415ab17
Mark some tests as (unconditonally) flaky (#6453)
Too many spurious failures on CI now. Will need to be tackled at some
point.
2023-04-27 17:09:46 +02:00
Dmitry Bushev
3401097266
Switch execution environment before the complete notification (#6449)
Fixes the runtime test

```
- should recompute expressions changing an execution environment *** FAILED ***
"[live]" did not equal "[design]" (RuntimeServerTest.scala:2721)
Analysis:
"[live]" -> "[design]"
```
2023-04-27 11:21:07 +00:00
James Dunkerley
0e51131809
Table Visualization and display text changes. (#6382)
- Missing tests from number parsing.
- Fix type signature on some warning methods.
- Fix warnings on `Standard.Database.Data.Table.parse_values`.
- Added test for `Nothing` and empty string on `use_first_row_as_names`.
- New API for `Number.format` taking a simple format string and `Locale`.
- Add ellipsis to truncated `Text.to_display_text`.
- Adjusted built-in `to_display_text` for numbers to not include type (but also to display BigInteger as value).
- Remove `Noise.Generator` interface type.
- Json: Added `to_display_text` to `JS_Object`.
- Time: Added `to_display_text` for `Date`, `Time_Of_Day`, `Date_Time`, `Duration` and `Period`.
- Text: Added `to_display_text` to `Locale`, `Case_Sensitivity`, `Encoding`, `Text_Sub_Range`, `Span`, `Utf_16_Span`.
- System: Added `to_display_text` to `File`, `File_Permissions`, `Process_Result` and `Exit_Code`.
- Network: Added `to_display_text` to `URI`, `HTTP_Status_Code` and `Header`.
- Added `to_display_text` to `Maybe`, `Regression`, `Pair`, `Range`, `Filter_Condition`.
- Added support for `to_js_object` and `to_display_text` to `Random_Number_Generator`.
- Verified all error types have `to_display_text`.
- Removed `BigInt`, `Date`, `Date_Time` and `Time_Of_Day` JS based rendering as using `to_display_text` now.
- Added support for rendering nested structures in the table viz.
2023-04-26 18:15:48 +00:00
Dmitry Bushev
0e5ee35aba
Skip redundant compilations (#6436)
related #6323

The engine can skip compilation when applying changes that do not require execution. It is more efficient to process the changes in a batch, than triggering compilations every time such edit is received.
2023-04-26 17:53:01 +00:00
Jaroslav Tulach
63de18e367
Turn null into UnexpectedExpression when Union type is incomplete (#6415)
Test and fix for #6401.
2023-04-25 14:49:26 +00:00
Radosław Waśko
b8f075a178
Ensure that IO.println does not fail if to_text returned a non-Text value. (#6223) 2023-04-25 10:55:39 +00:00
Jaroslav Tulach
8e08a3b634
Improve inlining of <| on (GraalVM EE) (#6384) 2023-04-25 12:02:51 +02:00
Jaroslav Tulach
e47eb49ea8
Removing need for asynchronous thread to execute ResourceManager finalizers (#6335)
While Enso runs single-threaded, its `ResourceManager` required additional asynchronous thread to execute its _"finalizers"_. What has been necessary back then is no longer needed since _GraalVM 21.1_. GraalVM now provides support for submitting `ThreadLocalAction` that gets then picked and executed via `TruffleSafepoint` locations. This PR uses such mechanism to _"inject"_ finalizer execution into already running Enso evaluation thread.

Requiring more than one thread has complicated Enso's co-existence with other Truffle language. For example Graal.js is strictly singlethreaded and used to refuse (simple) co-existence with Enso. By allowing Enso to perform all its actions in a single thread, the synergy with Graal.js becomes better.
2023-04-20 13:33:45 +02:00
Pavel Marek
4076a64f33
NaN can be used as a key in Map (#6301)
`Number.nan` can be used as a key in `Map`. This PR basically implements the support for [JavaScript's Same Value Zero Equality](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Equality_comparisons_and_sameness#same-value-zero_equality) so that `Number.nan` can be used as a key in `Map`.

# Important Notes
- For NaN, it holds that `Meta.is_same_object Number.nan Number.nan`, and `Number.nan != Number.nan` - inspired by JS spec.
- `Meta.is_same_object x y` implies `Any.== x y`, except for `Number.nan`.
2023-04-20 09:25:36 +00:00
Hubert Plociniczak
6d3151f32d
UnresolvedSymbol is now accepted by Vector.sort (#6334)
`Vector.sort` does some custom method dispatch logic which always expected a function as `by` and `on` arguments. At the same time, `UnresolvedSymbol` is treated like a (to be resolved) `Function` and under normal circumstances there would be no difference between `_.foo` and `.foo` provided as arguments.

Rather than adding an additional phase that does some form of eta-expansion, to accomodate for this custom dispatch, this change only fixes the problem locally. We accept `Function` and `UnresolvedSymbol` and perform the resolution on the fly. Ideally, we would have a specialization on the latter but again, it would be dependent on the contents of the `Vector` so unclear if that is better.

Closes #6276,

# Important Notes
There was a suggestion to somehow modify our codegen to accomodate for this scenario but I went against it. In fact a lot of name literals have `isMethod` flag and that information is used in the passes but it should not control how (late) codegen is done. If we were to make this more generic, I would suggest maybe to add separate eta-expansion pass. But it could affect other things and could be potentially a significant change with limited potential initially, so potential future work item.
2023-04-20 07:58:58 +00:00
Dmitry Bushev
8c9c2d79cd
Keep scheduled execution job on text edit (#6354)
close #6346

Changelog:
- fix: keep scheduled execution job in the queue when applying a text edit
2023-04-19 14:18:36 +00:00
Hubert Plociniczak
f288198b4b
Change order of arguments when controlling context (#6357)
https://github.com/orgs/enso-org/discussions/6344 requested to change the order of arguments when controlling context permissions.

# Important Notes
The change brings it closer to the design doc but IMHO also a bit cumbersome to use (see changed tests) - applications involving default arguments don't play well when the last argument is not the default 🤷 .`
2023-04-19 14:00:27 +00:00
Hubert Plociniczak
0088672f9b
Simplify test for 6248 (#6343)
As per suggestion https://github.com/enso-org/enso/pull/6275#discussion_r1167388449
2023-04-19 11:43:52 +02:00
Jaroslav Tulach
f52ef08be5
Avoid double WithWarnings wrapping (#6348) 2023-04-19 10:43:29 +02:00
Dmitry Bushev
7733b9bdac
Async Language Server resources initialization (#6336)
close #6306

Changelog:
- add: `AsyncResourceInitialization` component
- update: initialize language server resources asynchronously

# Important Notes
Speeds up `session/InitProtocolConnection` request by ~300ms (~10%) 3051ms (before) vs 2740ms (after) on my machine.
2023-04-18 15:55:44 +00:00
Jaroslav Tulach
e5a96b9782
Avoid NullPointerException in --dumpGraphs & test & docs (#6304) 2023-04-18 06:58:06 +02:00
Hubert Plociniczak
f720bd2516
Type of UnresolvedSymbol is Function (#6284)
`Meta.is_a` and `Meta.type_of` now recognize UnresolvedSymbol.
Closes #6277.
2023-04-17 13:44:15 +00:00
Jaroslav Tulach
413661b366
Splitting Gigantic EqualsNode into few Smaller Ones (#6280)
Splitting gigantic `EqualsNode` into few smaller ones
2023-04-17 11:29:54 +02:00
Pavel Marek
b42e910280
sort handles incomparable values (#5998)
* Update type ascriptions in some operators in Any

* Add @GenerateUncached to AnyToTextNode.

Will be used in another node with @GenerateUncached.

* Add tests for "sort handles incomparable types"

* Vector.sort handles incomparable types

* Implement sort handling for different comparators

* Comparison operators in Any do not throw Type_Error

* Fix some issues in Ordering_Spec

* Remove the remaining comparison operator overrides for numbers.

* Consolidate all sorting functionality into a single builtin node.

* Fix warnings attachment in sort

* PrimitiveValuesComparator handles other types than primitives

* Fix byFunc calling

* on function can be called from the builtin

* Fix build of native image

* Update changelog

* Add VectorSortTest

* Builtin method should not throw DataflowError.

If yes, the message is discarded (a bug?)

* TypeOfNode may not return only Type

* UnresolvedSymbol is not supported as `on` argument to Vector.sort_builtin

* Fix docs

* Fix bigint spec in LessThanNode

* Small fixes

* Small fixes

* Nothings and Nans are sorted at the end of default comparator group.

But not at the whole end of the resulting vector.

* Fix checking of `by` parameter - now accepts functions with default arguments.

* Fix changelog formatting

* Fix imports in DebuggingEnsoTest

* Remove Array.sort_builtin

* Add comparison operators to micro-distribution

* Remove Array.sort_builtin

* Replace Incomparable_Values by Type_Error in some tests

* Add on_incomparable argument to Vector.sort_builtin

* Fix after merge - Array.sort delegates to Vector.sort

* Add more tests for problem_behavior on Vector.sort

* SortVectorNode throws only Incomparable_Values.

* Delete Collections helper class

* Add test for expected failure for custom incomparable values

* Cosmetics.

* Fix test expecting different comparators warning

* isNothing is checked via interop

* Remove TruffleLogger from SortVectorNode

* Small review refactorings

* Revert "Remove the remaining comparison operator overrides for numbers."

This reverts commit 0df66b1080.

* Improve bench_download.py tool's `--compare` functionality.

- Output table is sorted by benchmark labels.
- Do not fail when there are different benchmark labels in both runs.

* Wrap potential interop values with `HostValueToEnsoNode`

* Use alter function in Vector_Spec

* Update docs

* Invalid comparison throws Incomparable_Values rather than Type_Error

* Number comparison builtin methods return Nothing in case of incomparables
2023-04-16 16:40:12 +02:00
Hubert Plociniczak
e163353d16
Deprecate Error's to_display_text builtin methods (#6275)
The primary motivation for this change was
https://github.com/enso-org/enso/issues/6248, which requested the possibility of defining `to_display_text` methods of common errors via regular method definitions. Until now one could only define them via builtins.

To be able to support that, polyglot invocation had to report `to_display_text` in the list of (invokable) members, which it didn't. Until now, it only considered fields of constructors and builtin methods. That is now fixed as indicated by the change in `Atom`.

Closes #6248.

# Important Notes
Once most of builtins have been translated to regular Enso code, it became apparent how the usage of `.` at the end of the message is not consistent and inflexible. The pure message should never follow with a dot or it makes it impossible to pretty print consistently for the purpose of error reporting. Otherwise we regularly end up with errors ending with `..` or worse. So I went medieval on the reasons for failures and removed all the dots.
The overall result is mostly the same except now we are much more consistent.

Finally, there was a bit of a good reason for using builtins as it simplified our testing.
Take for example `No_Such_Method.Error`. If we do not import `Errors.Common` module we only rely on builtin error types. The type obviously has the constructor but it **does not have** `to_display_text` in scope; the latter is no longer a builtin method but a regular method. This is not really a problem for users who will always import stdlib but our tests often don't. Hence the number of changes and sometimes lack of human-readable errors there.
2023-04-14 19:14:49 +00:00
Jaroslav Tulach
a74933d10f
Speed cascade of if statements up (#6255)
Fixes #5709. We have a test and a generic fix that improves inlining of every builtin. Everything seems to be faster.
2023-04-14 13:27:23 +00:00
Dmitry Bushev
9ebda56385
Language Server preserves the suggestion updates order (#6268)
Related #6138

Fixes an issue when the `SuggestionsHandler` can send suggestion updates without preserving the order in which they were received.
2023-04-14 12:39:02 +00:00
James Dunkerley
4e92d76665
Add Warning methods onto Any to make working with them easier. (#6176)
Adds `has_warnings`, `remove_warnings` and `throw_on_warning` extension methods.
2023-04-12 15:08:01 +00:00
Hubert Plociniczak
c8ea06c4a6
Provide Forbidden_Operation.to_display_text (#6246)
Added a missing `to_display_text` method which, by convention, is defined via a builtin method.

Also re-enabled and fixed pending tests.

Closes #6227.
2023-04-12 07:53:21 +00:00
Dmitry Bushev
b97fc39214
Remove SQL versions repo (#6242)
close #6232

Changelog:
- remove: `SqlVersionsRepo`
- update: `SuggestionsDatabaseModuleUpdateNotification` message removing the version
- update: cleanup versions repo usages in the language server
2023-04-11 19:22:30 +00:00
Jaroslav Tulach
5b3cf6f503
Consolidate Vector and Array methods (#6218)
Fixes #5011 by making sure the same methods that are on `Vector` are also available on `Array`.
2023-04-11 16:20:11 +00:00
Dmitry Bushev
6663125410
Language Server support for execution environments (#6217)
close #6133

Changelog:
- add: `executionContext/setExecutionEnvironment` request
- update: `executionContext/recompute` request to have optional `executionEnvironment` parameter.
2023-04-11 15:12:27 +00:00
Hubert Plociniczak
3cce3b3caf
Replace IOContexts with execution env and contexts (#6171)
As per design, IOContexts controlled via type signatures are going away. They are replaced by explicit `Context.if_enabled` runtime checks that will be added to particular method implementations.

`production`/`development` `IOPermissions` are replaced with `live` and `design` execution enviornment. Currently, the `live` env has a hardcoded list of allowed contexts i.e. `Input` and `Output`.

# Important Notes
As per design PR-55. Closes #6129. Closes #6131.
2023-04-06 15:47:40 +00:00
Dmitry Bushev
df4491de50
Batch insert suggestions (#6189)
close #6080

Changelog
- add: implement `SuggestionsRepo.insertAll` as a batch SQL insert
- update: `search/getSuggestionsDatabase` returns empty suggestions. Currently, the method is only used at startup and returns the empty response anyway because the libs are not loaded at that point.
- update: serialize only global (defined in the module scope) suggestions during the distribution building. There's no sense in storing the local library suggestions.
- update: sqlite dependency
- remove: unused methods from `SuggestionsRepo`
- remove: Arguments table

# Important Notes
Speeds up libraries loading by ~1 second.

![2023-04-03-173423_2086x324_scrot](https://user-images.githubusercontent.com/357683/229597470-19dcc010-2a34-43e1-87be-60af99afd275.png)
![2023-04-03-173514_2083x321_scrot](https://user-images.githubusercontent.com/357683/229597476-bf5b3c33-6321-4ac9-a0ca-2fb57d257857.png)
2023-04-06 07:47:23 +00:00
Jaroslav Tulach
741b394b0d
Suspended atom fields are evaluated only once (#6151)
Implements #6134.

# Important Notes
One can define lazy atom fields as:
```haskell
type Lazy
Value ~x ~y
```
the evaluation of the `x` and `y` fields is then delayed until they are needed. The evaluation happens once. Then the computed value is kept in the atom for further use.
2023-04-05 23:46:08 +00:00
Jaroslav Tulach
4805193428
Text.to_display_text is (shortened) identity (#6174)
Fixes #5971.
2023-04-05 19:53:07 +00:00
Jaroslav Tulach
519df66d39
Always try to resolve conversion for Any type (#6184)
Fixes #5898 by removing `Catch.panic` and speeding the `sieve.enso` benchmark from 1058 ms to 514 ms. Should there be no dedicated conversion, let's use one defined on `Any` type - e.g. defining a conversion `from(Any)` makes such a conversion is always available.
2023-04-04 09:24:35 +00:00
Jaroslav Tulach
310a2d8ae7
Delay creation of project root EnsoFile until it is really needed (#6149)
Delay creation of `EnsoFile` until it is needed.

# Important Notes
By putting breakpoint into `Atom` constructor I realized few `EnsoProjectNode` instances may be created when parsing the project. It makes no sense to also create `EnsoFile` for them - until it is needed.
2023-04-03 14:03:25 +00:00
Jaroslav Tulach
0d7682b6ec
Adjusting tests to new parser (#6143)
Modification to various tests disabled when #5917 was integrated to pass with new parser. Fixes #5894.

# Important Notes
Some tests can be fixed just by changes on the `IR` side. Some (especially error simulating ones) would benefit from changes in the `Tree` structure or at least @kazcw evaluation.
2023-04-02 21:24:50 +00:00
Radosław Waśko
6ddcb553e5
Date/time support for Postgres. Year/month/day operations on Columns. (#6153)
Closes #6115
2023-03-31 18:37:04 +00:00
Dmitry Bushev
2338e5d8e6
When renaming the project clean old modules instead of updating (#6148)
close #6139
close #6137

When the project is renamed, the engine cleans up affected modules and initiates modules re-indexing to fill the suggestions database with new records. This way it reduces the amount of information stored in the suggestions database and helps implement #6080 optimization.

Changelog:
- remove: rename features from the suggestions database
- update: rename command to initiate modules cleanup and project re-execution
- fix: #6137
2023-03-31 09:40:21 +00:00
Hubert Plociniczak
90612cfa35
Report errors when generating indexes (#6123)
`--compile` command would run the compilation pipeline but silently omit any encountered errors, thus skipping the serialization. This maybe was a good idea in the past but it was problematic now that we generate indexes on build time.
This resulted in rather obscure errors (#6092) for modules that were missing their caches.

The change should significantly improve developers' experience when working on stdlib.

# Important Notes
Making compilation more resilient to sudden cache misses is a separate item to be worked on.
2023-03-30 09:59:49 +00:00
Jaroslav Tulach
8c5cd6e35e
Always print stacktrace to provide some guidance of what is going on (#6128)
Avoid displaying just `Execution finished with an error: java.lang.NullPointerException` without any additional detail.

# Important Notes
When there is an execution error outside of Enso code (identified by the fact that all stack frames belong to `java` language), let's print the whole stack rather than printing nothing. To simulate one can:
```diff
diff --git engine/runtime/src/main/scala/org/enso/compiler/Compiler.scala engine/runtime/src/main/scala/org/enso/compiler/Compiler.scala
index ff0636bc66..42e5eae32e 100644
--- engine/runtime/src/main/scala/org/enso/compiler/Compiler.scala
+++ engine/runtime/src/main/scala/org/enso/compiler/Compiler.scala
@@ -453,7 +453,8 @@ class Compiler(
pool
)
} else {
-          CompletableFuture.completedFuture(ensureParsedAndAnalyzed(module))
+          // CompletableFuture.completedFuture(ensureParsedAndAnalyzed(module))
+          CompletableFuture.completedFuture(())
}
}
```
2023-03-30 08:05:28 +00:00
Jaroslav Tulach
d1c52fef0b
Turn missing body in a binding into a syntax error (#6107)
Test and fix for #5903.
2023-03-28 16:42:18 +00:00
Hubert Plociniczak
b3db805f92
More elaborate error message for invalid text edit (#6108)
It is sometimes impossible to figure out the real reason for invalid text edit request. Added a bit of context to failures to narrow down the cause of the failure.

# Important Notes
Should help with diagnosing issues like #6099.
2023-03-28 14:21:50 +00:00
Jaroslav Tulach
46bd6f6247
Serde testing infrastructure for the Enso compiler (#6062)
The primary delivery of this PR is a design of `SerdeCompilerTest` - a testing suite that allows us to write sample projects, parse them with and without caches and verify they still produce the same `IR`. This is a similar idea to #3723 which compared the old and new parser `IR`s.

With infrastructure like this we can start addressing #5567 without any (significant) fear of breaking something essential.
2023-03-28 08:46:07 +00:00
Dmitry Bushev
aa97787603
Update ZIO library (#6072)
close #6069

Changelog:
- update: ZIO major version bump
2023-03-28 07:58:59 +00:00
Pavel Marek
9bec3a4e71
Treat Boolean.False as false primitive (#6090)
Treat `Boolean.False` and `Boolean.True` as the corresponding primitives. Now, `Boolean.False == False` returns true.

# Important Notes
`False` and `True` constructs, that are converted to `ConstructorNode` during Truffle codegen, are handled specially in `ConstructorNode`. The easiest fix was to implement a similar special handling in `QualifiedAccessorNode`, although not the cleanest one.

A better solution would be to provide transformation of `Boolean.True` IR to a true literal in `ApplicationSaturation` compiler pass. But `ApplicationSaturation` pass does not handle `True`. Moreover, for our case, it is unnecessarily complicated.
2023-03-28 03:41:49 +00:00
Hubert Plociniczak
76409b285d
Ensure new and wrapper nodes inherit UUID (#6067)
Instrumentation of calls involving warning values never really worked because:
1) newly created nodes didn't set the UUID of their children
2) the instrumentable wrappers always had an empty (i.e. null) UUID and
they never referred `get`/`setId` calls to their delegates

On the surface, everything worked fine. Except when one actually relied on the instrumentation of values with warnings for proper setup. Then no instrumentation (replacement of nodes) was performed due to empty UUID (as required by `hasTag` of `FunctionCallInstrumentationNode`).

Closes #6045. Discovered in #5893.
2023-03-27 17:49:20 +00:00
James Dunkerley
bf2545fa04
Use new common parse method throwing less exceptions. (#6075)
Avoiding exceptions by not using parseBest.

Time now in CLI is 1.15s for 500k rows vs 1.65s in GUI.

CLI:
![image](https://user-images.githubusercontent.com/4699705/227711266-bc005b0d-5011-450f-964b-65dd2e437c2e.png)

GUI:
![image](https://user-images.githubusercontent.com/4699705/227711259-f7ddda29-86c7-4eef-a002-4bf0bda6063f.png)

Added it as a function in the shared library so used by both engine and polyglot.
2023-03-27 11:02:10 +00:00
Dmitry Bushev
4c62dc9061
Search suggestions by static attribute (#6036)
close #5874

Changelog:
- add: `isStatic` parameter to `search/completion` request to search by the `static` suggestion attribute
- update: search non-static suggestions when opening component browser

# Important Notes
Component browser doesn't show `Table.new` and `Table.from_rows` suggestions when a `Table` node is selected.

![2023-03-21-151117_1301x877_scrot](https://user-images.githubusercontent.com/357683/226874291-1ff99994-1bb6-41df-96b4-dc5c5178ba41.png)
2023-03-23 15:02:25 +00:00
James Dunkerley
dd009fd1af
Tidy up the public module level statics (#6032)
Tidies up a lot of PUBLIC module statics - marked some as PRIVATE, made some methods of types.
2023-03-22 18:02:37 +00:00
James Dunkerley
546cb0c4ab
Deal with warnings attached to value when making widgets. (#5994)
- Fixes InvokeCallableNode to support warnings.
- Strips warnings from annotations in `get_widget_json`.
- Remove `get_full_annotations_json`.
- Fix warnings on Dialect.
2023-03-22 10:50:19 +00:00
Hubert Plociniczak
8c6fd60aaf
Detect conflicts between exported types and FQNs (#5986)
Exporting types named the same as the module where they are defined in `Main` modules of library components may lead to accidental name conflicts. This became apparent when trying to access `Problem_Behavior` module via a fully qualified name and the compiler rejected it. This is due to the fact that `Main` module exported `Error` type defined in `Standard.Base.Error` module, thus making it impossible to access any other submodules of `Standard.Base.Error` via a fully qualified name.

This change adds a warning to FullyQualifiedNames pass that detects any such future problems.
While only `Error` module was affected, it was widely used in the stdlib, hence the number of changes.

Closes #5902.

# Important Notes
I left out the potential conflict in micro-distribution, thus ensuring we actually detect and report the warning.
2023-03-21 21:09:41 +00:00
Dmitry Bushev
e7ced7905a
feat: early init zio (#6029) 2023-03-21 17:34:02 +01:00
Dmitry Bushev
8125d89676
Delay initialization of JSON-RPC protocol (#5997)
close #5881

Changelog
- add: ProtocolFactory object that initializes and returns the protocol object
- update: Add protocol initialization to the initialization component
2023-03-20 11:04:16 +00:00
Dmitry Bushev
f3c5643f33
Delay initialization of ZIO executors (#5990)
close #5882

Changelog:
- update: move ZIO initialization logic to the initialization component
2023-03-17 23:37:47 +00:00
Dmitry Bushev
b2aa58bf20
Delay writing IR caches (#5957)
close #5911

In interactive mode, perform writing IR caches in the background jobs queue. Background jobs execution is delayed until the first execution is complete.
2023-03-17 22:47:15 +00:00
Jaroslav Tulach
b46be10f63
Introducing Meta.Type (#5956)
Fixing #5768 and #5765 and co. Introducing `Meta.Type` and giving it the desired methods.

# Important Notes
`Type` is no longer a `Meta.Atom`, but it has a dedicated `Meta.Type` representation.
2023-03-17 20:08:18 +00:00
Kaz Wesley
75fda33837
Fix precedence of -1.x (#5830)
Fixes #5826.

# Important Notes
- Change frontend representation of negation.
- Fix a precedence issue: The `.` operators in -1.x and -1.2 must have different precedences.
- Remove a no-longer-needed special case from backend translation.
- Add tests for this case after all translations.
2023-03-17 18:53:34 +00:00
Jaroslav Tulach
c696bf1d87
Don't persist UUIDs in IR caches (#5918)
Give Cache subclasses a chance to control the output stream format. Use that functionality to avoid persisting `UUID` into standard library `.ir` caches. Gets the number of caches down to 42MB from 48MB.

# Important Notes
I believe UUIDs are not really useful for standard libraries and can be omitted. Am I right?
2023-03-17 16:53:53 +00:00
Dmitry Bushev
265fb5d505
Delay initialization of SQL database (#5970)
close #5875

Move database initialization logic to the initialization component.
2023-03-17 13:38:11 +00:00
Dmitry Bushev
69d8864a1f
Fix qualified names in tag values (#5978)
close #5901

Changelog:
- fix: qualified names in tag values

# Important Notes
![2023-03-16-190913_892x345_scrot](https://user-images.githubusercontent.com/357683/225684530-0d60a7a0-8b95-40a9-9ff6-2c112d966991.png)
2023-03-16 17:36:15 +00:00
Hubert Plociniczak
a6487e5105
Don't log installed engines and runtimes in prod (#5900)
The `logAvailableComponentsForDebugging` will check and install all necessary components of GraalVM for every mentioned version. While not harmful, it adds up to startup time.
Additionally added an option in language server startup to skip installation of GraalVM components. The latter is already performed by project-manager when opening the project and it is unnecessary to do it twice. Due to LS' architecture this configuration has to be passed around via multiple configs.

Finally, skipped the attempt to install Python component on Windows - this is not supported by GraalVM atm.

Closes #5749.

# Important Notes
The impact of this problem could be really felt the more versions of Enso and GraalVM one had since it would go through all of them.
2023-03-16 10:36:55 +00:00
Jaroslav Tulach
e666d797c5
Dedicated Module.containsUUID method with a cache (#5945)
Fixes #5781 by caching all UUIDs in each module in a `Map`.
2023-03-15 17:14:59 +00:00
Kaz Wesley
e171fba301
New documentation parser (#5917)
Implement new Enso documentation parser; remove old Scala Enso parser.

Performance: Total time parsing documentation is now ~2ms.

# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2023-03-15 15:43:51 +00:00
Hubert Plociniczak
c21bd3b185
Generate import/export bindings in local cache (#5908)
When generating import/export bindings in local cache, they are included in the distribution.
Additionally, removed the hardcoded value for suggestions cache. Now one can generate them for local as well as for global cache, based on the presence or lack of `--no-global-cache` parameter.

Closes #5890.
2023-03-15 12:40:30 +00:00
Dmitry Bushev
5c6a5dcf60
Do not cancel detach visualization jobs (#5913)
close #5889

Changelog:
- update: make `DetachVisualizationJob` a unique job to make sure that they are not canceled during the re-compilation after `text/applyEdit` command.
2023-03-14 12:31:32 +00:00
Dmitry Bushev
0d82a2af89
Delay background jobs execution (#5904)
close #5892

Changelog:
add: feature to delay background jobs execution
add: start background jobs when program finishes
add: start background jobs on `search/completion` request
2023-03-14 11:26:11 +00:00
Jaroslav Tulach
888307f03f
Meta.get_constructor_fields works for Type (#5869)
Fixes #5805 by returning `[]` as list of fields of `Type`.

# Important Notes
`Type` is recognized as `Meta.is_atom` since #3671. However `Type` isn't an `Atom` internally. We have to provide special handling for it where needed.
2023-03-13 09:58:11 +00:00
Radosław Waśko
263c3ad651
Add a common-polyglot-core-utils project (#5855)
Adds a common project that allows sharing code between the `runtime` and `std-bits`.

Due to classpath separation and the way it is compiled, the classes will be duplicated - we will have one copy for the `runtime` classpath and another copy as a small JAR for `Standard.Base` library.

This is still much better than having the code duplicated - now at least we have a single source of truth for the shared implementations.

Due to the copying we should not expand this project too much, but I encourage to put here any methods that would otherwise require us to copy the code itself.

This may be a good place to put parts of the hashing logic to then allow sharing the logic between the `runtime` and the `MultiValueKey` in the `Table` library (cc: @Akirathan).
2023-03-11 09:27:26 +00:00
Dmitry Bushev
725841467f
Log exceptions with stack traces (#5871)
close #5070

Changelog:
- Include the original exception to log expressions
- Enable logging of Akka Actors' lifecycle events on debug logging level
- Decrease the severity of interruption log messages because interruptions are part of the workflow. The computation can be interrupted at any time, and still be recomputed after. Warnings are just misleading in this case.
2023-03-11 08:23:31 +00:00
Pavel Marek
5f7a4a5a39
Merge ordered and unordered comparators (#5845)
Merge _ordered_ and _unordered_ comparators into a single one.

# Important Notes
Comparator is now required to have only `compare` method:
```
type Comparator
comapre : T -> T -> (Ordering|Nothing)
hash : T -> Integer
```
2023-03-11 05:43:22 +00:00
Jaroslav Tulach
632a303089
Use SimpleFormatter.formatMessage to replace {0} with actual parameters (#5870)
Fixes #5801 to properly format Truffle log records before sending them for further processing.
2023-03-11 00:15:58 +00:00
James Dunkerley
7887fb8d40
Regex.Compile using Truffle Regex, update find, match and match_all (#5785) 2023-03-10 21:49:50 +00:00
Radosław Waśko
91ef8acf35
Review generated Column names (#5850)
Closes #5583 and closes #5157
2023-03-10 19:07:58 +00:00
Jaroslav Tulach
8bbdd1af5b
Meta.is_a consistent with case-type-of check (#5853)
Removing special handling of `AtomConstructor` in `Meta.is_a` check.

# Important Notes
A lot of tests are about to fail. Many of them indirectly call `Meta.is_a` with a constructor rather than type.
2023-03-10 07:41:04 +00:00
Hubert Plociniczak
6769ab0ee7
Downgrade hashing to SHA-1 and other optimizations (#5791)
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.

Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.

Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.

Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.

Finally fix an SBT warning when generating library cache.

Closes https://github.com/enso-org/enso/issues/5763

# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
2023-03-09 07:36:59 +00:00
Dmitry Bushev
9397a6ec2f
Pre compute suggestion db during build time (#5698)
Close #5068

Cache suggestions during the `buildEngineDistribution` command, and read them from the disk when the library is loaded. Initial graph coloring takes ~20 seconds vs ~25 seconds on the develop branch.

[peek-develop-branch.webm](https://user-images.githubusercontent.com/357683/223504462-e7d48262-4f5e-4724-b2b0-2cb97fc05140.webm)
[peek-suggestions-branch.webm](https://user-images.githubusercontent.com/357683/223504464-0fe86c04-8c4b-443c-ba96-6c5e2fb1e396.webm)
2023-03-08 12:37:48 +00:00
James Dunkerley
299bfd6b7d
Fixes from the Demo on 2nd March (#5823)
- Fix issue with Geo Map viz.
- Handle invalid format strings better in `Data_Formatter`.
- New constants for the ISO format strings (and a special ENSO_ZONED_DATE_TIME)
- Consistent Date Time format for parsing in all places.
- Avoid throwing exception in datetime parsing.
- Support for milliseconds (well nanoseconds) in Date_Time and Time_Of_Day.
- `Column.map` stays within Enso.
- Allow `Aggregate_Column.Group_By` in `cross_tab` group_by parameter.
2023-03-07 20:58:00 +00:00
Jaroslav Tulach
e3f0888c5f
Coerce values obtained from polyglot calls (#5832)
Coerce values obtained from polyglot calls to fix #5177.

# Important Notes
Adds `IntHolder` class into the `test/Tests` project to simulate access to a class with integer field.
2023-03-07 18:21:30 +00:00
Pavel Marek
b6e2319fcc
Comparators support partial ordering (#5778) 2023-03-07 04:16:38 +00:00
Radosław Waśko
da760aa27d
Review Text/Table.write problem behavior (#5816)
Closes #5114

Added tests for various problems scenarios when writing files.

And ensured that those tests are passing by fixing a few edge cases.
2023-03-07 02:25:13 +00:00
Jaroslav Tulach
161ea45019
Report AbstractTruffleException as a Panic (#5794)
Report `AbstractTruffleException` as a `Panic`. Fixes #5260.
2023-03-06 19:05:25 +00:00
Radosław Waśko
2d29456ed1
Review File/Data read and read_text warnings (#5799)
Closes #5113

Fixes a bug where read-only files would be overwritten if File.write was used in backup mode, and added tests to avoid such regression. To implement it, introduced a `is_writable` property on `File`.
2023-03-06 03:43:38 +00:00
Jaroslav Tulach
f64edd806d
Switch to Truffle PE mode before calling into Enso (#5783)
Enter partial evaluation mode via `CallTarget.call` before invoking `InteropLibrary`. Fixes #5782.
2023-03-03 14:39:37 +00:00
Hubert Plociniczak
941512e0ba
Optimize import/export resolution (#5700)
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.

When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.

The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.

Part of https://github.com/enso-org/enso/issues/5568 work.
2023-03-01 08:53:29 +00:00
Jaroslav Tulach
a150abbea8
Convert dangling comma into unexpected expression (#5776)
Test and fix for #5691.
2023-02-28 07:44:06 +00:00
Jaroslav Tulach
0536eeb0bb
Offer no suggestion for single constructor (#5747)
Resolving #5055 - avoid putting single constructor into suggestion database.

# Important Notes
Another way to fix #5055 is to keep the single constructor information in the suggestion database and let the IDE filter that out.
2023-02-28 05:58:36 +00:00
James Dunkerley
ba3d45e782
Adjust generateCheckedArgumentRead to return correct expected type (#5769)
Correctly get the expected type and return an IllegalArgument if no Enso type.

![image](https://user-images.githubusercontent.com/4699705/221263862-bbc122ca-b11f-49f3-b7cf-6294ea811a22.png)
2023-02-27 17:23:45 +00:00
James Dunkerley
9ee21e8812
Stability improvements in visualizations. (#5760)
- Handle `WithWarnings` in `IndirectInvokeCallableNode`.
- Handle no RootNode in `ErrorResolver`.
- Allow table vizualisation to cope if no `data` passed.
- Add `Warning.has_warnings` to check if warnings present.
- Adjust `set_value` for `JS_Object` so creates a new object each time.
2023-02-24 21:04:50 +00:00
Jaroslav Tulach
78aab133c7
Avoid nulls in Array.items (#5736)
Put `Nothing` into an empty array rather than `null`. When running with `assert` on (unit tests), check the content of the array and `AssertError` quickly when a `null` is found.
2023-02-22 17:53:36 +00:00
Pavel Marek
58c7ca5401
Performance improvements for Comparators (#5687)
Critical performance improvements after #4067

# Important Notes
- Replace if-then-else expressions in `Any.==` with case expressions.
- Fix caching in `EqualsNode`.
- This includes fixing specializations, along with fallback guard.
2023-02-21 00:56:11 +00:00
Jaroslav Tulach
97bcd87dc6
File.parent can return Nothing (#5699)
`File.parent` can be `Nothing`
2023-02-20 19:13:56 +00:00