Commit Graph

937 Commits

Author SHA1 Message Date
Jaroslav Tulach
a150abbea8
Convert dangling comma into unexpected expression (#5776)
Test and fix for #5691.
2023-02-28 07:44:06 +00:00
Jaroslav Tulach
0536eeb0bb
Offer no suggestion for single constructor (#5747)
Resolving #5055 - avoid putting single constructor into suggestion database.

# Important Notes
Another way to fix #5055 is to keep the single constructor information in the suggestion database and let the IDE filter that out.
2023-02-28 05:58:36 +00:00
James Dunkerley
ba3d45e782
Adjust generateCheckedArgumentRead to return correct expected type (#5769)
Correctly get the expected type and return an IllegalArgument if no Enso type.

![image](https://user-images.githubusercontent.com/4699705/221263862-bbc122ca-b11f-49f3-b7cf-6294ea811a22.png)
2023-02-27 17:23:45 +00:00
James Dunkerley
9ee21e8812
Stability improvements in visualizations. (#5760)
- Handle `WithWarnings` in `IndirectInvokeCallableNode`.
- Handle no RootNode in `ErrorResolver`.
- Allow table vizualisation to cope if no `data` passed.
- Add `Warning.has_warnings` to check if warnings present.
- Adjust `set_value` for `JS_Object` so creates a new object each time.
2023-02-24 21:04:50 +00:00
Jaroslav Tulach
78aab133c7
Avoid nulls in Array.items (#5736)
Put `Nothing` into an empty array rather than `null`. When running with `assert` on (unit tests), check the content of the array and `AssertError` quickly when a `null` is found.
2023-02-22 17:53:36 +00:00
Pavel Marek
58c7ca5401
Performance improvements for Comparators (#5687)
Critical performance improvements after #4067

# Important Notes
- Replace if-then-else expressions in `Any.==` with case expressions.
- Fix caching in `EqualsNode`.
- This includes fixing specializations, along with fallback guard.
2023-02-21 00:56:11 +00:00
Jaroslav Tulach
97bcd87dc6
File.parent can return Nothing (#5699)
`File.parent` can be `Nothing`
2023-02-20 19:13:56 +00:00
Jaroslav Tulach
172f72941b
Two findExceptionMessage methods to extract exception messages consistently (#5684)
Creating two `findExceptionMessage` methods in `HostEnsoUtils` and in `VisualizationResult`. Why two? Because one of them is using `org.graalvm.polyglot` SDK as it runs in _"normal Java"_ mode. The other one is using Truffle API as it is running inside of partially evaluated instrument.

There is a `FindExceptionMessageTest` to guarantee consistency between the two methods. It simulates some exceptions in Enso code and checks that both methods extract the same _"message"_ from the exception. The tests verifies hosted and well as Enso exceptions - however testing other polyglot languages is only possible in other modules - as such I created `PolyglotFindExceptionMessageTest` - but that one doesn't have access to Truffle API - e.g. it doesn't really check the consistency - just that a reasonable message is extracted from a JavaScript exception.

# Important Notes
This is not full fix of #5260 - something needs to be done on the IDE side, as the IDE seems to ignore the delivered JSON message - even if it contains properly extracted exception message.
2023-02-20 11:27:16 +00:00
Jaroslav Tulach
cb3bcbb495
Schedule initialization of JS context in a separate thread (#5680)
Implements the #5643 idea. As soon as `MainModule` creates `Context` for GraalVM execution, it schedules a background task to initialize JavaScript. The initialization finishes sooner than Enso compiler is ready to work, saving time when it is actually needed.

# Important Notes
Only modifies boot sequence of `MainModule` (used in the IDE) and `VerifyJavaScriptIsAvailableTest` (to verify the _"context passing logic"_ works OK between threads). Regular CLI execution remains unchanged for now assuming batch execution may not need JavaScript in all the cases and if it does the initialization speed isn't that critical.
2023-02-17 15:54:36 +00:00
Radosław Waśko
4dcf802831
Ensure that warnings are preserved on Nothing values passing back to Enso through polyglot boundary (#5677)
Fixes #5672

# Important Notes
- Added a subproject `enso-test-java-helpers` which allows the in-Enso tests to add Java helpers for testing.
2023-02-17 13:38:26 +00:00
Pavel Marek
3e8467c204
Update GraalVM to 22.3.1 (#5602)
Updates the engine to GraalVM 22.3.1 version, which contains fixes for:
- Chrome inspector issues - 38eb3b5932

# Important Notes
- Update to GraalVM 22.3.1
- Remove host object wrapping workaround
2023-02-14 15:51:17 +00:00
GregoryTravis
99b9728995
Improve Text.toDisplayString (#5045) (#5598) 2023-02-13 18:38:15 +00:00
Dmitry Bushev
7abc281bd8
Set edition when starting language server (#4117)
When the `project.yaml` specifies an unknown edition, the project is started with the fallback edition (current engine version). It works this way:

- IDE sends project/open request to project manager (specifying only which project to open)
- project manager resolves the project edition (based on `project.yaml`) or selects the fallback if the requested edition cannot be loaded
- project manager starts the language server specifying the edition with the `--server-edition` command line parameter
- language server specifies the correct edition when creates the runtime context
2023-02-11 00:32:01 +00:00
Kaz Wesley
d1af25793a
Port graph editor to new AST (#4113)
Use the Rust parser rather than the Scala parser to parse Enso code in the IDE.

Implements:
- https://www.pivotaltracker.com/story/show/182975925
- https://www.pivotaltracker.com/story/show/182988419
- https://www.pivotaltracker.com/story/show/182970096
- https://www.pivotaltracker.com/story/show/182973659
- https://www.pivotaltracker.com/story/show/182974161
- https://www.pivotaltracker.com/story/show/182974205

There is additional functionality needed before the transition is fully-completed, however I think it's time for this to see review and testing, so I've opened separate issues. In rough order of urgency (these issues are also linked from the corresponding disabled tests):
- #5573
- #5571
- #5572
- #5574

# Important Notes
The implementation is based partly on translation, and partly on new analysis. Method- and operator-related shapes are translated to the old `Ast` variants, so that all the analysis applied to them doesn't need to be ported at this time. Everything else (mostly "macros" in the old AST) is implemented with new analysis.
2023-02-10 18:05:40 +00:00
Hubert Plociniczak
b56d6d74b9
Fix serialization of FQNs (#5600)
Serialization of FQNs' metadata was broken - we attempted to serialiaze concrete modules and that's against the design.

Added a test illustrating the problem which would previously fail during serialization.

Closes #5037

# Important Notes
It is still hard to discover problems like this because SerializationManager creates system threads; when the exception occurs, it is typically not the real cause. Creating regular threads via `createThread` seems to be problematic for `native-image`. There is a simple workaround for the former but will leave it out for another PR to simplify the review.
2023-02-10 12:28:34 +00:00
Pavel Marek
1f8511dab2
Add Comparator conversion for all types (#4067)
Add `Comparator` type class emulation for all types. Migrate all the types in stdlib to this new `Comparator` API. The main documentation is in `Ordering.enso`.

Fixes these pivotals:
- https://www.pivotaltracker.com/story/show/183945328
- https://www.pivotaltracker.com/story/show/183958734
- https://www.pivotaltracker.com/story/show/184380208

# Important Notes
- The new Comparator API forces users to specify both `equals` and `hash` methods on their custom comparators.
- All the `compare_to` overrides were replaced by definition of a custom _ordered_ comparator.
- All the call sites of `x.compare_to y` method were replaced with `Ordering.compare x y`.
- `Ordering.compare` is essentially a shortcut for `Comparable.from x . compare x y`.
- The default comparator for `Any` is `Default_Unordered_Comparator`, which just forwards to the builtin `EqualsNode` and `HashCodeNode` nodes.
- For `x`, one can get its hash with `Comparable.from x . hash x`.
- This makes `hash` as _hidden_ as possible. There are no other public methods to get a hash code of an object.
- Comparing `x` and `y` can be done either by `Ordering.compare x y` or `Comparable.from x . compare x y` instead of `x.compare_to y`.
2023-02-10 09:22:11 +00:00
Dmitry Bushev
b58a5f458b
Send method pointers of constructors (#5593)
Closes #5043

Expression updates of constructors contain method pointers.
2023-02-09 11:49:39 +00:00
Hubert Plociniczak
53b3de1cb1
Recover from serialization failures (#5591)
An exception encountered during serialization prevents engine from continuing because it enters an infinite loop(!).

# Important Notes
The aim of this PR is to make it possible for engine to recover from the serialization failures. Any failure would mean that we enter an infinite loop in deserialization which is in turn waiting for the serialization to finish (which will never happen).
In this particular case FQNs are [referencing concrete modules](https://github.com/enso-org/enso/issues/5037). A separate PR will address that.
2023-02-09 08:22:28 +00:00
Hubert Plociniczak
472580df4d
Report type of expressions returning polyglot values (#4111)
Expressions returning polyglot values were not reporting the type of the result because we have to do additional magic that infers the correct Enso type. Since this is exactly what `TypeOfNode` does, I re-used the logic.

Straightforward solution failed in tests because of assertions:
```
[enso] WARNING: Execution of function main failed (Invalid library usage. Cached library must be adopted by a RootNode before it is executed.).
java.lang.AssertionError: Invalid library usage. Cached library must be adopted by a RootNode before it is executed.
```

That is why this PR replaces `ExecutionEventListener` with `ExecutionEventNodeFactory`.

# Important Notes
Usage of `TypeOfNode` for programs that **do not** import stdlib means that we report types that do not involve stdlib e.g.
`Standard.Builtins.Main.Integer` instead of `Standard.Base.Data.Numbers.Integer`. While surprising, this is correct and I would say desirable. While reviewing the code, notice the difference in expectations in our runtime tests.
2023-02-09 01:06:27 +00:00
James Dunkerley
1c821e22cf
Some fixed form the Anagrams experiment. (#5592)
- Fixes the display of Date, Time_Of_Day and Date_Time so doesn't wrap.
- Adjust serialization of large integer values for JS and display within table.
- Workaround for issue with using `.lines` in the Table (new bug filed).
- Disabled warning on no specified `separator` on `Concatenate`.

Does not include fix for aggregation on integer values outside of `long` range.
2023-02-08 22:17:00 +00:00
Hubert Plociniczak
4405be8a74
Reduce JGit logs verbosity in dev mode (#4106) 2023-02-08 14:03:16 +00:00
Dmitry Bushev
53d5487f47
Lookup method pointers in IDE (#5578)
Closes #5036

Move the logic that looks up method pointers from the language server to IDE. This way we can keep the suggestion updates and expression updates async, otherwise it will hurt the initial startup time of LS.
Fixes the issue when some expression updates does not contain the method pointer.
2023-02-08 00:57:32 +00:00
Hubert Plociniczak
cc7d024495
Stabilize flaky test that re-uses test setup (#5577)
Follow up on
16ba57d465 which marked the tests as flaky.

This test goes back to the original implementation where JGit did all its book-keeping synchronously. When we switched to asynchronous operation, cleaning up the test directory would sometimes not succeed because there would be a race-condition with the ongoing work of JGit.

# Important Notes
In the current setup tagging a test as flaky is only valid for Windows. The other platforms ignore the flag. So we were still getting random failures.
2023-02-07 21:17:17 +00:00
James Dunkerley
c39410b4d1
Adjusting annotations to use Widget types properly. (#5564)
Closes #5038

- Use the proper widget structure.
- Provide new method `get_widget_json` with whole structure, but keep `get_full_annotations_json` in old form.
- Start to get to some reusable functions.
- Added widget to JS_Object field selections.
2023-02-07 07:33:51 +00:00
Jaroslav Tulach
04415a2b5e
Write the log in XML format suitable for VisualVM 'UI Actions' (#4110)
Start `project-manager` with following options to provide first 20s of the startup sequence:
```
$ project-manager --profiling-events-log-path=start.log --profiling-path=start.npss --profiling-time=20
```
once the `start.log` and `start.npss` files are generated (next to each other), open them in GraalVM's VisualVM:
```
$ graalvm/bin/jvisualvm --openfile start.npss
```
analyze.
2023-02-05 06:36:16 +00:00
Paweł Grabarz
6b14ec5a63
Dynamic dropdown support (#4072)
Implementation of https://www.pivotaltracker.com/story/show/184012743

https://user-images.githubusercontent.com/919491/214082311-cf49e43c-1d1f-4654-903c-a4224cd954d8.mp4

This is also a step towards more general widget support. The widget metadata is queried using `Meta.get_annotation` method through a dedicated visualization. For now only `Single_Choice` case is handled, and always all suggestions are is returned.

# Important Notes
There are limitations as to which node segments receive a widget. Only chain method calls are supported now (`thing.method` syntax), and only outside of lambda scope. Widgets in lambdas will require support for visualisations of lambda subexpressions, which is currently missing in the engine. The IDE technically tries to place the widgets there, but the data never arrives. It should work once the engine support is added.

This PR includes a mock for `Meta.get_annotation` call that only supports `Table.at` method. Real implementation is a separate task that is already in progress.
2023-02-04 00:50:24 +00:00
Hubert Plociniczak
50f376e365
JGit should run its bookkeeping in the background (#4116)
JGit ops generally run fast (as in a few milliseconds) except for the first commit. The initialization + first commit was taking at least 3.5 seconds constistenly, but only in the first test case. Now, this led to frequent timeouts down the chain when the request was expected to finish fast.
The bookkeeping involved some timestamping and other expensive calls in order to calculate clock drift. The default appears to be to run it in a blocking mode, hence adding at least 3 seconds to the first command call.

Setting the job to run in the background makes the cost of the repo initialization acceptable (~300 milliseconds on a cold JVM). The other commands are unaffected and take < 10 milliseconds.

# Important Notes
Added a test to ensure that we don't introduce the regression. Marked it as potentially flaky because it uses timestamps and it is  therefore prone to random system hiccups.
2023-02-03 10:37:28 +00:00
Jaroslav Tulach
6735fa99c8
Prevent errors in log when closing a context (#4107)
Hold all executors associated with `EnsoContext` and close them when the context is being closed.
2023-02-01 16:27:45 +00:00
Hubert Plociniczak
a618dd6901
Don't attempt to access JGit config in VCS ops (#4105)
There is no need for JGit to try to save file attributes to its config in `<home>/jgit/config`. This change ensures that JGit operates on a stub file, without polluting users' configs, similarly to user's git config.

Additionally, noticed that all FS operations during `init` are rather slow and saw at least on one occassion when the handler timed out because of that (https://www.pivotaltracker.com/story/show/184359934). This change provides an init-specific timeout for `vcs/init`.

# Important Notes
This change should reduce chances of failures during VCS initialization.
2023-01-31 21:57:57 +00:00
James Dunkerley
0790ce494f
New set function, parse a column (#4097)
- New `set` function design - takes a `Column` and works with that more easily and supports control of `Set_Mode`.
- New simple `parse` API on `Column`.
- Separated expression support for `filter` to new `filter_by_expression` on `Table`.
- New `compute` function allowing creation of a column from an expression.
- Added case sensitivity argument to `Column` based on `starts_with`, `ends_with` and `contains`.
- Added case sensitivity argument to `Filter_Condition` for `Starts_With`, `Ends_With`, `Contains` and `Not_Contains`.
- Fixed the issue in JS Table visualisation where JavaScript date was incorrectly set.
- Some dynamic dropdown expressions - experimenting with ways to use them.
- Fixed issue with `.pretty` that wasn't escaping `\`.
- Changed default Postgres DB to `postgres`.
- Fixed SQLite support for starts_with, ends_with and contains to be consistent (using GLOB not LIKE).
2023-01-31 20:48:16 +00:00
Hubert Plociniczak
d3b350f460
Custom type conversion to double from large long (#4099)
When a large long would be passed to a host call expecting a double, it would crash with a
```
Cannot convert '<some long>'(language: Java, type: java.lang.Long) to Java type 'double': Invalid or lossy primitive coercion
```

That is unlikely to be expected by users. It also came up in the Statistics examples during Sum. One could workaround it by forcing the conversion manually with `.to_decimal` but it is not a permanent solution.

Instead this change adds a custom type mapping from Long to Double that will do it behind the scenes with no user interaction. The mapping kicks in only for really large longs.

# Important Notes
Note that the _safe_ range is hardcoded in Truffle and it is not accessible in enso packages. Therefore a simple c&p for that max safe long value was necessary.
2023-01-31 15:13:00 +00:00
Jaroslav Tulach
f54464881f
enso4igv can open engine/language-server & co. projects (#4098)
In order to investigate `engine/language-server` project, I need to be able to open its sources in IGV and NetBeans.

# Important Notes
By adding same Java source (this time `package-info.java`) and compiling with our Frgaal compiler the necessary `.enso-sources*` files are generated for `engine/language-server` and then the `enso4igv` plugin can open them and properly understand their compile settings.

![Logical View of language-server project](https://user-images.githubusercontent.com/26887752/215472696-ec9801f3-4692-4bdb-be92-c4d2ab552e60.png)

In addition to that this PR enhances the _"logical view"_ presentation of the project by including all source roots found under `src/*/*`.
2023-01-31 08:40:04 +00:00
Dmitry Bushev
3de488fc46
Fix vcs-restore edit notification (#4100)
Changelog:
- fix: vcs/restore sends proper text edit
- refactor: restore original `OpenFileNotification` name
2023-01-30 21:08:30 +00:00
Hubert Plociniczak
be91b1e8dd
Create static wrappers for builtin types (#4077)
https://github.com/enso-org/enso/pull/3764 introduced static wrappers for instance methods. Except it had a limitation to only be allowed for types with at least a single constructor.
That excluded builtin types as well which, by default, don't have them. This limitation is problematic for Array/Vector consolidation and makes builtin types somehow second-citizens.

This change lifts the limitation for builtin types only. Note that we do want to share the implementation of the generated builtin methods. At the same time due to the additional argument we have to adjust the starting index of the arguments.
This change avoids messing with the existing dispatch logic, to avoid unnecessary complexity.

As a result it is now possible to call builtin types' instance methods, statically:
```
arr = Array.new_1 42
Array.length arr
```
That would previously lead to missing method exception in runtime.

# Important Notes
The only exception is `Nothing`. Primarily because it requires `Nothing` to have a proper eigentype (`Nothing.type`) which would messed up a lot of existing logic for no obvious benefit (no more calling of `foo=Nothing` in parameters being one example).
2023-01-30 19:54:51 +00:00
Dmitry Bushev
6b912968e4
Send all pending expressions (#4006)
For graying out the nodes, the engine need to send IDE a set of values that will be computed before executing the program (and the IDE colors them gray). In general, it is tricky to do because we cannot know for sure which exactly nodes will be computed without running the program. But we can estimate based on the invalidated values, which nodes are expected to be executed during the next run, and send them to the IDE. This logic is simpler than the previous approach, and turned out working pretty well in practice.

[Peek-gray-out.webm](https://user-images.githubusercontent.com/357683/215092755-0010e41d-a2cf-447a-900e-4619408effa0.webm)
2023-01-30 11:46:44 +00:00
Jaroslav Tulach
53efcf0a17
Propagate subjectToInstrumentation flag via ExpressionProcessor (#4090)
`subjectToInstrumentation` needs to be propagated via `ExpressionProcessor`.
2023-01-30 05:48:19 +00:00
Dmitry Bushev
7e8f49e86f
vcs/restore triggers execution (#4095)
Changelog:
- update: `vcs/restore` command triggers program execution
2023-01-28 06:09:23 +00:00
Dmitry Bushev
b65dd1f3fe
Warnings API (#4093)
Add information about warngins associated with the value to the expression update payload.
2023-01-28 03:13:38 +00:00
Hubert Plociniczak
9b26077775
Provide better diagnostics when handlers crash (#4091)
In the event when the action of the underlying actor crashes really bad, the result of the future will be `Failure`. All such results will be then wrapped in `Status.Failure` via `pipeTo` (unlike `Success` which just forwards the response).

In some cases we don't handle `Status.Failure` messages, meaning we are rather left in the dark to the reason of the failure with a non-informative message `Received unknown message: class akka.actor.Status$Failure`.

I'm keeping this change small on purpose to keep this change self-contained. I think there are more cases but it needs careful investigation into how messages are being sent.

# Important Notes
This PR does not attempt to fix the underlying problem of the ticket, yet. We should however have a better overview where/why things go wrong.
2023-01-27 10:54:37 +00:00
Jaroslav Tulach
ca2f108513
Enable asserts in the tests (#4074)
Enso unit tests were running without `-ea` check enabled and as such various invariant checks in Truffle code were not executed. Let's turn the `-ea` flag on and fix all the code misbehaves.
2023-01-26 21:41:35 +00:00
Hubert Plociniczak
ad6419f204
Reload module's sources when LS reloads buffer (#4083)
LS needs to notify runtime that modules' sources need to be reloaded from FS, once its own buffer has been reloaded as well.

# Important Notes
Discovered during integration of https://github.com/enso-org/enso/pull/4050.
The test illustrates the problem if we don't reload module's sources - the sources essentially become stale even though they have changed.
2023-01-26 19:07:54 +00:00
Dmitry Bushev
3dbceef163
Default visualization preprocessor handles errors (#4080)
When the function is visualized with the `default_preprocessor` method, it returns a dataflow error that shows up in the logs as
```
Cannot encode class org.enso.interpreter.runtime.error.DataflowError to byte array.
```

PR handles dataflow errors in the preprocessor and fallbacks the `to_display_text` value. As a bonus, we get a function visualization.
2023-01-25 03:37:21 +00:00
Hubert Plociniczak
c85377f0ac
Eliminate various compiler warnings (#4079)
Slipped through review of https://github.com/enso-org/enso/pull/3862
2023-01-24 22:58:35 +00:00
Dmitry Bushev
bf9508603f
Add Meta.get_annotation (#4049)
- add: `GeneralAnnotation` IR node for `@name expression` annotations
- update: compilation pipeline to process the annotation expressions
- update: rewrite `OverloadsResolution` compiler pass so that it keeps the order of module definitions
- add: `Meta.get_annotation` builtin function that returns the result of annotation expression
- misc: improvements (private methods, lazy arguments, build.sbt cleanup)
2023-01-24 21:28:33 +03:00
Marcin Kostrzewa
242bd52942
Unboxed atoms (#3862)
Introduces unboxed (and arity-specialized) storage schemes for Atoms. It results in improvements both in memory consumption and runtime.
Memory wise: instead of using an array, we now use object fields. We also enable unboxing. This cuts a good few pointers in an unboxed object. E.g. a quadruple of integers is now 64 bytes (4x8 bytes for long fields + 16 bytes for layout and constructor pointers + 16 bytes for a class header). It used to be 168 bytes  (4x24 bytes for boxed Longs + 16 bytes for array header + 32 bytes for array contents +  8 bytes for constructor ptr  + 16 bytes for class header), so we're saving 104 bytes a piece. In the least impressive scenarios (all-boxed fields) we're saving 8 bytes per object (saving 16 bytes for array header, using 8 bytes for the new layout field). In the most-benchmarked case (list of longs), we save 32 bytes per cons-cell.
Time wise:
All list-summing benchmarks observe a ~2x speedup. List generation benchmarks get ~25x speedups, probably both due to less GC activity and better allocation characteristics (only allocating one object per Cons, rather than Cons + Object[] for fields). The "map-reverse" family gets a neat 10x speedup (part of the work is reading, which is 2x faster, the other is allocating, which is now 25x faster, we end up with 10x when combined).
2023-01-24 13:03:06 +00:00
Hubert Plociniczak
86eee6199d
Fix TextEdit range (#4069)
Typo: TextEdit's range should refer to the old buffer's range rather than the new one.
2023-01-20 11:31:16 +00:00
Kaz Wesley
591cacb79a
Reject @ as binary operator (#4021)
`@` should not be legal to use as a binary operator. I accepted it in the parser because it occurred in the .enso sources, but it was actually used to create a syntax error to test error recovery.

See: https://www.pivotaltracker.com/story/show/184054024
2023-01-19 20:31:14 +00:00
Pavel Marek
fcc2163ae3
All Enso objects are hasheable (#3878)
* Hash codes prototype

* Remove Any.hash_code

* Improve caching of hashcode in atoms

* [WIP] Add Hash_Map type

* Implement Any.hash_code builtin for primitives and vectors

* Add some values to ValuesGenerator

* Fix example docs on Time_Zone.new

* [WIP] QuickFix for HashCodeTest before PR #3956 is merged

* Fix hash code contract in HashCodeTest

* Add times and dates values to HashCodeTest

* Fix docs

* Remove hashCodeForMetaInterop specialization

* Introduce snapshoting of HashMapBuilder

* Add unit tests for EnsoHashMap

* Remove duplicate test in Map_Spec.enso

* Hash_Map.to_vector caches result

* Hash_Map_Spec is a copy of Map_Spec

* Implement some methods in Hash_Map

* Add equalsHashMaps specialization to EqualsAnyNode

* get and insert operations are able to work with polyglot values

* Implement rest of Hash_Map API

* Add test that inserts elements with keys with same hash code

* EnsoHashMap.toDisplayString use builder storage directly

* Add separate specialization for host objects in EqualsAnyNode

* Fix specialization for host objects in EqualsAnyNode

* Add polyglot hash map tests

* EconomicMap keeps reference to EqualsNode and HashCodeNode.

Rather than passing these nodes to `get` and `insert` methods.

* HashMapTest run in polyglot context

* Fix containsKey index handling in snapshots

* Remove snapshots field from EnsoHashMapBuilder

* Prepare polyglot hash map handling.

- Hash_Map builtin methods are separate nodes

* Some bug fixes

* Remove ForeignMapWrapper.

We would have to wrap foreign maps in assignments for this to be efficient.

* Improve performance of Hash_Map.get_builtin

Also, if_nothing parameter is suspended

* Remove to_flat_vector.

Interop API requires nested vector (our previous to_vector implementation). Seems that I have misunderstood the docs  the first time I read it.

- to_vector does not sort the vector by keys by default

* Fix polyglot hash maps method dispatch

* Add tests that effectively test hash code implementation.

Via hash map that behaves like a hash set.

* Remove Hashcode_Spec

* Add some polyglot tests

* Add Text.== tests for NFD normalization

* Fix NFD normalization bug in Text.java

* Improve performance of EqualsAnyNode.equalsTexts specialization

* Properly compute hash code for Atom and cache it

* Fix Text specialization in HashCodeAnyNode

* Add Hash_Map_Spec as part of all tests

* Remove HashMapTest.java

Providing all the infrastructure for all the needed Truffle nodes is no longer manageable.

* Remove rest of identityHashCode message implementations

* Replace old Map with Hash_Map

* Add some docs

* Add TruffleBoundaries

* Formatting

* Fix some tests to accept unsorted vector from Map.to_vector

* Delete Map.first and Map.last methods

* Add specialization for big integer hash

* Introduce proper HashCodeTest and EqualsTest.

- Use jUnit theories.
- Call nodes directly

* Fix some specializations for primitives in HashCodeAnyNode

* Fix host object specialization

* Remove Any.hash_code

* Fix import in Map.enso

* Update changelog

* Reformat

* Add truffle boundary to BigInteger.hashCode

* Fix performance of HashCodeTest - initialize DataPoints just once

* Fix MetaIsATest

* Fix ValuesGenerator.textual - Java's char is not Text

* Fix indent in Map_Spec.enso

* Add maps to datapoints in HashCodeTest

* Add specialization for maps in HashCodeAnyNode

* Add multiLevelAtoms to ValuesGenerator

* Provide a workaround for non-linear key inserts

* Fix specializations for double and BigInteger

* Cosmetics

* Add truffle boundaries

* Add allowInlining=true to some truffle boundaries.

Increases performance a lot.

* Increase the size of vectors, and warmup time for Vector.Distinct benchmark

* Various small performance fixes.

* Fix Geo_Spec tests to accept unsorted Map.to_vector

* Implement Map.remove

* FIx Visualization tests to accept unsorted Map.to_vector

* Treat java.util.Properties as Map

* Add truffle boundaries

* Invoke polyglot methods on java.util.Properties

* Ignore python tests if python lang is missing
2023-01-19 10:33:25 +01:00
Hubert Plociniczak
d463a43633
Resolve fully qualified names (#4056)
Added a separate pass, `FullyQualifiedNames`, that partially resolves fully qualified names. The pass only resolves the library part of the name and replaces it with a reference to the `Main` module.

There are 2 scenarios that could be potentially:
1) the code uses a fully qualified name to a component that has been
parsed/compiled
2) the code uses a fully qualified name to a component that has **not** be
imported

For the former case, it is sufficient to just check `PackageRepository` for the presence of the library name.
In the latter we have to ensure that the library has been already parsed and all its imports are resolved. That would require the reference to `Compiler` in the `FullyQualifiedNames` pass, which could then trigger a full compilation for missing library. Since it has some undesired consequences (tracking of dependencies becomes rather complex) we decided to exclude that scenario until it is really needed.

# Important Notes
With this change, one can use a fully qualified name directly.
e.g.
```
import Standard.Base
main =
Standard.Base.IO.println "Hello world!"
```
2023-01-18 20:19:36 +00:00
Dmitry Bushev
ed859c2682
Allow attaching visualizations to non-existent expressions (#4064)
Requested by @Frizi. IDE does not synchronize requests creating a node and attaching the visualization, and they can be sent in arbitrary order.
2023-01-18 18:56:32 +00:00
Hubert Plociniczak
5219ce5c5f
Fix clearing modules' sources during fileClose (#4059)
Unsetting literal sources during the `text/fileClose` operation would also unlink sources from the actual file. That would be never set back during the reopening of the file, resulting in an NPE during application of changes at a later point.

# Important Notes
Revealed during https://github.com/enso-org/enso/pull/4050 which was the first(?) to do close/open of the file.
The exception showing up in GUI and preventing further actions is now gone.
2023-01-18 17:48:51 +00:00
Jaroslav Tulach
cd172dad80
Remove all UnhandledEntity panics (#4054)
The new `TreeToIr` conversion class inherited usage of `UnhandledEntity` to signal parser errors from the previous `AstToIr` Scala based convertor. Over the time we were improving error recovery - however only _step by step_. This pull request exterminates _all the panics_ for once and forever!

# Important Notes
Unlike Scala, Java has concept of [checked exceptions](http://wiki.apidesign.org/wiki/Checked_exception) - an exception that has to be caught. This PR introduces new checked `SyntaxException`  and throws it instead of unchecked `UnhandledEntity`. Because the exception is checked, each method either needs to declare it in its signature or catch it and handle it. No exception can escape or disappear. The main conversion method `TreeToIr.translateModule` doesn't propagate the `SyntaxException`. That provably demonstrates - _all panic states_ are handled and reported as _syntax errors_ in the `IR`.
2023-01-17 08:24:07 +00:00
Hubert Plociniczak
74659301e7
Make sure import/export resolution does not bring duplicates (#4037)
Component Groups may add additional modules to compilation. This change ensures that whatever modules result from import/export resolution, they are not duplicate. Not only could that lead to unnecessary compilation but also to multiple reports of the same error which would be confusing to users.
This fixes https://www.pivotaltracker.com/story/show/184189980

# Important Notes
No easy way to write unit tests for this, so skipping that on purpose for now.
2023-01-16 12:18:34 +00:00
Dmitry Bushev
0a6e6237cd
Attach visualizations to sub-expressions (#4048)
Add ability to attach visualizations to sub-expressions.
2023-01-16 10:19:19 +00:00
Jaroslav Tulach
619974d97d
Avoid IllegalStateException in favor of syntax error in the IR (#4051)
Don't throw `IllegalStateException` when parsing.
2023-01-16 08:52:43 +00:00
Jaroslav Tulach
917176873d
visualizationResultToBytes converts anything that looks like a string (#4045)
Use `InteropLibrary.isString` and `asString` to convert any string value to `byte[]`

# Important Notes
Also contains a support for `Metadata.assertInCode` to help locating the right place in the code snippets.
2023-01-13 12:30:27 +00:00
Kaz Wesley
e15583fe65
Parser: Support annotations in type defs (#4036)
Support application of the new type of annotation to method bindings in type definitions.
2023-01-12 16:51:44 +00:00
James Dunkerley
c4c35c92b7
Align Vector API with design, add some extra functions from AoC (#4026)
**Vector**
- Adjusted `Vector.sort` to be `Vector.sort order on by`.
- Adjusted other sort to use `order` for direction argument.
- Added `insert`, `remove`, `index_of` and `last_index_of` to `Vector`.
- Added `start` and `if_missing` arguments to `find` on `Vector`, and adjusted default is `Not_Found` error.
- Added type checking to `+` on `Vector`.
- Altered `first`, `second` and `last` to error with `Index_Out_Of_Bounds` on `Vector`.
- Removed `sum`, `exists`, `head`, `init`, `tail`, `rest`, `append`, `prepend` from `Vector`.

**Pair**
- Added `last`, `any`, `all`, `contains`, `find`, `index_of`, `last_index_of`, `reverse`, `each`, `fold` and `reduce` to `Pair`.
- Added `get` to `Pair`.

**Range**
- Added `first`, `second`, `index_of`, `last_index_of`, `reverse` and `reduce` to `Range`.
- Added `at` and `get` to `Range`.
- Added `start` and `if_missing` arguments to `find` on `Range`.
- Simplified `last` and `length` of `Range`.
- Removed `exists` from `Range`.

**List**
- Added `second`, `find`, `index_of`, `last_index_of`, `reverse` and `reduce` to `Range`.
- Added `at` and `get` to `List`.
- Removed `exists` from `List`.
- Made `all` short-circuit if any fail on `List`.
- Altered `is_empty` to not compute the length of `List`.
- Altered `first`, `tail`, `head`, `init` and `last` to error with `Index_Out_Of_Bounds` on `List`.

**Others**
- Added `first`, `second`, `last`, `get` to `Text`.
- Added wrapper methods to the Random_Number_Generator so you can get random values more easily.
- Adjusted `Aggregate_Column` to operate on the first column by default.
- Added `contains_key` to `Map`.
- Added ALIAS to `row_count` and `order_by`.
2023-01-12 13:32:24 +00:00
Hubert Plociniczak
a3de151ff4
Workaround vcs/restore issues on Windows (#4042)
Potential workaround to line endings problems after `vcs/restore` operation is executed.

# Important Notes
Not really able to reproduce the problem myself so this PR has a lot of _leap of faith_ in it.
2023-01-12 12:28:09 +00:00
Jaroslav Tulach
fa22e467bf
Catch Panic and Error from Atom.to_text and yield detailed error message (#4034)
Don't propagate errors from `toDisplayString` - construct an error message with `Atom.toString`.

# Important Notes
> currently a failure in to_text is swallowed by `toString` and we cannot detect that something went wrong during the serialization

Not sure how satisfying the solution is, but the error swallowing happens in Truffle and there is little to do with it. We can just catch the error ourselves and produce some meaningful string.
2023-01-12 08:07:26 +00:00
Dmitry Bushev
2cd880f43d
Documentation for functions and locals (#4029)
Add documentation for functions and locals to suggestions database.
2023-01-10 16:59:53 +00:00
Jaroslav Tulach
bc66753627
Don't include type in tagValues (#4032)
`Text.trim` `what` argument offered `Text` in `tagValues` - that's wrong. Using the `Text` type isn't allowed value for `Text`

# Important Notes
I had to update three other tests to match the new behavior.
2023-01-10 14:10:46 +00:00
Hubert Plociniczak
ae0889e843
Make ArrayOverBuffer behave like an Array/Array.sort no longer mutates the Array (#4022)
Most of the problems with accessing `ArrayOverBuffer` have been resolved by using `CoerceArrayNode` (https://github.com/enso-org/enso/pull/3817). In `Array.sort` we still however specialized on Array which wasn't compatible with `ArrayOverBuffer`. Similarly sorting JS or Python arrays wouldn't work.

Added a specialization to `Array.sort` to deal with that case. A generic specialization (with `hasArrayElements`) not only handles `ArrayOverBuffer` but also polyglot arrays coming from JS or Python. We could have an additional specialization for `ArrayOverBuffer` only (removed in the last commit) that returns `ArrayOverBuffer` rather than `Array` although that adds additional complexity which so far is unnecessary.

Also fixed an example in `Array.enso` by providing a default argument.
2023-01-09 17:49:49 +00:00
Jaroslav Tulach
41b2aac39f
Removing Unsafe.set_atom_field (#4023)
Introducing `Meta.atom_with_hole` to create an `Atom` _with a hole_ that is then _safely_ filled in later.
2023-01-09 13:39:14 +00:00
Hubert Plociniczak
3379ce51f2
Report failed name resolutions in type signatures (#4030)
Compiler performed name resolution of literals in type signatures but would silently fail to report any problems.
This meant that wrong names or forgotten imports would sneak in to stdlib.

This change introduces 2 main changes:
1) failed name resolutions are appended in `TypeNames` pass
2) `GatherDiagnostics` pass also collects and reports failures from type
signatures IR

Updated stdlib so that it passes given the correct gatekeepers in place.
2023-01-09 10:35:36 +00:00
Hubert Plociniczak
3980c48d61
Sync file system and language server after restore (#4020)
VCS restore operation was correctly restoring the state of projects to the requested commit. Unfortunately, after the operation file system was becoming out-of-sync with language server's buffers (and IDE's content versions).

A few important changes are introduced here that complicate the interaction between components:
1) `vcs restore` returns an actual diff between the current state and the
requested commit
2) the response is forwarded to buffer registry first rather than to the client
3) the diff is used to identify appropriate collaborative editors and
notify them about the need to reload buffers from file system
4) all clients of affected open buffers are notified of the change via
`text/didChange` notification. If a file was removed and there were open buffers for it, clients will be notified via `file/event` and editor will be stopped
5) only then the client is notified about a successful restore operation

This PR addresses one of the two problems reported in https://www.pivotaltracker.com/story/show/184097084.

# Important Notes
We need to make sure that IDE correctly responds to `text/didChange` notifications.
2023-01-05 14:00:00 +00:00
Radosław Waśko
8c661fdb74
Database Joins (#4007)
Implements https://www.pivotaltracker.com/story/show/184032869

# Important Notes
- Currently we get failures in Full joins on Postgres which show a more serious problem - amending equality to ensure that `[NULL = NULL] == True` breaks hash/merge based indexing - so such joins will be extremely inefficient. All our joins currently rely on this notion of equality which will mean all of our DB joins will be extremely inefficient.
- We need to find a solution that will support nulls and still work OK with indices (but after exploring a few approaches: `COALESCE(a = b, a IS NULL AND b is NULL)`, `a IS NOT DISTINCT FROM b`, `(a = b) OR (a IS NULL AND b is NULL)`; all of which did not work (they all result in `ERROR: FULL JOIN is only supported with merge-joinable or hash-joinable join conditions`) I'm less certain that it is possible. Alternatively, we may need to change the NULL semantics to align it with SQL - this seems like likely the simpler solution, allowing us to generate simple, reliable SQL - the NULL=NULL solution will be cornering us into nasty workarounds very dependent on the particular backend.
2023-01-05 10:36:22 +00:00
Kaz Wesley
0a782a2ed1
AST support for new annotation syntax (#4018)
* rename Annotated -> AnnotatedBuiltin

* update Panic.catch test

* new annotations

* Revert "update Panic.catch test"

This reverts commit ccda46e3a7.
2023-01-04 09:50:38 -08:00
Dmitry Bushev
9df6448d85
Add Polyglot Support in Runtime Tests (#4016)
`runtime-with-instruments` project sets `-Dgraalvm.locatorDisabled=true` that disables the discovery of available polyglot languages (installed with `gu`). On the other hand, enabling locator makes polyglot languages available, but also makes the program classes and the test classes loaded with different classloaders. This way we're unable to use `EnsoContext` in tests to observe internal context state (there is an exception when you try to cast to `EnsoContext`).

The solution is to move tests with enabled polyglot support, but disabled `EnsoContext` introspection to a separate project.
2023-01-03 14:36:26 +00:00
Dmitry Bushev
1e5e2327ab
Improve performance of Text.compare_to (#4012)
PR adds a flag to `Text` implementation tracking whether it is in a FCD normal form. Then this information can be used in the `Normalizer.compare` method.



| Benchmark name | Old (ms) | With flag (ms)
| --- | --- | ---
| Unicode very short | 40.29 | 40.04
| Unicode medium | 9.07 | 1.99
| Unicode big - random | 115.39 | 0.35
| Unicode big - early difference | 107.02 | 0.54
| Unicode big - late difference | 749.81 | 94.73
| ASCII very short | 28.13 | 31.13
| ASCII medium | 4.58 | 2.26
| ASCII big - random | 42.68 | 0.26
| ASCII big - early difference | 30.91 | 0.32
| ASCII big - late difference | 66.29 | 42.72

Full benchmark output.
[bench_old.txt](https://github.com/enso-org/enso/files/10325202/bench_old.txt)
[bench_new.txt](https://github.com/enso-org/enso/files/10325201/bench_new.txt)
2023-01-02 17:09:03 +00:00
Hubert Plociniczak
d15bd8ab3b
Simplify compilation of nested patterns (#4005)
`NestedPatternMatch` pass desugared complex patterns in a very inefficient way resulting in an exponential generation of the number of `case` IR (and Truffle) nodes. Every failed nested pattern would copy all the remaining patterns of the original case expression, in a desugared form. While the execution itself of such deeply nested `case` expression might not have many problems, the time spent in compilation phases certainly was a blocker.

This change desugars deeply nested into individual cases with a fallthrough logic. However the fallthrough logic is implemented directly in Truffle nodes, rather than via IR. That way we can generate much simpler IR for nested patterns.

Consider a simple case of
```
case x of
Cons (Cons a b) Nil -> a + b
Cons a Nil -> a
_ -> 0
```

Before the change, the compiler would generate rather large IR even for those two patterns:
```
case x of
Cons w y -> case w of
Cons a b -> case y of
Nil -> a + b
_ -> case x of
Cons a z -> case z of
Nil -> a
_ -> case x of
_ -> 0
_ -> 0
_ -> case x of
Cons a z -> case z of
Nil -> a
_ -> case x of
_ -> 0
_ -> 0
Cons a z -> case z of
Nil -> a
_ -> case x of
_ -> 0
_ -> 0
```

Now we generate simple patterns with fallthrough semantics and no catch-all branches:
```
case x of
Cons w y -> case w of
Cons a b -> case y of   ## fallthrough on failed match ##
Nil -> a + b                ## fallthrough on failed match ##
Cons a z -> case z of
Nil -> a                          ## fallthrough on failed match ##
_ -> 0
```

# Important Notes
If you wonder how much does it improve, then @radeusgd's example in https://www.pivotaltracker.com/story/show/183971366/comments/234688327 used to take at least 8 minutes to compile and run.
Now it takes 5 seconds from cold start.

Also, the example in the benchmark includes compilation time on purpose (that was the main culprit of the slowdown).
For the old implementation I had to kill it after 15 minutes as it still wouldn't finish a single compilation.
Now it runs 2 seconds or less.

Bonus points: This PR will also fix problem reported in https://www.pivotaltracker.com/story/show/184071954 (duplicate errors for nested patterns)
2022-12-30 10:56:27 +00:00
Jaroslav Tulach
af57d14a8f
Nested type declaration shall yield a syntax error (#4011)
Nested type declaration shall yield a syntax error

# Important Notes
Now the Radek's sample:
```
type Foo
type Bar

main = 42
```
yields
```bash
$ enso --run test.enso
In module test:
Compiler encountered errors:
test.enso[2:9-2:16]: Unexpected declaration in the body of a type.
Aborting due to 1 errors and 0 warnings.
Execution finished with an error: Compilation aborted due to errors.
```
2022-12-30 09:51:21 +00:00
Pavel Marek
e6838bc90d
Convert Any.== to a builtin (#3956)
`Any.==` is a builtin method. The semantics is the same as it used to be, except that we no longer assume `x == y` iff `Meta.is_same_object x y`, which used to be the case and caused failures in table tests.

# Important Notes
Measurements from `EqualsBenchmarks` shows that the performance of `Any.==` for recursive atoms increased by roughly 20%, and the performance for primitive types stays roughly the same.
2022-12-29 21:20:00 +00:00
Dmitry Bushev
74742d3267
Make To Text Conversion Identity for Text (#4009)
First part of fixing `Text.to_text`.
- add: `pretty` method for pretty printing.
- update: make `Text.to_text` conversion identity for Text

In the next iterations `to_text` will be gradually replaced with `to Text` conversion once the related issues with conversions are fixed.
2022-12-29 12:21:24 +00:00
Jaroslav Tulach
7252af6d62
Enso.getMetaObject, Type.isMetaInstance and Meta.is_a consolidation (#3949)
Implements `getMetaObject` and related messages from Truffle interop for Enso values and types. Turns `Meta.is_a` into builtin and re-uses the same functionality.

# Important Notes
Adds `ValueGenerator` testing infrastructure to provide unified access to special Enso values and builtin types that can be reused by other tests, not just `MetaIsATest` and `MetaObjectTest`.
2022-12-22 08:00:06 +00:00
Radosław Waśko
c0c0abe4fe
Add benchmarks comparing ArrayProxy with elements generated ad-hoc with a regular Vector (#3831) 2022-12-21 20:15:39 +00:00
Dmitry Bushev
f16b902920
Fix collapsed node is not updating visualizations (#4002)
The culprit was cache invalidation.
2022-12-21 19:01:15 +00:00
Kaz Wesley
d24019aa57
Implement SKIP/FREEZE in parser/TreeToIr (#3942)
See: https://www.pivotaltracker.com/story/show/183919788

# Important Notes
`SKIP` would be simpler if implemented in the parser, but there is some work needed before the Rust AST and Java IR are able to represent the results of macro-expansion: https://www.pivotaltracker.com/story/show/184004555
2022-12-20 17:32:59 +00:00
James Dunkerley
ace459ed53
Let JavaScript parse JSON and write JSON ... (#3987)
Use JavaScript to parse and serialise to JSON. Parses to native Enso object.
- `.to_json` now returns a `Text` of the JSON.
- Json methods now `parse`, `stringify` and `from_pairs`.
- New `JSON_Object` representing a JavaScript Object.
- `.to_js_object` allows for types to custom serialize. Returning a `JS_Object`.
- Default JSON format for Atom now has a `type` and `constructor` property (or method to call for as needed to deserialise).
- Removed `.into` support for now.
- Added JSON File Format and SPI to allow `Data.read` to work.
- Added `Data.fetch` API for easy Web download.
- Default visualization for JS Object trunctes, and made Vector default truncate children too.

Fixes defect where types with no constructor crashed on `to_json` (e.g. `Matching_Mode.Last.to_json`.
Adjusted default visualisation for Vector, so it doesn't serialise an array of arrays forever.
Likewise, JS_Object default visualisation is truncated to a small subset.

New convention:
- `.get` returns `Nothing` if a key or index is not present. Takes an `other` argument allowing control of default.
- `.at` error if key or index is not present.
- `Nothing` gains a `get` method allowing for easy propagation.
2022-12-20 10:33:46 +00:00
Dmitry Bushev
f4ebdfdc71
Always send method pointer updates (#3988)
Remove duplicate update messages, and make sure engine always include method pointers in updates.
2022-12-20 00:15:41 +00:00
Hubert Plociniczak
49204e92cf
Simplify exception handling for polyglot exceptions (#3981)
This removes the special handling of polyglot exceptions and allows matching on Java exceptions in the same way as for any other types.

`Polyglot_Error`, `Panic.catch_java` and `Panic.catch_primitive` are gone

The change mostly deals with the backslash of removing `Polyglot_Error` and two `Panic` methods.
`Panic.catch` was implemented as a builtin instead of delegating to `Panic.catch_primitive` builtin that is now gone.

This fixes https://www.pivotaltracker.com/story/show/182844611
2022-12-19 19:16:43 +00:00
Jaroslav Tulach
4b4167fc06
Curried and lambda function invocation should be both fast (#3979)
Benchmark to compare _curried and lambda_ based function invocations and a fix to make _curried_ invocation (at least) as fast as the _lambda_ one. Allows us to use _curried invocations_ in standard library again without loosing any speed.

# Important Notes
Execute as:
```
sbt:runtime> benchOnly CurriedFunctionBenchmarks
```
Prior to subsequent bugfixes in this PR the benchmark results were:
- `averageCurried` runs in 0.290 ms
- `averageLambda` runs in 0.122 ms

e.g. _curried invocations_ is more than twice slow. That confirms our findings from the `Array_Proxy` vector benchmarks. The problem is that _function object is not compilation final_. After fixing it we have following results:
- `averageCurried` runs in 0.102 ms
- `averageLambda` runs in 0.111 ms

e.g. both operations are of similar complexity.
2022-12-16 07:12:24 +00:00
Dmitry Bushev
01f8e2976d
Fail runner when update manifest fails (#3990)
Otherwise the update manifest job gets stuck.
2022-12-15 16:12:09 +00:00
Radosław Waśko
b9bf958f2c
Efficient joining for Equals and Equals_Ignore_Case using a hashmap (#3978)
- Implemented https://www.pivotaltracker.com/story/show/183913276
- Refactored MultiValueIndex and MultiValueKeys to be more type-safe and more direct about using ordered or unordered maps.
- Added performance tests ensuring we use an efficient algorithm for the joins (the tests will fail for a full O(N*M) scan).
- Removed some duplicate code in the Table library.
- Added optional coloring of test results in terminal to make failures easier to spot.
2022-12-14 22:56:20 +00:00
James Dunkerley
77fe69dfd9
JSON Improvements, small Table stuff, Statistic in Enso not Java and few other minor bits. (#3964)
- Aligned `compare_to` so returns `Type_Error` if `that` is wrong type for `Text`, `Ordering` and `Duration`.
- Add `empty_object`, `empty_array`. `get_or_else`, `at`, `field_names` and `length` to `Json`.
- Fix `Json` serialisation of NaN and Infinity (to "null").
- Added `length`, `at` and `to_vector` to Pair (allowing it to be treated as a Vector).
- Added `running_fold` to the `Vector` and `Range`.
- Added `first` and `last` to the `Vector.Builder`.
- Allow `order_by` to take a single `Sort_Column` or have a mix of `Text` and `Sort_Column.Name` in a `Vector`.
- Allow `select_columns_helper` to take a `Text` value. Allows for a single field in group_by in cross_tab.
- Added `Patch` and `Custom` to HTTP_Method.
- Added running `Statistic` calculation and moved more of the logic from Java to Enso. Performance seems similar to pure Java version now.
2022-12-14 19:40:27 +00:00
Jaroslav Tulach
1dfcf1cafc
AvoidIdInstrumentationTagTest to control which nodes are instrumentable (#3977)
Fighting with _too many messages being delivered_ I wrote a test that dumps information about `AvoidIdInstrumentationTag` - every node that has `AvoidIdInstrumentationTag` is excluded from the instrumentation. However, when I look at the output for
```
from Standard.Base import all
import Standard.Visualization

run n = 0.up_to n . map i-> 1.noise * i
```
I see that `1.noise` didn't have the tag. Now there is [AvoidIdInstrumentationTagTest.java](https://github.com/enso-org/enso/pull/3973/files#diff-32cd9240bda2bfe0e5904695ced008daba86fefb3d137ac401997f4265fa50eb) which can be used to collect all programs where _too many messages is being delivered_. Just add a program, identify _isLambda_ and verify all nodes are properly tagged.
2022-12-14 04:00:38 +00:00
Jaroslav Tulach
4914f77bd0
Better handling of negative constants in case_of (#3976)
```
from Standard.Base import all

main =
value = 1

case value of
1 -> IO.println "one"
2 -> IO.println "two"
-1 -> IO.println "minus one"
_ -> IO.println "other"
```

# Important Notes
Had to write new `ExecCompilerTest` as comparing the AST with old compiler produced miserable results - the old AST wasn't ready for negative constants in `case of` at all.
2022-12-13 14:36:56 +00:00
Hubert Plociniczak
88effc41f8
Windows doesn't like resolving last commit with refs/heads/master (#3974)
Apparently
```
git --git-dir .enso/.vcs log refs/heads/master
fatal: ambiguous argument 'refs/heads/master': unknown revision or path not in the working tree.
```
but
```
git --git-dir .enso/.vcs log HEAD
fatal: ambiguous argument 'refs/heads/master': unknown revision or path not in the working tree.
```
works just fine on Windows.

Added some safeguards to avoid propagating weird errors because of retrieving element from an
empty Option.
2022-12-12 22:13:03 +00:00
Hubert Plociniczak
3d93fc24c0
Improvements to VCS setup (#3963)
The presence of `.git` in Enso's data directory (`.enso`), which should be under VCS as well, turned out to be really problematic. Git (and JGit) treat `.enso` as a submodule which prevents as from manual inspection of the repo. Additionally, every commit changed the submodule version, which in turn was reporting invalid/changed status always.

This change renames `.enso/.git` to `.enso/.vcs` which eliminates a lot of problems. Additionally, rather than doing `git add .` we selectively add files to the index, thus preventing most of the issues reported in the ticket.
This change also means that JGit's status needs to filter `.enso/.vcs` as those will be marked as untracked always.

# Important Notes
To manually test the status of the project's repo do
```
git --git-dir .enso/.vcs status
```
2022-12-11 23:15:15 +00:00
Jaroslav Tulach
ec047d6ac1
Speeding up Array_Proxy.from_proxy_object twice (#3969)
Using lambda instead of higher order function.

# Important Notes
Running:
```
sbt:runtime> benchOnly VectorBenchmarks.averageOverArrayProxy
```
speeds up from `0.038 ms/op` to `0.016 ms/op` on my computer. Which seems good enough.
2022-12-10 10:35:14 +00:00
Pavel Marek
b91ae2f5ef
ExpressionNodes are only wrapped in the presence of Chrome inspector (#3970)
Fixes bug in visualization of host polyglot values - `ExpressionNode` is only wrapped once Chrome inspector instrument is attached to the context. With this fix, when chromeinspector is attached (`enso --run --inspect ...`), all the host values are reinterpreted as text - the assumption is invalidated. But when running as language server, nothing is wrapped.
2022-12-10 09:49:55 +00:00
Jaroslav Tulach
308f4edc21
Tweaks to make benchmarking easier (#3961)
Enabling Frgaal for compilation of benchmarks. Using text blocks for Enso code. Making `--showCompilations` work on GraalVM 22.3 again.
2022-12-08 20:30:19 +00:00
Jaroslav Tulach
89b455fb6f
Empty block body shall result in a syntax error (#3960)
Represent a missing block body as syntax error.
2022-12-08 15:43:21 +00:00
Dmitry Bushev
43167c1617
Add executionContext/interrupt API command (#3952)
Implement the `executionContext/interrupt` API command that forcibly stops the program execution.
2022-12-08 00:04:46 +00:00
Pavel Marek
4641426ce9
Allow arbitrary expression evaluation in chromeinspector (#3941)
Allow arbitrary expression evaluation in the chromeinspector console. Moreover, allow modifications of any variable in any stack frame.

# Important Notes
- Implement inline parsing in `EnsoLanguage.parse(InlineParsingRequest)`.
- Debugging experience is affected by this [bug in Truffle](https://github.com/oracle/graal/issues/5513), which causes NPEs when a host object gets into chromeinspector. I tried to implement a workaround, but it does not work all the time. Nevertheless, it should not matter that much - if there is a NPE in the debugger, you can just ignore it, as it should be concealed in the debugger and should not be propagted outside. See comments in the `docs/debugger`.
2022-12-07 23:02:42 +00:00
Hubert Plociniczak
0855b74875
Vector should preserve warnings (#3938)
* Sequence literal (Vector) should preserve warnings

When Vector was created via a sequence literal, we simply dropped any
associated any warnings associated with it.
This change propagates Warnings during the creation of the Vector.
Ideally, it would be sufficient to propagate warnings from the
individual elements to the underlying storage but doesn't go well with
`Vector.fromArray`.

* update changelog

* Array-like structures preserver warnings

Added a WarningsLibrary that exposes `hasWarnings` and `getWarnings`
messages. That way we can have a single storage that defines how to
extract warnings from an Array and the others just delegate to it.

This simplifies logic added to sequence literals to handle warnings.

* Ensure polyglot method calls are warning-free

Since warnings are no longer automatically extracted from Array-like
structures, we delay the operation until an actual polyglot method call
is performed.

Discovered a bug in `Warning.detach_selected_warnings` which was missing
any usage or tests.

* nits

* Support multi-dimensional Vectors with warnings

* Propagate warnings from case branches

* nit

* Propagate all vector warnings when reading element

Previously, accessing an element of an Array-like structure would only
return warnings of that element or of the structure itself.
Now, accessing an element also returns warnings from all its elements as
well.
2022-12-07 11:10:11 +01:00
James Dunkerley
4cbd72a4eb
Some more tidying based on remaining tickets and PR comments. (#3946)
- Moved `to_default_visualization_data` to `Standard.Visualization`.
- Remove the use of `is_a` in favour of case statements.
- Stop exporting Standard.Base.Error.Common.
- Separate errors to own files.
- Change constructors to be called `Error`.
- Rename `Caught_Panic.Caught_Panic_Data` -> `Caught_Panic.Panic`.
- Rename `Project_Description.Project_Description_Data` ->`Project_Description.Value`
- Rename `Regex_Matcher.Regex_Matcher_Data` -> `Regex_Matcher.Value` (can't come up with anything better!).
- Rename `Range.Value` -> `Range.Between`.
- Rename `Interval.Value` -> `Interval.Between`.
- Rename `Column.Column_Data` -> `Column.Value`.
- Rename `Table.Table_Data` -> `Table.Value`.
- Align all the Error types in Table.
- Removed GEO Json bits from Table.
- `Json.to_table` doesn't have the GEO bits anymore.
- Added `Json.geo_json_to_table` to add the functions back in.

# Important Notes
No more exports from anywhere but Main!
No more `_Data` constructors!
2022-12-06 18:35:18 +00:00
Dmitry Bushev
768747a55e
Update suggestion argument doc (#3948)
Updated the doc to match the implemented API.
2022-12-05 13:29:18 +00:00
Jaroslav Tulach
a24dfe6adb
Recognize NamedApp in @Tail_Call (#3944)
Test and fix for `@Tail_Call` followed by `Tree.NamedApp`
2022-12-05 09:33:27 +00:00
James Dunkerley
0ad70c6332
Tidy Standard.Base part 5 of n ... (hopefully the end...) (#3929)
- Moved `Any`, `Error` and `Panic` to `Standard.Base`.
- Separated `Json` and `Range` extensions into own modules.
- Tidied `Case`, `Case_Sensitivity`, `Encoding`, `Matching`, `Regex_Matcher`, `Span`, `Text_Matcher`, `Text_Ordering` and `Text_Sub_Range` in `Standard.Base.Data.Text`.
- Tidied `Standard.Base.Data.Text.Extensions` and stopped it re-exporting anything.
- Tidied `Regex_Mode`. Renamed `Option` to `Regex_Option` and added type to export.
- Tidied up `Regex` space.
- Tidied up `Meta` space.
- Remove `Matching` from export.
- Moved `Standard.Base.Data.Boolean` to `Standard.Base.Boolean`.

# Important Notes
- Moved `to_json` and `to_default_visualization_data` from base types to extension methods.
2022-12-02 18:08:14 +00:00
Michał Wawrzyniec Urbańczyk
4de8e44871
Release process (#3909) 2022-12-02 02:56:22 +01:00
Jaroslav Tulach
099f045178
Integer.parse and Decimal.parse improvements (#3934)
Converting `Integer.parse` into a builtin and making sure it can parse big values like `100!`. Adding `locale` parameter to `Locale.parse` and making sure it parses `32,5` as `32.5` double in Czech locale.
2022-12-01 11:25:28 +00:00
Hubert Plociniczak
06bd69436b
Import modules' extension methods only with unqualified import statements (#3906)
# Important Notes
Note that one cannot
```
import Standard.Table as Table_Module
```
because of the 2-component name restriction that gets desugared to `Standard.Table.Main` and we have to write
```
import Standard.Table.Main as Table_Module
```
in a few places. Once we move `Json.to_table` extension this can be improved.
2022-12-01 10:13:34 +00:00
Jaroslav Tulach
030dbe4973
Disabling musl as it isn't capable to load dynamic library (#3917)
Disabling `musl` as it isn't capable to load dynamic library.

# Important Notes
With this change it is possible to:
```
$ sbt  bootstrap
$ sbt  engine-runner/buildNativeImage
$ ./runner --run ./engine/runner/src/test/resources/Factorial.enso 3
6
$ ./runner --run ./engine/runner/src/test/resources/Factorial.enso 4
24
$ ./runner --run ./engine/runner/src/test/resources/Factorial.enso 100
93326215443944152681699238856266700490715968264381621468592963895217599993229915608941463976156518286253697920827223758251185210916864000000000000000000000000
```

Is it OK, @radeusgd  to disable `musl`? If not, we would have to find a way to link the parser in statically, not dynamically.
2022-12-01 06:43:13 +00:00
Hubert Plociniczak
20c22f2422
from/all import must not include module in name resolution (#3931)
It appears that when were doing
`from XYZ import all`
the module `XYZ` was also being taken into account during name resolution.
This was unfortunate and became problematic when one had a type with the same name defined in it.
During pattern matching one could not simply do
```
from XYZ import all
...
case ... of
_ : XYZ -> ...
```
since the compiler would complain that we try to pattern match on a type but give it a module.

The module is now excluded from the name resolution, when importing everything from the module.
It appears that this "feature" was used in a number of our tests, so they had to be adapted.
This fixes task 4 in https://www.pivotaltracker.com/story/show/183833055
2022-11-30 16:28:57 +00:00
Pavel Marek
f5f5bff9a7
Improve undefined method error message on builtin types (#3907)
Improve undefined method error message for builtin types

### Important Notes
- Rename `org.enso.interpreter.runtime.Context` to `org.enso.interpreter.runtime.EnsoContext`.
- Rename `org.enso.interpreter.Language` to `or.enso.interpreter.EnsoLanguage`.
2022-11-30 13:37:17 +01:00
Hubert Plociniczak
dcba966754
Use a custom directory for VCS repo metadata (#3921)
Rather than hard-coding `.git` in the root of the project, VCS should save data into Enso's data directory (i.e. `.enso`).
This change reshuffles initialization and configuration to store Git VCS metadata by default at `.enso/.git`.
This is follow up to https://github.com/enso-org/enso/pull/3851

# Important Notes
Apparently a custom Git directory in JGit means that it always creates a `.git` **file** with `gitdir` pointing to the custom location.
This is not necessary in our case since all our commands provide that explicitly.
That is why `init` operation removes `.git` file, which may seem a bit counter-intuitive.
2022-11-29 14:40:36 +00:00
Jaroslav Tulach
a544618fbd
Better error recovery when body is null (#3925)
Parsing `foo : Integer ->` without throwing a `NullPointerException`.
2022-11-29 09:55:16 +00:00
Hubert Plociniczak
c87d1d6767
Don't export polyglot symbols (#3915)
By default all polyglot symbols that have been imported were always exported. This means that importing a module that had some polyglot imports brought them into the scope automatically. This didn't follow our desired semantics.

Fixes task 3 in https://www.pivotaltracker.com/story/show/183833055.
2022-11-28 19:11:31 +00:00
Kaz Wesley
336bbf505c
Parser: Newline normalization in text literals (#3903)
- Newlines in text literals are now normalized to `\n` when producing IR representation.
- Re-enabled tests that were dependent on the old behavior.
2022-11-27 09:40:44 +00:00
James Dunkerley
4e30b3036d
Tidy Standard.Base part 4 of n ... (#3898)
- Export all for `Problem_Behavior` (allowing for Report_Warning, Report_Error and Ignore to be trivially used).
- Renamed `Range.Range_Data` to `Range.Value` moved to using `up_to` wherever possible.
- Reviewed `Function`, `IO`, `Polyglot`, `Random`, `Runtime`, `System`.
- `File` now published as type. Some static methods moved to `Data` others into type. Removed `read_bytes` static.
- New `Data` module for reading input data in one place (e.g. `Data.read_file`) will add `Data.connect` later.
- Added `Random` module to the exports.
- Move static methods into `Warning` type and exporting the type not the module.

# Important Notes
- Sorted a few imports into order (ordering by direct import in project, then by from import in project then polyglot and finally self imports).
2022-11-25 02:00:16 +00:00
Jaroslav Tulach
f225a962ce
Allow conversion of EnsoBigInteger to double (#3865)
Make sure any Enso number (including `EnsoBigInteger`) can be passed to Java via Truffle interop and used on the Java side.
2022-11-24 10:00:16 +00:00
Jaroslav Tulach
402ebb2f8e
Upgrade to GraalVM 22.3.0 (#3663)
Upgrading to GraalVM 22.3.0.

# Important Notes
- Removed all deprecated `FrameSlot`, and replaced them with frame indexes - integers.
- Add more information to `AliasAnalysis` so that it also gathers these indexes.
- Add quick build mode option to `native-image` as default for non-release builds
- `graaljs` and `native-image` should now be downloaded via `gu` automatically, as dependencies.
- Remove `engine-runner-native` project - native image is now build straight from `engine-runner`.
- We used to have `engine-runner-native` without `sqldf` in classpath as a workaround for an internal native image bug.
- Fixed chrome inspector integration, such that it shows values of local variables both for current stack frame and caller stack frames.
- There are still many issues with the debugging in general, for example, when there is a polyglot value among local variables, a `NullPointerException` is thrown and no values are displayed.
- Removed some deprecated `native-image` options
- Remove some deprecated Truffle API method calls.
2022-11-23 14:30:48 +00:00
Hubert Plociniczak
deb670785c
Allow multiple exports of the same module (#3897)
Previously, when exporting the same module multiple times only the first statement would count and the rest would be discarded by the compiler.

This change allows for multiple exports of the same module e.g.,
```
export project.F1
from project.F1 export foo
```
Multiple exports may however lead to conflicts when combined with hiding names. Added logic in `ImportResolver` to detect such scenarios.

This fixes https://www.pivotaltracker.com/n/projects/2539304/stories/183092447

# Important Notes
Added a bunch of scenarios to simulate pos and neg results.
2022-11-23 11:40:59 +00:00
Dmitry Bushev
5a0aad16eb
Check that type names are resolved (#3895)
RuntimeStdlibTest now checks that names in type signatures are qualified (aka poor man's typechecker)

Will be merged after the stdlib tidying by James is complete.
2022-11-21 22:33:36 +00:00
Jaroslav Tulach
79329ef00f
Treat null Tree.OprApp.getRhs() as UnexpectedExpression syntax error (#3882)
Fixes `NullPointerException` in `TreeToIr` by detecting the case and raising `UnexpectedExpression` syntax error.

# Important Notes
This PR prevents the `NullPointerException`. It doesn't change the meaning of `x.` (followed by space) `length`. It just detects when `Tree.OprApp.getRhs() == null` and treats that as an error.
2022-11-21 08:48:21 +00:00
Dmitry Bushev
3d6f2b04f3
VcsManagerTest should compare UTC times (#3892)
`LocalDate.now()` returns time in system time zone, and git uses UTC. Sometimes it can cause test to fail

```
INFO ide_ci::program::command: sbtℹ️ [info] - must create a commit with a timestamp *** FAILED ***
INFO ide_ci::program::command: sbtℹ️ [info]   "2022-11-18T23:05:01.768499800Z" did not start with substring "2022-11-19" (VcsManagerTest.scala:201)
```
2022-11-19 09:43:26 +00:00
James Dunkerley
99bacc5c06
Tidy Standard.Base Part 2 of n... (#3889)
- Moved static methods into `Locale` type. Publishing type not module.
- Stop publishing `Nil` and `Cons` from `List`.
- Tidied up `Json` and merged static in to type. Sorted out various type signatures which used a `Constructor`. Now exporting type and extensions.
- Tidied up `Noise` and merge `Generator` into file. Export type not module.
- Moved static method of `Map` into type. Publishing type not module.

# Important Notes
- Move `Text.compare_to` into `Text`.
- Move `Text.to_json` into `Json`.
2022-11-19 08:01:45 +00:00
Kaz Wesley
8f3bfe8ce2
Parser: Fix @Ignored tests (#3879)
- 3 ignored cases were old-parser bugs (one of which was quite interesting)
- 1 was a new-lexer bug, fixed here

See: https://www.pivotaltracker.com/story/show/183797809
2022-11-18 20:58:41 +00:00
James Dunkerley
14dbe7287b
Tidy Standard.Base Part 1 of n... (#3884)
* Tidy Bound and Interval.

* Fix Interval tests.

* Fix Interval tests.

* Restructure Index_Sub_Range to new Type/Statics.

* Adjust for Vector exported as a type and static methods on it.

* Tidy Maybe.

* Fix issue with Line_Ending_Style.

* Revert Filter_Condition change.
Fix benchmark test issue.
Tidy imports on Index_Sub_Range.

* Revert Filter_Condition change.
Fix benchmark test issue.
Tidy imports on Index_Sub_Range.

* Can't export constructors unless exported from type in module.

* Fix failing tests.
2022-11-18 08:57:41 +00:00
Dmitry Bushev
52a8c72303
Message handler supervisor (#3881)
Changelog:
- Fix a potential `null` value in diagnostic messages
- Add a supervising strategy to the message handler
2022-11-17 19:02:04 +00:00
Dmitry Bushev
d54faab903
Git should ignore user config (#3883)
Ignore gitconfig located in the user's home directory when creating Git instance.

For example, I setup git to automatically sign commits. And all git related tests are failing because of that:

```
[info] - should return last X commits *** FAILED ***
[info]   org.eclipse.jgit.api.errors.ServiceUnavailableException: Signing service is not available
[info]   at org.eclipse.jgit.api.CommitCommand.sign(CommitCommand.java:328)
[info]   at org.eclipse.jgit.api.CommitCommand.call(CommitCommand.java:283)
[info]   at org.enso.languageserver.vcsmanager.GitSpec$InitialRepoSetup.setup(GitSpec.scala:360)
[info]   at org.enso.languageserver.vcsmanager.GitSpec$InitialRepoSetup.setup$(GitSpec.scala:348)
[info]   at org.enso.languageserver.vcsmanager.GitSpec$$anon$16.setup(GitSpec.scala:275)
[info]   at org.enso.languageserver.vcsmanager.GitSpec$InitialRepoSetup.$init$(GitSpec.scala:346)
[info]   at org.enso.languageserver.vcsmanager.GitSpec$$anon$16.<init>(GitSpec.scala:275)
[info]   at org.enso.languageserver.vcsmanager.GitSpec.$anonfun$new$35(GitSpec.scala:275)
[info]   at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
[info]   ...
```
2022-11-17 09:07:51 +00:00
James Dunkerley
c868ed5efe
Some minor fixes (#3874)
- Allow `Map` to store a `Nothing` key (fixes `Vector.distinct` with a `Nothing`).
- Add `column_names` method to `Table` as a shorthand.
- Return data flow error when comparing with Nothing (not a Panic or a Polyglot exception).
- Allow milli and micro second for DateTime and Time Of Day

# Important Notes
- Added a load of tests for the various comparison operators to Numbers_Spec.
2022-11-17 07:11:18 +00:00
Hubert Plociniczak
7b0759f8b3
Don't add module's builtins to the scope of a builtin type (#3791)
It appears that we were always adding builtin methods to the scope of the module and the builtin type that shared the same name.
This resulted in some methods being accidentally available even though they shouldn't.

This change treats differently builtins of types and modules and introduces auto-registration feature for builtins.
By default all builtin methods are registered with a type, unless explicitly defined in the annotation property.
Builtin methods that are auto-registered do not have to be explicitly defined and are registered with the underlying type.
Registration correctly infers the right type, depending whether we deal with static or instance methods.

Builtin methods that are not auto-registered have to be explicitly defined **always**. Modules' builtin methods are the prime example.

# Important Notes
Builtins now carry information whether they are static or not (inferred from the lack of `self` parameter).
They also carry a `autoRegister` property to determine if a builtin method should be automatically registered with the type.
2022-11-16 10:23:52 +00:00
Jaroslav Tulach
cd90a271d4
Ignore block with just spaces (#3871)
Two fixes in `EnsoCompilerTest` to align new parser with the previous behavior.

# Important Notes
Removes two `@Ignore` annotations. One test was already working. Another one is fixed by Kaz's idea to ignore _"empty block"_ application.
2022-11-15 05:52:23 +00:00
Kaz Wesley
a1db36b57c
Support mixed constructors/bindings in types (#3870)
Libraries: Revert changes that were necessitated by a new rule we have decided not to introduce.

Parser:
- Support mixed constructors/bindings in types.
- Disallow zero-length hex sequences in character escapes: `\x`, `\u`, `\u{}`, `\U`, `\U{}` are no longer legal synonyms for `\0` (matches old parser behavior).
2022-11-14 20:24:07 +00:00
Hubert Plociniczak
85d4337f26
Project save backed by git (#3851)
This change adds support for Version Controlled projects in language server.
Version Control supports operations:
- `init` - initialize VCS for a project
- `save` - commit all changes to the project in VCS
- `restore` - ability to restore project to some past `save`
- `status` - show the status of the project from VCS' perspective
- `list` - show a list of requested saves

# Important Notes
Behind the scenes, Enso's VCS uses git (or rather [jGit](https://www.eclipse.org/jgit/)) but nothing stops us from using a different implementation as long as it conforms to the establish API.
2022-11-14 17:32:39 +00:00
Jaroslav Tulach
ecd1fdc3f8
Caching the grapheme_length of a Text (#3864)
Computing length of a text takes time. Let's cache it after first computation.

# Important Notes
Wrote `StringBenchmarks` that sums lengths of (the same) `Text` present many time in a `Vector`. Initially it took `383.673 ms` per operation. Then it took `0.031 ms/op`. Looks like the `length` calls are returning instantly as they get cached.
2022-11-14 15:53:10 +00:00
Dmitry Bushev
8c148fd4c0
Runtime reports correct atom type (#3867)
Fix the Atom type reported by the runtime. I.e. the type of `Some 42` expression is now `Maybe`, and not `Some`.
2022-11-14 11:49:04 +00:00
Jaroslav Tulach
7f2d02aa7c
Use Rust Parser from Java instead of AST.scala (#3611) 2022-11-13 06:22:02 +00:00
Jaroslav Tulach
5ce173316b
More improvements that work with both parsers (#3868) 2022-11-12 02:34:14 +01:00
Jaroslav Tulach
20686dde4f
Construct IR.Syntax.Error rather than throwing an exception (#3861)
This PR mimics test cases from #3860 and makes sure `IR.Syntax.Error` is constructed at appropriate places rather than just yielding an `UnhandledEntity` exception.

# Important Notes
Merge before #3611 to minimize disruption when changing the parser.
2022-11-10 09:06:27 +00:00
Jaroslav Tulach
c2633bc137
Metadata, in context and imports (#3856)
Another set of improvements extracted from #3611. This time it includes a fix to the Rust part of the parser.

# Important Notes
After digging into metadata parsing I realized the positions used to query the BTree data structure are wrong. This PR tries to address that by re-arranging the order of serialized fields and passing `startCode` and `endCode` locations in.

Originally I though I need changes on the Rust side to support `in` operator. Turned out I can do that just with changes on the Java side.

Qualified names in imports were missing UUIDs. Fixed now.
2022-11-07 19:05:19 +00:00
Dmitry Bushev
14012a751f
Add parent type field to suggestion (#3846)
PR adds `parentType` field to the `Type` suggestion. All Enso types have parent type except `Any`.
2022-11-07 13:21:04 +00:00
Jaroslav Tulach
1273d8ac91
Updating the code to be more new parser friendly (#3853)
To minimize differences between #3611 and `develop` branch I propose to make following code changes that seem to work fine with the old as well as new parser. In addition to that there are new tests comparing the two parsers.

# Important Notes
Old parser is still used everywhere except `EnsoCompilerTest` where the test compares `IR`s constructed by both parsers.
2022-11-04 18:08:24 +00:00
Jaroslav Tulach
56a4b8815b
Relaxed CodeLocationsTest and co. (#3849)
Another set of fixes extracted from #3611. `CodeLocationsTest` test has been made more flexible to allow _"off by one"_ differences in the offset locations between the old and new parser.
2022-11-04 06:04:56 +00:00
Marcin Kostrzewa
1c9e57099c
Introduce the Self keyword (#3844)
Introduces the `Self` keyword, aliasing the current type.
2022-11-02 13:39:33 +00:00
Dmitry Bushev
a6ce49e8a5
Split Atom suggestion entry to Type and Constructor (#3835)
Changelog:
- update: split `Atom` suggestion to `Type` and `Constructor`
- update: gui API
- update: JSONRPC doc
2022-11-02 09:53:40 +00:00
Kaz Wesley
330612119a
Parse the standard library (#3830)
Fix bugs in `TreeToIr` (rewrite) and parser. Implement more undocumented features in parser. Emulate some old parser bugs and quirks for compatibility.

Changes in libs:
- Fix some bugs.
- Clean up some odd syntaxes that the old parser translates idiosyncratically.
- Constructors are now required to precede methods.

# Important Notes
Out of 221 files:
- 215 match the old parser
- 6 contain complex types the old parser is known not to handle correctly

So, compared to the old parser, the new parser parses 103% of files correctly.
2022-10-31 16:19:12 +00:00
Pavel Marek
f8a4e2a9d2
Add Period type (#3818)
This PR adds `Period` type, which is a date-only complement to `Duration` builtin type.

# Important Notes
- `Period` replaces `Date_Period`, and `Time_Period`.
- Added shorthand constructors for `Duration` and `Period`. For example: `Period.days 10` instead of `Period.new days=10`.
- `Period` can be compared to other `Period` in some cases, other cases throw an error.
2022-10-28 17:27:20 +00:00
Pavel Marek
28243a0fd1
Define Enso epoch start as 15th October 1582. (#3804)
Define start of Enso epoch as 15th of October 1582 - start of the Gregorian calendar.

# Important Notes
- Some (Gregorian) calendar related functionalities within `Date` and `Date_Time` now produces a warning  if the receiving Date/Date_Time is before the epoch start, e.g., `week_of_year`, `is_leap_year`, etc.
2022-10-27 10:16:43 +00:00
Marcin Kostrzewa
901760816c
State rework & IO Contexts (#3828)
1. Changes how we do monadic state – rather than a haskelly solution, we now have an implicit env with mutable data inside. It's better for the JVM. It also opens the possibility to have state ratained on exceptions (previously not possible) – both can now be implemented.
2. Introduces permission check system for IO actions.
2022-10-26 16:22:08 +00:00
Radosław Waśko
bb29833da5
Create a Table Row Type and expose as a Vector on In-Memory Table with .rows property (#3827)
Implements https://www.pivotaltracker.com/story/show/182307026
2022-10-26 11:21:33 +00:00
Dmitry Bushev
46441ca7a8
Add isStatic method field (#3829)
PR adds `isStatic` field to suggestion. The field is required for Component Browser.
2022-10-26 09:12:45 +00:00
Hubert Plociniczak
55b9bea352
Initialize Builtins at Native Image build time (#3821)
Moved loading of Builtin Types and Methods to a static initializer. That way the information is available at Native Image build time and one does not have to update a corresponding entry in `reflect-config` which would be a real pain when we start using it in anger.
Fixes https://www.pivotaltracker.com/story/show/183374932
2022-10-25 18:28:10 +00:00
Hubert Plociniczak
f6379fb1f7
Accept Array-like objects seamlessly in builtins (#3817)
Most of the time, rather than defining the type of the parameter of the builtin, we want to accept every Array-like object i.e. Vector, Array, polyglot Array etc.
Rather than writing all possible combinations, and likely causing bugs on the way anyway as we already saw, one should use `CoerceArrayNode` to convert to Java's `Object[]`.
Added various test cases to illustrate the problem.
2022-10-25 12:44:48 +00:00
Jaroslav Tulach
d8882f606d
Few more properly parsed files (#3826)
Another part of #3611 with few more `TreeToIr` improvements.

# Important Notes
Unofficial `LoadParser.sh` check from #3611 of all library files now reports just 54 failures out of 222 files - e.g. 75% success rate.
2022-10-24 08:53:37 +00:00
Jaroslav Tulach
42d0e40db7
Order of types in method signatures and more robust comment handling (#3825)
Another part of #3611 to integrate into `develop` branch.
2022-10-22 04:19:13 +00:00
Jaroslav Tulach
7bcd76ac5c
Test to make sure number of files accepted by the new parser only grows (#3814)
A new test to verify that some `Standard.Base` library files can be parsed by the new parser is added. Use it as:
```bash
$ sbt bootstrap
$ sbt buildEngineDistribution
$ sbt "runtime/testOnly *ParseStdLibTest"
```
By having this test instantly available we can block integration of PRs that would decrease the number of parseable files or introduce new files that aren't paseable.

# Important Notes
The test contains a _black list_ of classes that are currently known to fail. Over the time the _black list_ shall get smaller and smaller.
2022-10-21 15:09:32 +00:00
Hubert Plociniczak
6c440beecc
Move logic calculating the index in Vector.at to a builtin method to make the performance of Vector to be on par with Array (#3811)
The main culprit of a Vector slowdown (when compared to Array) was the normalization of the index when accessing the elements. Turns out that the Graal was very persistent on **not** inlining that particular fragment and that was degrading the results in benchmarks.

Being unable to force it to do it (looks like a combination of thunk execution and another layer of indirection) we resorted to just moving the normalization to the builtin method. That makes Array and Vector perform roughly the same.

Moved all handling of invalid index into the builtin as well, simplifying the Enso implementation. This also meant that `Vector.unsafe_at` is now obsolete.
Additionally, added support for negative indices in Array, to behave in the same way as for Vector.

# Important Notes
Note that this workaround only addresses this particular perf issue. I'm pretty sure we will have more of such scenarios.
Before the change `averageOverVector` benchmark averaged around `0.033 ms/op` now it does consistently `0.016 ms/op`, similarly to `averageOverArray`.
2022-10-20 12:50:44 +00:00
Jaroslav Tulach
21fe0b8865
Process documentation in type definitions (#3806)
Another part of #3611 ready for integration into `develop` branch.

# Important Notes
Test `org.enso.compiler.EnsoCompilerTest.testTestGroup` is ignored as it has problems with source offsets - identifiers don't have the appropriate names due to `Tree.codeRepr()` being _off_.
2022-10-19 09:19:42 +00:00
Kaz Wesley
feb8eb4f83
fix span bug in doc comments (#3808) 2022-10-18 20:37:36 +00:00
Pavel Marek
a53fbc79be
Improve Unsupported_Argument_Types message. (#3803)
Improve `Unsupported_Argument_Types` error so that it includes the message from the original exception. `arguments` field is retained, but not included in `to_display_text` method.
2022-10-18 12:03:25 +00:00
Kaz Wesley
28daf14f75
Parse case-by-type, add an old-lambda syntax rule (#3802)
- Special precedence rules for case-of so that `:` operator works without parens or nospace-grouping.
- Support an old-lambda syntax: `x->x-> x`. According to the usual rules, the first nospace group would be parsed as an operator section. The expression now parses as a lambda that contains a lambda.
- Match old parser treatment of # in doc comments.
- Tweak precedence so (a : B = c) works.
- Documented constructors.
2022-10-17 22:46:52 +00:00
Pavel Marek
47148a2ff1
Missing foreign language generates proper Enso error. (#3798)
Trying to invoke a foreign method with non-installed language (either not enabled in the Truffle `Context`, or not installed in the GraalVM distribution) results in `Polyglot_Error`, rather than crashing the entire engine.
2022-10-17 09:59:31 +00:00
Kaz Wesley
2740406f93
Lex doc comments and attach text to AST (#3795)
- New `Documented` node attaches documentation, lexed as a raw text literal, to a statement.
- Handle a case of lambdas with body blocks.
2022-10-15 06:13:32 +00:00
Pavel Marek
e9260227c4
Duration type is a builtin type (#3759)
- Reimplement the `Duration` type to a built-in type.
- `Duration` is an interop type.
- Allow Enso method dispatch on `Duration` interop coming from different languages.

# Important Notes
- The older `Duration` type should now be split into new `Duration` builtin type and a `Period` type.
- This PR does not implement `Period` type, so all the `Period`-related functionality is currently not working, e.g., `Date - Period`.
- This PR removes `Integer.milliseconds`, `Integer.seconds`, ..., `Integer.years` extension methods.
2022-10-14 18:08:08 +00:00
Jaroslav Tulach
5873af88c5
More coding patterns converted to IR (#3796)
Few more coding patterns found in stdlib converted from `Tree` to `IR`.
2022-10-14 14:14:54 +00:00
Hubert Plociniczak
14ed2812ee
Don't crash interpreter on missing method (#3787)
When trying to resolve an invalid method of a polyglot array we were reaching a state where no specialization applied.
Turns out we can now simplify the logic of inferring polyglot call type for arrays and avoid the crash.
2022-10-14 13:17:13 +02:00
Kaz Wesley
0e412044f6
Macro contexts (#3792)
- Implement macro-contexts-lite (`from` is now only a keyword at the beginning of a line)
- Support special nospace-group handling for old lambdas (so expressions like this work: `x-> y-> x + y`)
- Fix a text-escape incompatibility

# Important Notes
- There is now an `OperatorFunction`, which is like a `Function` but has an operator for a name, and likewise an `OperatorTypeSignature`.
2022-10-13 22:47:02 +00:00
Kaz Wesley
5668cbcc24
Qualified defs (#3785)
Allow qualified names in LHS of type signatures and method definitions.
2022-10-12 17:40:16 +00:00
Jaroslav Tulach
102dd9a790
Supporting self attribute, type methods and operations (#3789)
Supporting self attribute, type methods and operations in the Rust parser to `IR` conversion. Another part extracted out of #3611.
2022-10-12 09:50:32 +00:00
Jaroslav Tulach
5386a8be07
Convert annotations provided by the new parser to IR (#3784)
Builds on top of #3780 and converts the new `Tree.Annotated` to IR.
[ci no changelog needed]
2022-10-10 20:37:39 +00:00
Marcin Kostrzewa
b3dd778eed
Static, but instance, but static (#3764)
Adds the ability to write `Foo.method (Mk_Foo 123)` as a synonym of `(Mk_Foo 123).method` because Rust.
2022-10-10 19:28:33 +00:00
Hubert Plociniczak
7e0ab8908c
Fix for perf degradation in method calls on polyglot arrays (#3781) 2022-10-10 16:35:38 +00:00
Jaroslav Tulach
ea60cd5fab
Supporting more of the Enso language syntax in the newly built IR (#3777)
Supporting more of the Enso language syntax in the IR built by the new Rust parser.
2022-10-07 13:32:44 +00:00
Kaz Wesley
5d46cb184d
Old translateArgumentDefinitions is not necessary for Tree.ArgumentDefinition (#3773) 2022-10-07 05:22:31 +00:00
Hubert Plociniczak
55f9e96a78
Fix repl (#3767)
Can't instantiate the associated type:
```
Exception in thread "main" java.lang.UnsupportedOperationException: Unsupported operation Value.newInstance(Object...) for 'Internal_Repl_Module___'(language: Java, type: com.oracle.truffle.polyglot.PolyglotMap). You can ensure that the operation is supported using Value.canInstantiate().
at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotEngineException.unsupported(PolyglotEngineException.java:137)
at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatch.unsupported(PolyglotValueDispatch.java:1257)
at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatch.newInstanceUnsupported(PolyglotValueDispatch.java:613)
at org.graalvm.truffle/com.oracle.truffle.polyglot.PolyglotValueDispatch$InteropValue$NewInstanceNode.doCached(PolyglotValueDispatch.java:4382)
...
```

# Important Notes
I thought we had a CI check that would prevent us from introducing such regressions?
EDIT: we don't
2022-10-06 19:53:54 +00:00
Hubert Plociniczak
486874216b
Always report errors for all modules (#3765)
`.find(...)` wasn't equivalent to `.map(...).exists(identity)` in this case because of side-effects related to diagnostics reporting. That is why we were reporting errors only for the first erroneous module.

/cc @jdunkerley
2022-10-06 14:11:32 +00:00
Jaroslav Tulach
4485a6c804
Supporting more complicated case_of and other patterns (#3771)
Bigger alignment between Rust based and old parser.
2022-10-06 10:11:44 +00:00
Jaroslav Tulach
792ef2520a
Associate the argument root nodes with their function name (#3763)
I found it hard to see all those `<argument-0>` or `<argument-1>` nodes in the IGV. Associating them with their _scope_.

[ci no changelog needed]
2022-10-05 17:19:42 +00:00
Hubert Plociniczak
0f4f4fc4d1
Fix type pattern matching benchmark (#3757)
c&p failure, so it was generating a very small Vector. Surprisingly, it indicates similar problems with `@ExplodeLoop` in `CatchTypeBranchNode`.

Kudos to @JaroslavTulach for reporting it.
2022-10-05 13:24:42 +00:00
Kaz Wesley
44a031f9f0
Parser: Full constructor syntax for type definitions; Field syntax; Complex operator sections; Template functions; Text improvements; Operator methods; eliminate Unsupported; better ArgumentDefinitions (#3716)
I believe all parse failures remaining after these changes are because the new parser is intentionally stricter about some things. I'll be reviewing those failures and opening a bug to change the library/tests code.

Implements:
- https://www.pivotaltracker.com/story/show/182941610: full type def syntax
- https://www.pivotaltracker.com/story/show/182497490: field syntax
- https://www.pivotaltracker.com/story/show/182497395: complex operator sections
- https://www.pivotaltracker.com/story/show/182497236: template functions
- `codeRepr` without leading whitespace
- text literals: interpret escape sequences in lexer
- the multiline text-literal left-trim algorithm
- type operator-methods
- the `<=` operator is no longer treated as a modifier
- https://www.pivotaltracker.com/story/show/183315038: eliminate Unsupported
- use ArgumentDefinition for type constructor arguments
- more detailed ArgumentDefinition type
2022-10-05 04:45:31 +00:00
Hubert Plociniczak
c460b7c4a4
Fix for native image generation (#3755)
Missing `@TruffleBoundary` annotations and entries in the configs were preventing us from generating native image. Again.

# Important Notes
@mwu-tow This check is easy to miss so it would be good to have it in CI.
2022-10-04 19:03:45 +00:00
Dmitry Bushev
f6b5438e9e
Fix Warnings visualization (#3754)
Changelog
- fix reporting of runtime type for values annotated with warning
- fix visualizations of values annotated with warnings
- fix `Runtime.get_stack_trace` failure in interactive mode
2022-10-04 17:27:13 +00:00
Hubert Plociniczak
ae66087bb9
By-type pattern matching (#3742)
This change brings by-type pattern matching to Enso.
One can pattern match on Enso types as well as on polyglot types.

For example,
```
case x of
_ : Integer -> ...
_ : Text -> ...
_ -> ...
```
as well as Java's types
```
case y of
_ : ArrayList -> ...
_ : List -> ...
_ : AbstractList -> ...
_ -> ..
```

It is no longer possible to match a value with a corresponding type constructor.
For example
```
case Date.now of
Date -> ...
```
will no longer match and one should match on the type (`_ : Date`) instead.
```
case Date of
Date -> ...
```
is fine though, as requested in the ticket.

The change required further changes to `type_of` logic which wasn't dealing well with polyglot values.

Implements https://www.pivotaltracker.com/story/show/183188846

# Important Notes
~I discovered late in the game that nested patterns involving type patterns, such as `Const (f : Foo) tail -> ...` are not possible due to the old parser logic.
I would prefer to add it in a separate PR because this one is already getting quite large.~ This is now supported!
2022-10-04 09:23:58 +00:00
Dmitry Bushev
11acad5cff
Unique job for visualizations (#3752)
PR adds special kind of jobs for visualizations. It should
- prevent cancelling visualization jobs when the program is re-executed (resulting in the fact that visualization is not showing up)
- omit unnecessary executions of visualization jobs (the case when the user goes through menu items in the component browser)

I skipped the tests because testing of the last scenario involves indeterminism. I.e. when preparing the test, we can't control which of submitted visualization jobs will be actually executed, and which will be cancelled.
2022-10-03 08:26:54 +00:00
Jaroslav Tulach
9b40bd5ab8
Parse text literal, lambda and multi-parameter function (#3747)
Construct `IR` for text literal, lambda and multi-parameter function using the new Rust parser.
2022-09-30 10:56:24 +00:00
Jaroslav Tulach
835ac05218
Engine should send notification about node status (#3729)
When nodes get invalidated in the cache, they have to be recomputed. Let the IDE know which of the nodes are pending by sending `Api.ExpressionUpdate.Payload.Pending` message.

# Important Notes
This PR introduces new `Api.ExpressionUpdate.Payload.Pending` message. This message is delivered before re-computation of nodes. Later `Api.ExpressionUpdate.Payload.Value` or other is sent to notify the IDE that a value for given node is available.

Trivial implementation of of the `Api.ExpressionUpdate.Payload.Pending` message in the IDE is provided by this PR to (improperly) visualize pending node status - further improvements needed in follow up PRs.
2022-09-28 12:35:12 +00:00
Radosław Waśko
7da4d61484
Update Case Sensitivity arguments (#3741)
Implements https://www.pivotaltracker.com/story/show/183314956
2022-09-27 20:15:52 +00:00
Jaroslav Tulach
3239815957
Registering SliceVectorMethodGen for reflection (#3737)
Each builtin `makeFunction` method has to be currently registered for reflection. Adding registration of `SliceVectorMethodGen` to make following example work ag

```bash
enso$ sbt engine-runner-native/buildNativeImage
enso$ ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
```

# Important Notes
Regression caused by #3724 - Once the above code is executed in the CI, we'll discover such breakages before integration.
2022-09-27 17:24:58 +00:00
Marcin Kostrzewa
726bfeb883
Wip/mk/statics (#3740)
Makes statics static. A type and its instances have different methods defined on them, as it should be. Constructors are now scoped in types, and can be imported/exported.

# Important Notes
The method of fixing stdlib chosen here is to just not. All the conses are exported to make all old code work. All such instances are marked with `TODO Dubious constructor export` so that it can be found and fixed.
2022-09-27 14:23:06 +00:00
Hubert Plociniczak
7a6ee0c200
Implement type_of (#3722)
This change implements a simple `type_of` method that returns a type of a given value, including for polyglot objects.

The change also allows for pattern matching on various time-related instances. It is a nice-to-have on its own, but it was primarily needed here to write some tests. For equality checks on types we currently can't use `==` due to a known _feature_ which essentially does wrong dispatching. This will be improved in the upcoming statics PR so we agreed that there is no point in duplicating that work and we can replace it later.

Also, note that this PR changes `Meta.is_same_object`. Comparing types revealed that it was wrong when comparing polyglot wrappers over the same value.
2022-09-26 16:01:39 +00:00
James Dunkerley
a3de3c6128
Use ArraySlice to slice a Vector (#3724)
Use an `ArraySlice` to slice `Vector`.
Avoids memory copying for the slice function.

# Important Notes
| Test | Ref | New |
| --- | --- | --- |
| New Vector | 71.9 | 71.0 |
| Append Single | 26.0 | 27.7 |
| Append Large | 15.1 | 14.9 |
| Sum | 156.4 | 165.8 |
| Drop First 20 and Sum | 171.2 | 165.3 |
| Drop Last 20 and Sum | 170.7 | 163.0 |
| Filter | 76.9 | 76.9 |
| Filter With Index | 166.3 | 168.3 |
| Partition | 278.5 | 273.8 |
| Partition With Index | 392.0 | 393.7 |
| Each | 101.9 | 102.7 |

- Note: the performance of New and Append has got slower from previous tests.
2022-09-23 15:13:16 +00:00
Hubert Plociniczak
096fcfee82
Generate native image for engine-runner (#3638)
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m4.503s
user	0m9.248s
sys	0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m0.176s
user	0m0.042s
sys	0m0.038s
```

# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.

Will need to add the above command to CI.
2022-09-22 14:45:10 +00:00
Jaroslav Tulach
9134f9b2d7
EnsoCompilerTest to verify compatibility of parsers (#3723)
Adding new _compatibility test_ `EnsoCompilerTest` to verify the new Rust based parser can produce the same `IR` as the original `AST` based one. The simplest way to execute the test from an empty repository is:
```bash
enso$ sbt bootstrap
enso$ sbt "testOnly *EnsoCompilerTest"
```

There are [GitHub Actions run](https://github.com/enso-org/enso/actions/runs/3087664644/jobs/4993266212#step:9:5187) on Linux as well as [run on Windows](https://github.com/enso-org/enso/actions/runs/3087664644/jobs/4993266370#step:9:5254) that show `EnsoCompilerTest` is being executed by the CI (good, as that means `.so` was properly built and linked to the JVM running the test). The [linux](https://github.com/enso-org/enso/actions/runs/3087664644/jobs/4993266212#step:9:5187) as well as [windows](https://github.com/enso-org/enso/actions/runs/3087664644/jobs/4993266370#step:9:5254) runs also demonstrate that failures in the `EnsoCompilerTest` suite fail the CI.

# Important Notes
Right now [there are five test failures](https://github.com/enso-org/enso/actions/runs/3087664644/jobs/4993266212#step:9:5187) - waiting for @kazcw to make sure `codeRepr()` doesn't contain spaces. However, as this PR is more about the infrastructure, I am disabling the currently failing tests in [031169b](031169bd05)
2022-09-20 15:50:27 +00:00
Dmitry Bushev
4443ccc0a9
Fix expression updates for builtin types (#3721)
Changelog:
- add missing cases to runtime Types check
- create an appropriate test suite
2022-09-19 13:56:51 +00:00
Hubert Plociniczak
0e5df935d3
Don't rename imported Main module that only imports names (#3710)
Turns that if you import a two-part import we had special code that would a) add Main submodule b) add an explicit rename.

b) is problematic because sometimes we only want to import specific names.
E.g.,
```
from Bar.Foo import Bar, Baz
```
would be translated to
```
from Bar.Foo.Main as Foo import Bar, Baz
```
and it should only be translated to
```
from Bar.Foo.Main import Bar, Baz
```

This change detects this scenario and does not add renames in that case.

Fixes [183276486](https://www.pivotaltracker.com/story/show/183276486).
2022-09-16 13:01:06 +00:00
Hubert Plociniczak
a04425576a
Force recompilation if imported module has changed (#3703)
IR cache never really took into account a situation when a binding from the imported module has changed. In other words, it would continue to happily use the serialized metadata without noticing that it changed.

This change forces cache invalidation when any of the imported modules was invalidated (or rather not loaded from cache).

# Important Notes
Added simple test infrastructure that simulates file modifications that would trigger the initial cache invalidation.
If they succeed, cache invalidation is propagated thus causing an error.
2022-09-15 13:41:58 +00:00
Hubert Plociniczak
65b27447e6
Minor cleanups in tests (#3697)
* Minor cleanups in tests

Removing some leftovers from big PRs.

* More tweaks

* Print failed status to stderr
2022-09-15 14:13:40 +02:00
Dmitry Bushev
77bcb87f7c
Fix execution error message (#3702) 2022-09-13 15:37:24 +03:00
Radosław Waśko
b304402d8e
Add Period Start and End functions to Date and DateTime (#3695)
Implements https://www.pivotaltracker.com/story/show/183081152
2022-09-13 09:51:08 +00:00
Hubert Plociniczak
fba5047acc
Improved Vector/Array interop (#3667)
`Vector` type is now a builtin type. This requires a bunch of additional builtin methods for its creation:
- Use `Vector.from_array` to convert any array-like structure into a `Vector` [by copy](f628b28f5f)
- Use (already existing) `Vector.from_polyglot_array` to convert any array-like structure into a `Vector` **without** copying
- Use (already existing) `Vector.fill 1 item` to create a singleton `Vector`

Additional, for pattern matching purposes, we had to implement a `VectorBranchNode`. Use following to match on `x` being an instance of `Vector` type:
```
import Standard.Base.Data.Vector

size = case x of
Vector.Vector -> x.length
_ -> 0
```

Finally, `VectorLiterals` pass that transforms `[1,2,3]` to (roughly)
```
a1 = 1
a2 = 2
a3 = 3
Vector (Array (a1,a2, a3))
```
had to be modified to generate
```
a1 = 1
a2 = 2
a3 = 3
Vector.from_array (Array (a1, a2, a3))
```
instead to accomodate to the API changes. As of 025acaa676 all the known CI checks passes. Let's start the review.

# Important Notes
Matching in `case` statement is currently done via `Vector_Data`. Use:
```
case x of
Vector.Vector_Data -> True
```
until a better alternative is found.
2022-09-13 03:07:17 +00:00
Jaroslav Tulach
40ef4a01f1
Print result of main (if not Nothing) (#3696)
The goal of this request is to simplify hello world and other trivial Enso programs. No need to learn any standard library functions, enough to write:
```
main = "Hello World!"
```
and the result is going to be printed:
```bash
enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run hello.enso
'Hello World!'
```
the result is only printed, if it is not `Nothing`. E.g. if the last statement is `IO.print ...` (which returns `Nothing`), then no value is printed at the end by the launcher.
2022-09-12 15:37:46 +00:00
Jaroslav Tulach
af24f5e543
Prevent reading broken IR caches by changing their serialVersionUID (#3692)
Many engine PRs modify builtins or other engine internals and then they are subject to [incremental CI runtime errors](https://www.pivotaltracker.com/n/projects/2539304/stories/182868680) as outdated `IR` caches from global space at `~/.local/share/enso/cache/ir/Standard/Builtins/0.0.0-dev/0.0.0-dev/Standard/Builtins` are read in.

This PR provides solution for that problem by explicitly defining `IR.Module` `serialVersionUID`. By changing the `serialVersionUID` one prevents previously saved `IR` caches to be loaded into the running process. Change the `serialVersionUID` whenever you see errors caused by reading outdated `IR` caches in the CI.

# Important Notes
Whenever one needs to avoid loading previous `IR` caches, go to `case class IR.Module` and change the `@SerialVersionUID(3692L)` to **number of your pull request**.
2022-09-09 12:01:53 +00:00
Jaroslav Tulach
9967dd3da1
Check methods of a type are present on its Atoms (#3689)
Found a bug when accessing keys via `get(constructor)`. Providing a test and a fix.

# Important Notes
Marcin, is it correct that the whole set of members of `End` is: `[head, tail, Int, is_empty, IntList]`? What does `Int` and `IntList` do there? Shall test test check for their presence? **Answer**: rename `Int` and `IntList` to lowercase and yes, then the members shall be there. Done in [ca9f42a](ca9f42a2b8).
2022-09-07 11:03:09 +00:00
Dmitry Bushev
05a6415ec0
create inmem database (#3690) 2022-09-07 12:59:18 +03:00
Dmitry Bushev
de0a231417
IDE uses new visualization API (#3661) 2022-09-01 15:33:46 +03:00
Marcin Kostrzewa
2a8201eb15
Remove a leftover debug print (#3676) 2022-08-31 12:28:41 +00:00
Marcin Kostrzewa
4fc6dcced0
Get rid of free-floating atoms. Everything has a type now! (#3671)
This is a step towards the new language spec. The `type` keyword now means something. So we now have
```
type Maybe a
Some (from_some : a)
None
```
as a thing one may write. Also `Some` and `None` are not standalone types now – only `Maybe` is.
This halfway to static methods – we still allow for things like `Number + Number` for backwards compatibility. It will disappear in the next PR.

The concept of a type is now used for method dispatch – with great impact on interpreter code density.

Some APIs in the STDLIB may require re-thinking. I take this is going to be up to the libraries team – some choices are not as good with a semantically different language. I've strived to update stdlib with minimal changes – to make sure it still works as it did.

It is worth mentioning the conflicting constructor name convention I've used: if `Foo` only has one constructor, previously named `Foo`, we now have:
```
type Foo
Foo_Data f1 f2 f3
```

This is now necessary, because we still don't have proper statics. When they arrive, this can be changed (quite easily, with SED) to use them, and figure out the actual convention then.

I have also reworked large parts of the builtins system, because it did not work at all with the new concepts.

It also exposes the type variants in SuggestionBuilder, that was the original tiny PR this was based on.

PS I'm so sorry for the size of this. No idea how this could have been smaller. It's a breaking language change after all.
2022-08-30 22:54:53 +00:00
James Dunkerley
a20d43390e
Adding DateTime part functions (#3669)
- Added `Zone`, `Date_Time` and `Time_Of_Day` to `Standard.Base`.
- Renamed `Zone` to `Time_Zone`.
- Added `century`.
- Added `is_leap_year`.
- Added `length_of_year`.
- Added `length_of_month`.
- Added `quarter`.
- Added `day_of_year`.
- Added `Day_Of_Week` type and `day_of_week` function.
- Updated `week_of_year` to support ISO.

# Important Notes
- Had to pass locale to formatter for date/time tests to work on my PC.
- Changed default of `week_of_year` to use ISO.
2022-08-26 15:47:58 +00:00
Radosław Waśko
fd318cfa96
Remove Array.set_at (#3634)
Implements https://www.pivotaltracker.com/story/show/182879865

# Important Notes
Note that removing `set_at` still does not make our arrays fully immutable - `Array.copy` can still be used to mutate them.
2022-08-26 09:34:33 +00:00
Hubert Plociniczak
d87a32d019
Builtin Date_Time, Time_Of_Day, Zone (#3658)
* Builtin Date_Time, Time_Of_Day, Zone

Improved polyglot support for Date_Time (formerly Time), Time_Of_Day and
Zone. This follows the pattern introduced for Enso Date.

Minor caveat - in tests for Date, had to bend a lot for JS Date to pass.
This is because JS Date is not really only a Date, but also a Time and
Timezone, previously we just didn't consider the latter.
Also, JS Date does not deal well with setting timezones so the trick I
used is to first call foreign function returning a polyglot JS Date,
which is converted to ZonedDateTime and only then set the correct
timezone. That way none of the existing tests had to be changes or
special cased.

Additionally, JS deals with milliseconds rather than nanoseconds so
there is loss in precision, as noted in Time_Spec.

* Add tests for Java's LocalTime

* changelog

* Make date formatters in table happy

* PR review, add more tests for zone

* More tests and fixed a bug in column reader

Column reader didn't take into account timezone but that was a mistake
since then it wouldn't map to Enso's Date_Time.
Added tests that check it now.

* remove redundant conversion

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Time.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* First round of addressing PR review

* don't leak java exceptions in Zone

* Move Date_Time to top-level module

* PR review

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2022-08-24 12:31:29 +02:00
Hubert Plociniczak
4b9c91626e
Use Vector.from_polyglot_array to make Vectors backed by polyglot arrays (#3628)
Use Proxy_Polyglot_Array as a proxy for polyglot arrays, thus unifying
the way the underlying array is accessed in Vector.

Used the opportunity to cleanup builtin lookup, which now actually
respects what is defined in the body of @Builtin_Method annotation.

Also discovered that polyglot null values (in JS, Python and R) were leaking to Enso.
Fixed that by doing explicit translation to `Nothing`.

https://www.pivotaltracker.com/story/show/181123986
2022-08-23 21:13:16 +00:00
Dmitry Bushev
f10081d905
Increase library upload timeout in tests (#3649) 2022-08-15 12:40:46 +03:00
Hubert Plociniczak
8575b76b0a
Support pattern matching on constants (#3641)
This change adds support for matching on constants by:
1) extending parser to allow literals in patterns
2) generate branch node for literals

Related to https://www.pivotaltracker.com/story/show/182743559
2022-08-12 13:18:58 +00:00
Hubert Plociniczak
3fa78afd10
Support Autosave for open buffers (#3637)
This change adds Autosave action for open buffers. The action is scheduled
after every edit request and is cancelled by every explicit save file request, if
necessary. Successful autosave also notifies any active clients of the buffer.

Related to https://www.pivotaltracker.com/story/show/182721656

# Important Notes
WIP
2022-08-11 11:45:12 +00:00
Jaroslav Tulach
a7bc3c6c89
Verify benchmarks compile and execute in the gate (#3640)
Execution of `sbt runtime/bench` doesn't seem to be part of the gate. As such it can happen a change into the Enso language syntax, standard libraries, runtime & co. can break the benchmarks suite without being noticed. Integrating such PR causes unnecessary disruptions to others using the benchmarks.

Let's make sure verification of the benchmarks (e.g. that they compile and can execute without error) is part of the CI.

# Important Notes
Currently the gate shall fail. The fix is being prepared in parallel PR - #3639. When the two PRs are combined, the gate shall succeed again.
2022-08-11 07:21:44 +00:00
Dmitry Bushev
1083a2532e
Fix benchmarks (#3639) 2022-08-10 13:31:28 +00:00
Dmitry Bushev
98d30bccf3
Enable caching in visualization functions (#3618)
PR allows to attach metod pointers as a visualization expressions. This way it allows to attach a runtime instrument that enables caching of intermediate expressions.

# Important Notes
ℹ️ API is backward compatible.

To attach the visualization with caching support, the same `executionContext/attachVisualisation` method is used, but `VisualisationConfig` message should contain the message pointer.
While `VisualisationConfiguration` message has changed, the language server accepts both new and old formats to keep visualisations working in IDE.

#### Old format

```json
{
"executionContextId": "UUID",
"visualisationModule": "local.Unnamed.Main",
"expression": "x -> x.to_text"
}
```

#### New format

```json
{
"executionContextId": "UUID",
"expression": {
"module": "local.Unnamed.Main",
"definedOnType": "local.Unnamed.Main",
"name": "encode"
}
}
```
2022-08-10 12:01:33 +00:00
Dmitry Bushev
5e114acbb5
Update Scala to 2.13.8 (#3631)
Update Scala compiler and libraries.
2022-08-08 19:32:55 +00:00
Hubert Plociniczak
1eec315ce1
Fix REPL runner (#3624)
`main` method is now special because it is trully static i.e. no
implicit `self` is being generated for it.
But since REPL's `main` isn't called `main` its invocation was missing
an argument.

Follow up on https://github.com/enso-org/enso/pull/3569

# Important Notes
Will work on adding a CI test for REPL to avoid problems with REPL in a
follow up PR.
2022-08-08 08:27:53 +00:00
Hubert Plociniczak
42dbd8bb59
Allow for importing methods (#3633)
Importing individual methods didn't work as advertised because parser
would allow them but later drop that information. This slipped by because we never had mixed atoms and methods in stdlib.

# Important Notes
Added some basic tests but we need to ensure that the new parser allows for this.
@jdunkerley will be adding some changes to stdlib that will be testing this functionality as well.
2022-08-05 16:25:51 +00:00
Dmitry Bushev
fb4f9ab193
Add text/openBuffer command (#3623)
PR adds the `text/openBuffer` command required for lazy visualizations. This is an implementation PR, API documentation has been added previously.
2022-08-05 14:45:43 +00:00
Radosław Waśko
0a2fea925c
Create Index_Sub_Range type and update Text.take and Text.drop (#3617) 2022-08-03 11:41:34 +00:00
Hubert Plociniczak
d59714a29d
Support module imports using a qualified name (#3608)
This change allows for importing modules using a qualified name and deals with any conflicts on the way.
Given a module C defined at `A/B/C.enso` with
```
type C
type C a
```
it is now possible to import it as
```
import project.A
...
val x = A.B.C 10
```

Given a module located at `A/B/C/D.enso`, we will generate
intermediate, synthetic, modules that only import and export the successor module along the path.
For example, the contents of a synthetic module B will look like
```
import <namespace>.<pkg-name>.A.B.C
export <namespace>.<pkg-name>.A.B.C
```
If module B is defined already by the developer, the compiler will _inject_ the above statements to the IR.

Also removed the last elements of some lowercase name resolution that managed to survive recent
changes (`Meta.Enso_Project` would now be ambiguous with `enso_project` method).

Finally, added a pass that detects shadowing of the synthetic module by the type defined along the path.
We print a warning in such a situation.

Related to https://www.pivotaltracker.com/n/projects/2539304

# Important Notes
There was an additional request to fix the annoying problem with `from` imports that would always bring
the module into the scope. The changes in stdlib demonstrate how it is now possible to avoid the workaround of
```
from X.Y.Z as Z_Module import A, B
```
(i.e. `as Z_Module` part is almost always unnecessary).
2022-07-29 14:19:07 +00:00
Hubert Plociniczak
206dbc425a
here, be gone! (#3619)
Noticed benchmarks were failing to compile and quick check revealed that
`here` managed to survive [my crusade](https://github.com/enso-org/enso/pull/3538).
2022-07-29 09:31:26 +00:00
Hubert Plociniczak
26018e4969
Make ConditionProfiles values final (#3612)
That way the underlying values can be treated as constant.
Problem discovered initially by @JaroslavTulach .
2022-07-28 13:23:37 +00:00
Jaroslav Tulach
58827954b0
Make sure the plain Java exception wrapper always provides a message (#3610) 2022-07-28 06:36:21 +00:00
Hubert Plociniczak
f63e40df1b
Explicit self (#3569)
This change modifies the current language by requiring explicit `self` parameter declaration
for methods. Methods without `self` parameter in the first position should be treated as statics
although that is not yet part of this PR. We add an implicit self to all methods
This obviously required updating the whole stdlib and its components, tests etc but the change
is pretty straightforward in the diff.

Notice that this change **does not** change method dispatch, which was removed in the last changes.
This was done on purpose to simplify the implementation for now. We will likely still remove all
those implicit selfs to bring true statics.
Minor caveat - since `main` doesn't actually need self, already removed that which simplified
a lot of code.
2022-07-27 17:45:36 +00:00
Radosław Waśko
ee91656f30
Remove duplicate Line_Ending_Style and update defaults (#3597)
Implements https://www.pivotaltracker.com/story/show/182749831
2022-07-27 09:43:51 +00:00
Radosław Waśko
16fd038c1a
Add support for .pgpass to PostgreSQL (#3593)
Implements https://www.pivotaltracker.com/story/show/182582924
2022-07-21 13:32:37 +00:00
Jaroslav Tulach
4465d63dd8
Improved polyglot Date support (#3559)
Significantly improves the polyglot Date support (as introduced by #3374). It enhances the `Date_Spec` to run it in four flavors:
- with Enso Date (as of now)
- with JavaScript Date
- with JavaScript Date wrapped in (JavaScript) array
- with Java LocalDate allocated directly

The code is then improved by necessary modifications to make the `Date_Spec` pass.

# Important Notes
James has requested in [#181755990](https://www.pivotaltracker.com/n/projects/2539304/stories/181755990) - e.g. _Review and improve InMemory Table support for Dates, Times, DateTimes, BigIntegers_ the following program to work:
```
foreign js dateArr = """
return [1, new Date(), 7]

main =
IO.println <| (dateArr.at 1).week_of_year
```
the program works with here in provided changes and prints `27` as of today.

@jdunkerley has provided tests for proper behavior of date in `Table` and `Column`. Those tests are working as of [f16d07e](f16d07e640). One just needs to accept `List<Value>` and then query `Value` for `isDate()` when needed.

Last round of changes is related to **exception handling**. 8b686b12bd makes sure `makePolyglotError` accepts only polyglot values. Then it wraps plain Java exceptions into `WrapPlainException` with `has_type` method - 60da5e70ed - the remaining changes in the PR are only trying to get all tests working in the new setup.

The support for `Time` isn't part of this PR yet.
2022-07-21 06:32:40 +00:00
Dmitry Bushev
2c780ef6ba
Add command for changing expression value (#3562)
Implement `text/applyExpressionValue` command.
2022-07-13 15:59:47 +00:00
Jaroslav Tulach
891eef0f92
Cannot debug recursive/infinite structure (#3556)
There is an Unsafe.set_atom_field operation in Standard library. That operation allows one to create an infinite data structure. Store following program in ones.enso:
```
import Standard.Base.IO
import Standard.Base.Runtime.Unsafe

type Gen
type Empty
type Generator a:Int tail:Gen

ones : Gen
ones =
g = Generator 1 Empty
Unsafe.set_atom_field g 1 g
g

main =
IO.println here.ones
```

running such program then leads to (probably expectable) stack overflow exception:
```
Execution finished with an error: Resource exhausted: Stack overflow
at <enso> Any.to_text(Internal)
...
at <enso> Any.to_text(Internal)
at <enso> Any.to_text(Internal)
at <enso> Any.to_text(Internal)
at <enso> IO.println(Internal)
at <enso> g.main(g.enso:15:5-24)
```
However the bigger problem is that it also crashes our debugger. While producing guest Stack overflow when the guest program is running maybe OK, crashing the engine doesn't seem tolerable.

Try:
```
enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --inspect --run ones.enso
```
and navigate Chrome dev tools to the line 11 as shown on the attached picture.

Stepping over that line generates following error:
```
at org.enso.interpreter.runtime.callable.atom.Atom.toString(Atom.java:84)
at org.enso.interpreter.runtime.callable.atom.Atom.lambda$toString$0(Atom.java:79)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
```
Stack overflow in the engine when computing `Atom.toString()` - I want to prevent that.

# Important Notes
I am able to see a stacktrace in the debugger and I can _step in_ and _step over_, @kustosz:

![image](https://user-images.githubusercontent.com/26887752/176626989-fdc2979e-f86c-42bc-a4df-c533cf7a4839.png)

However there are extra items like `case_branch` which I'd like to avoid, would you know how to do that?
2022-07-11 13:03:19 +00:00
Dmitry Bushev
b59a496589
Create execution context with provided id (#3575) 2022-07-11 14:59:38 +03:00
Jaroslav Tulach
735053c218
Implementing basic functions (#3554)
The language specification suggests to add [five basic functions into the standard library](https://github.com/enso-org/design/blob/wip/wd/enso-spec/epics/enso-spec-1.0/05.%20Functions.md#useful-functions-in-the-standard-library). `identity`, `flip`, `const`, `curry` & `uncurry`.

# Important Notes
The new functions are being added into existing `Function.enso` file. That may not be the best place, but it is not clear from the [design spec](https://github.com/enso-org/design/blob/wip/wd/enso-spec/epics/enso-spec-1.0/05.%20Functions.md#useful-functions-in-the-standard-library) how they are supposed to be imported. I can move them wherever needed.

There is a documentation provided for each of the functions, but I am not sure how to verify it is correct. Do we generate the documentation for stdlib somehow?
2022-07-11 10:30:44 +00:00
Radosław Waśko
28513a3389
Allow filtering caught error type in Error.catch (#3574)
More and more often I need a way to only recover a specific type of a dataflow error (in a similar manner as with panics). So the API for `Error.catch` has been amended to more closely resemble `Panic.catch`, allowing to handle only specific types of dataflow errors, passing others through unchanged. The default is `Any`, meaning all errors are caught by default, and the behaviour of `x.catch` remains unchanged.
2022-07-11 08:26:44 +00:00
Hubert Plociniczak
96e50648dd
Remove 'here' and make method name resolution case-sensitive (#3538)
Modified UppercaseNames to now resolve methods without an explicit `here` to point to the current module.
`here` was also often used instead of `self` which was allowed by the compiler.
Therefore UppercaseNames pass is now GlobalNames and does some extra work -
it translated method calls without an explicit target into proper applications.

# Important Notes
There was a long-standing bug in scopes usage when compiling standalone expressions.
This resulted in AliasAnalysis generating incorrect graphs and manifested itself only in unit tests
and when running `eval`, thus being a bit hard to locate.
See `runExpression` for details.

Additionally, method name resolution is now case-sensitive.

Obsolete passes like UndefinedVariables and ModuleThisToHere were removed. All tests have been adapted.
2022-07-07 10:31:06 +00:00
James Dunkerley
16e6f2fa08
Adding Append support to Excel.Write (#3558)
Adds support for appending to an existing Excel table.

# Important Notes
- Renamed `Column_Mapping` to `Column_Name_Mapping`
- Changed new type name to `Map_Column`
- Added last modified time and creation time to `File`.
2022-07-07 06:41:33 +00:00
Dmitry Bushev
509663f7ab
Disable flaky create_process test (#3561) 2022-07-05 11:25:47 +03:00
James Dunkerley
4ca2097488
Adding write support to File_Format.Excel (#3551)
Support for writing tables to Excel.

# Important Notes
Has custom support for Error mode as will allow appending a new table in this mode to the file.
2022-07-04 18:32:16 +00:00
Michał Wawrzyniec Urbańczyk
43a893cae6
Bump the build script (#3535) 2022-07-01 03:58:14 +02:00
Jaroslav Tulach
abc2d37b12
Disable MultiTier compilation in benchmarks (#3548)
Truffle is using [MultiTier compilation](190e0b2bb7) by default since 21.1.0. That mean every `RootNode` is compiled twice. However benchmarks only care about peak performance. Let's return back the previous behavior and compile only once after profiling in interpreter.

# Important Notes
This change does not influence the peak performance. Just the amount of IGV graphs produced from benchmarks when running:
```
enso$ sbt  "project runtime" "withDebug --dumpGraphs benchOnly -- AtomBenchmark"
```
is cut to half.
2022-06-28 04:55:12 +00:00
Jaroslav Tulach
096d8fdca0
Avoid crashing the engine when JS debugger statement is used (#3547)
Try following enso program:
```
main =
here.debug

foreign js debug = """
debugger;

```
it crashes the engine with exception:
```
Execution finished with an error: java.lang.ClassCastException: class com.oracle.truffle.js.runtime.builtins.JSFunctionObject$Unbound cannot be cast to class org.enso.interpreter.runtime.callable.CallerInfo (com.oracle.truffle.js.runtime.builtins.JSFunctionObject$Unbound and org.enso.interpreter.runtime.callable.CallerInfo are in unnamed module of loader com.oracle.graalvm.locator.GraalVMLocator$GuestLangToolsLoader @55cb6996)
at <java> org.enso.interpreter.runtime.callable.function.Function$ArgumentsHelper.getCallerInfo(Function.java:352)
at <java> org.enso.interpreter.instrument.ReplDebuggerInstrument$ReplExecutionEventNodeImpl.onEnter(ReplDebuggerInstrument.java:179)
at <java> org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode$EventProviderChainNode.innerOnEnter(ProbeNode.java:1397)
at <java> org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode$EventChainNode.onEnter(ProbeNode.java:912)
at <java> org.graalvm.truffle/com.oracle.truffle.api.instrumentation.ProbeNode.onEnter(ProbeNode.java:216)
at <java> com.oracle.truffle.js.nodes.JavaScriptNodeWrapper.execute(JavaScriptNodeWrapper.java:44)
at <java> com.oracle.truffle.js.nodes.control.DiscardResultNode.execute(DiscardResultNode.java:88)
at <java> com.oracle.truffle.js.nodes.function.FunctionBodyNode.execute(FunctionBodyNode.java:73)
at <java> com.oracle.truffle.js.nodes.JavaScriptNodeWrapper.execute(JavaScriptNodeWrapper.java:45)
at <java> com.oracle.truffle.js.nodes.function.FunctionRootNode.executeInRealm(FunctionRootNode.java:150)
at <java> com.oracle.truffle.js.runtime.JavaScriptRealmBoundaryRootNode.execute(JavaScriptRealmBoundaryRootNode.java:93)
at <js> <js> poly_enso_eval(Unknown)
at <epb> <epb> null(Unknown)
at <enso> Prg.debug(Prg.enso:27-28)
```
2022-06-25 05:58:24 +00:00
Hubert Plociniczak
8f9a0b33d5
Fix System.nanoTime definition (#3543)
While redesigning builtins a bug was accidentally introduced.
Stumbled upon accidentally while trying to run `test/Benchmarks`.
2022-06-24 10:46:31 +00:00
Dmitry Bushev
206933ec57
Implement execute parameter of applyEdit command (#3541)
PR implements new `execute` parameter designed in #3532. It allows IDE to control the execution after applying edits to files.
2022-06-23 15:33:45 +00:00
Jaroslav Tulach
de89673e8f
Fix broken AtomBenchmarks (#3546)
`AtomBenchmarks` are broken since the introduction of [micro distribution](https://github.com/enso-org/enso/pull/3531). The micro distribution doesn't contain `Range` and as such one cannot use `1.up_to` method.

# Important Notes
I have rewritten enso code to manual `generator`. The results of the benchmark seem comparable. Executed as:
```
sbt:runtime> benchOnly AtomBenchmarks
```
2022-06-23 13:52:46 +00:00
Jaroslav Tulach
d0124fbb02
Fail bin/enso --run when there is an error in the script (#3544)
I was modifying `Date_Spec.enso` today and made a mistake. When executing with I could see the error, but the process got stuck...
```
enso/test/Tests/src/Data/Time$ ../../../../built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run Date_Spec.enso In module Date_Spec:
Compiler encountered warnings:
Date_Spec.enso[14:29-14:37]: Unused function argument parseDate.
Compiler encountered errors:
Date_Spec.enso[18:13-18:20]: Variable `debugger` is not defined.
Exception in thread "main" Compilation aborted due to errors.
at org.graalvm.sdk/org.graalvm.polyglot.Value.invokeMember(Value.java:932)
at org.enso.polyglot.Module.getAssociatedConstructor(Module.scala:19)
at org.enso.runner.Main$.runMain(Main.scala:755)
at org.enso.runner.Main$.runSingleFile(Main.scala:695)
at org.enso.runner.Main$.run(Main.scala:582)
at org.enso.runner.Main$.main(Main.scala:1031)
at org.enso.runner.Main.main(Main.scala)
^C
```
...had to kill it with Ctrl-C. This PR fixes the problem by moving `getAssociatedConstructor` call into `try` block, catching the exception and exiting properly.
2022-06-23 11:16:20 +00:00
Jaroslav Tulach
479f74cc84
Enso Integration with Ideal Graph Visualizer (#3533)
This PR adds sources for Enso language support in IGV (and NetBeans). The support is based on TextMate grammar shown in the editor and registration of the Enso language so IGV can find it. Then this PR adds new GitHub Actions workflow file to build the project using Maven.
2022-06-23 04:43:49 +00:00
Hubert Plociniczak
22a371a9c6
Substitute this with self (#3524)
A semi-manual s/this/self appied to the whole standard library.
Related to https://www.pivotaltracker.com/story/show/182328601

In the compiler promoted to use constants instead of hardcoded
`this`/`self` whenever possible.

# Important Notes
The PR **does not** require explicit `self` parameter declaration for methods as this part
of the design is still under consideration.
2022-06-21 10:53:52 +00:00
Jaroslav Tulach
8f6367966b
Single (+patchable) LiteralNode (#3536)
This PR merges existing variants of `LiteralNode` (`Integer`, `BigInteger`, `Decimal`, `Text`) into a single `LiteralNode`. It adds `PatchableLiteralNode` variant (with non `final` `value` field) and uses `Node.replace` to modify the AST to be patchable. With such change one can remove the `UnwindHelper` workaround as `IdExecutionInstrument` now sees _patched_ return values without any tricks.
2022-06-21 09:42:23 +00:00
Dmitry Bushev
1e7047f26f
Use system dispatcher for ZIO (#3534)
Changelog:
- update: reuse system dispatcher for ZIO library instead of creating an extra one.
2022-06-20 07:57:06 +00:00
Jaroslav Tulach
c72a6582bc
Avoid whole source reparsing when the IDE performs a simple edit (#3508)
Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
2022-06-16 16:02:57 +02:00
Marcin Kostrzewa
ec3fa32fec
introduce a micro stdlib for testing (#3531)
This introduces a tiny alternative to our stdlib, that can be used for testing the interpreter. There are 2 main advantages of such a solution:
1. Performance: on my machine, `runtime-with-intstruments/test` drops from 146s to 65s, while `runtime/test` drops from 165s to 51s. >6 mins total becoming <2 mins total is awesome. This alone means I'll drink less coffee in these breaks and will be healthier.
2. Better separation of concepts – currently working on a feature that breaks _all_ enso code. The dependency of interpreter tests on the stdlib means I have no means of incremental testing – ALL of stdlib must compile. This is horrible, rendered my work impossible, and resulted in this PR.
2022-06-16 10:25:24 +00:00
Jaroslav Tulach
dc30e44b60
Register instruments/language in their own compilation units to fix the sbt build issues (#3509)
New plan to [fix the `sbt` build](https://www.pivotaltracker.com/n/projects/2539304/stories/182209126) and its annoying:
```
log.error(
"Truffle Instrumentation is not up to date, " +
"which will lead to runtime errors\n" +
"Fixes have been applied to ensure consistent Instrumentation state, " +
"but compilation has to be triggered again.\n" +
"Please re-run the previous command.\n" +
"(If this for some reason fails, " +
s"please do a clean build of the $projectName project)"
)
```
When it is hard to fix `sbt` incremental compilation, let's restructure our project sources so that each `@TruffleInstrument` and `@TruffleLanguage` registration is in individual compilation unit. Each such unit is either going to be compiled or not going to be compiled as a batch - that will eliminate the `sbt` incremental compilation issues without addressing them in `sbt` itself.

fa2cf6a33ec4a5b2e3370e1b22c2b5f712286a75 is the first step - it introduces `IdExecutionService` and moves all the `IdExecutionInstrument` API up to that interface. The rest of the `runtime` project then depends only on `IdExecutionService`. Such refactoring allows us to move the `IdExecutionInstrument` out of `runtime` project into independent compilation unit.
2022-06-13 14:09:08 +00:00
Hubert Plociniczak
fd46e84e8d
Towards a full-blown builtins DSL (part 3) (#3471)
Auto-generate all builtin methods for builtin `File` type from method signatures.
Similarly, for `ManagedResource` and `Warning`.
Additionally, support for specializations for overloaded and non-overloaded methods is added.
Coverage can be tracked by the number of hard-coded builtin classes that are now deleted.

## Important notes

Notice how `type File` now lacks `prim_file` field and we were able to get rid off all of those
propagating method calls without writing a single builtin node class.
Similarly `ManagedResource` and `Warning` are now builtins and `Prim_Warnings` stub is now gone.
2022-06-13 11:48:34 +00:00
Radosław Waśko
a04825a5ce
Add Text.write Function (#3518)
Implements https://www.pivotaltracker.com/story/show/182309026
2022-06-13 09:11:46 +00:00
Hubert Plociniczak
b1db359f19
Minor compilation improvements (#3512)
Drop `Core` implementation (replacement for IR) as it (sadly) looks increasingly
unlikely this effort will be continued. Also, it heavily relies
on implicits which increases some compilation time (~1sec from `clean`)

Related to https://www.pivotaltracker.com/story/show/182359029
2022-06-07 14:48:50 +00:00
Dmitry Bushev
0e867c663e
Keep the components order (#3505)
PR ensures that the component groups are returned in the same order that the packages were loaded.
2022-06-03 18:45:56 +00:00