Commit Graph

1119 Commits

Author SHA1 Message Date
Jaroslav Tulach
4222ddccb5
Canonicalize file names before comparing them (#8670) 2024-01-04 15:21:13 +01:00
Hubert Plociniczak
20531d51df
Initialize builtin methods lazily in non-AOT mode (#8654)
Registration of builtin methods is done upfront, similarly to AOT mode, but the initialization is delayed until the first use.
One still has to read the full list of builtin methods and register them in the builtins scope or we would be getting the `NoSuchMethod` errors otherwise. But the most expensive operation, initialization of classes and getting a method via a reflection, is all done on the first usage.

The result is an improved startup and negligible performance impact in regular usage.

Closes #8423.

# Important Notes
Visible speedup.
1. (old) Program returning plain "Hello World" (no stdlib loading)
![Screenshot from 2024-01-02 17-34-35](https://github.com/enso-org/enso/assets/292128/b8348cbb-baf6-4292-8c0e-c8a3492c6583)
2. (new) Program returning plain "Hello World" (no stdlib loading)
![Screenshot from 2024-01-02 17-34-54](https://github.com/enso-org/enso/assets/292128/63dd6c4f-49e2-4a5e-ab95-2460c155b91c)
About 10% improvement for the overall execution.

3. (old) Program printing "Hello World" (some stdlib loading)
![Screenshot from 2024-01-02 17-40-21](https://github.com/enso-org/enso/assets/292128/cf3f464c-8f00-4adf-9499-a6ac86430da0)
4. (new) Program printing "Hello World" (some stdlib loading)
![Screenshot from 2024-01-02 17-40-41](https://github.com/enso-org/enso/assets/292128/3ef99893-ccaf-4ea7-90ba-aa090f904a50)
About 2% improvement for the overall execution.

Minor price to pay for lazy initialization:
![Screenshot from 2024-01-02 17-56-12](https://github.com/enso-org/enso/assets/292128/faa682d8-5370-43d2-9fc9-3454af5749a4)
2024-01-04 08:55:21 +00:00
Jaroslav Tulach
5e3480b1e8
Apply SectionsToBinOp pass to arguments of Section (#8655)
Fixes #8436 by making sure that `SectionsToBinOp` pass is applied also to arguments of `Section`.
2024-01-03 14:37:11 +00:00
Dmitry Bushev
cfab344fbe
Unable to access constructor value in interactive mode (#8626)
close #7184

The constructor value was not accessible because during the re-compilation a new instance of the type was registered in runtime. Then during the execution, an old cached instance of the type was used in method resolution.

Changelog:
- update: the registration of types in runtime
- update: invalidate cached nodes that became a resolution error after applying the edit
2024-01-03 13:23:42 +00:00
Pavel Marek
819741bb4a
Increase warmup for EqualsBenchmarks. (#8636)
After #8620, there is a noticeable slowdown in `EqualsBenchmarks.equalsTrees` as suggested in https://github.com/enso-org/enso/pull/8620#issuecomment-1870776609. After some digging, I realized that the number of warmup iterations is most probably insufficient. Let's increase the warmup for this benchmark and see if we can get its score down again.
2023-12-29 08:47:36 +00:00
Jaroslav Tulach
27431c126c
Make IR caches smaller by storing Graph as a reference (#8635) 2023-12-29 06:15:48 +01:00
Pavel Marek
48a3c14ee5
Format Java sources in benchmark configuration (#8638)
I noticed that sources in `runtime/bench` are not formatted at all. Turns out that the `JavaFormatterPlugin` does not override `javafmt` task for the `Benchmark` configuration. After some failed attempts, I have just redefined the `Benchmark/javafmt` task in the `runtime` project. After all, the `runtime` project is almost the only project where we have any Java benchmarks.

# Important Notes
`javafmtAll` now also formats sources in `runtime/bench/src/java`.
2023-12-28 17:06:06 +00:00
Dmitry Bushev
2d628263ff
Update cache invalidation commands (#8634) 2023-12-26 12:39:17 +00:00
Jaroslav Tulach
07d58f2c02
DataflowError.withoutTrace shall not store a trace (#8608) 2023-12-24 11:07:32 +01:00
Pavel Marek
74436830ce
Fix build of engine benchmarks (#8620)
After #8467, Engine benchmarks are broken, they cannot compile - https://github.com/enso-org/enso/actions/runs/7268987483/job/19805862815#logs

This PR fixes the benchmark build

# Important Notes
Apart from fixing the build of `engine/bench`:
- Don't assemble any fat jars in `runtime/bench`.
- Use our `TestLogProvider` in the benches instead of NOOP provider.
- So that we can at least see warnings and errors in benchmarks.
2023-12-22 11:40:32 +00:00
Pavel Marek
edc98593b9
All unit tests use the same testing logging provider (#8593)
Make sure that the correct test logging provider is loaded in `project-manager/Test`, so that only WARN and ERROR log messages are displayed. Also, make sure that the test log provider parses the correct configuration file - Rename all the `application.conf` files in the test resources to `application-test.conf`.

The problem was introduced in #8467
2023-12-21 13:45:33 +00:00
Radosław Waśko
dfdb547616
Better context info in Type_Error raised from return type checks (#8566)
- Followup to #8502 that adds better error messages
2023-12-20 18:22:47 +00:00
Pavel Marek
4cb2439890
Submodules can be private (#8581) 2023-12-19 19:13:44 +01:00
Jaroslav Tulach
7daad75dd9
Avoid WithWarnings without any warnings (#8583) 2023-12-19 17:03:16 +01:00
Radosław Waśko
f0c2a5fa7f
Opt-in return type checks (#8502)
- Closes #8240
2023-12-19 15:32:30 +00:00
Hubert Plociniczak
1ad11104d3
Stop parsing METADATA section for IdMap (#8574)
We infer old IDs from module's IRs location, instead. This workarounds ill-constructed IdMaps at the start of the project.

If IDE or backend ends up with invalid ID map, I suppose application of edits might still fail.

# Important Notes
Closes #8500 by not doing the parsing at all. Since I couldn't figure why we had invalid metadata in the first place, this might still bite us at some point.
2023-12-19 12:03:33 +00:00
Pavel Marek
21d164ec3e
Run unit tests with truffle-compiler (#8467) 2023-12-18 18:22:16 +01:00
Dmitry Bushev
b2a1cd55d2
Syntax error results in silent error (#8560)
close #7555

Compiler passes after `GenerateMethodBodies` expect the method body to be a function.

After fixing the pass, the compilation returns a proper compiler error:
```
built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Table/0.0.0-dev/src/Data/Column.enso:869:22: error: Methods must have only one definition of the `this` argument, and it must be the first.
869 |     round self round self (decimal_places:Integer = 0) (use_bankers:Boolean = False) = Value_Type.expect_numeric self <|
|                      ^~~~
Aborting due to 1 errors and 0 warnings.
```
2023-12-18 13:12:31 +00:00
Pavel Marek
c1098865f2
Update java formatter sbt plugin (#8543)
Add a local clone of javaFormatter plugin. The upstream is not maintained anymore. And we need to update it to use the newest Google java formatter because the old one, that we use, cannot format sources with Java 8+ syntax.

# Important Notes
Update to Google java formatter 1.18.1 - https://github.com/google/google-java-format/releases/tag/v1.18.1
2023-12-15 14:45:23 +00:00
Dmitry Bushev
56cc9561b1
Fix serialization of alias analysis graph (#8550)
close #8431

Fixes the scenario:
- user sends `executionContext/executeExpression`
- program execution is scheduled
- during the compilation the already compiled `IR` is loaded from the cache (reading invalid alias analysis graph)
- during the codegen the local scope with that aliasing graph is propagated to the runtime
- `EvalNode` compiles the expression to execute with the local scope containing an invalid aliasing graph
- compilation fails in the `AliasAnalysis` pass because of the clashing IDs in the graph
2023-12-15 12:54:52 +00:00
Pavel Marek
4b65e44ef3
EpbLanguage re-uses other TruffleContext support to run tests with assertions enabled (#7882) 2023-12-15 13:31:32 +01:00
Radosław Waśko
95f11abe2c
Try reporting diagnostics in non-strict mode (#8272)
- Makes sure that the compiler will print the diagnostics even if non-strict mode is used
- To prevent printing unexpected stuff to stdout in interactive mode, the diagnostics are printed as 'warning' level log messages in that mode. In strict mode, the messages are printed like before, without changes.
2023-12-14 15:41:35 +00:00
GregoryTravis
1c815a3d45
Better Error Trapping in map (#8307)
* tests

* wip

* wip

* additional warnings

* wip

* wip

* cleanup

* nested wrapping

* multiple nestings

* wraps_error uses looks_for, test for should_fail_with

* wip

* stack trace line fix

* use catch_primitive internally

* fix warning mapping, dtf spec

* just one wrapper checker, vector spec

* missing ctor, back to non-primitive catch

* back to c_p

* put old map back

* wip

* unnest tests

* Array.map on_problems

* wip

* Revert "wip"

This reverts commit c30d171457.

* better test names

* warning logging

* wip

* wip

* move logic into ALH

* doc

* constant

* My_Error.Error

* nested

* doc

* map_primtiive in warning mapper

* composition

* ref spec

* Remove warnings prior to matching on the value

If an expression has warnings and is matched we:
1) extract the warnings
2) execute the branch of a pattern that matches the value
3) attach extracted warnings to the result

This caused warnings to reappear when doing the custom warnings
manipulation.
This is also consistent with how `CaseNode`'s `doWarning` specialization
is defined.

* fix 1

* do not auto unwrap in test error checkers

* nested error matcher

* in problems too

* dtf

* v

* statistics

* wip

* Table_Spec, map_with_index_primitive

* Column_Operations_Spec

* disable warning wrapping and Report_Warning

* unimpl test

* Warnings_Spec

* DCS

* ACG JP

* zip_primitive

* join_helpers

* Lookup_Helpers

* Table

* Data_Formatter

* Value_Type_Helpers

* revert check types changes

* table_helpers

* table tests

* remove st

* do not remove warnings from value

* vec docs, tests for zip, mwi, flat_map

* docs, fixes

* remove nested_error_matcher

* cleanup

* benchmark

* one error

* alter

* add bench to main

* review

* review

* review

* tail call

* changelog

* tail call was not a tail call

* ws

* bad import

* Added missing import

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Array.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* review, ref example

* lazy benchmark data

* extra paren

* check outside of catch

* review

* vector too

* actually lazy

* disambiguate Map_Error

* finish rename

* move to extensions

* combine Additional_Warnings error

* rename to map_no_wrap

* do not catch and rethrow

* review

* wip

* remove _primitives entirely

* remove unused should_fail_with function options

* remove expected_warning as function in Problems

---------

Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2023-12-13 09:38:09 -05:00
Pavel Marek
1bde4c5699
Add org.jline as module to component directory (#8496)
Add `jline` module to the distribution so that our REPL is usable again.

# Important Notes
- No more: "WARNING: Unable to create a system terminal, creating a dumb terminal " warning when starting REPL
- Arrow keys works as expected in REPL
- Back search (the default shortcut `CTRL + R`) works as expected.
2023-12-13 13:47:50 +00:00
Kaz Wesley
ce6c770fd7
Parse inline function signatures (#8470)
Implements #6166.

# Important Notes
- More consistent handling of `default` arguments. `default` is a valid identifier, and only has special meaning when it isn't bound in scope. Since distinguishing the builtin `default` from an identifier called `default` cannot be done until alias analysis has been performed, `default` is now represented in the AST as a regular identifier.
- `TreeToIr`: Remove `insideTypeAscription`. It was only used for bug-for-bug compatibility with the old parser during the transition.
2023-12-12 14:48:44 +00:00
Jaroslav Tulach
80f94a21e1
Compare long and double and BigInteger properly (#8510) 2023-12-12 06:41:43 +01:00
Jaroslav Tulach
e4b2b56a40
Turning Sieve benchmarks into Enso benchmarks (#8475) 2023-12-08 10:27:52 +01:00
Hubert Plociniczak
a14ebd6259
Increase timeout for running launcher command (#8486)
We've been experiencing consistently failures on MacOS due to timeouts.
Doubling the timeout, hoping this will be sufficient to eliminate such
false failures. Will seek alternative solutions if that does not rememdy
the problem on CI.
2023-12-07 16:34:55 +01:00
Hubert Plociniczak
021ff2b40e
Mark some tests as flaky on Windows (#8446) 2023-12-06 11:17:34 +00:00
Hubert Plociniczak
a11bddcb74
Inline execution should support FQNs (#8454)
* Test illustrating problems with FQNs

Inline execution fails with `Compile error: The name `Standard` could
not be found.`.

* Ensure InlineContext carries Package Repos info

Previously, there was no requirement that inline execution should allow
for FQNs. This meant that the omission of Package Repository info went
unnoticed.

In order to be able to refer to `Standard.Visualization.Preprocessor` it
has to be exported as well.
2023-12-06 10:03:06 +01:00
Jaroslav Tulach
7f0cb88fa1
Consistent simple and qualified type name (#8448)
Fixes #8255 by unifying `get_qualified_type_name` and `get_simple_type_name` implementations.
2023-12-06 04:30:24 +00:00
Dmitry Bushev
98d2221b13
Detach oneshot visualization when the evaluation failed (#8463)
close #8429

Changelog:
- fix: detach oneshot visualization in case when the evaluation was failed
2023-12-05 16:15:12 +00:00
Pavel Marek
a67297aebf
Add graalpy packages to the component directory (#8351)
Adds these JAR modules to the `component` directory inside Engine distribution:
- `graal-language-23.1.0`
- `org.bouncycastle.*` - these need to be added for graalpy language

# Important Notes
- Remove `org.bouncycastle.*` packages from `runtime.jar` fat jar.
- Make sure that the `./run` script preinstalls GraalPy standalone distribution before starting engine tests
- Note that using `python -m venv` is only possible from standalone distribution, we cannot distribute `graalpython-launcher`.
- Make sure that installation of `numpy` and its polyglot execution example works.
- Convert `Text` to `TruffleString` before passing to GraalPy - 8ee9a2816f
2023-12-04 11:50:59 +00:00
Hubert Plociniczak
5b38aff294
Part II of making sure that tests clean up after each run (#8432)
* Ensure runtime tests cleanup after run

Context is closed, maps/lists are cleaned up.

* DRY
2023-12-04 10:10:01 +01:00
Jaroslav Tulach
2f676963da
Properly report errors on duplicated constructor names (#8438) 2023-12-01 17:30:34 +01:00
Jaroslav Tulach
b1be8c0faa
Multi line chained operator syntax (#8415) 2023-12-01 11:48:37 +01:00
Jaroslav Tulach
65daaf6f0c
Replacing usage of Source by (Location => String) (#8437) 2023-12-01 08:41:24 +01:00
Jaroslav Tulach
81f06456bf
400x faster with linear hashing of the hash map entries (#8425)
Fixes #5233 by removing `EconomicMap` & co. and using plain old good _linear hashing_. Fixes #8090 by introducing `StorageEntry.removed()` rather than copying the builder on each removal.
2023-12-01 06:43:13 +00:00
Hubert Plociniczak
b6bdf901a9
Cleanup after each test run (#8418)
Reducing leaks when running our test suite.

Potentially fixes #8408.

# Important Notes
Managed to keep thread count and memory in between runs relatively stable.
Initially:
![Screenshot from 2023-11-29 11-06-04](https://github.com/enso-org/enso/assets/292128/af437d8a-9111-4bd6-9033-a59030c7ebed)
Now:
![Screenshot from 2023-11-29 15-57-34](https://github.com/enso-org/enso/assets/292128/3c1f8aef-fe7e-4f5b-a236-12c86ea8b906)

The screenshot illustrates for `RuntimeVisualizationsTest` only. Will need to be applied in other places as well.

Applying the same style to `runtime-with-instruments`.
Before:
![Screenshot from 2023-11-29 16-55-11](https://github.com/enso-org/enso/assets/292128/385e11be-0265-431d-b0d7-c5096df11c50)
After:
![Screenshot from 2023-11-29 16-50-07](https://github.com/enso-org/enso/assets/292128/5e8c28ea-e921-484a-a82c-9f2d3e827e8b)
2023-11-29 19:02:59 +00:00
Jaroslav Tulach
f1cd51ab6c
Convert constants to Enso values (#8416) 2023-11-29 14:57:45 +01:00
Jaroslav Tulach
c889c8e83f
More robust work with caches (#8393) 2023-11-28 09:03:15 +01:00
Hubert Plociniczak
0402e8bafb
Ensure compilation is run with a compilation lock (#8395)
Evaluating visualization expression may trigger a full compilation. A change in #7042 went a bit too far and led to a situation when there could be compilations running at the same time leading to a rather obscure `RedefinedMethodException` when the compilation on one thread already finished. This will make the logic correct again at the price of potentially slowing the processing of visualization.

Closes #8296.

# Important Notes
Should make visualizations a bit more stable as well.
2023-11-27 14:01:23 +00:00
Jaroslav Tulach
7a9a5ba1ff
Rewriting MetadataStorage into Java (#8366) 2023-11-27 13:55:40 +01:00
Jaroslav Tulach
893965ed5c
3% speedup with LazyMap and MetadataStorage (#8359) 2023-11-27 10:28:12 +01:00
Hubert Plociniczak
36996c8938
Don't NPE on op ._ when translating tree to IR (#8381)
Encountered a random NPE when playing with bookclubs. Test case demonstrating the problem is attached.

Threw in a bunch of minor tweaks to logs to make life of the person debugging code more pleasant.
2023-11-23 19:56:56 +00:00
Dmitry Bushev
f1825f3f32
Gather diagnostics from annotations (#8380)
close #8289

Changelog
- feat: `GatherDiagnostics` compiler pass works with annotations
2023-11-23 17:18:39 +00:00
Dmitry Bushev
4b3ba78b52
Add shortcuts to start and stop the backend profiling (#8358)
close #8329

Changelog:
- add: `cmd`+`shift`+`,` and `cmd`+`shift`+`.` shortcuts to start and stop the backend profiling. Profiling data is stored on disk.
2023-11-23 15:31:17 +00:00
Pavel Marek
a9099ddce5
Fix Java log initialization in std-table (#8364) 2023-11-23 14:10:22 +01:00
Hubert Plociniczak
af5be19be8
Minor tweaks post jdk21 upgrade (#8373) 2023-11-23 11:58:49 +00:00
Pavel Marek
268e595ec1
Add Chrome devtools and DAP tools for debugging (#8344)
Adds chrome-inspector tool and Debug Adapter protocol tool for debugging Enso.

# Important Notes
The chrome devtools seems to be broken (tracked in https://github.com/oracle/graal/issues/7636). Even `graalpy --inspect ...` fails (Chrome devtools freezes after connecting to the server). This PR adds Debug adapter protocol tool for debugging as a workaround for chrome inspector. There is docs in [dap.md](https://github.com/enso-org/enso/pull/8344/files#diff-421574b50574cfe546e86d4b3d32d79b8b2087f2fe204f68e5cf2693af43bbe1)
2023-11-22 17:18:41 +00:00