Commit Graph

1006 Commits

Author SHA1 Message Date
Jaroslav Tulach
a6fbdb11ce
Avoid using assert in Scala, use assertInJvm to elide it in production (#11027)
Fixes #11022 and in general fixes #5787 by avoiding calls to Scala's `assert` function and using regular one written in Java that uses `assert` keyword and is thus sensitive to `-ea` command line option of the JVM. Use `assertInJvm` in newly written Scala code to get typical JVM behavior on `assert`.
2024-09-16 06:32:47 +00:00
Jaroslav Tulach
8b5c6338f7
Log details when to_display_text invocation fails (#11025)
Closes #10770 by dumping out more information when `to_display_text` invocations fails.
2024-09-11 03:18:06 +00:00
Dmitry Bushev
ab0a5e0c36
Improve the speed of preorder operation on IR (#11019)
close #10537

Changelog:
- add: implement `IR.preorder` method with callback
- update: update `IR.preorder` method usages

# Important Notes
Shows ~10% speed improvement during the compilation
2024-09-10 14:47:50 +00:00
Pavel Marek
19ff2a2bb7
Prepare for JPMS - rename packages (#10974)
* Rename packages in logging-utils-akka

* Migrate buildInfo to Java

* Rename packages in logging

* Rename package in scala-yaml

* No usage of CompilerDirectives inside pkg

* log errors of initialization of directory watcher

* HashCodeNode does not use com.google.common.base.Objects

* Rename rest of the packages

* fmt

* Fix dependencies on version-output

* Add necessary dependencies to testkit

* Rename instruments in runtime-fat-jar module-info

* Fix compilation errors because of BuildVersion

* Fix logger renames

* Use java.util.List directly

* Fixes after merge

* Improve error message in NativeLauncherSpec

* Fix logger renames

* Fix json version formatting

* Revert "No usage of CompilerDirectives inside pkg"

This reverts commit cc7e078416.

* fmt
2024-09-06 10:27:59 +02:00
Jaroslav Tulach
2fc553065f
Get ready for python and js being optional when running test/Base_Tests (#10975)
Let's modify the `Base_Tests` to be able to succeed without JavaScript being installed - **done** in 3298b07dca
2024-09-05 13:00:44 +00:00
Jaroslav Tulach
d37b8f3786
Provide names of local variables via FramePointerAnalysis (#10906) 2024-09-05 10:02:35 +02:00
Jaroslav Tulach
339c275338
Benchmark and speed processing of polyglot java imports up (#10899) 2024-08-29 17:20:53 +02:00
Jaroslav Tulach
1500849c32
When there are no pendingItems for a KEEP_ALIVE period, stop RequestQueue observer thread (#10890) 2024-08-27 10:10:07 +02:00
Jaroslav Tulach
b6f0675563
IO.print without new line (#10858) 2024-08-24 07:00:16 +02:00
Dmitry Bushev
09722b36dc
Provide missing arguments when stepping into a function (#10879)
close #10792

Changelog:
- update: replace missing arguments with `Nothing` when schedule a function execution in interactive mode
2024-08-23 19:13:58 +00:00
Dmitry Bushev
3bfe963e32
Lazy Diagnostic storage allocation (#10852)
close #10727



Changelog:
- add: implement lazy allocation of `DiagnosticStorage` objects
- update: compiler phases to make sure they don't allocate redundant `DiagnosticStorage` objects
- update: cleanup `DiagnosticStorage` class

# Important Notes
As a result, all 10MB of redundant `DiagnosticStorage` allocations are gone

#### Before
![2024-08-21-170148_2092x171_scrot](https://github.com/user-attachments/assets/c1fd34d5-019d-472f-b523-a5c31b87f454)

#### After
![2024-08-21-170058_1693x131_scrot](https://github.com/user-attachments/assets/10d71d81-42b7-4b3c-a49f-ca6267bc6ccf)
2024-08-23 17:48:25 +00:00
Jaroslav Tulach
e5f865f1df
Print out warnings associated with local variables (#10842)
Fixes #9749 by:
- [x] Adding `fn` option to `enso-debug-server` instrument - eb3b76e
- [x] Print warnings (if any) to stderr - 4fda04b
- [x] Improving output of `:list` to print out warnings - dbe3c45480
- [x] Print errors to stderr - 1312546
- [x] Exiting on `DataflowError` - 2cc7ef5 and e6fbf73
- [x] Using all of that inside of `runner/*Main` - 7df58ef

The core of the change is in instrumentation that wraps the `main` method and at its end checks for _warnings or errors_ among local variables. When an error is found, it wraps the original return value of `main` with a proxy that delegates to the original value, but also pretends to be _exit exception_ with exit code 173. That one is detected in `Main` launcher to exit the process with exit code 173.

# Important Notes
As a side-effect of this change, one can request an invocation of REPL at the end of any method just by providing a property to the VM:
```bash
$ enso --vm.D=polyglot.enso-debug-server.method-break-point=err_test.main --run err_test.enso --repl
```
stops at the end of `main` method of `err_test.enso` file.
2024-08-21 07:00:54 +00:00
Jaroslav Tulach
384e01903a
Delay loading of AliasAnalysis data (#10837)
Continuation of #10729 and a step towards #10833 to actually speed things up by 10% by delaying loading of `AliasAnalysis` data.
2024-08-19 09:33:05 +00:00
Pavel Marek
53e99809e2
Store FramePointer in IR (#10729)
* Add stub of FramePointerAnalysis pass

* Add stub of FramePointerAnalysisTest

* Add more framepointer tests

* FramePointer has constructor for better debugging

* Update tests - FramePointer metadata is also in Info.Def

* FramePointerAnalysis traverses the whole IR.

* Fix test - check for metadata

* Reorder methods in FramePointerAnalysis

* Fix argument processing

* Improve some toString methods for debugging

* Fix parent levels in tests

* FramePointerAnalysis does not copy IRs

* FramePointerAnalysis metadata is Persistable

* FramePointerAnalysis attaches metadata to atom constructors

* Synthetic self argument is handled specially

* IRDumperPass is run as the last compiler pass

This ensure that one can see all the metadata on the IR.

* IRDumper can dump alias analysis metadata

* FramePointerMeta has meaningful toString

* FramePointerAnalysis process case expressions

* Fix javadoc links

* FramePointer is attached to GenericAnnotation expression

* IRDumper handles case type patterns

* FramePointer is attached to argument default value expressions

* Add test for default argument value metadata

* Attach frame pointer to default value expressions even if they have no alias graph

* FramePointerAnalysis can run inline on expression

* Fixes after rebase

* Fix annotation processing

* Add tests for no frame pointers on global symbol usages

* Do not use AliasAnalysis in IrToTruffle.

Also refactor NameResolutionAlgorithm to not use AliasMetadata

* Remove unused methods from LocalScope

* Remove unused import

* fmt

* Fix scala.MatchError for case patterns

* Do not use absolute code locations in the test.

Test on Windows fails because of different length of new lines.
2024-08-16 14:28:37 +02:00
Jaroslav Tulach
8e765953a4
Detect outdated launcher (#10779) 2024-08-16 07:37:04 +00:00
GregoryTravis
e836373d9b
Mixed Decimal/Float operations throw error or attach warning (#10725) 2024-08-14 16:45:28 +00:00
Hubert Plociniczak
ff7e31c237
Reducing memory leaks in runtime-integration-tests (#10793)
Inspired by the revert done in https://github.com/enso-org/enso/pull/10778, started looking into apparent memory leaks in `runtime-integration-tests`, written in Java.

Initial state:
![Screenshot from 2024-08-09 14-36-29](https://github.com/user-attachments/assets/39abd48f-503b-49d8-af97-da051352c70d)

After:
![Screenshot from 2024-08-12 16-33-40](https://github.com/user-attachments/assets/2bf4cc2d-7e0e-4d22-8810-c2e7e5c3b065)

# Important Notes
Some remaining issues:
- [ ] [TCK tests](https://github.com/enso-org/enso/tree/develop/engine/runtime-integration-tests/src/test/java/org/enso/interpreter/test/tck) appear to have some memory leaks but we are essentially enabling them via simple inheritance
- [ ] [RuntimeManagementTest](https://github.com/enso-org/enso/blob/develop/engine/runtime-integration-tests/src/test/scala/org/enso/interpreter/test/semantic/RuntimeManagementTest.scala) appears to be broken as it doesn't seem to shutdown properly created threads:
![Screenshot from 2024-08-12 16-29-09](https://github.com/user-attachments/assets/d90aca62-0562-4287-88b7-6d4719e5cf50)

Leaving this for now, as it will probably need to be taken care by initial authors of those tests, if possible. Plus this PR leaves tests in a much better state than before.
2024-08-13 08:01:31 +00:00
Pavel Marek
a79a393ed7
Flatten object hierarchy of AliasAnalysis metadata so it is accessible from Java (#10780)
Flatten hierarchy of alias analysis metadata (`or.genso.compiler.pass.alias.Info`) so that it is easily accessible from Java and rename it.

# Important Notes
- Rename `org.enso.compiler.alias.Graph.Occurence` to `org.enso.compiler.alias.graph.GraphOccurence` and flatten the hierarchy.
- Rename `org.enso.compiler.pass.alias.Info` to `org.enso.compiler.pass.alias.AliasMetadata` and flatten the hierarchy.
2024-08-12 15:03:41 +00:00
Hubert Plociniczak
0f5a3a78b1
Revert "Import all available libraries in --repl mode (#10746)" (#10778)
* Revert "Import all available libraries in --repl mode (#10746)"

This reverts commit 71285e6ff8.

* trigger build
2024-08-09 14:38:05 +02:00
Jaroslav Tulach
8de4b8c9a5
Appending warning to DataflowError is noop (#10765) 2024-08-09 08:39:31 +00:00
Pavel Marek
c2c6712b77
WithWarnings uses EnsoHashMap to speed things up (#10555)
Majority of warnings handling is now done via newly introduced nodes. Moreover, the underlying representation of warnings storage in `WithWarnings` was changed from `Warning[]` to `EnsoHashMap`.

# Important Notes
- Remove `ArrayRope`.
2024-08-07 15:29:57 +00:00
Jaroslav Tulach
71285e6ff8
Import all available libraries in --repl mode (#10746)
Continuation of REPL work (#10709): Import all available libraries when running `--repl`.
2024-08-07 07:36:39 +00:00
Pavel Marek
f0de43a970
Add org.enso.compiler.dumpIr system prop (#10740)
Working on compiler IR is a daunting task. I have therefore added a new system property `enso.compiler.dumpIr` that will help with that. It dumps the encountered IRs to `ir-dumps` directory in the [GraphViz](www.graphviz.org) format. More info in updated docs.

Note that all the functionality to dump IRs to `dot` files was already implemented. This PR just adds the command line option and updates docs.

# Important Notes
- `--dump-graphs` cmd line option is removed as per [Jaroslav's request](https://github.com/enso-org/enso/pull/10740#pullrequestreview-2216676140).
- To dump graphs, use `-Dgraal.Dump=Truffle:2` system property passed via `JAVA_OPTS` env var.

If you run `env JAVA_OPTS='-Denso.compiler.dumpIr=true' enso --run tmp.enso` where `tmp.enso` is, e.g.:
```
from Standard.Base import all
main = 42
```
You will then have something like:
```
$ ls ir-dumps
Standard.Base.Data.Filter_Condition.dot     Standard.Base.Data.Time.dot              Standard.Base.System.Advanced.dot       Standard.Base.Warning.dot
Standard.Base.Data.Locale.dot               Standard.Base.Enso_Cloud.Enso_File.dot   Standard.Base.System.File.Advanced.dot  tmp.dot
Standard.Base.Data.Numeric.dot              Standard.Base.Errors.dot                 Standard.Base.System.File.dot
Standard.Base.Data.Numeric.Internal.dot     Standard.Base.Network.HTTP.Internal.dot  Standard.Base.System.File.Generic.dot
Standard.Base.Data.Text.Regex.Internal.dot  Standard.Base.Runtime.dot                Standard.Base.System.Internal.dot
```
You can then visualize any of these with `dot -Tsvg -O ir-dumps/tmp.dot`.

An example how that could look like is
![image.svg](https://github.com/user-attachments/assets/26ab8415-72cf-46da-bc63-f475e9fa628e)
2024-08-06 12:00:27 +00:00
Radosław Waśko
9b2f611402
Relative paths are relative to current project locally and in Cloud (#10660)
- Close #10622
- Changes `project-manager` and `ensoup` launcher to run the engine/language-server with working directory set to the directory containing currently running project.
- If the working directory is _not_ "the directory containing currently running project", a warning is written to logs. This can happen if the raw `/bin/enso` engine runner is used in a different directory.
- In the Cloud, the `File.new` interprets relative paths as cloud paths relative to the Cloud directory containing the current project. Absolute paths are unaffected.
2024-07-31 09:43:17 +00:00
Hubert Plociniczak
d43ad7ce13
Disable linting passes for IDE and repl (#10705)
There is no need to generate unused variables warnings or other linting for IDE and repl users. By default linting is enabled during compilation and for those use-cases it is now disabled via runtime options.

Closes #9883
2024-07-31 08:49:45 +00:00
Jaroslav Tulach
07bc728b6d
Avoid race condition in GC_Example test (#10665)
Re-enables https://github.com/enso-org/enso/pull/10602/files#r1690919173 - uses `IO.println` to allow us to see what the CI actually does.
2024-07-29 09:37:21 +00:00
Jaroslav Tulach
cb72487cc9
ContextFactory reused to initialize language-server context (#10670) 2024-07-29 09:49:14 +02:00
Dmitry Bushev
9e19009158
When reopening a project no dynamic widgets available (#10674)
close #10595

Changelog:
- fix: storing IdMap from the request
- update: apply updated external ids after loading the module from cache
- update: simplify the node invalidation logic

# Important Notes
https://github.com/user-attachments/assets/c93df161-3bc5-4ecf-8696-239a2498dfb8
2024-07-27 08:21:05 +00:00
GregoryTravis
f0e9616d39
Implement Runtime.Context.Dataflow_Stack_Trace for dataflow errors thrown from Enso (#9625) 2024-07-26 19:20:41 +00:00
Pavel Marek
f849634db3
Atom fields are visible in debugger (#10661)
Atom fields are now visible in the debugger (both chrome inspector and VSCode's debug adapter protocol):
![image](https://github.com/user-attachments/assets/c3d19475-c271-46b6-a44e-e9aebf414b8d)
2024-07-26 06:54:39 +00:00
Jaroslav Tulach
c6da3a93ba
Treat WithWarnings via dedicated @Specialization (#10650) 2024-07-24 15:31:27 +02:00
Jaroslav Tulach
b6bbfc5cda
Prevent re-entrant execution of finalizers (#10602)
Fixes #10211 by avoiding re-entrant execution of finalizers.
2024-07-22 20:11:54 +00:00
Jaroslav Tulach
c0f5759e07
Speeding up Ordering_Comparator (#10593) 2024-07-19 04:08:53 +02:00
Pavel Marek
451d7cb452
System.exit does proper context hard exit. (#10363)
The `System.exit 42` component is treated the same way as any other Panic error - it does not interfere with other component evaluation:
![image](https://github.com/user-attachments/assets/516490b5-755f-453e-8dc9-744437dc51bd)

After removing the `System.exit 42` component, the workflow works as expected. I have also tried opening the project with the component and then removing it.
2024-07-18 18:10:36 +00:00
Jaroslav Tulach
c20eab2af9
Detect compilation while benchmarking (#10574)
Enables `engine.TruffleCompilation` in `std-benchmarks`, collects the logs and dumps compilation into to `System.err` when a benchmark is influenced by dynamic compilation.
2024-07-18 15:49:16 +00:00
Radosław Waśko
4e310d723d
Refactor: split IrToTruffle::processModule into smaller sub-methods (#10587)
- This PR only re-arranges code, splitting the **huge** `processModule` function into a few smaller ones.
- I decided to do it, because when I was working with `processModule` on #9812 I was constantly getting lost in this huge method (this **one** method had 570 lines!) - there is too much happening at once there. Now it's been split into smaller methods, each dealing with one thing.
2024-07-18 12:14:11 +00:00
Hubert Plociniczak
39898da664
Reduce the number of log INFO events (#10500)
Reducing the number of log events that we spam regular users. Not the last PR in that area, but already a progress.
Also replaced `fileWalk` with a stream approach hoping that maybe it is a bit more stable on Windows.
2024-07-17 14:29:48 +00:00
Jaroslav Tulach
a33f632a73
Make EnsoOutputStream fast again! (#10559)
Will fix #10550 by properly using `@TruffleBoundary` annotation
2024-07-16 17:30:56 +00:00
Pavel Marek
0f9852aab2
Replace all from ... export all with explicit exports (#10369)
Replace all exports with explicit exports.
2024-07-11 19:34:25 +02:00
Jaroslav Tulach
077b86f98c
Speed up EnsoInputStream again (#10515)
Fix #10503 by creating a benchmark and then speeding it up by making sure usage of `InteropLibrary` reminds in partially evaluated code and isn't hidden behind `@TruffleBoundary`.
2024-07-11 10:08:26 +00:00
Jaroslav Tulach
220b40a1cd
Enforce conversion method return type & introduce Comparable.new (#10468) 2024-07-11 06:58:51 +02:00
Adam Obuchowicz
60c1a0e1f6
Record control tweaks (#10509)
Fixes #10388

[Screencast from 2024-07-10 13-55-46.webm](https://github.com/enso-org/enso/assets/3919101/4ad7c039-84f6-4e42-aad9-5e287ccd88bb)

# Important Notes
Also, removed unnecessary  setExecutionEnvironment call on startup
2024-07-10 20:19:23 +00:00
James Dunkerley
4b3e4ae15e
Rename Map to Dictionary and Set to Hashset. (#10474)
- Rename `Map` to `Dictionary`.
- Rename `Set` to `Hashset`.
- Add a deprecated place holder for the static method of `Map`.
2024-07-09 09:12:23 +00:00
Jaroslav Tulach
515d8238bb
Support for --jvm option in Enso runner (#10374)
Addresses one of two concerns of #5298 - adds support for `--jvm` argument to allow us to switch from _native image_ built Enso binary (as developed by #10126) to regular JVM based Enso execution. This change _doesn't affect production builds_. The _native executable_ continues to be only built by `engine-runner/buildNativeImage` which is tested on CI, but not in the production jobs.
2024-07-06 07:02:20 +00:00
Jaroslav Tulach
489de72828
getPublicTruffleFile may yield an exception (#10380) 2024-06-28 09:58:50 +02:00
Radosław Waśko
db4f7ab3b5
Fixing Database tests and Snowflake Dialect - part 2 out of ... (#10319)
- Part of #9486
- Fixing our tests to not rely on deterministic ordering of created Tables in Database backends
- Before, SQLite and Postgres used to mostly return rows in the order they were inserted in, but Snowflake does not.
- Fixing various parts of Snowflake dialect.
2024-06-27 14:54:00 +00:00
Pavel Marek
9010cf93be
Extension methods can be exported by name (#10274)
Ultimately, we want to forbid the `from ... export all` syntax. This PR starts by providing a way to explicitly export extension and conversion methods by name.

Stdlib code will be modified in upcoming PR.

# Important Notes
A single name can refer to multiple extension or conversion methods. Exports are not qualified. For example,
```
type My_Type
type Other_Type
My_Type.ext_method x = x
Other_Type.ext_method x = x
```
```
from project.Mod export ext_method
```
will export both `My_Type.ext_method` and `Other_Type.ext_method`.
2024-06-25 12:08:22 +00:00
Jaroslav Tulach
fe2cf49568
Run whole test/Base_Tests in native image runner (#10296) 2024-06-21 06:03:53 +02:00
Hubert Plociniczak
114b3a5c5e
Cleaning up YAML parsing in preparation for circe-yaml removal (#10309)
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.

As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.

Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.

In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
2024-06-20 13:07:54 +00:00
Dmitry Bushev
04a92ef765
TextEdit with custom IdMap (#10283)
close #10182

Changelog:
- add: IdMap parameter to the `text/applyEdit` request
- add: IdMap to the runtime module
- update: set IdMap during the interactive compilation
- update: set the IR identifiers in the `TreeToIR` parsing step
2024-06-15 18:03:18 +00:00