Fixes#9749 by:
- [x] Adding `fn` option to `enso-debug-server` instrument - eb3b76e
- [x] Print warnings (if any) to stderr - 4fda04b
- [x] Improving output of `:list` to print out warnings - dbe3c45480
- [x] Print errors to stderr - 1312546
- [x] Exiting on `DataflowError` - 2cc7ef5 and e6fbf73
- [x] Using all of that inside of `runner/*Main` - 7df58ef
The core of the change is in instrumentation that wraps the `main` method and at its end checks for _warnings or errors_ among local variables. When an error is found, it wraps the original return value of `main` with a proxy that delegates to the original value, but also pretends to be _exit exception_ with exit code 173. That one is detected in `Main` launcher to exit the process with exit code 173.
# Important Notes
As a side-effect of this change, one can request an invocation of REPL at the end of any method just by providing a property to the VM:
```bash
$ enso --vm.D=polyglot.enso-debug-server.method-break-point=err_test.main --run err_test.enso --repl
```
stops at the end of `main` method of `err_test.enso` file.
* Add stub of FramePointerAnalysis pass
* Add stub of FramePointerAnalysisTest
* Add more framepointer tests
* FramePointer has constructor for better debugging
* Update tests - FramePointer metadata is also in Info.Def
* FramePointerAnalysis traverses the whole IR.
* Fix test - check for metadata
* Reorder methods in FramePointerAnalysis
* Fix argument processing
* Improve some toString methods for debugging
* Fix parent levels in tests
* FramePointerAnalysis does not copy IRs
* FramePointerAnalysis metadata is Persistable
* FramePointerAnalysis attaches metadata to atom constructors
* Synthetic self argument is handled specially
* IRDumperPass is run as the last compiler pass
This ensure that one can see all the metadata on the IR.
* IRDumper can dump alias analysis metadata
* FramePointerMeta has meaningful toString
* FramePointerAnalysis process case expressions
* Fix javadoc links
* FramePointer is attached to GenericAnnotation expression
* IRDumper handles case type patterns
* FramePointer is attached to argument default value expressions
* Add test for default argument value metadata
* Attach frame pointer to default value expressions even if they have no alias graph
* FramePointerAnalysis can run inline on expression
* Fixes after rebase
* Fix annotation processing
* Add tests for no frame pointers on global symbol usages
* Do not use AliasAnalysis in IrToTruffle.
Also refactor NameResolutionAlgorithm to not use AliasMetadata
* Remove unused methods from LocalScope
* Remove unused import
* fmt
* Fix scala.MatchError for case patterns
* Do not use absolute code locations in the test.
Test on Windows fails because of different length of new lines.
Flatten hierarchy of alias analysis metadata (`or.genso.compiler.pass.alias.Info`) so that it is easily accessible from Java and rename it.
# Important Notes
- Rename `org.enso.compiler.alias.Graph.Occurence` to `org.enso.compiler.alias.graph.GraphOccurence` and flatten the hierarchy.
- Rename `org.enso.compiler.pass.alias.Info` to `org.enso.compiler.pass.alias.AliasMetadata` and flatten the hierarchy.
Majority of warnings handling is now done via newly introduced nodes. Moreover, the underlying representation of warnings storage in `WithWarnings` was changed from `Warning[]` to `EnsoHashMap`.
# Important Notes
- Remove `ArrayRope`.
Working on compiler IR is a daunting task. I have therefore added a new system property `enso.compiler.dumpIr` that will help with that. It dumps the encountered IRs to `ir-dumps` directory in the [GraphViz](www.graphviz.org) format. More info in updated docs.
Note that all the functionality to dump IRs to `dot` files was already implemented. This PR just adds the command line option and updates docs.
# Important Notes
- `--dump-graphs` cmd line option is removed as per [Jaroslav's request](https://github.com/enso-org/enso/pull/10740#pullrequestreview-2216676140).
- To dump graphs, use `-Dgraal.Dump=Truffle:2` system property passed via `JAVA_OPTS` env var.
If you run `env JAVA_OPTS='-Denso.compiler.dumpIr=true' enso --run tmp.enso` where `tmp.enso` is, e.g.:
```
from Standard.Base import all
main = 42
```
You will then have something like:
```
$ ls ir-dumps
Standard.Base.Data.Filter_Condition.dot Standard.Base.Data.Time.dot Standard.Base.System.Advanced.dot Standard.Base.Warning.dot
Standard.Base.Data.Locale.dot Standard.Base.Enso_Cloud.Enso_File.dot Standard.Base.System.File.Advanced.dot tmp.dot
Standard.Base.Data.Numeric.dot Standard.Base.Errors.dot Standard.Base.System.File.dot
Standard.Base.Data.Numeric.Internal.dot Standard.Base.Network.HTTP.Internal.dot Standard.Base.System.File.Generic.dot
Standard.Base.Data.Text.Regex.Internal.dot Standard.Base.Runtime.dot Standard.Base.System.Internal.dot
```
You can then visualize any of these with `dot -Tsvg -O ir-dumps/tmp.dot`.
An example how that could look like is
- Close#10622
- Changes `project-manager` and `ensoup` launcher to run the engine/language-server with working directory set to the directory containing currently running project.
- If the working directory is _not_ "the directory containing currently running project", a warning is written to logs. This can happen if the raw `/bin/enso` engine runner is used in a different directory.
- In the Cloud, the `File.new` interprets relative paths as cloud paths relative to the Cloud directory containing the current project. Absolute paths are unaffected.
There is no need to generate unused variables warnings or other linting for IDE and repl users. By default linting is enabled during compilation and for those use-cases it is now disabled via runtime options.
Closes#9883
The `System.exit 42` component is treated the same way as any other Panic error - it does not interfere with other component evaluation:
After removing the `System.exit 42` component, the workflow works as expected. I have also tried opening the project with the component and then removing it.
Enables `engine.TruffleCompilation` in `std-benchmarks`, collects the logs and dumps compilation into to `System.err` when a benchmark is influenced by dynamic compilation.
- This PR only re-arranges code, splitting the **huge** `processModule` function into a few smaller ones.
- I decided to do it, because when I was working with `processModule` on #9812 I was constantly getting lost in this huge method (this **one** method had 570 lines!) - there is too much happening at once there. Now it's been split into smaller methods, each dealing with one thing.
Reducing the number of log events that we spam regular users. Not the last PR in that area, but already a progress.
Also replaced `fileWalk` with a stream approach hoping that maybe it is a bit more stable on Windows.
Fix#10503 by creating a benchmark and then speeding it up by making sure usage of `InteropLibrary` reminds in partially evaluated code and isn't hidden behind `@TruffleBoundary`.
Addresses one of two concerns of #5298 - adds support for `--jvm` argument to allow us to switch from _native image_ built Enso binary (as developed by #10126) to regular JVM based Enso execution. This change _doesn't affect production builds_. The _native executable_ continues to be only built by `engine-runner/buildNativeImage` which is tested on CI, but not in the production jobs.
- Part of #9486
- Fixing our tests to not rely on deterministic ordering of created Tables in Database backends
- Before, SQLite and Postgres used to mostly return rows in the order they were inserted in, but Snowflake does not.
- Fixing various parts of Snowflake dialect.
Ultimately, we want to forbid the `from ... export all` syntax. This PR starts by providing a way to explicitly export extension and conversion methods by name.
Stdlib code will be modified in upcoming PR.
# Important Notes
A single name can refer to multiple extension or conversion methods. Exports are not qualified. For example,
```
type My_Type
type Other_Type
My_Type.ext_method x = x
Other_Type.ext_method x = x
```
```
from project.Mod export ext_method
```
will export both `My_Type.ext_method` and `Other_Type.ext_method`.
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.
As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.
Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.
In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
close#10182
Changelog:
- add: IdMap parameter to the `text/applyEdit` request
- add: IdMap to the runtime module
- update: set IdMap during the interactive compilation
- update: set the IR identifiers in the `TreeToIR` parsing step
- Fixes#9980
- Adds some tests to ensure types like `|` or `&` (in addition to `!` from the ticket) correctly work in return type check.
- Fixes a weird behaviour where we used to avoid processing type related IR transformations inside of type ascriptions.
- Adds parentheses to type representations if they are more complex: `A | B & C` is unclear as it can either mean `A | (B & C)` or `(A | B) & C` which have different meanings. If we now have an operation with such nesting, the sub expressions are wrapped in parentheses to disambiguate.
Fixes `Standard.Base.Meta.Enso_Project.enso_project` to return a project descriptor for the *main* project, i.e., the one configured as a *root* for the engine.
# Important Notes
`enso_project` builtin no longer iterates the stack frames to infer the project descriptor. It derives it from the default package repository.
- Remove remnants of deprecated Scala parser
- The following projects are now JPMS modules provided on system module-path (in components directory):
- `ydoc-server`
- `profiling-utils`
- `syntax-rust-definition`
- The contents of the aforementioned modules are excluded from both `runner.jar` and `runtime.jar` fat jars.
- Suggestions are serialized and deserialized with our Persistance framework, rather than via the default Java OutputObjectWriter.
Refactored mutable parts of `ModuleScope` into builder to make it easier to reduce unnecessary locks.
# Important Notes
Elements of ModuleScope (types, imports etc) are used while _building_ of it may still be in progress. In order to make static typing happy, every `ModuleScope.Builder` can be exposed as (unmodifiable) `ModuleScope`.
While working on #10056 I realized the names of method and closure nodes are incomprehensible to anyone. This PR replaces the infamous `<anonymous>` with a name hinting where the method actually is.
# Important Notes
I assume this change will be visible not only in IGV, but also in _stacktraces_ and we may need to adjust few tests.
Add support for private methods. Most of the changes are in parser and compiler. The runtime checking of private functions was already present since #9692
# Important Notes
- Only top-level methods can be declared `private`.
- private method cannot be called from different project
- private method cannot be accessed from polyglot code (private method does not exist for polyglot code)
- related #7954
Changelog:
- update: Ydoc starts with the language server on the `localhost:1234` by default. The hostname and ports can be configured by setting environment variables `LANGUAGE_SERVER_YDOC_HOSTNAME` and `LANGUAGE_SERVER_YDOC_PORT`
- update: by default `npm dev run` uses the node Ydoc server. You can control it with `POLYGLOT_YDOC_SERVER` env variable. For example,
```
env POLYGLOT_YDOC_SERVER='true' npm --workspace=enso-gui2 run dev
```
To connect to the Ydoc server running on the 1234 port (the one started with the language server)
⠀
```
env POLYGLOT_YDOC_SERVER='ws://127.0.0.1:1235' npm --workspace=enso-gui2 run dev
```
To connect to the provided URL. Can be useful for debugging when you start a separate Ydoc process.
- update: run `npm install` before the engine build. It is required to create the Ydoc JS bundle.
close#9172
Changelog:
- update: insert `Empty` IR node for the empty method definition bodies
- update: generate node resulting in `Nothing` for `Empty` IR nodes
- refactor: remove redundant checked exceptions in `EnsoParserTest`
Fixes#8735 by making sure compilation errors contain `SourceSection` identifying location where they occurred. This behavior is required by Truffle TCK. The TCK allocates its own `Context` without any other configuration and still requires a syntax error to be reported. Thus changing the default mode to _strict errors_.
In certain cases, when the `action` of `Panic.catch` is tail-call-optimized (via `@Tail_Call`) annotation, the panic is not caught. Fixed by ensuring that the `action` of `Panic.catch` is executed as `NOT_TAIL` rather than `TAIL_DIRECT`.
# Important Notes
The `handler` parameter of `Panic.catch` is executed as `NOT_TAIL` as well, just to be sure.
Closes#8836.
Atom constructors can be declared as private (project-private). project-private constructors can be called only from the same project. See the encapsulation.md docs for more info.
---------
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
Resolves#9607 by computing `Number.hash` by converting given number to `Float` first and then computing the hash. Also the conversion from `Float.to Decimal` is exact - done via `new BigDecimal(double)`. There is `Decimal.new` that handles the user-friendly conversion. However as a result `Decimal.from 2.1 != Decimal.new 2.1` - that's the only way to ensure consistency between hash code and conversions.
While investigating #9749 a JavaScript call to `Polyglot.eval("enso", ....).eval_expression("id")` was made. It crashed as JavaScript isn't using `String` but `TruffleString` to represent strings.
1. Experimenting with invalidating modules' indexes without requiring full write-context locks. That should significantly improve the execution.
2. Improving performance by making background job executor run in a larger threadpool than 1.
`42 == (Error.throw "foo")` now correctly returns an `Error` rather than False
# Important Notes
The error was in the wrong usage of the `org.enso.interpreter.dsl.AcceptsError` DSL annotation.
close#9351
Changelog:
- update: deprecate the `reexport` suggestion field
- add: `reexports` suggestion field containing the list of modules re-exporting this symbol
- update: exports logic to gather all the symbols exported from a given module
`ExecuteJob` can now be interrupted.
We now have a separate threadpool for visualization-related jobs.
# Important Notes
In a lock step situation, a job or command could have been interrupted while waiting for one of the locks. As locks ensured only that they were released once all of them have been acquired this could leave engine in a broken state.
Once `ExecuteJob` could be interrupted this became a blocker as it prevented project startup almost in every case.
The change also makes it careful to avoid constant `ExecuteJob` restarts.
Addresses #9278. There will be follow up work.
- Closes#9300
- Now the Enso libraries are themselves capable of refreshing the access token, thus there is no more problems if the token expires during a long running workflow.
- Adds `get_optional_field` sibling to `get_required_field` for more unified parsing of JSON responses from the Cloud.
- Adds `expected_type` that checks the type of extracted fields. This way, if the response is malformed we get a nice Enso Cloud error telling us what is wrong with the payload instead of a `Type_Error` later down the line.
- Fixes `Test.expect_panic_with` to actually catch only panics. Before it used to also handle dataflow errors - but these have `.should_fail_with` instead. We should distinguish these scenarios.
