Commit Graph

1366 Commits

Author SHA1 Message Date
Jaroslav Tulach
8e765953a4
Detect outdated launcher (#10779) 2024-08-16 07:37:04 +00:00
James Dunkerley
422fa8c16b
Adding support for creating Atoms in expressions (#10820)
- Enables the `..` autoscoping style for creating Atoms in expressions.
- Add type checking to methods in columns.
- Auto wrap returns from method in expressions into a column as needed.
- Remove `Time_Period.Day` to remove confusion..
2024-08-15 15:52:30 +00:00
GregoryTravis
e836373d9b
Mixed Decimal/Float operations throw error or attach warning (#10725) 2024-08-14 16:45:28 +00:00
Hubert Plociniczak
f8eb922c9d
Cache self argument (#10785)
In order for widgets not to invalidate expression's results and trigger computations, we now cache self argument to which visualizations should be attached to.

It should help with #10730 but there is still a bug in GUI.
2024-08-14 10:43:33 +00:00
Radosław Waśko
fc3ac6ced1
Various fixes to visualizations (#10745)
- Closes #10716 by un-marking the `Snowflake_Connection` as `private`.
- Does the same to `SQLServer_Connection`.
- Makes sure that `Snowflake_Connection` has correct visualization and enables links for it.
- Adds a fallback in `to_default_visualization_data` and in Table's `prepare_visualization` so that if `to_js_object` of a given type is malformed, we still get some kind of visualization + debug info about the underlying error.
2024-08-13 15:20:55 +00:00
Pavel Marek
3ff5281722
Add --vm.D cmdline option (#10798)
* Add --vm.D cmdline option

* EngineMainTest uses TemporaryFolder rule

* Test system properties set

* Add note about ignored --jvm option

* Ignore --vm.D options in --jvm mode
2024-08-13 13:28:43 +02:00
Hubert Plociniczak
ff7e31c237
Reducing memory leaks in runtime-integration-tests (#10793)
Inspired by the revert done in https://github.com/enso-org/enso/pull/10778, started looking into apparent memory leaks in `runtime-integration-tests`, written in Java.

Initial state:
![Screenshot from 2024-08-09 14-36-29](https://github.com/user-attachments/assets/39abd48f-503b-49d8-af97-da051352c70d)

After:
![Screenshot from 2024-08-12 16-33-40](https://github.com/user-attachments/assets/2bf4cc2d-7e0e-4d22-8810-c2e7e5c3b065)

# Important Notes
Some remaining issues:
- [ ] [TCK tests](https://github.com/enso-org/enso/tree/develop/engine/runtime-integration-tests/src/test/java/org/enso/interpreter/test/tck) appear to have some memory leaks but we are essentially enabling them via simple inheritance
- [ ] [RuntimeManagementTest](https://github.com/enso-org/enso/blob/develop/engine/runtime-integration-tests/src/test/scala/org/enso/interpreter/test/semantic/RuntimeManagementTest.scala) appears to be broken as it doesn't seem to shutdown properly created threads:
![Screenshot from 2024-08-12 16-29-09](https://github.com/user-attachments/assets/d90aca62-0562-4287-88b7-6d4719e5cf50)

Leaving this for now, as it will probably need to be taken care by initial authors of those tests, if possible. Plus this PR leaves tests in a much better state than before.
2024-08-13 08:01:31 +00:00
Pavel Marek
a79a393ed7
Flatten object hierarchy of AliasAnalysis metadata so it is accessible from Java (#10780)
Flatten hierarchy of alias analysis metadata (`or.genso.compiler.pass.alias.Info`) so that it is easily accessible from Java and rename it.

# Important Notes
- Rename `org.enso.compiler.alias.Graph.Occurence` to `org.enso.compiler.alias.graph.GraphOccurence` and flatten the hierarchy.
- Rename `org.enso.compiler.pass.alias.Info` to `org.enso.compiler.pass.alias.AliasMetadata` and flatten the hierarchy.
2024-08-12 15:03:41 +00:00
Hubert Plociniczak
0f5a3a78b1
Revert "Import all available libraries in --repl mode (#10746)" (#10778)
* Revert "Import all available libraries in --repl mode (#10746)"

This reverts commit 71285e6ff8.

* trigger build
2024-08-09 14:38:05 +02:00
Jaroslav Tulach
8de4b8c9a5
Appending warning to DataflowError is noop (#10765) 2024-08-09 08:39:31 +00:00
Pavel Marek
c2c6712b77
WithWarnings uses EnsoHashMap to speed things up (#10555)
Majority of warnings handling is now done via newly introduced nodes. Moreover, the underlying representation of warnings storage in `WithWarnings` was changed from `Warning[]` to `EnsoHashMap`.

# Important Notes
- Remove `ArrayRope`.
2024-08-07 15:29:57 +00:00
Jaroslav Tulach
71285e6ff8
Import all available libraries in --repl mode (#10746)
Continuation of REPL work (#10709): Import all available libraries when running `--repl`.
2024-08-07 07:36:39 +00:00
Kaz Wesley
e47bdd3e17
Implement full new-lambda syntax (#10756)
Implement full `ArgumentDefinition` syntax for new-lambda arguments, e.g `\a=1 (b:Integer = 23)-> a + b`; add backend support for new lambdas.

Emit an error when any syntactic operator is used outside of its associated syntax (fixes #10473).

Phase out complex arguments for old-lambdas: It is now a syntax error to specify default arguments for an old-lambda. This capability had no usage in real code; affected tests have been updated to test new lambdas. For now, old lambdas can continue to be used with simple arguments; if default arguments are desired, a new-style lambda can be used.
2024-08-06 17:02:32 +00:00
Pavel Marek
f0de43a970
Add org.enso.compiler.dumpIr system prop (#10740)
Working on compiler IR is a daunting task. I have therefore added a new system property `enso.compiler.dumpIr` that will help with that. It dumps the encountered IRs to `ir-dumps` directory in the [GraphViz](www.graphviz.org) format. More info in updated docs.

Note that all the functionality to dump IRs to `dot` files was already implemented. This PR just adds the command line option and updates docs.

# Important Notes
- `--dump-graphs` cmd line option is removed as per [Jaroslav's request](https://github.com/enso-org/enso/pull/10740#pullrequestreview-2216676140).
- To dump graphs, use `-Dgraal.Dump=Truffle:2` system property passed via `JAVA_OPTS` env var.

If you run `env JAVA_OPTS='-Denso.compiler.dumpIr=true' enso --run tmp.enso` where `tmp.enso` is, e.g.:
```
from Standard.Base import all
main = 42
```
You will then have something like:
```
$ ls ir-dumps
Standard.Base.Data.Filter_Condition.dot     Standard.Base.Data.Time.dot              Standard.Base.System.Advanced.dot       Standard.Base.Warning.dot
Standard.Base.Data.Locale.dot               Standard.Base.Enso_Cloud.Enso_File.dot   Standard.Base.System.File.Advanced.dot  tmp.dot
Standard.Base.Data.Numeric.dot              Standard.Base.Errors.dot                 Standard.Base.System.File.dot
Standard.Base.Data.Numeric.Internal.dot     Standard.Base.Network.HTTP.Internal.dot  Standard.Base.System.File.Generic.dot
Standard.Base.Data.Text.Regex.Internal.dot  Standard.Base.Runtime.dot                Standard.Base.System.Internal.dot
```
You can then visualize any of these with `dot -Tsvg -O ir-dumps/tmp.dot`.

An example how that could look like is
![image.svg](https://github.com/user-attachments/assets/26ab8415-72cf-46da-bc63-f475e9fa628e)
2024-08-06 12:00:27 +00:00
Kaz Wesley
aafdef1aeb
Improve parser contextualization (#10734) 2024-08-05 15:46:58 +00:00
Radosław Waśko
4b35e0a99e
Propagate FS errors when creating a new project (#10708)
- Part of #10662
2024-08-05 10:58:03 +00:00
Radosław Waśko
9b2f611402
Relative paths are relative to current project locally and in Cloud (#10660)
- Close #10622
- Changes `project-manager` and `ensoup` launcher to run the engine/language-server with working directory set to the directory containing currently running project.
- If the working directory is _not_ "the directory containing currently running project", a warning is written to logs. This can happen if the raw `/bin/enso` engine runner is used in a different directory.
- In the Cloud, the `File.new` interprets relative paths as cloud paths relative to the Cloud directory containing the current project. Absolute paths are unaffected.
2024-07-31 09:43:17 +00:00
Hubert Plociniczak
d43ad7ce13
Disable linting passes for IDE and repl (#10705)
There is no need to generate unused variables warnings or other linting for IDE and repl users. By default linting is enabled during compilation and for those use-cases it is now disabled via runtime options.

Closes #9883
2024-07-31 08:49:45 +00:00
Jaroslav Tulach
b9214f052c
Must specify --repl to enable debug server (#10709) 2024-07-29 18:29:55 +02:00
Jaroslav Tulach
07bc728b6d
Avoid race condition in GC_Example test (#10665)
Re-enables https://github.com/enso-org/enso/pull/10602/files#r1690919173 - uses `IO.println` to allow us to see what the CI actually does.
2024-07-29 09:37:21 +00:00
Jaroslav Tulach
cb72487cc9
ContextFactory reused to initialize language-server context (#10670) 2024-07-29 09:49:14 +02:00
Hubert Plociniczak
73cb5d1dd7
Don't report stacktraces for interruptions (#10698)
Interruptions are OK now and shouldn't be reported as a warning.
This is confusing to devs.
2024-07-28 16:41:53 +02:00
Dmitry Bushev
9e19009158
When reopening a project no dynamic widgets available (#10674)
close #10595

Changelog:
- fix: storing IdMap from the request
- update: apply updated external ids after loading the module from cache
- update: simplify the node invalidation logic

# Important Notes
https://github.com/user-attachments/assets/c93df161-3bc5-4ecf-8696-239a2498dfb8
2024-07-27 08:21:05 +00:00
GregoryTravis
f0e9616d39
Implement Runtime.Context.Dataflow_Stack_Trace for dataflow errors thrown from Enso (#9625) 2024-07-26 19:20:41 +00:00
Hubert Plociniczak
0f055c7fb3
Additional info to debug AccessDenied problems (#10663)
Plain `Storage failure [AccessDenied].` was rather uninformative when it comes to debugging the underlying problem.
Added more detailed error messages and runners' failures should now sometimes print a detailed message.
References #10662.
2024-07-26 16:01:44 +00:00
Pavel Marek
f849634db3
Atom fields are visible in debugger (#10661)
Atom fields are now visible in the debugger (both chrome inspector and VSCode's debug adapter protocol):
![image](https://github.com/user-attachments/assets/c3d19475-c271-46b6-a44e-e9aebf414b8d)
2024-07-26 06:54:39 +00:00
Dmitry Bushev
2e56ffa63a
More detailed logs for locking logic (#10656)
related: #10595

Locking log message now always includes the caller class.
2024-07-25 12:10:12 +00:00
Kaz Wesley
8b48637691
Provide syntax warnings to Java (#10645)
Translate syntax warnings and attach to IR when translating operator applications.

We should ensure that all Trees are checked for warnings and every warning is attached to some IR. That would require a bit of refactoring: In TreeToIr, we could define helpers wrapping every IR constructor and accepting a `Tree` parameter. The `Tree` could be used to populate the `IdentifiedLocation` when constructing the IR type, and then to attach all warnings after constructing the IR object.

# Important Notes
- Update JNI dependency.
- Introduces a `cargo bench` runner for parser.
2024-07-24 17:54:23 +00:00
Jaroslav Tulach
c6da3a93ba
Treat WithWarnings via dedicated @Specialization (#10650) 2024-07-24 15:31:27 +02:00
Kaz Wesley
e5b85bf16e
Space-precedence does not apply to value-level operators (#10597)
In a sequence of value-level operators, whitespace does not affect relative precedence. Functional operators still follow the space-precedence rules.

The "functional" operators are: `>> << |> |>> <| <<| : .`, application, and any operator containing `<-` or `->`. All other operators are considered value-level operators.

Asymmetric whitespace can still be used to form *operator sections* of value-level operators, e.g. `+2 * 3` is still equivalent to `x -> (x+2) * 3`.

Precedence of application is unchanged, so `f x+y` is still equivalent to `f (x + y)` and `f x+y * z` is still equivalent to `(f (x + y)) * z`.

Any attempt to use spacing to override value-level operator precedence will be caught by the new enso linter. Mixed spacing (for clarity) in value-operator expressions is allowed, as long as it is consistent with the precedences of the operators.

Closes #10366.

# Important Notes
Precedence warnings:
- The parser emits a warning if the whitespace in an expression is inconsistent with its effective precedence.
- A new enso linter can be run with `./run libraries lint`. It parses all `.enso` files in `distribution/lib` and `test`, and reports any errors or warnings. It can also be run on individual files: `cargo run --release --bin check_syntax -- file1 file2...` (the result may be easier to read than the `./run` output).
- The linter is also run as part of `./run lint`, so it is checked in CI.

Additional language change:
- The exponentiation operator (`^`) now has higher precedence than the multiplication class (`*`, `/`, `%`). This change did not affect any current enso files.

Library changes:
- The libraries have been updated. The new warnings were used to identify all affected code; the changes themselves have not been programmatically verified (in many cases their equivalence relies on the commutativity of string concatenation).
2024-07-24 10:55:44 +00:00
Hubert Plociniczak
aae8370977
Disable Graal-based Ydoc temporarily (#10627)
Graal Ydoc implementation is currently not being used locally or in the cloud and giving an impression of a slower startup.
Plus it appears that there some issues in the local connection as well.
To limit the impact of it now, let's make it controllable by the same env variable as GUI is.
2024-07-23 12:29:26 +00:00
Adam Obuchowicz
71bae7e4b0
initProtocol endpoint returns success when already initialized (#10542)
* initProtocol endpoint returns success when already initialized

* Fix test

* Docs and return error when clientId differs

* fmt

* fix: session management test

* misc: json connection controller

---------

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>

Issue with Vector nothing to do with this.
2024-07-23 07:45:53 +01:00
Jaroslav Tulach
b6bbfc5cda
Prevent re-entrant execution of finalizers (#10602)
Fixes #10211 by avoiding re-entrant execution of finalizers.
2024-07-22 20:11:54 +00:00
Jaroslav Tulach
c0f5759e07
Speeding up Ordering_Comparator (#10593) 2024-07-19 04:08:53 +02:00
Pavel Marek
451d7cb452
System.exit does proper context hard exit. (#10363)
The `System.exit 42` component is treated the same way as any other Panic error - it does not interfere with other component evaluation:
![image](https://github.com/user-attachments/assets/516490b5-755f-453e-8dc9-744437dc51bd)

After removing the `System.exit 42` component, the workflow works as expected. I have also tried opening the project with the component and then removing it.
2024-07-18 18:10:36 +00:00
Jaroslav Tulach
c20eab2af9
Detect compilation while benchmarking (#10574)
Enables `engine.TruffleCompilation` in `std-benchmarks`, collects the logs and dumps compilation into to `System.err` when a benchmark is influenced by dynamic compilation.
2024-07-18 15:49:16 +00:00
Kaz Wesley
4cff789b69
Refactor precedence for whitespace changes (#10569)
Single-phase whitespace-aware precedence resolution.

#### Performance
![newplot(4)](https://github.com/user-attachments/assets/9822b0dc-17c3-4d2d-adf7-eb8b1c240522)
Since this is a major refactor of the core of the parser, I benchmarked it; it's about 3% faster.

# Important Notes
- Move operator-identifier recognition to lexer.
- Move compound-token assembly out of precedence resolver
2024-07-18 15:48:20 +00:00
Radosław Waśko
4e310d723d
Refactor: split IrToTruffle::processModule into smaller sub-methods (#10587)
- This PR only re-arranges code, splitting the **huge** `processModule` function into a few smaller ones.
- I decided to do it, because when I was working with `processModule` on #9812 I was constantly getting lost in this huge method (this **one** method had 570 lines!) - there is too much happening at once there. Now it's been split into smaller methods, each dealing with one thing.
2024-07-18 12:14:11 +00:00
Hubert Plociniczak
700a638f1e
Ignore ProcessBuilder logs (#10578) 2024-07-17 21:38:16 +00:00
Hubert Plociniczak
39898da664
Reduce the number of log INFO events (#10500)
Reducing the number of log events that we spam regular users. Not the last PR in that area, but already a progress.
Also replaced `fileWalk` with a stream approach hoping that maybe it is a bit more stable on Windows.
2024-07-17 14:29:48 +00:00
Jaroslav Tulach
a33f632a73
Make EnsoOutputStream fast again! (#10559)
Will fix #10550 by properly using `@TruffleBoundary` annotation
2024-07-16 17:30:56 +00:00
Jaroslav Tulach
c367e76e44
Renaming launcher executable to ensoup (#10535)
Closes #10476.

# Important Notes
Let's see what exactly fails on the CI and fix that then...
2024-07-16 14:30:23 +00:00
Hubert Plociniczak
a992c8aa4f
Fix context lock removal (#10273)
* Fix context lock removal

Removal of context lock assumed that one still holds a lock on it. This
is no longer the case when using a `withContextLock` block that
correctly manages the resource. This change fixes the
`IllegalMonitorStateException`.
Closes #10354.

* address review

* Add missing file

* Hiding ContextLock internals from other than ReentrantLocking classes

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2024-07-16 09:58:03 +02:00
Hubert Plociniczak
e585904291
Mitigate Language Server visualizations DOS scenario on startup (#10549)
* Mitiigate LS DDOS scenario on startup

For a medium-size project with a lot of visualizations, Language Server
will be flooded with visualization requests on startup. Due to an
existing limit for the job execution engine, some of those requests
might have been silently dropped.
This change lifts that limit until a better fix can be invented.

Additionally, a slow startup might lead to a timeout when serving open
file request. This change adds some retries as a fallback
mechanism/progress monitoring.

* add runtime-fat-jar to a list of aggregates
2024-07-15 16:45:07 +02:00
Pavel Marek
79a1fdbed5
Add ExportImportResolutionBenchmark (#10043)
Vast majority of CPU time in ExportsResolution is spent in [BindingsMap.SymbolRestriction.optimize](9a37357247/engine/runtime-compiler/src/main/scala/org/enso/compiler/phase/ExportsResolution.scala (L173)). #10369 dealt with this. This PR only adds the `ExportImportResolutionBenchmark`.

# Important Notes
- Introduce new [ExportImportResolutionBenchmark](9e70f675d8/engine/runtime-benchmarks/src/main/java/org/enso/compiler/benchmarks/exportimport/ExportImportResolutionBenchmark.java) that measures the performance of import and export resolution only.
- Note that the already existing [ImportStandardLibrariesBenchmark](4d49b00375/engine/runtime-benchmarks/src/main/java/org/enso/compiler/benchmarks/module/ImportStandardLibrariesBenchmark.java) is probably fine for the purpose, but I just wanted to be sure that **ONLY** the import/export resolution is measured and nothing else, so I have isolated that into a new benchmark.
2024-07-15 10:38:05 +00:00
Jaroslav Tulach
aaa3495546
engine-runner shall not depend on language-server (#10536) 2024-07-12 17:17:07 +02:00
Pavel Marek
0f9852aab2
Replace all from ... export all with explicit exports (#10369)
Replace all exports with explicit exports.
2024-07-11 19:34:25 +02:00
Jaroslav Tulach
077b86f98c
Speed up EnsoInputStream again (#10515)
Fix #10503 by creating a benchmark and then speeding it up by making sure usage of `InteropLibrary` reminds in partially evaluated code and isn't hidden behind `@TruffleBoundary`.
2024-07-11 10:08:26 +00:00
Jaroslav Tulach
220b40a1cd
Enforce conversion method return type & introduce Comparable.new (#10468) 2024-07-11 06:58:51 +02:00
Adam Obuchowicz
60c1a0e1f6
Record control tweaks (#10509)
Fixes #10388

[Screencast from 2024-07-10 13-55-46.webm](https://github.com/enso-org/enso/assets/3919101/4ad7c039-84f6-4e42-aad9-5e287ccd88bb)

# Important Notes
Also, removed unnecessary  setExecutionEnvironment call on startup
2024-07-10 20:19:23 +00:00