While investigating behavior of
```
sbt:std-benchmarks> withDebug benchOnly --dumpGraphs -- Vector_Operations.Max_Stat
```
in IGV I realized there is a deep chain of nodes when reading an element of `Vector` related to work with warnings. There is an invocation of `WarningsLibrary` on `this` - that's probably unnecessary as we know how it is going to resolve. This PR skips such one level of indirection by directly delegating to `this.storage`.
However I haven't seen any effect of this change on peak performance. The library overhead seems to disappear. Anyway I wanted to bring this finding to your attention and perform independent measurement on our CI server.
Fixes#7213 by reacting to new `isBigInteger` and `asBigInteger` messages. Adjusts tests to require `BigInteger` when appropriate. `EnsoBigInteger` no longer extends `Number`. Enso tests to come.
- Closes#5951
- Ensures any SQL warnings reported by the database through the JDBC driver are processed and forwarded to the user.
- These warnings show issues like the implicit name truncation that this PR is also solving. It's good to make sure they are visible as they can help avoid and understand unexpected problems. They should not show up in most standard workflows.
- Adds simple history to our REPL.
Fixes#7279 by detecting missing `getClose()` and yielding an `UnclosedTextLiteral`.
# Important Notes
Special care must be taken for _text blocks_. They have `null` `getClose()`.
- Previous GraalVM update: https://github.com/enso-org/enso/pull/6750
Removed warnings:
- Remove deprecated `ConditionProfile.createCountingProfile()`.
- Add `@Shared` to some `@Cached` parameters (Truffle now emits warnings about potential `@Share` usage).
- Specialization method names should not start with execute
- Add limit attribute to some specialization methods
- Add `@NeverDefault` for some cached initializer expressions
- Add `@Idempotent` or `@NonIdempotent` where appropriate
BigInteger and potential Node inlining are tracked in follow-up issues.
# Important Notes
For `SDKMan` users:
```
sdk install java 17.0.7-graalce
sdk use java 17.0.7-graalce
```
For other users - download link can be found at https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-17.0.7
Release notes: https://www.graalvm.org/release-notes/JDK_17/
R component was dropped from the release 23.0.0, only `python` is available to install via `gu install python`.
This PR does three related things:
- Fails more gracefully when a non-string is passed to compile_regex
- Don't pass a non-string to compile_regex
- Allow a Regex param to parse_to_table
This PR modifies the builtin method processor such that it forbids arrays of non-primitive and non-guest objects in builtin methods. And provides a proper implementation for the builtin methods in `EnsoFile`.
- Remove last `to_array` calls from `File.enso`
Less dependencies on `EnsoContext` - `Module` shall exist without it. `Module` is a result of a `Compiler` and shall be created before its execution - e.g. requiring `EnsoContext` (with all its runtime information) is a bit too _demanding_.
# Important Notes
The only reason why `Module` wanted `EnsoContext` was to create its (associated) `Type`. `Type`'s constructor needed a parent type and the code was asking for `Any` from the context. That's unnecessary at creation time - we can just use some constant (like `null`) and turning it into `Any` during execution. Benchmarks show that there is no slowdown doing so.
On a quest to avoid dependencies on `EnsoContext` from `Compiler`. Step one. The ultimate goal is to move `Compiler` and all its `IRPasses` into a dedicated `runtime/compiler` module that could be used from #7054.
- Removed Array methods: `new`, `copy` and `new_[1234]`.
- New builtins for `Vector.insert`, `Vector.remove` and `Vector.flatten`.
- Replaced `Vector_Builder` use of `Array.copy` to a `Vector.Builder` approach.
Package's config information, once loaded, never changed. While there is typically no need for it, this was problematic when the config became out-of-sync with the filesystem, like in the case of project rename action.
In rename, the config's properties would be updated in the FS, but that would never be reflected in module's package. Therefore further compilations would continue to ask for the old namespace.
Most of the changes are cosmetic (s/`.config`/`.getConfig()`) except for the new `reloadConfig` method on `Package` that is being called in `RenameProjectCmd` handler.
Closes#7062.
# Important Notes
The reported `ExecutionFailed` error should have been mostly fixed already via #7143. This change makes sure that all the related warnings are gone as well and the compiler uses the updated namespace.
Mostly stuff to tidy up the static methods in the CB.
- Remove default pattern from `parse_to_table` (caused IDE to freeze).
- Rename any `_` arguments to what they are.
- Merge `Date.now` into `Date.today`
- Merge the Interval constructors into a single constructor.
- Hide various methods.
Fixes#6955 by:
- using `visualisationModule` to specify the module where the visualization is to be used
- referring to method in `Meta.get_annotation` with `.method_name` - e.g. unresolved symbol notation
- evaluating arguments to `Meta.get_annotation` in the context of the user module (which can access the extension functions)
Partially revert https://github.com/enso-org/enso/pull/6849, which introduced a regression in TCO in the presence of warnings. Rather than modifying the tail call status, `TailCallException` now propagates the extracted warnings and appends them to the final result.
Closes#7093
# Important Notes
Compared to the previous attempt we don't pay the penalty of adding the warnings or even checking for them because it is being dealt in a separate specialization.
Function bodies cannot be instrumented even if the function is right
inside a binding. Consider a scenario when a function is assigned to a
variable and then applied to a `map` method of a really large vector.
The instrumentation will render execution extremely slow.
Alternatively we would still support instrumenting function bodies in
this limited case but take into account the number of times function is
actually called.
Previously, static method calls on `Any` have not worked as expected. For example, `Any.to_text` returned Function instead of Text. That is because the function resolution for `Any.to_text` finds `Any.type.to_text` method on eigentype which expects two `self` arguments, but only one argument is provided.
Note that `Boolean.to_text` worked previously, and returned "Boolean" as expected. This is because the method resolution finds `Any.to_text` method that takes just one `self` argument.
This PR solves this issue by introducing special handling for static method dispatch on `Any`. Simply put, an additional `self` argument is prepended to the argument list.
# Important Notes
A new child node is introduced to `InvokeMethodNode`. This child node is a copy of the current `invokeFunctionNode` with one more `CallArgumentInfo` in its schema.
There is an issue that after a clean build, the compiler is unable to resolve some types, i.e
```py
$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run ~/enso/projects/Unnamed/
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:11: error: The name `Connection_Details` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
| ^~~~~~~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:33: error: The name `Connection_Options` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
| ^~~~~~~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:55: error: The name `Connection` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
| ^~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:24:68: error: The name `SQL_Error` could not be found.
24 | connect : Connection_Details -> Connection_Options -> Connection ! SQL_Error
| ^~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:25:25: error: The name `Connection_Options` could not be found.
25 | connect details options=Connection_Options.Value =
| ^~~~~~~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:29:29: error: The name `Widget` could not be found.
29 | connection_details_widget : Widget
| ^~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:31:28: error: The name `Vector` could not be found.
31 | default_constructors = Vector.from_polyglot_array <|
| ^~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:32:63: error: The name `False` could not be found.
32 | DatabaseConnectionDetailsSPI.get_default_constructors False
| ^~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:36:9: error: The name `Option` could not be found.
36 | Option name code
| ^~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:37:5: error: The name `Single_Choice` could not be found.
37 | Single_Choice display=Display.Always values=choices
| ^~~~~~~~~~~~~
/home/dbushev/projects/luna/enso/built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/lib/Standard/Database/0.0.0-dev/src/Connection/Database.enso:37:27: error: The name `Display` could not be found.
37 | Single_Choice display=Display.Always values=choices
| ^~~~~~~
Aborting due to 11 errors and 0 warnings.
Execution finished with an error: Compilation aborted due to errors.
```
The compiler can't resolve those symbols because the `IR` cache for `Standard.Database.Connection.Database` is missing. What happens during the `buildEngineDistribution` is:
- Compiler processes libraries one by one
- Compiler processes (and compiles and generates caches) `Standard.Database` library
- Compiler starts processing `Standard.Visualization` library
- During the compilation of `Standard.Database.Connection.Database` module, it sees that the module was loaded from the cache. But some of the required dependencies from `Standard.Table` library were not loaded from the cache. But at this point, the `Standard.Table` library has not been processed yet and the caches for it don't exist. The compiler decides that the `Database` file was changed, and the cache is invalid and should be cleaned.
- Removed `module` argument from `enso_project` (new `Project_Description.new` API).
- Removed the custom option from date and time parse/format dropdowns.
- The `format` dropdown uses the value to create the dropdown. (Screenshot below)
- Removed `StorageType` coalescing rules and replaced them with simpler logic in `ObjectStorage`.
- Update signature for `add_row_number` and add aliases.
Request Timeouts started plaguing IDE due to numerous `executionContext/***Visualization` requests. While caused by a bug they revealed a bigger problem in the Language Server when serving large amounts of requests:
1) Long and short lived jobs are fighting for various locks. Lock contention leads to some jobs waiting for a longer than desired leading to unexpected request timeouts. Increasing timeout value is just delaying the problem.
2) Requests coming from IDE are served almost instantly and handled by various commands. Commands can issue further jobs that serve request. We apparently have and always had a single-thread thread pool for serving such jobs, leading to immediate thread starvation.
Both reasons increase the chances of Request Timeouts when dealing with a large number of requests. For 2) I noticed that while we used to set the `enso-runtime-server.jobParallelism` option descriptor key to some machine-dependent value (most likely > 1), the value set would **only** be available for instrumentation. `JobExecutionEngine` where it is actually used would always get the default, i.e. a single-threaded ThreadPool. This means that this option descriptor was simply misused since its introduction. Moved that option to runtime options so that it can be set and retrieved during normal operation.
Adding parallelism intensified problem 1), because now we could execute multiple jobs and they would compete for resources. It also revealed a scenario for a yet another deadlock scenario, due to invalid order of lock acquisition. See `ExecuteJob` vs `UpsertVisualisationJob` order for details.
Still, a number of requests would continue to randomly timeout due to lock contention. It became apparent that
`Attach/Modify/Detach-VisualisationCmd` should not wait until a triggered `UpsertVisualisationJob` sends a response to the client; long and short lived jobs will always compete for resources and we cannot guarantee that they will not timeout that way. That is why the response is sent immediately from the command handler and not from the job executed after it.
This brings another problematic scenario:
1. `AttachVisualisationCmd` is executed, response sent to the client, `UpsertVisualisationJob` scheduled.
2. In the meantime `ModifyVisualisationCmd` comes and fails; command cannot find the visualization that will only be added by `UpsertVisualisationJob`, which might have not yet been scheduled to run.
Remedied that by checking visualisation-related jobs that are still in progress. It also allowed for cancelling jobs which results wouldn't be used anyway (`ModifyVisualisationCmd` sends its own `UpsertVisualisationJob`). This is not a theoretical scenario, it happened frequently on IDE startup.
This change does not fully solve the rather problematic setup of numerous locks, which are requested by short and long lived jobs. A better design should still be investigated. But it significantly reduces the chances of Request Timeouts which IDE had to deal with.
With this change I haven't been able to experience Request Timeouts for relatively modest projects anymore.
I added the possibility of logging wait times for locks to better investigate further problems.
Closes#7005
- Add type detection for `Mixed` columns when calling column functions.
- Excel uses column name for missing headers.
- Add aliases for parse functions on text.
- Adjust `Date`, `Time_Of_Day` and `Date_Time` parse functions to not take `Nothing` anymore and provide dropdowns.
- Removed built-in parses.
- All support Locale.
- Add support for missing day or year for parsing a Date.
- All will trim values automatically.
- Added ability to list AWS profiles.
- Added ability to list S3 buckets.
- Workaround for Table.aggregate so default item added works.
