Commit Graph

198 Commits

Author SHA1 Message Date
Michał Wawrzyniec Urbańczyk
90bbee352e
Bump Rust Toolchain (#9517)
This PR updates the Rust toolchain to recent nightly.

Most of the changes are related to fixing newly added warnings and adjusting the feature flags. Also the formatter changed its behavior slightly, causing some whitespace changes.

Other points:
* Changed debug level of the `buildscript` profile to `lint-tables-only` — this should improve the build times and space usage somewhat.
* Moved lint configuration to the worksppace `Cargo.toml` definition. Adjusted the formatter appropriately.
* Removed auto-generated IntelliJ run configurations, as they are not useful anymore.
* Added a few trivial stdlib nightly functions that were removed to our codebase.
* Bumped many dependencies but still not all:
* `clap` bump encountered https://github.com/clap-rs/clap/issues/5407 — for now the warnings were silenced by the lint config.
* `octocrab` — our forked diverged to far with the original, needs more refactoring.
* `derivative` — is unmaintained and has no updated version, despite introducing warnings in the generated code. There is no direct replacement.
2024-03-24 23:45:55 +00:00
Michał Wawrzyniec Urbańczyk
c983d081d9
[CI] Split GUI tests into a separate workflow (#9507)
As discussed in the team.
2024-03-21 20:42:58 +00:00
Michał Wawrzyniec Urbańczyk
de9f2764f9
Remove ensogl-pack (#9407)
This PR removes enso-pack (ensogl-pack) crate.
It still keeps the `enso-runner` JS package, as it is used for CLI argument parser and logger. The runner should be probably refactored (and possible removed altogether).

# Important Notes
I've temporarily extracted the `enso-runner` to `lib/js` directory, as I wanted to avoid keeping pure JS library under `lib/rust`. Attempts at integrating this with `app/ide-desktop` and family caused too much trouble for this PR. The expectation is that the package will be removed or moved elsewhere soon anyway.
2024-03-18 12:18:18 +00:00
Pavel Marek
c4029eed4f
dry-run benchmarks exits when some benchmark fails (#9397)
If some benchmark fails in dry-run (compileOnly) mode, the whole process exits with non-zero return code. Also fixes failing engine compiler benchmarks.

# Important Notes
Manually added failure:
```diff
diff --git a/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java b/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java
index c8d86cecc..f9f4d7cbc 100644
--- a/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java
+++ b/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java
@@ -95,7 +95,8 @@ public class ArrayProxyBenchmarks {

@Benchmark
public void sumOverComputingProxy(Blackhole matter) {
-    performBenchmark(matter);
+    //performBenchmark(matter);
+    throw new AssertionError("My error");
}

@Benchmark
```
Run with `sbt "-Dbench.compileOnly=true runtime-benchmarks/benchOnly org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy"` fails with:
```
[info] Running benchmarks [org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy] in compileOnly mode
[info] # JMH version: 1.36
[info] # VM version: JDK 21.0.2, Java HotSpot(TM) 64-Bit Server VM, 21.0.2+13-LTS-jvmci-23.1-b30
[info] # VM invoker: /home/pavel/.sdkman/candidates/java/21.0.2-graal/bin/java
[info] # VM options: -XX:ThreadPriorityPolicy=1 -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCIProduct -XX:-UnlockExperimentalVMOptions -Dslf4j.provider=org.slf4j.nop.NOPServiceProvider -Dbench.compileOnly=true --module-path=/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/nativeimage/23.1.2/nativeimage-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/word/23.1.2/word-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/jniutils/23.1.2/jniutils-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/collections/23.1.2/collections-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/polyglot/polyglot/23.1.2/polyglot-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-api/23.1.2/truffle-api-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-runtime/23.1.2/truffle-runtime-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-compiler/23.1.2/truffle-compiler-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/js/js-language/23.1.2/js-language-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/regex/regex/23.1.2/regex-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/shadowed/icu4j/23.1.2/icu4j-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/python/python-language/23.1.2/python-language-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/python/python-resources/23.1.2/python-resources-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcutil-jdk18on/1.76/bcutil-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcpkix-jdk18on/1.76/bcpkix-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcprov-jdk18on/1.76/bcprov-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/llvm/llvm-api/23.1.2/llvm-api-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-nfi/23.1.2/truffle-nfi-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-nfi-libffi/23.1.2/truffle-nfi-libffi-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/tools/profiler-tool/23.1.2/profiler-tool-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/shadowed/json/23.1.2/json-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/2.0.9/slf4j-api-2.0.9.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-nop/2.0.9/slf4j-nop-2.0.9.jar:/home/pavel/dev/enso/runtime.jar --add-modules=org.enso.runtime --add-exports=org.slf4j.nop/org.slf4j.nop=org.slf4j
[info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
[info] # Warmup: <none>
[info] # Measurement: 1 iterations, 1 s each
[info] # Timeout: 10 min per iteration
[info] # Threads: 1 thread, will synchronize iterations
[info] # Benchmark mode: Average time, time/op
[info] # Benchmark: org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy
[info] # Run progress: 0.00% complete, ETA 00:00:01
[info] # Fork: N/A, test runs in the host VM
[info] # *** WARNING: Non-forked runs may silently omit JVM options, mess up profilers, disable compiler hints, etc. ***
[info] # *** WARNING: Use non-forked runs only for debugging purposes, not for actual performance runs. ***
[error] SLF4J: Attempting to load provider "org.slf4j.nop.NOPServiceProvider" specified via "slf4j.provider" system property
[info] Iteration   1: <failure>
[info] java.lang.AssertionError: My error
[info] 	at org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy(ArrayProxyBenchmarks.java:99)
[info] 	at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_avgt_jmhStub(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:232)
[info] 	at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_AverageTime(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:173)
[info] 	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[info] 	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[info] 	at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:475)
[info] 	at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:458)
[info] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[info] 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
[info] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[info] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[info] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] Benchmark run failed: Benchmark caught the exception
[info] 	at java.base/java.lang.Thread.run(Thread.java:1583)
[error] org.openjdk.jmh.runner.RunnerException: Benchmark caught the exception
[error] 	at org.openjdk.jmh.runner.Runner.runBenchmarks(Runner.java:575)
[error] 	at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:310)
[error] 	at org.openjdk.jmh.runner.Runner.run(Runner.java:209)
[error] 	at org.enso.interpreter.bench.BenchmarksRunner.runCompileOnly(BenchmarksRunner.java:93)
[error] 	at org.enso.interpreter.bench.BenchmarksRunner.run(BenchmarksRunner.java:36)
[error] 	at org.enso.interpreter.bench.benchmarks.RuntimeBenchmarksRunner.main(RuntimeBenchmarksRunner.java:8)
[error] Caused by: org.openjdk.jmh.runner.BenchmarkException: Benchmark error during the run
[error] 	at org.openjdk.jmh.runner.BenchmarkHandler.runIteration(BenchmarkHandler.java:424)
[error] 	at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:281)
[error] 	at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:233)
[error] 	at org.openjdk.jmh.runner.BaseRunner.doSingle(BaseRunner.java:138)
[error] 	at org.openjdk.jmh.runner.BaseRunner.runBenchmarksEmbedded(BaseRunner.java:110)
[error] 	at org.openjdk.jmh.runner.Runner.runBenchmarks(Runner.java:555)
[error] 	... 5 more
[error] 	Suppressed: java.lang.AssertionError: My error
[error] 		at org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy(ArrayProxyBenchmarks.java:99)
[error] 		at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_avgt_jmhStub(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:232)
[error] 		at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_AverageTime(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:173)
[error] 		at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
[error] 		at java.base/java.lang.reflect.Method.invoke(Method.java:580)
[error] 		at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:475)
[error] 		at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:458)
[error] 		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] 		at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
[error] 		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
[error] 		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[error] 		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[error] 		at java.base/java.lang.Thread.run(Thread.java:1583)
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 5 s, completed Mar 13, 2024, 12:49:59 PM
```
2024-03-14 15:21:38 +00:00
Michał Wawrzyniec Urbańczyk
d9ca6cf023
Build Script Post-Gui1 Cleanup (#9376)
This PR:
* removes much of logic related to building and packaging the gui1;
* made `./run gui` and `./run ide` work with the new gui;
* rename numerous references to the "gui2" or "new gui" in favor of simply "gui", same for "ide".
2024-03-12 20:25:51 +01:00
Pavel Marek
a5a729a7d6
Add some engine jobs that run with Oracle GraalVM (#9322)
Adds `Oracle GraalVM` configuration for some backend jobs. `Oracle GraalVM` jobs run only on Linux so far. The old jobs use `GraalVM CE`.

### Important Notes

- The JDK to download and use is deduced from the `JAVA_VENDOR` environment variable. By default, `GraalVM CE` is used.
- sbt can be started with both GraalVM CE and Oracle GraalVM without any warnings.
  - If you try to start sbt with JDK from a different vendor, but with the same Java version, a warning is printed.

Current list of jobs in the `Engine CI` workflow (these jobs are visible on this PR, because they are scheduled to run on every PR):
- Engine (GraalVM CE) (linux, x86_64)
- Engine (GraalVM CE) (macos, x86_64)
- Engine (GraalVM CE) (windows, x86_64)
- **Engine (Oracle GraalVM) (linux, x86_64)**
- Scala Tests (GraalVM CE) (linux, x86_64)
- Scala Tests (GraalVM CE) (macos, x86_64)
- Scala Tests (GraalVM CE) (windows, x86_64)
- **Scala Tests (Oracle GraalVM) (linux, x86_64)**
- Standard Library Tests (GraalVM CE) (linux, x86_64)
- Standard Library Tests (GraalVM CE) (macos, x86_64)
- Standard Library Tests (GraalVM CE) (windows, x86_64)
- **Standard Library Tests (Oracle GraalVM) (linux x86_64)**
- Verify License Packages (linux, x86_64)

Benchmark Engine workflow (not visible on this PR, cannot schedule manually yet):
- Benchmark Engine (GraalVM CE)
- **Benchmark Engine (Oracle GraalVM)**

Benchmark Standard Libraries workflow (not visible on this PR, cannot schedule manually yet):
- Benchmark Standard Libraries (GraalVM CE)
- **Benchmark Standard Libraries (Oracle GraalVM)**
2024-03-12 20:25:26 +01:00
Michał Wawrzyniec Urbańczyk
d17d1ed484
[CI] Upload the directory as an artifact, not just its content (#9323) 2024-03-11 17:58:46 +00:00
Michał Wawrzyniec Urbańczyk
e7ca4aaf25
Removed enso-types and a some unused dependencies. (#9342)
Removed `enso-types` crate which had only one reference in unused part of the code. Removed some unused dependencies from `Cargo.toml` files.

# Important Notes
CI has a similar hiccup as before. Please disregard this for now in the review.
2024-03-11 17:44:39 +00:00
somebody1234
d2f6b1026a
Refactor configuration keys out of repository (#8895)
- Close https://github.com/enso-org/cloud-v2/issues/866
- Remove *all* references to client keys and API base URLs from the codebase.
- The app can still be built by external contributors. *However*, the cloud backend (among some other things) will be completely disabled, as the required keys and base URLs will be missing.
- Add entry to `.gitignore` to allow `*.env` files in `app/ide-desktop/lib/dashboard/`

# Important Notes
- Tested (no `.env`; `.env` with prod backend; `.pbuchu.env`) on:
- `npm run dev` in `app/ide-desktop/lib/dashboard/`
- `./run ide build`
- `./run ide2 build`
- `./run gui watch`
2024-03-08 03:14:26 +00:00
Michael Mauderer
7c68bf170d
Remove gui1 codebase (#9242)
Removes the old GUI1 code base and reduces the Rust code footprint by removing unused code.

# Important Notes
Updates build scripts and reformats part of the codebase with the autoformatter.
2024-03-07 02:20:21 +00:00
Michał Wawrzyniec Urbańczyk
e930738e02
[CI] Engine CI Rework, Part 1 (#9295)
I have created PR with the first set of changes for the Engine CI. The changes are small and effectively consist of:
1. Spltting the `verifyLicensePackages`. It is now run only on Linux. There are hardly any time benefits, as the actual job cost is dominated by the overhead of spinning a new job — but it is not expensive in the big picture.
2. Splitting the Scala Tests into separate job. This is probably the biggest "atomic" piece of work we have.
3. Splitting the Standard Library Tests into a separate job.

The time is nicely split across the jobs now. The last run has:
* 27 min for Scala tests;
* 25 min for Standard Library tests;
* 24 min for the "rest": the old job containing everything that has not been split.

While total CPU time has increased (as jobs are not effectively reusing the same build context), the wall time has decreased significantly. Previously we had ~1 hour of wall time for the old monolithic job, so we are getting more than 2x speedup.

The now-slowest Scala tests job is currently comparable with the native Rust tests (and they should improve when the old gui is gone) — which are the slowest job across all CI checks.

The PR is pretty minimal. Several future improvements can be made:
* Reorganizing and splitting other "heavy" jobs, like the native image generation.
* Reusing the built Engine distribution. However, this is probably a lower priority than I initially thought.
* Building package takes several minutes, so duplicating this job is not that expensive.
* The package is OS-specific.
* Scala tests don't really benefit from it, they'd need way more compilation artifacts.
It'd make sense to reuse the distribution if we, for example, decided to split more jobs that actually benefit from it, like Standard Library tests.
* Reusing the Rust build script binary.
* As our self-hosted runners reuse environment, we effectively get this for free. Especially when Rust part of codebase is less frequently changed.
* This is however significant cost for the GitHub-hosted runners, affecting our macOS runners. Reusing the binary does not save wall time for jobs that are run in parallel (as we have enough runners), but if we introduce job dependencies that'd force sequential execution of jobs on macOS, this would be a significant need.
2024-03-06 18:56:13 +00:00
Michał Wawrzyniec Urbańczyk
1a76f6383d
[CI] Checkbox for requesting a clean build when manually dispatching a workflow (#9270)
This PR allows requesting a clean build when triggering the workflow through the manual dispatch.
Previously it was possible only by creating PR and adding the label to it.
2024-03-05 15:40:46 +00:00
Adam Obuchowicz
ba9b7f199a
E2E test fixing and bring back reports (#9238)
After investigating some errors, I found another two missing awaits in our tests. Because those are so easy to overlook, I added a lint rule which makes failure on unhandled promise (for e2e tests only).

Also, enabled HTML reports again, with traces this time, to enable closer investigation of any failure in the future. @mwu-tow added code for uploading them in GH.
2024-03-05 07:06:11 +00:00
Pavel Marek
c2842df8d8
Fix benchmark run build (#9220) 2024-02-29 14:03:13 +01:00
Radosław Waśko
47c64167ef
Clearer warnings in license review (#9134)
- Closes #9120
- Reorders CI steps to do the license check last (to avoid it preventing tests from running which are more important than the license check)
- Tries to reword the warnings to be clearer
- Adds some CSS to the report to more clearly indicate which elements can be clicked.
2024-02-27 16:32:08 +00:00
Pavel Marek
f48caac586
Upload native-image argfiles (#9094)
In PR #8953, in commit ba0a69de6e, I have introduced argument files to the `native-image`. In this PR, let's try to upload these argfiles as artifacts on GH, so that we can inspect them later.
2024-02-26 19:25:37 +00:00
Michał Wawrzyniec Urbańczyk
11a9e8d044
CI: Release links the new IDE (#9110)
This tweaks the release template to link to the new IDE builds.
2024-02-20 19:15:25 +00:00
Michał Wawrzyniec Urbańczyk
2e06b4baa0
CI: disable sbt server autostart, but if it tries to start, force it (#9059) 2024-02-17 11:52:18 +01:00
Michał Wawrzyniec Urbańczyk
64490ec0af
CI: Fix for the runtime docker image build following #8925 (#9044)
Since #8925 `--build-context` flag must be given to build the runtime image. However, it can be used only with BuildKit.
2024-02-13 18:37:40 +00:00
Pavel Marek
5919eda753
Fix incremental compilation of runtime/test (#8975) 2024-02-13 10:05:31 +01:00
Michał Wawrzyniec Urbańczyk
cf19115432
CI: Remove Manual Stdlib Compilation (#9033)
CI currently manually compiles standard library. However, this became redundant since the `buildEngineDistribution` included the very same behavior.
Because of that, we ended up compiling the libraries twice.
2024-02-13 08:50:07 +00:00
Michał Wawrzyniec Urbańczyk
357f2aadd4
CI: Tweaks for backend tests (#9001)
* Use glob pattern to discover stdlib tests (rather than a hardcoded list).
* Don't fail CI check immediately after failing Scala test.
* Remove meta test suite tests.

# Important Notes
The meta test suite tests are removed following the discussion with @radeusgd. In short, these were failing anyway and were supposed to be rewritten (probably using a different technology, like JUnit). The current code will be a useful reference but it doesn't have to be kept on a repository head. The relevant information and references shall be added to the task.
2024-02-12 13:05:41 +00:00
Michał Wawrzyniec Urbańczyk
b5a9ec1f96
CI: Retry npm install in case of failure. (#8992)
Retry running `npm install` in case it fails.

This should help avoiding issues like this one: https://github.com/enso-org/enso/actions/runs/7786527973/job/21231561437?pr=8972#step:8:314
2024-02-07 21:52:14 +00:00
Michał Wawrzyniec Urbańczyk
9083ebfbbc
CI: Retry Artifact Upload Finalization (#8970)
This has been observed to be the most random error-prone part of the Rust build scripts.

This adds several retries to the patching of the artifact size (which finalizes the upload).
Additional diagnostics was added, so we observe if the retries are actually helping, so we can better understand the issue if this is not enough to fix it.
2024-02-05 16:14:11 +00:00
Michał Wawrzyniec Urbańczyk
655ae46b8f
CI: Hotfix aarch64 build (#8944) 2024-02-02 13:21:35 +01:00
Michał Wawrzyniec Urbańczyk
9a2bb19a89
CI: Native aarch64 macOS builds, refactoring workflow generation (#8864)
This PR adds a native aarch64 target to our release process.

It also includes refactoring of workflow generation and minor tweaks:
* removing some workarounds in the generated action code that are not needed anymore;
* some version bumps that are harmless;
* release builds have cleaning enabled unconditionally.
2024-02-01 13:53:50 +01:00
somebody1234
4f7032f966
Add dashboard tests to GUI2 CI; fix dashboard tests (#8922)
- Close #8911
- Add dashboard unit tests to GUI2 CI
- Add dashboard E2E tests to GUI2 CI
- Fix (minor) issues in dashboard unit tests

# Important Notes
None
2024-02-01 11:03:58 +00:00
Michał Wawrzyniec Urbańczyk
06f1c772d8
CI: Use GH Actions concurrency setting to manage cancelling. (#8901) 2024-01-31 12:12:08 +01:00
Michał Wawrzyniec Urbańczyk
0b16db4399
CI: Unconditional cleaning for benchmark runs. (#8839)
Now the `clean` CI steps are run always for benchmarking jobs. We run the full `./run git-clean` before and after benchmarks. Benchmarks take long enough to make any savings by not cleaning negligible.

### Important Notes
This PR brings partial refactoring in the workflow generating code which was very dirty. I'll build on this further soon when adding proper aarch64 macOS support.

Also, some minor tweaks to the generation were made:
* not writing `always() &&` twice;
* run only the latter cleaning step for canceled jobs.
2024-01-29 13:02:02 +01:00
Pavel Marek
d0fdeca6df
Refactor Table_Tests to the builder API (#8622)
Refactor `test/Table_Test` to the builder API. The builder API is in a new library called `Test_New` that is alongside the old `Test` library. There will be follow-up PRs that will migrate the rest of the tests. Meanwhile, let's keep these two libraries, and merge them after the last PR.

# Important Notes
- For a brief introduction into the new API, see **Prototype 1** section in https://github.com/enso-org/enso/pull/8622#issuecomment-1889706168
- When executing all the tests, the behavior should be the same as with the old library. With the only exception that if `ENSO_TEST_ANSI_COLORS` env var is set, the output is more colorful than it used to be.
2024-01-26 12:08:24 +00:00
Michał Wawrzyniec Urbańczyk
32d0459534
CI: Remove bench-report.xml in the whole repo subtree (#8762)
As requested in https://github.com/enso-org/enso/pull/8707#issuecomment-1882512361 and discussed on Discord.
2024-01-18 15:32:00 +00:00
Michał Wawrzyniec Urbańczyk
3c2418ded1
CI: Don't dry-run (check) benchmarks on Windows. (#8790)
As [discussed on Discord](https://discord.com/channels/401396655599124480/1197160815040667779) the dry-run of benchmarks shall be performed only on Linux.

This is meant to avoid CI issues that currently block #[8781](https://github.com/enso-org/enso/pull/8781).
2024-01-17 17:33:42 +00:00
Radosław Waśko
71427c3319
Rename Tests to Base_Tests to be more descriptive (#8781) 2024-01-17 16:19:19 +01:00
Michał Wawrzyniec Urbańczyk
184128949e
CI: Better free port finding (#8780)
Replace our port-finding code with `portpicker` crate.

We expect that it'll greatly reduce possibility of race conditions, as the port will be picked at random, so they won't collide as easily when we use the routine more than once.
2024-01-17 07:54:37 +00:00
somebody1234
da7dcd8bd5
Rename index.js.gz to entrypoint.js.gz (#8757)
This is required because `index.ts` was renamed to `entrypoint.ts` in order to avoid colliding with the main export of the `enso-dashboard` module, which is used by `gui2` as `dashboard.run()`. This file was renamed in #8587, causing the issue.

Build script changes also included thanks to @mwu-tow.

# Important Notes
This should only affect projects opened against the remote (cloud) backend, meaning that to test this, you should open a project against the cloud backend.
2024-01-16 19:10:49 +00:00
Michał Wawrzyniec Urbańczyk
75ebcb0f38
Fix the build error (#8748) 2024-01-12 10:38:22 +01:00
Michał Wawrzyniec Urbańczyk
ed410a8332
CI: don't build native runner on gui workflow (#8726) 2024-01-12 04:59:32 +01:00
Michał Wawrzyniec Urbańczyk
4776a8344d
CI: Enable sanity test for native Runner (#8724)
Co-authored-by: Pavel Marek <pavel.marek@enso.org>
2024-01-12 04:59:07 +01:00
Michał Wawrzyniec Urbańczyk
11510eae2b
CI: Remove old GUI building and packaging from PR checks (#8721) 2024-01-10 20:23:36 +01:00
Radosław Waśko
724f8d2a56
Add tests for Enso Cloud auth + simple API mock for Enso_User (#8511)
- Closes #8354
- Extends `simple-httpbin` with a simple mock of the Cloud API (currently it checks the token and serves the `/users` endpoint).
- Renames `simple-httpbin` to `http-test-helper`.
2023-12-19 17:41:09 +00:00
Pavel Marek
4b65e44ef3
EpbLanguage re-uses other TruffleContext support to run tests with assertions enabled (#7882) 2023-12-15 13:31:32 +01:00
somebody1234
f5c3713f87
Fixes for Enso Font in GUI2 (#8508)
- Fixes issue reported in Discord.

# Important Notes
None
2023-12-13 22:17:12 +00:00
Adam Obuchowicz
be2ae3279c
E2E Tests in CI (#8462)
Fixes #8433 

* Adds E2E test to `test` script in gui2 (without server)
* Add options to `run` script to specify what test should be run: `unit` (default), `e2e`, or `ci` (which runs both unit and e2e without watching/spawning report server).
* The CI test step now checks e2e tests.

### Important Notes

~~One of e2e tests was disabled because it caught the regression on develop: #8476 ~~
2023-12-12 15:27:40 +01:00
Adam Obuchowicz
ee519e979b
Fix package (#8458)
Fixes #8457
2023-12-05 14:40:51 +00:00
Pavel Marek
a67297aebf
Add graalpy packages to the component directory (#8351)
Adds these JAR modules to the `component` directory inside Engine distribution:
- `graal-language-23.1.0`
- `org.bouncycastle.*` - these need to be added for graalpy language

# Important Notes
- Remove `org.bouncycastle.*` packages from `runtime.jar` fat jar.
- Make sure that the `./run` script preinstalls GraalPy standalone distribution before starting engine tests
- Note that using `python -m venv` is only possible from standalone distribution, we cannot distribute `graalpython-launcher`.
- Make sure that installation of `numpy` and its polyglot execution example works.
- Convert `Text` to `TruffleString` before passing to GraalPy - 8ee9a2816f
2023-12-04 11:50:59 +00:00
Michał Wawrzyniec Urbańczyk
a04a2cc849
Build script: git-clean and cwd fixes (#8424) 2023-11-29 20:35:06 +01:00
Michał Wawrzyniec Urbańczyk
8021109d8c
Build script fixes & improvements (#8285) 2023-11-23 21:19:31 +01:00
Pavel Marek
5a7ad6bfe4
Upgrade enso to GraalVM for jdk 21 (#7991)
Upgrade to GraalVM JDK 21.
```
> java -version
openjdk version "21" 2023-09-19
OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15)
OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing)
```

With SDKMan, download with `sdk install java 21-graalce`.

# Important Notes
- After this PR, one can theoretically run enso with any JRE with version at least 21.
- Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution.
- `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded.
- sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368
- Which means that we can have `module-info.java` files only for Java-only projects.
- Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar.
- `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent.
- Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory.
- The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module.
- The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725)
- [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 18:02:36 +00:00
Michał Wawrzyniec Urbańczyk
4b181c38d0
Bump electron-builder on CI macOS runners. (#8284)
This PR avoids #8119 by selectively bumping the `electron-builder` on macOS CI runners. We do this only on macOS, as we do not want to trigger https://github.com/electron-userland/electron-builder/issues/6865 on Windows.
2023-11-16 19:07:29 +00:00
Michał Wawrzyniec Urbańczyk
a9118ee0c3
Use the new notarization tool from Apple (#8192) 2023-11-01 20:58:28 +01:00