enso/project/GraalVM.scala

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

192 lines
6.7 KiB
Scala
Raw Normal View History

import sbt.Keys._
import sbt._
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
import sbt.internal.util.ManagedLogger
import sbt.io.IO
import sbt.librarymanagement.{ConfigurationFilter, DependencyFilter}
import scala.collection.immutable.Seq
/** A collection of utility methods for everything related to the GraalVM and Truffle.
*/
object GraalVM {
/** Has the user requested to use Espresso for Java interop? */
private def isEspressoMode(): Boolean =
"espresso".equals(System.getenv("ENSO_JAVA"))
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// Keep in sync with graalMavenPackagesVersion in build.sbt
private val version: String = "24.0.0"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
/** The list of modules that are included in the `component` directory in engine distribution.
* When invoking the `java` command, these modules need to be put on the module-path.
*/
val modules: Seq[ModuleID] = Seq(
"org.graalvm.sdk" % "nativeimage" % version,
"org.graalvm.sdk" % "word" % version,
"org.graalvm.sdk" % "jniutils" % version,
"org.graalvm.sdk" % "collections" % version,
"org.graalvm.polyglot" % "polyglot" % version,
"org.graalvm.truffle" % "truffle-api" % version,
"org.graalvm.truffle" % "truffle-runtime" % version,
"org.graalvm.truffle" % "truffle-compiler" % version
)
private val sdkPkgs = Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.sdk" % "polyglot-tck" % version,
"org.graalvm.sdk" % "nativeimage" % version,
"org.graalvm.sdk" % "word" % version,
"org.graalvm.sdk" % "jniutils" % version,
"org.graalvm.sdk" % "collections" % version
)
private val polyglotPkgs = Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.polyglot" % "polyglot" % version
)
private val trufflePkgs = Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.truffle" % "truffle-api" % version,
"org.graalvm.truffle" % "truffle-runtime" % version,
"org.graalvm.truffle" % "truffle-compiler" % version,
"org.graalvm.truffle" % "truffle-dsl-processor" % version
)
/** Manually maintained GraalVM languages and their dependencies. Optimally,
* we would use 'org.graalvm.polyglot:js-community' or 'org.graavm.polyglot:python-community'
* maven artifacts and all their transitive dependencies, but we have to copy all these artifacts
* into engine distribution build, so we have to maintain these manually.
*/
private val pythonPkgs =
Seq(
"org.graalvm.python" % "python-language" % version,
"org.graalvm.python" % "python-resources" % version,
"org.bouncycastle" % "bcutil-jdk18on" % "1.76",
"org.bouncycastle" % "bcpkix-jdk18on" % "1.76",
"org.bouncycastle" % "bcprov-jdk18on" % "1.76",
"org.graalvm.llvm" % "llvm-api" % version,
"org.graalvm.truffle" % "truffle-nfi" % version,
"org.graalvm.truffle" % "truffle-nfi-libffi" % version,
"org.graalvm.regex" % "regex" % version,
"org.graalvm.tools" % "profiler-tool" % version,
"org.graalvm.shadowed" % "json" % version,
"org.graalvm.shadowed" % "icu4j" % version,
"org.graalvm.shadowed" % "xz" % version
)
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val jsPkgs =
Seq(
"org.graalvm.js" % "js-language" % version,
"org.graalvm.regex" % "regex" % version,
"org.graalvm.shadowed" % "icu4j" % version
)
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val chromeInspectorPkgs = Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.tools" % "chromeinspector-tool" % version,
"org.graalvm.shadowed" % "json" % version,
"org.graalvm.tools" % "profiler-tool" % version
)
private val debugAdapterProtocolPkgs = Seq(
"org.graalvm.tools" % "dap-tool" % version
)
val insightPkgs = Seq(
"org.graalvm.tools" % "insight-tool" % version,
"org.graalvm.tools" % "insight-heap-tool" % version
)
private val espressoPkgs =
Seq(
"org.graalvm.truffle" % "truffle-nfi" % version,
"org.graalvm.truffle" % "truffle-nfi-libffi" % version,
"org.graalvm.espresso" % "espresso-language" % version,
"org.graalvm.espresso" % "espresso-libs-resources-linux-amd64" % version,
"org.graalvm.espresso" % "espresso-runtime-resources-linux-amd64" % version
)
val toolsPkgs = chromeInspectorPkgs ++ debugAdapterProtocolPkgs ++ insightPkgs
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val langsPkgs =
if (isEspressoMode()) {
espressoPkgs
} else {
jsPkgs ++ pythonPkgs
}
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
Add some engine jobs that run with Oracle GraalVM (#9322) Adds `Oracle GraalVM` configuration for some backend jobs. `Oracle GraalVM` jobs run only on Linux so far. The old jobs use `GraalVM CE`. ### Important Notes - The JDK to download and use is deduced from the `JAVA_VENDOR` environment variable. By default, `GraalVM CE` is used. - sbt can be started with both GraalVM CE and Oracle GraalVM without any warnings. - If you try to start sbt with JDK from a different vendor, but with the same Java version, a warning is printed. Current list of jobs in the `Engine CI` workflow (these jobs are visible on this PR, because they are scheduled to run on every PR): - Engine (GraalVM CE) (linux, x86_64) - Engine (GraalVM CE) (macos, x86_64) - Engine (GraalVM CE) (windows, x86_64) - **Engine (Oracle GraalVM) (linux, x86_64)** - Scala Tests (GraalVM CE) (linux, x86_64) - Scala Tests (GraalVM CE) (macos, x86_64) - Scala Tests (GraalVM CE) (windows, x86_64) - **Scala Tests (Oracle GraalVM) (linux, x86_64)** - Standard Library Tests (GraalVM CE) (linux, x86_64) - Standard Library Tests (GraalVM CE) (macos, x86_64) - Standard Library Tests (GraalVM CE) (windows, x86_64) - **Standard Library Tests (Oracle GraalVM) (linux x86_64)** - Verify License Packages (linux, x86_64) Benchmark Engine workflow (not visible on this PR, cannot schedule manually yet): - Benchmark Engine (GraalVM CE) - **Benchmark Engine (Oracle GraalVM)** Benchmark Standard Libraries workflow (not visible on this PR, cannot schedule manually yet): - Benchmark Standard Libraries (GraalVM CE) - **Benchmark Standard Libraries (Oracle GraalVM)**
2024-03-12 22:25:26 +03:00
private val allowedJavaVendors = Seq(
"GraalVM Community",
"Oracle Corporation"
)
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
/** Augments a state transition to do GraalVM version check.
*
* @param graalVersion the GraalVM version that should be used for
* building this project
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
* @param graalPackagesVersion Version of Truffle and GraalVM packages that
* will be downloaded from Maven
* @param javaVersion Version of the Java source code
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
* @return an augmented state transition that does all the state changes of
* oldTransition but also runs the version checks
*/
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
def versionCheck(
graalVersion: String,
graalPackagesVersion: String,
javaVersion: String,
oldState: State
): State = {
val log = oldState.log
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
if (graalPackagesVersion != version) {
log.error(
s"Expected GraalVM packages version $version, but got $graalPackagesVersion. " +
s"Version specified in build.sbt and GraalVM.scala must be in sync"
)
return oldState.fail
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
}
val javaVendor = System.getProperty("java.vendor")
Add some engine jobs that run with Oracle GraalVM (#9322) Adds `Oracle GraalVM` configuration for some backend jobs. `Oracle GraalVM` jobs run only on Linux so far. The old jobs use `GraalVM CE`. ### Important Notes - The JDK to download and use is deduced from the `JAVA_VENDOR` environment variable. By default, `GraalVM CE` is used. - sbt can be started with both GraalVM CE and Oracle GraalVM without any warnings. - If you try to start sbt with JDK from a different vendor, but with the same Java version, a warning is printed. Current list of jobs in the `Engine CI` workflow (these jobs are visible on this PR, because they are scheduled to run on every PR): - Engine (GraalVM CE) (linux, x86_64) - Engine (GraalVM CE) (macos, x86_64) - Engine (GraalVM CE) (windows, x86_64) - **Engine (Oracle GraalVM) (linux, x86_64)** - Scala Tests (GraalVM CE) (linux, x86_64) - Scala Tests (GraalVM CE) (macos, x86_64) - Scala Tests (GraalVM CE) (windows, x86_64) - **Scala Tests (Oracle GraalVM) (linux, x86_64)** - Standard Library Tests (GraalVM CE) (linux, x86_64) - Standard Library Tests (GraalVM CE) (macos, x86_64) - Standard Library Tests (GraalVM CE) (windows, x86_64) - **Standard Library Tests (Oracle GraalVM) (linux x86_64)** - Verify License Packages (linux, x86_64) Benchmark Engine workflow (not visible on this PR, cannot schedule manually yet): - Benchmark Engine (GraalVM CE) - **Benchmark Engine (Oracle GraalVM)** Benchmark Standard Libraries workflow (not visible on this PR, cannot schedule manually yet): - Benchmark Standard Libraries (GraalVM CE) - **Benchmark Standard Libraries (Oracle GraalVM)**
2024-03-12 22:25:26 +03:00
if (!allowedJavaVendors.contains(javaVendor)) {
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
log.warn(
s"Running on non-GraalVM JVM (The actual java.vendor is $javaVendor). " +
Add some engine jobs that run with Oracle GraalVM (#9322) Adds `Oracle GraalVM` configuration for some backend jobs. `Oracle GraalVM` jobs run only on Linux so far. The old jobs use `GraalVM CE`. ### Important Notes - The JDK to download and use is deduced from the `JAVA_VENDOR` environment variable. By default, `GraalVM CE` is used. - sbt can be started with both GraalVM CE and Oracle GraalVM without any warnings. - If you try to start sbt with JDK from a different vendor, but with the same Java version, a warning is printed. Current list of jobs in the `Engine CI` workflow (these jobs are visible on this PR, because they are scheduled to run on every PR): - Engine (GraalVM CE) (linux, x86_64) - Engine (GraalVM CE) (macos, x86_64) - Engine (GraalVM CE) (windows, x86_64) - **Engine (Oracle GraalVM) (linux, x86_64)** - Scala Tests (GraalVM CE) (linux, x86_64) - Scala Tests (GraalVM CE) (macos, x86_64) - Scala Tests (GraalVM CE) (windows, x86_64) - **Scala Tests (Oracle GraalVM) (linux, x86_64)** - Standard Library Tests (GraalVM CE) (linux, x86_64) - Standard Library Tests (GraalVM CE) (macos, x86_64) - Standard Library Tests (GraalVM CE) (windows, x86_64) - **Standard Library Tests (Oracle GraalVM) (linux x86_64)** - Verify License Packages (linux, x86_64) Benchmark Engine workflow (not visible on this PR, cannot schedule manually yet): - Benchmark Engine (GraalVM CE) - **Benchmark Engine (Oracle GraalVM)** Benchmark Standard Libraries workflow (not visible on this PR, cannot schedule manually yet): - Benchmark Standard Libraries (GraalVM CE) - **Benchmark Standard Libraries (Oracle GraalVM)**
2024-03-12 22:25:26 +03:00
s"Expected Java vendors: ${allowedJavaVendors.mkString(", ")}."
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
)
}
val javaSpecVersion = System.getProperty("java.specification.version")
if (javaSpecVersion != javaVersion) {
log.error(
s"Running on Java version $javaSpecVersion. " +
s"Expected Java version $javaVersion."
)
return oldState.fail
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
}
val vmVersion = System.getProperty("java.vm.version")
tryParseJavaVMVersion(vmVersion) match {
case Some(version) =>
if (version != graalVersion) {
log.error(
s"Running on GraalVM version $version. " +
s"Expected GraalVM version $graalVersion."
)
oldState.fail
} else {
oldState
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
}
case None =>
log.error(
s"Could not parse GraalVM version from java.vm.version: $vmVersion."
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
)
oldState.fail
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
}
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
}
private def tryParseJavaVMVersion(
version: String
): Option[String] = {
if (version.contains('+')) {
Some(version.split('+')(0))
} else {
None
}
}
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
}