enso/build.sbt

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

3328 lines
125 KiB
Plaintext
Raw Normal View History

import LibraryManifestGenerator.BundledLibrary
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
import org.enso.build.BenchTasks.*
2019-09-10 19:53:23 +03:00
import org.enso.build.WithDebugCommand
import org.apache.commons.io.FileUtils
import sbt.Keys.{libraryDependencies, scalacOptions}
import sbt.addCompilerPlugin
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
import sbt.complete.DefaultParsers.*
import sbt.complete.Parser
import sbt.nio.file.FileTreeView
import sbt.internal.util.ManagedLogger
import src.main.scala.licenses.{
DistributionDescription,
SBTDistributionComponent
}
// This import is unnecessary, but bit adds a proper code completion features
// to IntelliJ.
import JPMSPlugin.autoImport.*
2021-02-22 16:32:55 +03:00
import java.io.File
2020-04-30 22:30:55 +03:00
// ============================================================================
// === Global Configuration ===================================================
// ============================================================================
2019-06-07 14:49:47 +03:00
val scalacVersion = "2.13.11"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// source version of the Java language
val javaVersion = "21"
// version of the GraalVM JDK
val graalVersion = "21.0.2"
// Version used for the Graal/Truffle related Maven packages
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// Keep in sync with GraalVM.version. Do not change the name of this variable,
// it is used by the Rust build script via regex matching.
val graalMavenPackagesVersion = "23.1.2"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val targetJavaVersion = "17"
val defaultDevEnsoVersion = "0.0.0-dev"
val ensoVersion = sys.env.getOrElse(
"ENSO_VERSION",
defaultDevEnsoVersion
) // Note [Engine And Launcher Version]
val currentEdition = sys.env.getOrElse(
"ENSO_EDITION",
defaultDevEnsoVersion
) // Note [Default Editions]
// Note [Stdlib Version]
val stdLibVersion = defaultDevEnsoVersion
val targetStdlibVersion = ensoVersion
2020-08-07 12:18:09 +03:00
// Inspired by https://www.scala-sbt.org/1.x/docs/Howto-Startup.html#How+to+take+an+action+on+startup
lazy val startupStateTransition: State => State = { s: State =>
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
GraalVM.versionCheck(
graalVersion,
graalMavenPackagesVersion,
javaVersion,
s
Java and Graal versions are checked before the build starts (#9106) Add checks of Java and GraalVM versions before the `sbt` project is fully loaded. This ensures that all the devs have exactly the version specified in our `build.sbt`. # Important Notes Trying to start `sbt` with a different java versions now results in: ``` $ java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) $ sbt [info] welcome to sbt 1.9.7 (GraalVM Community Java 21) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [error] Running on GraalVM version 21. Expected GraalVM version 21.0.2. [error] Total time: 0 s, completed Feb 20, 2024, 1:06:18 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ``` ``` $ java -version openjdk version "17.0.10" 2024-01-16 OpenJDK Runtime Environment JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17) OpenJDK 64-Bit Server VM JBR-17.0.10+1-1087.17-jcef (build 17.0.10+1-b1087.17, mixed mode) $ sbt [info] welcome to sbt 1.9.7 (JetBrains s.r.o. Java 17.0.10) [info] loading settings for project enso-build from plugins.sbt ... [info] loading project definition from /home/pavel/dev/enso/project [info] compiling 44 Scala sources to /home/pavel/dev/enso/project/target/scala-2.12/sbt-1.0/classes ... [info] loading settings for project enso from build.sbt ... [info] resolving key references (65272 settings) ... [info] set current project to enso (in build file:/home/pavel/dev/enso/) [warn] Running on non-GraalVM JVM (The actual java.vendor is JetBrains s.r.o.). Expected GraalVM Community java.vendor. [error] Running on Java version 17. Expected Java version 21. [error] Total time: 0 s, completed Feb 20, 2024, 1:07:40 PM [warn] Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore? (default: r) ```
2024-02-21 13:33:32 +03:00
)
}
Global / onLoad := {
val old = (Global / onLoad).value
startupStateTransition compose old
}
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
2020-08-07 12:18:09 +03:00
/* Note [Engine And Launcher Version]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Currently both Engine and Launcher versions are tied to each other - each new
* releases contains the Engine and the Launcher and thus the version number is
* shared. If the version numbers ever diverge, make sure to update the build
2020-08-07 12:18:09 +03:00
* scripts at .github/workflows accordingly.
*/
2021-07-08 16:38:20 +03:00
/* Note [Default Editions]
* ~~~~~~~~~~~~~~~~~~~~~~~
* Currently, the default edition to use is inferred based on the engine
* version. Each Enso version has an associated default edition name and the
* `currentEdition` field specifies the default edition name for the upcoming
* release.
*
* Thus the `library-manager` needs to depend on the `version-output` to get
* this defaults from the build metadata.
*
* In the future we may automate generating this edition number when cutting a
* release.
*/
/* Note [Stdlib Version]
* ~~~~~~~~~~~~~~~~~~~~~
* The `stdlibVersion` variable stores the version at which standard library is
* stored within the source tree, which is currently set to a constant of
* `0.0.0-dev`.
*
* When distributions are built, the library versions are updated to match the
* current Enso version.
*/
2021-05-14 15:08:39 +03:00
ThisBuild / organization := "org.enso"
ThisBuild / scalaVersion := scalacVersion
2022-07-01 04:58:14 +03:00
/* Tag limiting the concurrent access to tools/simple-library-server in tests.
*/
val simpleLibraryServerTag = Tags.Tag("simple-library-server")
Global / concurrentRestrictions += Tags.limit(simpleLibraryServerTag, 1)
lazy val gatherLicenses =
taskKey[Unit]("Gathers licensing information for relevant dependencies")
gatherLicenses := {
2021-02-22 16:32:55 +03:00
val _ = GatherLicenses.run.value
}
lazy val verifyLicensePackages =
taskKey[Unit](
"Verifies if the license package has been generated, " +
"has no warnings and is up-to-date with dependencies."
)
verifyLicensePackages := GatherLicenses.verifyReports.value
2020-12-09 16:58:11 +03:00
lazy val verifyGeneratedPackage =
inputKey[Unit](
"Verifies if the license package in a generated distribution is " +
"up-to-date with the one from the report."
)
verifyGeneratedPackage := GatherLicenses.verifyGeneratedPackage.evaluated
2021-07-08 16:38:20 +03:00
def makeStdLibDistribution(
name: String,
components: Seq[SBTDistributionComponent]
): DistributionDescription =
Distribution(
name,
file(s"distribution/lib/Standard/$name/$stdLibVersion/THIRD-PARTY"),
components
)
GatherLicenses.distributions := Seq(
Distribution(
"launcher",
file("distribution/launcher/THIRD-PARTY"),
Distribution.sbtProjects(launcher)
),
Distribution(
"engine",
file("distribution/engine/THIRD-PARTY"),
Distribution.sbtProjects(
runtime,
`engine-runner`,
`language-server`
)
),
2020-12-09 16:58:11 +03:00
Distribution(
"project-manager",
file("distribution/project-manager/THIRD-PARTY"),
Distribution.sbtProjects(`project-manager`)
),
2021-07-08 16:38:20 +03:00
makeStdLibDistribution("Base", Distribution.sbtProjects(`std-base`)),
2021-09-03 22:41:12 +03:00
makeStdLibDistribution(
"Google_Api",
Distribution.sbtProjects(`std-google-api`)
),
2021-07-08 16:38:20 +03:00
makeStdLibDistribution("Table", Distribution.sbtProjects(`std-table`)),
makeStdLibDistribution("Database", Distribution.sbtProjects(`std-database`)),
makeStdLibDistribution("Image", Distribution.sbtProjects(`std-image`)),
makeStdLibDistribution("AWS", Distribution.sbtProjects(`std-aws`))
)
2021-07-08 16:38:20 +03:00
GatherLicenses.licenseConfigurations := Set("compile")
GatherLicenses.configurationRoot := file("tools/legal-review")
lazy val openLegalReviewReport =
taskKey[Unit](
"Gathers licensing information for relevant dependencies and opens the " +
"report in review mode in the browser."
)
openLegalReviewReport := {
val _ = gatherLicenses.value
GatherLicenses.runReportServer()
}
2019-09-12 17:47:25 +03:00
lazy val analyzeDependency = inputKey[Unit]("...")
analyzeDependency := GatherLicenses.analyzeDependency.evaluated
val packageBuilder = new DistributionPackage.Builder(
ensoVersion = ensoVersion,
graalVersion = graalMavenPackagesVersion,
graalJavaVersion = graalVersion,
artifactRoot = file("built-distribution")
)
2019-10-29 17:32:50 +03:00
Global / onChangedBuildSource := ReloadOnSourceChanges
Suppress pointless warnings coming from SBT (#3499) This change introduces a custom LogManager for console that allows for excluding certain log messages. The primarily reason for introducing such LogManager/Appender is to stop issuing hundreds of pointless warnings coming from the analyzing compiler (wrapper around javac) for classes that are being generated by annotation processors. The output looks like this: ``` [info] Cannot install GraalVM MBean due to Failed to load org.graalvm.nativebridge.jni.JNIExceptionWrapperEntryPoints [info] compiling 129 Scala sources and 395 Java sources to /home/hubert/work/repos/enso/enso/engine/runtime/target/scala-2.13/classes ... [warn] Unexpected javac output: warning: File for type 'org.enso.interpreter.runtime.type.ConstantsGen' created in the last round will not be subject to annotation processing. [warn] 1 warning. [info] [Use -Dgraal.LogFile=<path> to redirect Graal log output to a file.] [info] Cannot install GraalVM MBean due to Failed to load org.graalvm.nativebridge.jni.JNIExceptionWrapperEntryPoints [info] foojavac Filer [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.decimal.CeilMethodGen [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.resource.TakeNodeGen [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.error.ThrowErrorMethodGen [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.smallInteger.MultiplyMethodGen [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.warning.GetWarningsNodeGen [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.number.smallInteger.BitAndMethodGen [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.error.ErrorToTextNodeGen [warn] Could not determine source for class org.enso.interpreter.node.expression.builtin.warning.GetValueMethodGen [warn] Could not determine source for class org.enso.interpreter.runtime.callable.atom.AtomGen$MethodDispatchLibraryExports$Cached .... ``` The output now has over 500 of those and there will be more. Much more (generated by our and Truffle processors). There is no way to tell SBT that those are OK. One could potentially think of splitting compilation into 3 stages (Java processors, Java and Scala) but that will already complicate the non-trivial build definition and we may still end up with the initial problem. This is a fix to make it possible to get reasonable feedback from compilation without scrolling mutliple screens *every single time*. Also fixed a spurious warning in javac processor complaining about creating files in the last round. Related to https://www.pivotaltracker.com/story/show/182138198
2022-06-01 16:50:46 +03:00
Global / excludeLintKeys += logManager
2019-10-29 17:32:50 +03:00
2020-04-30 22:30:55 +03:00
// ============================================================================
// === Compiler Options =======================================================
// ============================================================================
2019-09-12 17:47:25 +03:00
2021-05-14 15:08:39 +03:00
ThisBuild / javacOptions ++= Seq(
"-encoding", // Provide explicit encoding (the next line)
"UTF-8", // Specify character encoding used by Java source files
"-deprecation", // Shows a description of each use or override of a deprecated member or class
"-g", // Include debugging information
"-Xlint:unchecked", // Enable additional warnings
"-proc:full" // Annotation processing is enabled
)
2021-05-14 15:08:39 +03:00
ThisBuild / scalacOptions ++= Seq(
"-deprecation", // Emit warning and location for usages of deprecated APIs.
"-encoding", // Provide explicit encoding (the next line)
"utf-8", // Specify character encoding used by Scala source files.
"-explaintypes", // Explain type errors in more detail.
"-feature", // Emit warning and location for usages of features that should be imported explicitly.
"-language:existentials", // Existential types (besides wildcard types) can be written and inferred
"-language:experimental.macros", // Allow macro definition (besides implementation and application)
"-language:higherKinds", // Allow higher-kinded types
"-language:implicitConversions", // Allow definition of implicit functions called views
"-unchecked", // Enable additional warnings where generated code depends on assumptions.
"-Vimplicits", // Prints implicit resolution chains when no implicit can be found.
"-Vtype-diffs", // Prints type errors as coloured diffs between types.
"-Xcheckinit", // Wrap field accessors to throw an exception on uninitialized access.
"-Xfatal-warnings", // Make warnings fatal so they don't make it onto main (use @nowarn for local suppression)
"-Xlint:adapted-args", // Warn if an argument list is modified to match the receiver.
"-Xlint:constant", // Evaluation of a constant arithmetic expression results in an error.
"-Xlint:delayedinit-select", // Selecting member of DelayedInit.
"-Xlint:doc-detached", // A Scaladoc comment appears to be detached from its element.
"-Xlint:inaccessible", // Warn about inaccessible types in method signatures.
"-Xlint:infer-any", // Warn when a type argument is inferred to be `Any`.
"-Xlint:missing-interpolator", // A string literal appears to be missing an interpolator id.
"-Xlint:nullary-unit", // Warn when nullary methods return Unit.
"-Xlint:option-implicit", // Option.apply used implicit view.
"-Xlint:package-object-classes", // Class or object defined in package object.
"-Xlint:poly-implicit-overload", // Parameterized overloaded implicit methods are not visible as view bounds.
"-Xlint:private-shadow", // A private field (or class parameter) shadows a superclass field.
"-Xlint:stars-align", // Pattern sequence wildcard must align with sequence component.
"-Xlint:type-parameter-shadow", // A local type parameter shadows a type already in scope.
"-Xmacro-settings:-logging@org.enso", // Disable the debug logging globally.
"-Ywarn-dead-code", // Warn when dead code is identified.
"-Ywarn-extra-implicit", // Warn when more than one implicit parameter section is defined.
"-Ywarn-numeric-widen", // Warn when numerics are widened.
"-Ywarn-unused:implicits", // Warn if an implicit parameter is unused.
"-Ywarn-unused:imports", // Warn if an import selector is not referenced.
"-Ywarn-unused:locals", // Warn if a local definition is unused.
"-Ywarn-unused:params", // Warn if a value parameter is unused.
"-Ywarn-unused:patvars", // Warn if a variable bound in a pattern is unused.
"-Ywarn-unused:privates" // Warn if a private member is unused.
)
2020-04-30 22:30:55 +03:00
ThisBuild / Test / testOptions ++=
Suppress logs for test cases unless a failure is reported (#8694) The change adds a convenient trait `ReportLogsOnFailure` that, when merged with the test class, will keep logs in memory and only delegate to the underlying appender on failure. For now we only support forwarding to the console which is sufficient. A corresponding entry in `application-test.conf` has to point to the new `memory` appender. The additional complexity in the implementation ensures that if someone forgets to mixin `ReportLogsOnFailure` logs appear as before i.e. they respect the log level. As a bonus fixed arguments passed to ScalaTest in build.sbt so that we are now, again, showing timings of individual tests. Closes #8603. # Important Notes Before: ``` [info] VcsManagerTest: [info] Initializing project [ERROR] [2024-01-04 17:27:03,366] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must create a repository (3 seconds, 538 milliseconds) [info] - must fail to create a repository for an already existing project (141 milliseconds) [info] Save project [ERROR] [2024-01-04 17:27:08,346] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must create a commit with a timestamp (198 milliseconds) [ERROR] [2024-01-04 17:27:08,570] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must create a commit with a name (148 milliseconds) [ERROR] [2024-01-04 17:27:08,741] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must force all pending saves (149 milliseconds) [info] Status project [ERROR] [2024-01-04 17:27:08,910] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must report changed files since last commit (148 milliseconds) [info] Restore project [ERROR] [2024-01-04 17:27:09,076] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must reset to the last state with committed changes (236 milliseconds) [ERROR] [2024-01-04 17:27:09,328] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must reset to a named save (pending) [ERROR] [2024-01-04 17:27:09,520] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must reset to a named save and notify about removed files *** FAILED *** (185 milliseconds) [info] Right({ [info] "jsonrpc" : "2.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) did not equal Right({ [info] "jsonrpc" : "1.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) (VcsManagerTest.scala:1343) [info] Analysis: [info] Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }])) [ERROR] [2024-01-04 17:27:09,734] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] List project saves [info] - must return all explicit commits (146 milliseconds) [info] Run completed in 9 seconds, 270 milliseconds. [info] Total number of tests run: 9 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1 [info] *** 1 TEST FAILED *** ``` After: ``` [info] VcsManagerTest: [info] Initializing project [info] - must create a repository (3 seconds, 554 milliseconds) [info] - must fail to create a repository for an already existing project (164 milliseconds) [info] Save project [info] - must create a commit with a timestamp (212 milliseconds) [info] - must create a commit with a name (142 milliseconds) [info] - must force all pending saves (185 milliseconds) [info] Status project [info] - must report changed files since last commit (142 milliseconds) [info] Restore project [info] - must reset to the last state with committed changes (202 milliseconds) [info] - must reset to a named save (pending) [ERROR] [2024-01-04 17:24:55,738] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/8456553964637757156]. [info] - must reset to a named save and notify about removed files *** FAILED *** (186 milliseconds) [info] Right({ [info] "jsonrpc" : "2.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) did not equal Right({ [info] "jsonrpc" : "1.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) (VcsManagerTest.scala:1343) [info] Analysis: [info] Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }])) [info] List project saves [info] - must return all explicit commits (131 milliseconds) [info] Run completed in 9 seconds, 400 milliseconds. [info] Total number of tests run: 9 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1 [info] *** 1 TEST FAILED *** ```
2024-01-09 12:59:10 +03:00
Seq(Tests.Argument(TestFrameworks.ScalaTest, "-oID")) ++
sys.env
.get("ENSO_TEST_JUNIT_DIR")
.map { junitDir =>
Tests.Argument(TestFrameworks.ScalaTest, "-u", junitDir)
}
2021-05-14 15:08:39 +03:00
Compile / console / scalacOptions ~= (_ filterNot (_ == "-Xfatal-warnings"))
2020-04-30 22:30:55 +03:00
// ============================================================================
// === Benchmark Configuration ================================================
// ============================================================================
2019-09-10 19:53:23 +03:00
lazy val Benchmark = config("bench") extend sbt.Test
// Native Image Generation
2020-12-09 16:58:11 +03:00
lazy val rebuildNativeImage = taskKey[Unit]("Force to rebuild native image")
2019-08-28 18:40:08 +03:00
lazy val buildNativeImage =
2020-12-09 16:58:11 +03:00
taskKey[Unit]("Ensure that the Native Image is built.")
2019-06-07 14:49:47 +03:00
2020-04-30 22:30:55 +03:00
// ============================================================================
// === Global Project =========================================================
// ============================================================================
2019-09-12 17:47:25 +03:00
lazy val enso = (project in file("."))
.settings(version := "0.1")
.aggregate(
`persistance-dsl`,
`persistance`,
`interpreter-dsl`,
`interpreter-dsl-test`,
`json-rpc-server-test`,
`json-rpc-server`,
`language-server`,
`polyglot-api`,
`project-manager`,
`syntax-definition`,
`syntax-rust-definition`,
`text-buffer`,
pkg,
cli,
`task-progress-notifications`,
`profiling-utils`,
`logging-utils`,
`logging-config`,
`logging-service`,
`logging-service-logback`,
`logging-utils-akka`,
filewatcher,
2021-07-22 09:24:06 +03:00
`logging-truffle-connector`,
`locking-test-helper`,
`akka-native`,
2020-07-10 13:57:42 +03:00
`version-output`,
`refactoring-utils`,
`engine-runner`,
runtime,
searcher,
2020-07-10 13:57:42 +03:00
launcher,
2021-07-22 09:24:06 +03:00
downloader,
`runtime-integration-tests`,
`runtime-benchmarks`,
`runtime-parser`,
`runtime-compiler`,
`runtime-language-epb`,
`runtime-language-arrow`,
`runtime-instrument-common`,
`runtime-instrument-id-execution`,
`runtime-instrument-repl-debugger`,
`runtime-instrument-runtime-server`,
`runtime-version-manager`,
`runtime-version-manager-test`,
editions,
semver,
`distribution-manager`,
2021-07-22 09:24:06 +03:00
`edition-updater`,
2021-08-12 17:55:23 +03:00
`edition-uploader`,
`library-manager`,
`library-manager-test`,
`connected-lock-manager`,
`connected-lock-manager-server`,
syntax,
testkit,
`common-polyglot-core-utils`,
`std-base`,
`std-database`,
`std-google-api`,
`std-image`,
`std-table`,
`std-aws`,
`http-test-helper`,
`enso-test-java-helpers`,
`exploratory-benchmark-java-helpers`,
2023-10-27 18:32:04 +03:00
`benchmark-java-helpers`,
`benchmarks-common`,
2023-10-27 18:32:04 +03:00
`bench-processor`
)
2019-09-12 17:47:25 +03:00
.settings(Global / concurrentRestrictions += Tags.exclusive(Exclusive))
.settings(
commands ++= Seq(packageBuilder.makePackages, packageBuilder.makeBundles)
)
2019-09-12 17:47:25 +03:00
2020-04-30 22:30:55 +03:00
// ============================================================================
// === Dependency Versions ====================================================
// ============================================================================
2019-09-12 17:47:25 +03:00
2020-04-30 22:30:55 +03:00
/* Note [Dependency Versions]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~
* Please maintain the following section in alphabetical order for the bundles
* of dependencies. Additionally, please keep the 'Other' subsection in
* alphabetical order.
*
* Furthermore, please keep the following in mind:
* - Wherever possible, we should use the same version of a dependency
* throughout the project.
* - If you need to include a new dependency, please define its version in this
* section.
* - If that version is not the latest, please include a note explaining why
* this is the case.
* - If, for some reason, you need to use a dependency version other than the
* global one, please include a note explaining why this is the case, and the
* circumstances under which the dependency could be upgraded to use the
* global version
*/
2020-04-30 22:30:55 +03:00
// === Akka ===================================================================
2020-02-13 17:33:39 +03:00
def akkaPkg(name: String) = akkaURL %% s"akka-$name" % akkaVersion
def akkaHTTPPkg(name: String) = akkaURL %% s"akka-$name" % akkaHTTPVersion
2020-04-30 22:30:55 +03:00
val akkaURL = "com.typesafe.akka"
val akkaVersion = "2.6.20"
val akkaHTTPVersion = "10.2.10"
2020-04-30 22:30:55 +03:00
val akkaMockSchedulerVersion = "0.5.5"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val logbackClassicVersion = JPMSUtils.logbackClassicVersion
val logbackPkg = Seq(
"ch.qos.logback" % "logback-classic" % logbackClassicVersion,
"ch.qos.logback" % "logback-core" % logbackClassicVersion
2020-07-03 15:02:27 +03:00
)
val akkaActor = akkaPkg("actor")
val akkaStream = akkaPkg("stream")
val akkaTyped = akkaPkg("actor-typed")
val akkaTestkit = akkaPkg("testkit")
val akkaSLF4J = akkaPkg("slf4j")
val akkaTestkitTyped = akkaPkg("actor-testkit-typed") % Test
val akkaHttp = akkaHTTPPkg("http")
val akkaSpray = akkaHTTPPkg("http-spray-json")
val logbackTest = logbackPkg.map(_ % Test)
2020-04-30 22:30:55 +03:00
val akka =
2020-07-01 14:21:13 +03:00
Seq(
akkaActor,
akkaStream,
akkaHttp,
akkaSpray,
akkaTyped
)
2020-04-30 22:30:55 +03:00
// === Cats ===================================================================
val catsVersion = "2.9.0"
2019-09-12 17:47:25 +03:00
2020-04-30 22:30:55 +03:00
// === Circe ==================================================================
2019-09-12 17:47:25 +03:00
val circeVersion = "0.14.5"
val circeYamlVersion = "0.14.2"
val enumeratumCirceVersion = "1.7.2"
val circeGenericExtrasVersion = "0.14.2"
val circe = Seq("circe-core", "circe-generic", "circe-parser")
.map("io.circe" %% _ % circeVersion)
2019-09-12 17:47:25 +03:00
2020-04-30 22:30:55 +03:00
// === Commons ================================================================
val commonsCollectionsVersion = "4.4"
2021-12-31 17:50:32 +03:00
val commonsLangVersion = "3.12.0"
val commonsIoVersion = "2.12.0"
val commonsTextVersion = "1.10.0"
2020-04-30 22:30:55 +03:00
val commonsMathVersion = "3.6.1"
val commonsCompressVersion = "1.23.0"
2021-12-31 17:50:32 +03:00
val commonsCliVersion = "1.5.0"
2020-04-30 22:30:55 +03:00
val commons = Seq(
"org.apache.commons" % "commons-collections4" % commonsCollectionsVersion,
"org.apache.commons" % "commons-lang3" % commonsLangVersion,
"commons-io" % "commons-io" % commonsIoVersion,
"org.apache.commons" % "commons-text" % commonsTextVersion,
"org.apache.commons" % "commons-math3" % commonsMathVersion,
"commons-cli" % "commons-cli" % commonsCliVersion
)
// === Jackson ================================================================
2019-09-12 17:47:25 +03:00
val jacksonVersion = "2.15.2"
2020-04-30 22:30:55 +03:00
val jackson = Seq(
"com.fasterxml.jackson.dataformat" % "jackson-dataformat-cbor" % jacksonVersion,
"com.fasterxml.jackson.core" % "jackson-databind" % jacksonVersion,
2020-06-24 20:02:42 +03:00
"com.fasterxml.jackson.module" %% "jackson-module-scala" % jacksonVersion
2020-04-30 22:30:55 +03:00
)
2020-07-01 14:21:13 +03:00
// === JAXB ================================================================
val jaxbVersion = "4.0.0"
2020-07-01 14:21:13 +03:00
val jaxb = Seq(
"jakarta.xml.bind" % "jakarta.xml.bind-api" % jaxbVersion % Benchmark,
"com.sun.xml.bind" % "jaxb-impl" % jaxbVersion % Benchmark
)
2020-04-30 22:30:55 +03:00
// === JMH ====================================================================
2019-09-12 17:47:25 +03:00
val jmhVersion = "1.36"
2019-09-12 17:47:25 +03:00
val jmh = Seq(
2020-04-30 22:30:55 +03:00
"org.openjdk.jmh" % "jmh-core" % jmhVersion % Benchmark,
"org.openjdk.jmh" % "jmh-generator-annprocess" % jmhVersion % Benchmark
2019-09-12 17:47:25 +03:00
)
2019-06-07 14:49:47 +03:00
2020-04-30 22:30:55 +03:00
// === Scala Compiler =========================================================
val scalaCompiler = Seq(
"org.scala-lang" % "scala-reflect" % scalacVersion,
"org.scala-lang" % "scala-compiler" % scalacVersion
)
// === std-lib ================================================================
val antlrVersion = "4.13.0"
val awsJavaSdkV1Version = "1.12.480"
val awsJavaSdkV2Version = "2.20.78"
val icuVersion = "73.1"
val poiOoxmlVersion = "5.2.3"
val redshiftVersion = "2.1.0.15"
val univocityParsersVersion = "2.9.1"
val xmlbeansVersion = "5.1.1"
2020-04-30 22:30:55 +03:00
// === ZIO ====================================================================
val zioVersion = "2.0.14"
val zioInteropCatsVersion = "23.0.0.6"
2020-04-30 22:30:55 +03:00
val zio = Seq(
"dev.zio" %% "zio" % zioVersion,
"dev.zio" %% "zio-interop-cats" % zioInteropCatsVersion
)
// === Other ==================================================================
2021-12-31 17:50:32 +03:00
val bcpkixJdk15Version = "1.70"
val declineVersion = "2.4.1"
val directoryWatcherVersion = "0.18.0"
2020-08-07 12:18:09 +03:00
val flatbuffersVersion = "1.12.0"
val guavaVersion = "32.0.0-jre"
val jlineVersion = "3.23.0"
Eliminate VCS TimeoutExceptions on startup (#8080) Having a modest-size files in a project would lead to a timeout when the project was first initialized. This became apparent when testing delivered `.enso-project` files with some data files. After some digging there was a bug in JGit (https://bugs.eclipse.org/bugs/show_bug.cgi?id=494323) which meant that adding such files was really slow. The implemented fix is not on by default but even with `--renormalization` turned off I did not see improvement. In the end it didn't make sense to add `data` directory to our version control, or any other files than those in `src` or some meta files in `.enso`. Not including such files eliminates first-use initialization problems. # Important Notes To test, pick an existing Enso project with some data files in it (> 100MB) and remove `.enso/.vcs` directory. Previously it would timeout on first try (and work in successive runs). Now it works even on the first try. The crash: ``` [org.enso.languageserver.requesthandler.vcs.InitVcsHandler] Initialize project request [Number(2)] for [f9a7cd0d-529c-4e1d-a4fa-9dfe2ed79008] failed with: null. java.util.concurrent.TimeoutException: null at org.enso.languageserver.effect.ZioExec$.<clinit>(Exec.scala:134) at org.enso.languageserver.effect.ZioExec.$anonfun$exec$3(Exec.scala:60) at org.enso.languageserver.effect.ZioExec.$anonfun$exec$3$adapted(Exec.scala:60) at zio.ZIO.$anonfun$foldCause$4(ZIO.scala:683) at zio.internal.FiberRuntime.runLoop(FiberRuntime.scala:904) at zio.internal.FiberRuntime.evaluateEffect(FiberRuntime.scala:381) at zio.internal.FiberRuntime.evaluateMessageWhileSuspended(FiberRuntime.scala:504) at zio.internal.FiberRuntime.drainQueueOnCurrentThread(FiberRuntime.scala:220) at zio.internal.FiberRuntime.run(FiberRuntime.scala:139) at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49) at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1395) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ```
2023-10-18 12:34:08 +03:00
val jgitVersion = "6.7.0.202309050840-r"
val diffsonVersion = "4.4.0"
2021-12-31 17:50:32 +03:00
val kindProjectorVersion = "0.13.2"
val mockitoScalaVersion = "1.17.14"
2020-08-07 12:18:09 +03:00
val newtypeVersion = "0.4.4"
val pprintVersion = "0.8.1"
val pureconfigVersion = "0.17.4"
val scalacheckVersion = "1.17.0"
val scalacticVersion = "3.3.0-SNAP4"
2021-12-31 17:50:32 +03:00
val scalaLoggingVersion = "3.9.4"
2020-08-07 12:18:09 +03:00
val scalameterVersion = "0.19"
val scalatestVersion = "3.3.0-SNAP4"
val shapelessVersion = "2.3.10"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val slf4jVersion = JPMSUtils.slf4jVersion
val slickVersion = "3.4.1"
val sqliteVersion = "3.42.0.0"
val tikaVersion = "2.4.1"
val typesafeConfigVersion = "1.4.2"
val junitVersion = "4.13.2"
val junitIfVersion = "0.13.2"
val hamcrestVersion = "1.3"
val netbeansApiVersion = "RELEASE180"
val fansiVersion = "0.4.0"
val httpComponentsVersion = "4.4.1"
val apacheArrowVersion = "14.0.1"
2020-04-30 22:30:55 +03:00
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// ============================================================================
// === Utility methods =====================================================
// ============================================================================
lazy val componentModulesPaths =
taskKey[Def.Classpath](
"Gathers all component modules (Jar archives that should be put on module-path" +
" as files"
)
(ThisBuild / componentModulesPaths) := {
val runnerCp = (`engine-runner` / Runtime / fullClasspath).value
val runtimeCp = (LocalProject("runtime") / Runtime / fullClasspath).value
val fullCp = (runnerCp ++ runtimeCp).distinct
val log = streams.value.log
JPMSUtils.filterModulesFromClasspath(
fullCp,
JPMSUtils.componentModules,
log,
shouldContainAll = true
)
}
lazy val compileModuleInfo = taskKey[Unit]("Compiles `module-info.java`")
2020-04-30 22:30:55 +03:00
// ============================================================================
// === Internal Libraries =====================================================
// ============================================================================
lazy val `syntax-definition` =
project in file("lib/scala/syntax/definition")
2019-09-12 17:47:25 +03:00
lazy val syntax = (project in file("lib/scala/syntax/specialization"))
.dependsOn(`syntax-definition`)
.settings(
2020-10-15 17:52:26 +03:00
commands += WithDebugCommand.withDebug,
2019-11-26 16:02:50 +03:00
testFrameworks := Nil,
scalacOptions ++= Seq("-Ypatmat-exhaust-depth", "off"),
2021-05-14 15:08:39 +03:00
Compile / run / mainClass := Some("org.enso.syntax.text.Main"),
2019-09-12 17:47:25 +03:00
version := "0.1",
logBuffered := false,
2019-11-26 16:02:50 +03:00
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % scalatestVersion % Test
),
2020-07-01 14:21:13 +03:00
(Compile / compile) := (Compile / compile)
.dependsOn(RecompileParser.run(`syntax-definition`))
.value
)
lazy val `text-buffer` = project
2020-07-03 16:42:45 +03:00
.in(file("lib/scala/text-buffer"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= Seq(
"org.typelevel" %% "cats-core" % catsVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.scalacheck" %% "scalacheck" % scalacheckVersion % Test
)
)
lazy val rustParserTargetDirectory =
SettingKey[File]("target directory for the Rust parser")
(`syntax-rust-definition` / rustParserTargetDirectory) := {
// setting "debug" for release, because it isn't yet safely integrated into
// the parser definition
val versionName = if (BuildInfo.isReleaseMode) "debug" else "debug"
target.value / "rust" / versionName
}
val generateRustParserLib =
TaskKey[Seq[File]]("generateRustParserLib", "Generates parser native library")
`syntax-rust-definition` / generateRustParserLib := {
val log = state.value.log
val libGlob =
(`syntax-rust-definition` / rustParserTargetDirectory).value.toGlob / "libenso_parser.so"
val allLibs = FileTreeView.default.list(Seq(libGlob)).map(_._1)
if (
sys.env.get("CI").isDefined ||
allLibs.isEmpty ||
(`syntax-rust-definition` / generateRustParserLib).inputFileChanges.hasChanges
) {
val os = System.getProperty("os.name")
val baseArguments = Seq(
"build",
"-p",
"enso-parser-jni",
"-Z",
"unstable-options",
"--out-dir",
(`syntax-rust-definition` / rustParserTargetDirectory).value.toString
)
val adjustedArguments = baseArguments ++
(if (BuildInfo.isReleaseMode)
Seq("--release")
else Seq())
Cargo.run(adjustedArguments, log)
}
FileTreeView.default.list(Seq(libGlob)).map(_._1.toFile)
}
`syntax-rust-definition` / generateRustParserLib / fileInputs +=
(`syntax-rust-definition` / baseDirectory).value.toGlob / "jni" / "src" / "*.rs"
val generateParserJavaSources = TaskKey[Seq[File]](
"generateParserJavaSources",
"Generates Java sources for Rust parser"
)
`syntax-rust-definition` / generateParserJavaSources := {
generateRustParser(
(`syntax-rust-definition` / Compile / sourceManaged).value,
(`syntax-rust-definition` / generateParserJavaSources).inputFileChanges,
state.value.log
)
}
`syntax-rust-definition` / generateParserJavaSources / fileInputs +=
(`syntax-rust-definition` / baseDirectory).value.toGlob / "generate-java" / "src" / ** / "*.rs"
`syntax-rust-definition` / generateParserJavaSources / fileInputs +=
(`syntax-rust-definition` / baseDirectory).value.toGlob / "src" / ** / "*.rs"
def generateRustParser(
base: File,
changes: sbt.nio.FileChanges,
log: ManagedLogger
): Seq[File] = {
import scala.jdk.CollectionConverters._
import java.nio.file.Paths
val syntaxPkgs = Paths.get("org", "enso", "syntax2").toString
val fullPkg = Paths.get(base.toString, syntaxPkgs).toFile
if (!fullPkg.exists()) {
fullPkg.mkdirs()
}
if (changes.hasChanges) {
val args = Seq(
"run",
"-p",
"enso-parser-generate-java",
"--bin",
"enso-parser-generate-java",
fullPkg.toString
)
Cargo.run(args, log)
}
FileUtils.listFiles(fullPkg, Array("scala", "java"), true).asScala.toSeq
}
lazy val `syntax-rust-definition` = project
.in(file("lib/rust/parser"))
.configs(Test)
.settings(
Compile / sourceGenerators += generateParserJavaSources,
Compile / resourceGenerators += generateRustParserLib,
Compile / javaSource := baseDirectory.value / "generate-java" / "java",
frgaalJavaCompilerSetting
)
2020-07-03 16:42:45 +03:00
lazy val pkg = (project in file("lib/scala/pkg"))
.settings(
2021-05-14 15:08:39 +03:00
Compile / run / mainClass := Some("org.enso.pkg.Main"),
frgaalJavaCompilerSetting,
2019-09-12 17:47:25 +03:00
version := "0.1",
libraryDependencies ++= circe ++ Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"io.circe" %% "circe-yaml" % circeYamlVersion,
"org.apache.commons" % "commons-compress" % commonsCompressVersion,
"commons-io" % "commons-io" % commonsIoVersion
)
)
2021-06-18 17:39:45 +03:00
.dependsOn(editions)
2020-07-10 13:57:42 +03:00
lazy val `akka-native` = project
.in(file("lib/scala/akka-native"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
akkaActor
),
// Note [Native Image Workaround for GraalVM 20.2]
libraryDependencies += "org.graalvm.nativeimage" % "svm" % graalMavenPackagesVersion % "provided"
)
lazy val `profiling-utils` = project
.in(file("lib/scala/profiling-utils"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
"org.netbeans.api" % "org-netbeans-modules-sampler" % netbeansApiVersion
exclude ("org.netbeans.api", "org-openide-loaders")
exclude ("org.netbeans.api", "org-openide-nodes")
exclude ("org.netbeans.api", "org-netbeans-api-progress-nb")
exclude ("org.netbeans.api", "org-netbeans-api-progress")
exclude ("org.netbeans.api", "org-openide-util-lookup")
exclude ("org.netbeans.api", "org-openide-util")
exclude ("org.netbeans.api", "org-openide-dialogs")
exclude ("org.netbeans.api", "org-openide-filesystems")
exclude ("org.netbeans.api", "org-openide-util-ui")
exclude ("org.netbeans.api", "org-openide-awt")
exclude ("org.netbeans.api", "org-openide-modules")
exclude ("org.netbeans.api", "org-netbeans-api-annotations-common"),
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
)
)
lazy val `logging-utils` = project
.in(file("lib/scala/logging-utils"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.slf4j" % "slf4j-api" % slf4jVersion
) ++ logbackTest
)
lazy val `logging-service` = project
.in(file("lib/scala/logging-service"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"com.typesafe" % "config" % typesafeConfigVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.dependsOn(`logging-utils`)
.dependsOn(`logging-config`)
lazy val `logging-config` = project
.in(file("lib/scala/logging-config"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
"com.typesafe" % "config" % typesafeConfigVersion,
"org.slf4j" % "slf4j-api" % slf4jVersion
)
)
lazy val `logging-service-logback` = project
.in(file("lib/scala/logging-service-logback"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"io.sentry" % "sentry-logback" % "6.28.0",
"io.sentry" % "sentry" % "6.28.0",
Suppress logs for test cases unless a failure is reported (#8694) The change adds a convenient trait `ReportLogsOnFailure` that, when merged with the test class, will keep logs in memory and only delegate to the underlying appender on failure. For now we only support forwarding to the console which is sufficient. A corresponding entry in `application-test.conf` has to point to the new `memory` appender. The additional complexity in the implementation ensures that if someone forgets to mixin `ReportLogsOnFailure` logs appear as before i.e. they respect the log level. As a bonus fixed arguments passed to ScalaTest in build.sbt so that we are now, again, showing timings of individual tests. Closes #8603. # Important Notes Before: ``` [info] VcsManagerTest: [info] Initializing project [ERROR] [2024-01-04 17:27:03,366] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must create a repository (3 seconds, 538 milliseconds) [info] - must fail to create a repository for an already existing project (141 milliseconds) [info] Save project [ERROR] [2024-01-04 17:27:08,346] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must create a commit with a timestamp (198 milliseconds) [ERROR] [2024-01-04 17:27:08,570] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must create a commit with a name (148 milliseconds) [ERROR] [2024-01-04 17:27:08,741] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must force all pending saves (149 milliseconds) [info] Status project [ERROR] [2024-01-04 17:27:08,910] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must report changed files since last commit (148 milliseconds) [info] Restore project [ERROR] [2024-01-04 17:27:09,076] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must reset to the last state with committed changes (236 milliseconds) [ERROR] [2024-01-04 17:27:09,328] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must reset to a named save (pending) [ERROR] [2024-01-04 17:27:09,520] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] - must reset to a named save and notify about removed files *** FAILED *** (185 milliseconds) [info] Right({ [info] "jsonrpc" : "2.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) did not equal Right({ [info] "jsonrpc" : "1.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) (VcsManagerTest.scala:1343) [info] Analysis: [info] Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "cd84a4a3-fa50-4ead-8d80-04f6d0d124a3", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }])) [ERROR] [2024-01-04 17:27:09,734] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/3607843843826594318]. [info] List project saves [info] - must return all explicit commits (146 milliseconds) [info] Run completed in 9 seconds, 270 milliseconds. [info] Total number of tests run: 9 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1 [info] *** 1 TEST FAILED *** ``` After: ``` [info] VcsManagerTest: [info] Initializing project [info] - must create a repository (3 seconds, 554 milliseconds) [info] - must fail to create a repository for an already existing project (164 milliseconds) [info] Save project [info] - must create a commit with a timestamp (212 milliseconds) [info] - must create a commit with a name (142 milliseconds) [info] - must force all pending saves (185 milliseconds) [info] Status project [info] - must report changed files since last commit (142 milliseconds) [info] Restore project [info] - must reset to the last state with committed changes (202 milliseconds) [info] - must reset to a named save (pending) [ERROR] [2024-01-04 17:24:55,738] [org.enso.languageserver.search.SuggestionsHandler] Cannot read the package definition from [/tmp/8456553964637757156]. [info] - must reset to a named save and notify about removed files *** FAILED *** (186 milliseconds) [info] Right({ [info] "jsonrpc" : "2.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) did not equal Right({ [info] "jsonrpc" : "1.0", [info] "method" : "file/event", [info] "params" : { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] } [info] }) (VcsManagerTest.scala:1343) [info] Analysis: [info] Right(value: Json$JObject(value: object[jsonrpc -> "2.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }] -> object[jsonrpc -> "1.0",method -> "file/event",params -> { [info] "path" : { [info] "rootId" : "965ed5c8-1760-4284-91f2-1376406fde0d", [info] "segments" : [ [info] "src", [info] "Bar.enso" [info] ] [info] }, [info] "kind" : "Removed" [info] }])) [info] List project saves [info] - must return all explicit commits (131 milliseconds) [info] Run completed in 9 seconds, 400 milliseconds. [info] Total number of tests run: 9 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 8, failed 1, canceled 0, ignored 0, pending 1 [info] *** 1 TEST FAILED *** ```
2024-01-09 12:59:10 +03:00
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided"
) ++ logbackPkg
)
.dependsOn(`logging-config`)
.dependsOn(`logging-service`)
lazy val `logging-utils-akka` = project
.in(file("lib/scala/logging-utils-akka"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"com.typesafe.akka" %% "akka-actor" % akkaVersion
)
)
lazy val filewatcher = project
.in(file("lib/scala/filewatcher"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
libraryDependencies ++= Seq(
"io.methvin" % "directory-watcher" % directoryWatcherVersion,
"commons-io" % "commons-io" % commonsIoVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
),
Test / fork := true,
Test / javaOptions ++= testLogProviderOptions
)
.dependsOn(testkit % Test)
.dependsOn(`logging-service-logback` % "test->test")
2021-07-06 01:27:14 +03:00
lazy val `logging-truffle-connector` = project
.in(file("lib/scala/logging-truffle-connector"))
.settings(
frgaalJavaCompilerSetting,
2021-07-06 01:27:14 +03:00
version := "0.1",
libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided"
2021-07-06 01:27:14 +03:00
)
)
.dependsOn(`logging-utils`)
.dependsOn(`polyglot-api`)
lazy val cli = project
.in(file("lib/scala/cli"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
version := "0.1",
2021-07-22 09:24:06 +03:00
libraryDependencies ++= circe ++ Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.typelevel" %% "cats-core" % catsVersion
),
2021-05-14 15:08:39 +03:00
Test / parallelExecution := false
)
lazy val `task-progress-notifications` = project
.in(file("lib/scala/task-progress-notifications"))
.configs(Test)
.settings(
version := "0.1",
libraryDependencies ++= Seq(
"com.beachape" %% "enumeratum-circe" % enumeratumCirceVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
),
Test / parallelExecution := false
)
.dependsOn(cli)
.dependsOn(`json-rpc-server`)
2020-07-10 13:57:42 +03:00
lazy val `version-output` = (project in file("lib/scala/version-output"))
.settings(
version := "0.1"
)
.settings(
frgaalJavaCompilerSetting,
2020-07-10 13:57:42 +03:00
Compile / sourceGenerators += Def.task {
val file = (Compile / sourceManaged).value / "buildinfo" / "Info.scala"
BuildInfo
.writeBuildInfoFile(
2022-02-15 18:34:33 +03:00
file = file,
log = state.value.log,
defaultDevEnsoVersion = defaultDevEnsoVersion,
ensoVersion = ensoVersion,
scalacVersion = scalacVersion,
graalVersion = graalVersion,
currentEdition = currentEdition
)
}.taskValue
2020-07-10 13:57:42 +03:00
)
lazy val `refactoring-utils` = project
.in(file("lib/scala/refactoring-utils"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
commands += WithDebugCommand.withDebug,
version := "0.1",
libraryDependencies ++= Seq(
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
)
)
.dependsOn(`runtime-parser`)
.dependsOn(`text-buffer`)
.dependsOn(testkit % Test)
2020-07-03 16:42:45 +03:00
lazy val `project-manager` = (project in file("lib/scala/project-manager"))
.enablePlugins(JPMSPlugin)
2019-11-18 14:18:16 +03:00
.settings(
(Compile / mainClass) := Some("org.enso.projectmanager.boot.ProjectManager")
2019-11-18 14:18:16 +03:00
)
.settings(
frgaalJavaCompilerSetting,
(Compile / run / fork) := true,
(Test / fork) := true,
(Compile / run / connectInput) := true,
commands += WithDebugCommand.withDebug,
libraryDependencies ++= akka ++ Seq(akkaTestkit % Test),
2020-03-18 13:41:55 +03:00
libraryDependencies ++= circe,
libraryDependencies ++= Seq(
"com.typesafe" % "config" % typesafeConfigVersion,
"com.github.pureconfig" %% "pureconfig" % pureconfigVersion,
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"dev.zio" %% "zio" % zioVersion,
"dev.zio" %% "zio-interop-cats" % zioInteropCatsVersion,
"commons-cli" % "commons-cli" % commonsCliVersion,
"commons-io" % "commons-io" % commonsIoVersion,
"org.apache.commons" % "commons-lang3" % commonsLangVersion,
"com.beachape" %% "enumeratum-circe" % enumeratumCirceVersion,
"com.miguno.akka" %% "akka-mock-scheduler" % akkaMockSchedulerVersion % Test,
"org.mockito" %% "mockito-scala" % mockitoScalaVersion % Test,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.hamcrest" % "hamcrest-all" % hamcrestVersion % Test
),
addCompilerPlugin(
2020-04-30 22:30:55 +03:00
"org.typelevel" %% "kind-projector" % kindProjectorVersion cross CrossVersion.full
2020-03-18 13:41:55 +03:00
)
2019-11-18 14:18:16 +03:00
)
/** Fat jar assembly settings
*/
2020-05-15 14:05:44 +03:00
.settings(
2021-05-14 15:08:39 +03:00
assembly / assemblyJarName := "project-manager.jar",
assembly / test := {},
assembly / assemblyOutputPath := file("project-manager.jar"),
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// Exclude all the Truffle/Graal related artifacts from the fat jar
assembly / assemblyExcludedJars := {
val pkgsToExclude = GraalVM.modules
val ourFullCp = (Runtime / fullClasspath).value
JPMSUtils.filterModulesFromClasspath(
ourFullCp,
pkgsToExclude,
streams.value.log
)
},
2021-05-14 15:08:39 +03:00
assembly / assemblyMergeStrategy := {
2020-05-15 14:05:44 +03:00
case PathList("META-INF", file, xs @ _*) if file.endsWith(".DSA") =>
MergeStrategy.discard
case PathList("META-INF", file, xs @ _*) if file.endsWith(".SF") =>
MergeStrategy.discard
case PathList("META-INF", "MANIFEST.MF", xs @ _*) =>
MergeStrategy.discard
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// This fat Jar must not be an explicit module, so discard all the module-info classes
case PathList(xs @ _*) if xs.last.contains("module-info") =>
MergeStrategy.discard
2020-05-15 14:05:44 +03:00
case "application.conf" => MergeStrategy.concat
case "reference.conf" => MergeStrategy.concat
case _ => MergeStrategy.first
}
)
/** JPMS related settings for tests
*/
.settings(
Test / fork := true,
// These dependencies are here so that we can use them in `--module-path` later on.
libraryDependencies ++= {
val necessaryModules =
GraalVM.modules.map(_.withConfigurations(Some(Test.name))) ++
GraalVM.langsPkgs.map(_.withConfigurations(Some(Test.name)))
necessaryModules
2020-05-15 14:05:44 +03:00
},
Test / addModules := Seq(
(`runtime-fat-jar` / javaModuleName).value
),
Test / modulePath := {
val updateReport = (Test / update).value
val requiredModIds =
GraalVM.modules ++ GraalVM.langsPkgs ++ logbackPkg ++ Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion
)
val requiredMods = JPMSUtils.filterModulesFromUpdate(
updateReport,
requiredModIds,
streams.value.log,
shouldContainAll = true
)
val runtimeMod =
(`runtime-fat-jar` / Compile / productDirectories).value.head
requiredMods ++ Seq(runtimeMod)
},
Test / javaOptions ++= testLogProviderOptions
)
.settings(
2020-12-09 16:58:11 +03:00
rebuildNativeImage := NativeImage
.buildNativeImage(
"project-manager",
staticOnLinux = true,
initializeAtRuntime = Seq(
"scala.util.Random",
"zio.internal.ZScheduler$$anon$4",
"zio.Runtime$",
"zio.FiberRef$"
)
2020-12-09 16:58:11 +03:00
)
.dependsOn(VerifyReflectionSetup.run)
2020-12-09 16:58:11 +03:00
.dependsOn(assembly)
.value,
buildNativeImage := NativeImage
.incrementalNativeImageBuild(
rebuildNativeImage,
"project-manager"
)
.value
2020-05-15 14:05:44 +03:00
)
2020-12-09 16:58:11 +03:00
.dependsOn(`akka-native`)
2020-07-10 13:57:42 +03:00
.dependsOn(`version-output`)
2021-06-18 17:39:45 +03:00
.dependsOn(editions)
2021-08-12 17:55:23 +03:00
.dependsOn(`edition-updater`)
2021-06-18 17:39:45 +03:00
.dependsOn(cli)
.dependsOn(`task-progress-notifications`)
.dependsOn(`polyglot-api`)
.dependsOn(`runtime-version-manager`)
2021-06-18 17:39:45 +03:00
.dependsOn(`library-manager`)
.dependsOn(`logging-utils-akka`)
.dependsOn(`logging-service`)
2021-06-18 17:39:45 +03:00
.dependsOn(pkg)
2020-03-18 13:41:55 +03:00
.dependsOn(`json-rpc-server`)
.dependsOn(`logging-service-logback` % Runtime)
2020-03-18 13:41:55 +03:00
.dependsOn(`json-rpc-server-test` % Test)
2020-07-14 14:44:40 +03:00
.dependsOn(testkit % Test)
.dependsOn(`runtime-version-manager-test` % Test)
.dependsOn(`logging-service-logback` % "test->test")
2019-11-18 14:18:16 +03:00
/* Note [Classpath Separation]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~
* Projects using the language runtime do not depend on it directly, but instead
* the language runtime is put on the Truffle classpath, rather than the
* standard classpath. This is the recommended way of handling this and we
* strive to use such structure everywhere.
* See
* https://www.graalvm.org/docs/graalvm-as-a-platform/implement-language#graalvm
*
* Currently the only exception to this are the tests of the runtime project
* which have classpath separation disabled, because they need direct access to
* the runtime's instruments.
*
* To ensure correct handling of dependencies by sbt, the classpath appended to
* Java options, should be based on `(runtime / Compile / fullClasspath).value`
* wherever possible. Using a key from the runtime project enables sbt to see
* the dependency.
*
* Assembly tasks that build JAR files which need `runtime.jar` to run should
* also add a dependency on `runtime / assembly`.
*/
lazy val `json-rpc-server` = project
2020-07-03 16:42:45 +03:00
.in(file("lib/scala/json-rpc-server"))
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= akka ++ logbackTest,
libraryDependencies ++= circe,
libraryDependencies ++= Seq(
"io.circe" %% "circe-literal" % circeVersion,
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
2022-05-24 16:01:26 +03:00
akkaTestkit % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.apache.httpcomponents" % "httpclient" % httpComponentsVersion % Test,
"org.apache.httpcomponents" % "httpcore" % httpComponentsVersion % Test,
"commons-io" % "commons-io" % commonsIoVersion % Test
)
)
2019-11-05 17:12:33 +03:00
lazy val `json-rpc-server-test` = project
2020-07-03 16:42:45 +03:00
.in(file("lib/scala/json-rpc-server-test"))
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= akka,
libraryDependencies ++= circe,
libraryDependencies ++= Seq(
"io.circe" %% "circe-literal" % circeVersion,
akkaTestkit,
"org.scalatest" %% "scalatest" % scalatestVersion,
"org.gnieh" %% "diffson-circe" % diffsonVersion
)
)
.dependsOn(`json-rpc-server`)
2020-07-14 14:44:40 +03:00
lazy val testkit = project
.in(file("lib/scala/testkit"))
.settings(
frgaalJavaCompilerSetting,
2020-07-14 14:44:40 +03:00
libraryDependencies ++= Seq(
"org.apache.commons" % "commons-lang3" % commonsLangVersion,
"commons-io" % "commons-io" % commonsIoVersion,
"org.scalatest" %% "scalatest" % scalatestVersion,
"junit" % "junit" % junitVersion,
"com.github.sbt" % "junit-interface" % junitIfVersion
)
2020-07-14 14:44:40 +03:00
)
lazy val searcher = project
2020-07-03 16:42:45 +03:00
.in(file("lib/scala/searcher"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
2020-07-06 16:55:21 +03:00
libraryDependencies ++= jmh ++ Seq(
"com.typesafe.slick" %% "slick" % slickVersion,
"org.xerial" % "sqlite-jdbc" % sqliteVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
) ++ logbackTest
)
2020-07-06 16:55:21 +03:00
.configs(Benchmark)
.settings(
inConfig(Benchmark)(Defaults.testSettings),
2021-05-14 15:08:39 +03:00
Benchmark / fork := true
2020-07-06 16:55:21 +03:00
)
2020-07-14 14:44:40 +03:00
.dependsOn(testkit % Test)
.dependsOn(`polyglot-api`)
lazy val `persistance` = (project in file("lib/java/persistance"))
.settings(
version := "0.1",
frgaalJavaCompilerSetting,
Compile / javacOptions := ((Compile / javacOptions).value),
libraryDependencies ++= Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
)
)
.dependsOn(`persistance-dsl` % Test)
lazy val `persistance-dsl` = (project in file("lib/java/persistance-dsl"))
.settings(
version := "0.1",
frgaalJavaCompilerSetting,
Compile / javacOptions := ((Compile / javacOptions).value ++
// Only run ServiceProvider processor and ignore those defined in META-INF, thus
// fixing incremental compilation setup
Seq(
"-processor",
"org.netbeans.modules.openide.util.ServiceProviderProcessor"
)),
libraryDependencies ++= Seq(
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided"
)
)
2020-07-03 16:42:45 +03:00
lazy val `interpreter-dsl` = (project in file("lib/scala/interpreter-dsl"))
.settings(
version := "0.1",
frgaalJavaCompilerSetting,
Compile / javacOptions := ((Compile / javacOptions).value ++
2022-07-01 04:58:14 +03:00
// Only run ServiceProvider processor and ignore those defined in META-INF, thus
// fixing incremental compilation setup
Seq(
"-processor",
"org.netbeans.modules.openide.util.ServiceProviderProcessor"
)),
Move Builtin Types and Methods to stdlib (#3363) This PR replaces hard-coded `@Builtin_Method` and `@Builtin_Type` nodes in Builtins with an automated solution that a) collects metadata from such annotations b) generates `BuiltinTypes` c) registers builtin methods with corresponding constructors. The main differences are: 1) The owner of the builtin method does not necessarily have to be a builtin type 2) You can now mix regular methods and builtin ones in stdlib 3) No need to keep track of builtin methods and types in various places and register them by hand (a source of many typos or omissions as it found during the process of this PR) Related to #181497846 Benchmarks also execute within the margin of error. ### Important Notes The PR got a bit large over time as I was moving various builtin types and finding various corner cases. Most of the changes however are rather simple c&p from Builtins.enso to the corresponding stdlib module. Here is the list of the most crucial updates: - `engine/runtime/src/main/java/org/enso/interpreter/runtime/builtin/Builtins.java` - the core of the changes. We no longer register individual builtin constructors and their methods by hand. Instead, the information about those is read from 2 metadata files generated by annotation processors. When the builtin method is encountered in stdlib, we do not ignore the method. Instead we lookup it up in the list of registered functions (see `getBuiltinFunction` and `IrToTruffle`) - `engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java` has now information whether it corresponds to the builtin type or not. - `engine/runtime/src/main/scala/org/enso/compiler/codegen/RuntimeStubsGenerator.scala` - when runtime stubs generator encounters a builtin type, based on the @Builtin_Type annotation, it looks up an existing constructor for it and registers it in the provided scope, rather than creating a new one. The scope of the constructor is also changed to the one coming from stdlib, while ensuring that synthetic methods (for fields) also get assigned correctly - `engine/runtime/src/main/scala/org/enso/compiler/codegen/IrToTruffle.scala` - when a builtin method is encountered in stdlib we don't generate a new function node for it, instead we look it up in the list of registered builtin methods. Note that Integer and Number present a bit of a challenge because they list a whole bunch of methods that don't have a corresponding method (instead delegating to small/big integer implementations). During the translation new atom constructors get initialized but we don't want to do it for builtins which have gone through the process earlier, hence the exception - `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor not only generates the actual code fpr nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization - `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor no longer generates only (root) nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization - `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/TypeProcessor.java` - Similar to MethodProcessor but handles @Builtin_Type annotations. It doesn't, **yet**, generate any builtin objects. It also collects the names, as present in stdlib, if any, so that we can generate the names automatically (see generated `types/ConstantsGen.java`) - `engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin` - various classes annotated with @BuiltinType to ensure that the atom constructor is always properly registered for the builitn. Note that in order to support types fields in those, annotation takes optional `params` parameter (comma separated). - `engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala` - drop manual creation of test list which seemed to be a relict of the old design
2022-05-05 21:18:06 +03:00
libraryDependencies ++= Seq(
"org.apache.commons" % "commons-lang3" % commonsLangVersion,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
2022-07-01 04:58:14 +03:00
"com.google.guava" % "guava" % guavaVersion exclude ("com.google.code.findbugs", "jsr305")
Move Builtin Types and Methods to stdlib (#3363) This PR replaces hard-coded `@Builtin_Method` and `@Builtin_Type` nodes in Builtins with an automated solution that a) collects metadata from such annotations b) generates `BuiltinTypes` c) registers builtin methods with corresponding constructors. The main differences are: 1) The owner of the builtin method does not necessarily have to be a builtin type 2) You can now mix regular methods and builtin ones in stdlib 3) No need to keep track of builtin methods and types in various places and register them by hand (a source of many typos or omissions as it found during the process of this PR) Related to #181497846 Benchmarks also execute within the margin of error. ### Important Notes The PR got a bit large over time as I was moving various builtin types and finding various corner cases. Most of the changes however are rather simple c&p from Builtins.enso to the corresponding stdlib module. Here is the list of the most crucial updates: - `engine/runtime/src/main/java/org/enso/interpreter/runtime/builtin/Builtins.java` - the core of the changes. We no longer register individual builtin constructors and their methods by hand. Instead, the information about those is read from 2 metadata files generated by annotation processors. When the builtin method is encountered in stdlib, we do not ignore the method. Instead we lookup it up in the list of registered functions (see `getBuiltinFunction` and `IrToTruffle`) - `engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java` has now information whether it corresponds to the builtin type or not. - `engine/runtime/src/main/scala/org/enso/compiler/codegen/RuntimeStubsGenerator.scala` - when runtime stubs generator encounters a builtin type, based on the @Builtin_Type annotation, it looks up an existing constructor for it and registers it in the provided scope, rather than creating a new one. The scope of the constructor is also changed to the one coming from stdlib, while ensuring that synthetic methods (for fields) also get assigned correctly - `engine/runtime/src/main/scala/org/enso/compiler/codegen/IrToTruffle.scala` - when a builtin method is encountered in stdlib we don't generate a new function node for it, instead we look it up in the list of registered builtin methods. Note that Integer and Number present a bit of a challenge because they list a whole bunch of methods that don't have a corresponding method (instead delegating to small/big integer implementations). During the translation new atom constructors get initialized but we don't want to do it for builtins which have gone through the process earlier, hence the exception - `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor not only generates the actual code fpr nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization - `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor no longer generates only (root) nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization - `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/TypeProcessor.java` - Similar to MethodProcessor but handles @Builtin_Type annotations. It doesn't, **yet**, generate any builtin objects. It also collects the names, as present in stdlib, if any, so that we can generate the names automatically (see generated `types/ConstantsGen.java`) - `engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin` - various classes annotated with @BuiltinType to ensure that the atom constructor is always properly registered for the builitn. Note that in order to support types fields in those, annotation takes optional `params` parameter (comma separated). - `engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala` - drop manual creation of test list which seemed to be a relict of the old design
2022-05-05 21:18:06 +03:00
)
2020-07-03 16:42:45 +03:00
)
lazy val `interpreter-dsl-test` =
(project in file("engine/interpreter-dsl-test"))
.configs(Test)
.settings(
version := "0.1",
frgaalJavaCompilerSetting,
Test / fork := true,
Test / javaOptions ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"-Dpolyglotimpl.DisableClassPathIsolation=true"
),
Test / javacOptions ++= Seq(
"-s",
(Test / sourceManaged).value.getAbsolutePath
),
Compile / logManager :=
sbt.internal.util.CustomLogManager.excludeMsg(
"Could not determine source for class ",
Level.Warn
),
commands += WithDebugCommand.withDebug,
libraryDependencies ++= Seq(
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided",
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
)
)
.dependsOn(`interpreter-dsl`)
.dependsOn(`runtime`)
2020-04-30 22:30:55 +03:00
// ============================================================================
// === Sub-Projects ===========================================================
// ============================================================================
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val benchOnlyOptions = if (java.lang.Boolean.getBoolean("bench.compileOnly")) {
Seq(
"-Dbench.compileOnly=true"
)
} else {
Seq(
"-Dbench.compileOnly=false"
)
}
/** Truffle-related settings for test running.
*/
val truffleRunOpts = Seq(
"-Dpolyglot.compiler.IterativePartialEscape=true",
"-Dpolyglot.compiler.BackgroundCompilation=false"
)
val truffleRunOptionsSettings = Seq(
fork := true,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
javaOptions ++= "-ea" +: benchOnlyOptions
)
/** Explicitly provide `application-test.conf` as the resource that should be used for
* parsing the logging configuration. Explicitly setting `config.resource` prevents
* the potential conflicts with other *.conf files.
*/
val testLogProviderOptions = Seq(
"-Dslf4j.provider=org.enso.logger.TestLogProvider",
"-Dconfig.resource=application-test.conf"
)
lazy val `polyglot-api` = project
2020-02-13 17:33:39 +03:00
.in(file("engine/polyglot-api"))
.settings(
frgaalJavaCompilerSetting,
2020-02-13 17:33:39 +03:00
Test / fork := true,
2021-09-18 15:48:13 +03:00
commands += WithDebugCommand.withDebug,
2021-07-08 16:38:20 +03:00
Test / envVars ++= distributionEnvironmentOverrides,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
Test / javaOptions ++= Seq(
"-Dpolyglot.engine.WarnInterpreterOnly=false",
"-Dpolyglotimpl.DisableClassPathIsolation=true"
),
// Append enso language on the class-path
Test / unmanagedClasspath :=
(LocalProject(
"runtime-fat-jar"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
) / Compile / fullClasspath).value,
2020-02-13 17:33:39 +03:00
libraryDependencies ++= Seq(
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
"com.google.flatbuffers" % "flatbuffers-java" % flatbuffersVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.scalacheck" %% "scalacheck" % scalacheckVersion % Test
),
2020-04-30 22:30:55 +03:00
libraryDependencies ++= jackson,
GenerateFlatbuffers.flatcVersion := flatbuffersVersion,
2021-05-14 15:08:39 +03:00
Compile / sourceGenerators += GenerateFlatbuffers.task
)
2020-02-13 17:33:39 +03:00
.dependsOn(pkg)
.dependsOn(`text-buffer`)
2021-05-18 11:23:13 +03:00
.dependsOn(`logging-utils`)
2021-06-18 17:39:45 +03:00
.dependsOn(testkit % Test)
2020-02-13 17:33:39 +03:00
lazy val `language-server` = (project in file("engine/language-server"))
.enablePlugins(JPMSPlugin)
2020-02-13 17:33:39 +03:00
.settings(
commands += WithDebugCommand.withDebug,
frgaalJavaCompilerSetting,
libraryDependencies ++= akka ++ circe ++ Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"io.circe" %% "circe-generic-extras" % circeGenericExtrasVersion,
"io.circe" %% "circe-literal" % circeVersion,
"dev.zio" %% "zio" % zioVersion,
"com.beachape" %% "enumeratum-circe" % enumeratumCirceVersion,
"com.google.flatbuffers" % "flatbuffers-java" % flatbuffersVersion,
"commons-io" % "commons-io" % commonsIoVersion,
akkaTestkit % Test,
"com.typesafe.akka" %% "akka-http-testkit" % akkaHTTPVersion % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.scalacheck" %% "scalacheck" % scalacheckVersion % Test,
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion % "provided",
"org.eclipse.jgit" % "org.eclipse.jgit" % jgitVersion,
"org.bouncycastle" % "bcutil-jdk18on" % "1.76" % Test,
"org.bouncycastle" % "bcpkix-jdk18on" % "1.76" % Test,
"org.bouncycastle" % "bcprov-jdk18on" % "1.76" % Test,
"org.apache.tika" % "tika-core" % tikaVersion % Test
),
2021-05-14 15:08:39 +03:00
Test / testOptions += Tests
.Argument(TestFrameworks.ScalaCheck, "-minSuccessfulTests", "1000"),
Test / envVars ++= distributionEnvironmentOverrides,
GenerateFlatbuffers.flatcVersion := flatbuffersVersion,
2021-05-14 15:08:39 +03:00
Compile / sourceGenerators += GenerateFlatbuffers.task
2020-02-28 16:17:48 +03:00
)
.configs(Benchmark)
.settings(
inConfig(Compile)(truffleRunOptionsSettings),
2020-02-28 16:17:48 +03:00
inConfig(Benchmark)(Defaults.testSettings),
2021-05-14 15:08:39 +03:00
bench := (Benchmark / test).value,
2020-04-30 22:30:55 +03:00
libraryDependencies += "com.storm-enroute" %% "scalameter" % scalameterVersion % "bench",
2020-02-28 16:17:48 +03:00
testFrameworks ++= List(
new TestFramework("org.scalameter.ScalaMeterFramework")
)
2020-02-13 17:33:39 +03:00
)
.settings(
Test / fork := true,
// These dependencies are here so that we can use them in `--module-path` later on.
libraryDependencies ++= {
val necessaryModules =
GraalVM.modules.map(_.withConfigurations(Some(Test.name))) ++
GraalVM.langsPkgs.map(_.withConfigurations(Some(Test.name)))
necessaryModules
},
Test / addModules := Seq(
(`runtime-fat-jar` / javaModuleName).value
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
),
Test / modulePath := {
val updateReport = (Test / update).value
val requiredModIds =
GraalVM.modules ++ GraalVM.langsPkgs ++ logbackPkg ++ Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion
)
val requiredMods = JPMSUtils.filterModulesFromUpdate(
updateReport,
requiredModIds,
streams.value.log,
shouldContainAll = true
)
val runtimeMod =
(`runtime-fat-jar` / Compile / productDirectories).value.head
requiredMods ++ Seq(runtimeMod)
},
Test / javaOptions ++= testLogProviderOptions,
Test / patchModules := {
/** All these modules will be in --patch-module cmdline option to java, which means that
* for the JVM, it will appear that all the classes contained in these sbt projects are contained
* in the `org.enso.runtime` module. In this way, we do not have to assembly the `runtime.jar`
* fat jar.
*/
val modulesToPatchIntoRuntime: Seq[File] =
(LocalProject(
"runtime-instrument-common"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-instrument-id-execution"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-instrument-repl-debugger"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-instrument-runtime-server"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-language-epb"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-compiler"
) / Compile / productDirectories).value ++
(LocalProject("runtime-parser") / Compile / productDirectories).value ++
(LocalProject(
"interpreter-dsl"
) / Compile / productDirectories).value ++
// We have to patch the `runtime` project as well, as it contains BuiltinTypes.metadata in
// runtime/target/classes/META-INF directory
(LocalProject("runtime") / Compile / productDirectories).value ++
(LocalProject(
"syntax-rust-definition"
) / Compile / productDirectories).value
val extraModsToPatch = JPMSUtils.filterModulesFromUpdate(
(Test / update).value,
Seq(
"org.apache.tika" % "tika-core" % tikaVersion
),
streams.value.log,
shouldContainAll = true
)
Map(
(`runtime-fat-jar` / javaModuleName).value -> (modulesToPatchIntoRuntime ++ extraModsToPatch)
)
},
Test / addReads := {
Map(
(`runtime-fat-jar` / javaModuleName).value -> Seq("ALL-UNNAMED")
)
}
)
.settings(
Test / compile := (Test / compile)
.dependsOn(`runtime-fat-jar` / Compile / compileModuleInfo)
.value,
Test / envVars ++= Map(
"ENSO_EDITION_PATH" -> file("distribution/editions").getCanonicalPath
)
)
2020-03-18 13:41:55 +03:00
.dependsOn(`json-rpc-server-test` % Test)
.dependsOn(`json-rpc-server`)
.dependsOn(`task-progress-notifications`)
.dependsOn(`library-manager`)
.dependsOn(`connected-lock-manager-server`)
2021-08-12 17:55:23 +03:00
.dependsOn(`edition-updater`)
.dependsOn(`logging-utils-akka`)
.dependsOn(`logging-service`)
.dependsOn(`polyglot-api`)
.dependsOn(`searcher`)
.dependsOn(`text-buffer`)
.dependsOn(`version-output`)
.dependsOn(pkg)
.dependsOn(`profiling-utils`)
.dependsOn(filewatcher)
2020-07-14 14:44:40 +03:00
.dependsOn(testkit % Test)
.dependsOn(`logging-service-logback` % "test->test")
.dependsOn(`library-manager-test` % Test)
.dependsOn(`runtime-version-manager-test` % Test)
2021-02-22 16:32:55 +03:00
lazy val cleanInstruments = taskKey[Unit](
"Cleans fragile class files to force a full recompilation and preserve" +
"consistency of instrumentation configuration."
)
2021-07-08 16:38:20 +03:00
/** Overrides for the environment variables related to the distribution, so that
* a local installation does not interfere with runtime tests.
*/
val distributionEnvironmentOverrides = {
val fakeDir = file("target/fake_dir").getAbsolutePath
Map(
"ENSO_DATA_DIRECTORY" -> fakeDir,
"ENSO_CONFIG_DIRECTORY" -> fakeDir,
"ENSO_RUNTIME_DIRECTORY" -> file("target/run").getAbsolutePath,
"ENSO_LOG_DIRECTORY" -> file("target/logs").getAbsolutePath,
"ENSO_HOME" -> fakeDir,
"ENSO_EDITION_PATH" -> "",
"ENSO_LIBRARY_PATH" -> "",
"ENSO_AUXILIARY_LIBRARY_CACHES" -> ""
)
}
val frgaalSourceLevel = FrgaalJavaCompiler.sourceLevel
lazy val truffleDslSuppressWarnsSetting = Seq(
Compile / javacOptions ++= Seq(
"-Atruffle.dsl.SuppressWarnings=truffle-inlining"
)
)
/** A setting to replace javac with Frgaal compiler, allowing to use latest Java features in the code
* and still compile down to JDK 17
*/
lazy val frgaalJavaCompilerSetting =
customFrgaalJavaCompilerSettings(targetJavaVersion)
def customFrgaalJavaCompilerSettings(targetJdk: String) = Seq(
Compile / compile / compilers := FrgaalJavaCompiler.compilers(
(Compile / dependencyClasspath).value,
compilers.value,
targetJdk
),
// This dependency is needed only so that developers don't download Frgaal manually.
// Sadly it cannot be placed under plugins either because meta dependencies are not easily
// accessible from the non-meta build definition.
libraryDependencies += FrgaalJavaCompiler.frgaal,
// Ensure that our tooling uses the right Java version for checking the code.
2022-07-01 04:58:14 +03:00
Compile / javacOptions ++= Seq(
"-source",
frgaalSourceLevel,
"--enable-preview"
)
)
lazy val instrumentationSettings = frgaalJavaCompilerSetting ++ Seq(
version := ensoVersion,
commands += WithDebugCommand.withDebug,
2022-07-01 04:58:14 +03:00
Compile / logManager :=
sbt.internal.util.CustomLogManager
.excludeMsg("Could not determine source for class ", Level.Warn),
Compile / javacOptions --= Seq(
"-source",
frgaalSourceLevel,
"--enable-preview"
),
libraryDependencies ++= Seq(
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided"
),
(Compile / javacOptions) ++= Seq(
"-s",
(Compile / sourceManaged).value.getAbsolutePath,
2022-07-01 04:58:14 +03:00
"-Xlint:unchecked"
)
)
2022-07-01 04:58:14 +03:00
lazy val `runtime-language-epb` =
(project in file("engine/runtime-language-epb"))
.settings(
frgaalJavaCompilerSetting,
2022-07-01 04:58:14 +03:00
inConfig(Compile)(truffleRunOptionsSettings),
truffleDslSuppressWarnsSetting,
commands += WithDebugCommand.withDebug,
fork := true,
Test / javaOptions ++= Seq(),
instrumentationSettings,
libraryDependencies ++= Seq(
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided"
)
2022-07-01 04:58:14 +03:00
)
lazy val `runtime-language-arrow` =
(project in file("engine/runtime-language-arrow"))
.enablePlugins(JPMSPlugin)
.settings(
crossPaths := false,
autoScalaLibrary := false,
inConfig(Compile)(truffleRunOptionsSettings),
instrumentationSettings,
libraryDependencies ++= GraalVM.modules ++ Seq(
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.slf4j" % "slf4j-nop" % slf4jVersion % Test,
"org.slf4j" % "slf4j-api" % slf4jVersion % Test,
"org.apache.arrow" % "arrow-vector" % apacheArrowVersion % Test,
"org.apache.arrow" % "arrow-memory-netty" % apacheArrowVersion % Test
),
javaModuleName := "org.enso.interpreter.arrow",
modulePath := {
val updateReport = (Test / update).value
JPMSUtils.filterModulesFromUpdate(
updateReport,
GraalVM.modules,
streams.value.log,
shouldContainAll = true
) ++ Seq(
(LocalProject(
"runtime-language-arrow"
) / Compile / productDirectories).value.head
)
},
Test / patchModules := {
val testClassesDir = (Test / productDirectories).value.head
Map(javaModuleName.value -> Seq(testClassesDir))
},
Test / addModules := Seq(javaModuleName.value),
Test / javaOptions ++= Seq(
s"--add-opens=java.base/java.nio=${javaModuleName.value}", // DirectByteBuffer in MemoryUtil init is in-accessible
"--add-opens=java.base/java.nio=ALL-UNNAMED" // Tests use Apache Arrow
),
Test / addReads := {
Map(javaModuleName.value -> Seq("ALL-UNNAMED"))
}
)
/** `runtime-test-instruments` project contains Truffle instruments that are used solely for testing.
* It is compiled into an explicit Java module. Note that this project cannot have compile-time dependency on `runtime`
* project, so if you need access to classes from `runtime`, you need to use reflection.
*/
lazy val `runtime-test-instruments` =
(project in file("engine/runtime-test-instruments"))
.enablePlugins(JPMSPlugin)
.settings(
inConfig(Compile)(truffleRunOptionsSettings),
truffleDslSuppressWarnsSetting,
instrumentationSettings,
javaModuleName := "org.enso.runtime.test",
modulePath := {
JPMSUtils.filterModulesFromUpdate(
update.value,
GraalVM.modules ++ Seq(
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck-common" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck-tests" % graalMavenPackagesVersion,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion
),
streams.value.log,
shouldContainAll = true
)
},
libraryDependencies ++= GraalVM.modules,
libraryDependencies ++= Seq(
Add Truffle TCK libraryDependencies to runtime-test-instruments. (#8766) Since #8685, there is the following error (warning) message from JPMS plugin when building `runtime-test-instruments`: ``` [error] Returned (10): Vector(/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/nativeimage/23.1.0/nativeimage-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/word/23.1.0/word-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/jniutils/23.1.0/jniutils-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/collections/23.1.0/collections-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/polyglot/polyglot/23.1.0/polyglot-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-api/23.1.0/truffle-api-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-runtime/23.1.0/truffle-runtime-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-compiler/23.1.0/truffle-compiler-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/polyglot-tck/23.1.0/polyglot-tck-23.1.0.jar, /home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/netbeans/api/org-openide-util-lookup/RELEASE180/org-openide-util-lookup-RELEASE180.jar) [error] Expected: (13): List(org.graalvm.sdk:nativeimage:23.1.0, org.graalvm.sdk:word:23.1.0, org.graalvm.sdk:jniutils:23.1.0, org.graalvm.sdk:collections:23.1.0, org.graalvm.polyglot:polyglot:23.1.0, org.graalvm.truffle:truffle-api:23.1.0, org.graalvm.truffle:truffle-runtime:23.1.0, org.graalvm.truffle:truffle-compiler:23.1.0, org.graalvm.sdk:polyglot-tck:23.1.0, org.graalvm.truffle:truffle-tck:23.1.0, org.graalvm.truffle:truffle-tck-common:23.1.0, org.graalvm.truffle:truffle-tck-tests:23.1.0, org.netbeans.api:org-openide-util-lookup:RELEASE180) ``` This PR removes this error message by providing appropriate `libraryDependencies` to `runtime-test-instruments`.
2024-01-16 13:44:50 +03:00
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck-common" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck-tests" % graalMavenPackagesVersion,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided"
)
)
lazy val runtime = (project in file("engine/runtime"))
.enablePlugins(JPMSPlugin)
.settings(
frgaalJavaCompilerSetting,
truffleDslSuppressWarnsSetting,
2022-07-01 04:58:14 +03:00
Compile / logManager :=
sbt.internal.util.CustomLogManager.excludeMsg(
"Could not determine source for class ",
Level.Warn
),
2020-05-15 11:07:58 +03:00
version := ensoVersion,
2019-09-12 17:47:25 +03:00
commands += WithDebugCommand.withDebug,
2019-11-05 17:12:33 +03:00
inConfig(Compile)(truffleRunOptionsSettings),
libraryDependencies ++= GraalVM.langsPkgs ++ Seq(
"org.apache.commons" % "commons-lang3" % commonsLangVersion,
"org.apache.tika" % "tika-core" % tikaVersion,
"com.lihaoyi" %% "fansi" % fansiVersion,
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided",
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided",
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
"org.scalacheck" %% "scalacheck" % scalacheckVersion % Test,
"org.scalactic" %% "scalactic" % scalacticVersion % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.hamcrest" % "hamcrest-all" % hamcrestVersion % Test,
"org.slf4j" % "slf4j-api" % slf4jVersion % Test
),
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// Add all GraalVM packages with Runtime scope - we don't need them for compilation,
// just provide them at runtime (in module-path).
libraryDependencies ++= {
val necessaryModules =
GraalVM.modules.map(_.withConfigurations(Some(Runtime.name)))
val langs =
GraalVM.langsPkgs.map(_.withConfigurations(Some(Runtime.name)))
val tools =
GraalVM.toolsPkgs.map(_.withConfigurations(Some(Runtime.name)))
necessaryModules ++ langs ++ tools
}
)
.settings(
(Compile / javacOptions) ++= Seq(
"-s",
(Compile / sourceManaged).value.getAbsolutePath,
"-Xlint:unchecked"
)
)
.settings(
(Compile / compile) := (Compile / compile)
.dependsOn(Def.task { (Compile / sourceManaged).value.mkdirs })
.value
)
2020-09-30 14:33:57 +03:00
.settings(
(Runtime / compile) := (Runtime / compile)
.dependsOn(`std-base` / Compile / packageBin)
.dependsOn(`enso-test-java-helpers` / Compile / packageBin)
.dependsOn(`benchmark-java-helpers` / Compile / packageBin)
.dependsOn(`exploratory-benchmark-java-helpers` / Compile / packageBin)
.dependsOn(`std-image` / Compile / packageBin)
.dependsOn(`std-database` / Compile / packageBin)
2021-09-03 22:41:12 +03:00
.dependsOn(`std-google-api` / Compile / packageBin)
.dependsOn(`std-table` / Compile / packageBin)
.dependsOn(`std-aws` / Compile / packageBin)
.value
2020-09-30 14:33:57 +03:00
)
.dependsOn(`common-polyglot-core-utils`)
.dependsOn(`edition-updater`)
2020-06-24 20:02:42 +03:00
.dependsOn(`interpreter-dsl`)
.dependsOn(`persistance-dsl` % "provided")
.dependsOn(`library-manager`)
.dependsOn(`logging-truffle-connector`)
.dependsOn(`polyglot-api`)
.dependsOn(`text-buffer`)
.dependsOn(`runtime-compiler`)
.dependsOn(`connected-lock-manager`)
.dependsOn(testkit % Test)
/** A project holding all the runtime integration tests. These tests require, among other things,
* the `org.enso.runtime` JPMS module, so it is easier to keep them in a separate project.
* For standard unit tests, use `runtime/Test`.
*/
lazy val `runtime-integration-tests` =
(project in file("engine/runtime-integration-tests"))
.enablePlugins(JPMSPlugin)
.settings(
frgaalJavaCompilerSetting,
Compile / logManager :=
sbt.internal.util.CustomLogManager.excludeMsg(
"Could not determine source for class ",
Level.Warn
),
commands += WithDebugCommand.withDebug,
libraryDependencies ++= GraalVM.modules ++ GraalVM.langsPkgs ++ GraalVM.insightPkgs ++ logbackPkg ++ Seq(
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided",
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided",
"org.graalvm.truffle" % "truffle-tck" % graalMavenPackagesVersion % Test,
"org.graalvm.truffle" % "truffle-tck-common" % graalMavenPackagesVersion % Test,
"org.graalvm.truffle" % "truffle-tck-tests" % graalMavenPackagesVersion % Test,
"org.scalacheck" %% "scalacheck" % scalacheckVersion % Test,
"org.scalactic" %% "scalactic" % scalacticVersion % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.hamcrest" % "hamcrest-all" % hamcrestVersion % Test,
"org.slf4j" % "slf4j-api" % slf4jVersion % Test
),
Test / javacOptions ++= Seq(
"-s",
(Compile / sourceManaged).value.getAbsolutePath,
"-Xlint:unchecked"
),
Test / compile := (Test / compile)
.dependsOn(Def.task { (Compile / sourceManaged).value.mkdirs })
.dependsOn(`runtime-fat-jar` / Compile / compileModuleInfo)
.value,
Test / fork := true,
Test / parallelExecution := false,
Test / logBuffered := false,
Test / envVars ++= distributionEnvironmentOverrides ++ Map(
"ENSO_TEST_DISABLE_IR_CACHE" -> "false",
"ENSO_EDITION_PATH" -> file("distribution/editions").getCanonicalPath
),
inConfig(Test)(truffleRunOptionsSettings),
Test / javaOptions ++= Seq(
"-Dtck.values=java-host,enso",
"-Dtck.language=enso",
"-Dtck.inlineVerifierInstrument=false",
"-Dpolyglot.engine.AllowExperimentalOptions=true"
),
Test / javaOptions ++= testLogProviderOptions,
Test / addModules := Seq(
(`runtime-test-instruments` / javaModuleName).value,
(`runtime-fat-jar` / javaModuleName).value
),
Test / modulePath := {
val updateReport = (Test / update).value
val requiredModIds =
GraalVM.modules ++ GraalVM.langsPkgs ++ GraalVM.insightPkgs ++ logbackPkg ++ Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion,
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck-common" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-tck-tests" % graalMavenPackagesVersion
)
val requiredMods = JPMSUtils.filterModulesFromUpdate(
updateReport,
requiredModIds,
streams.value.log,
shouldContainAll = true
)
val runtimeTestInstrumentsMod =
(`runtime-test-instruments` / Compile / exportedProducts).value.head.data
val runtimeMod =
(`runtime-fat-jar` / Compile / exportedProducts).value.head.data
requiredMods ++
Seq(runtimeTestInstrumentsMod) ++
Seq(runtimeMod)
},
Test / patchModules := {
/** All these modules will be in --patch-module cmdline option to java, which means that
* for the JVM, it will appear that all the classes contained in these sbt projects are contained
* in the `org.enso.runtime` module. In this way, we do not have to assembly the `runtime.jar`
* fat jar.
*/
val modulesToPatchIntoRuntime: Seq[File] =
(LocalProject(
"runtime-instrument-common"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-instrument-id-execution"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-instrument-repl-debugger"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-instrument-runtime-server"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-language-epb"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-compiler"
) / Compile / productDirectories).value ++
(LocalProject(
"refactoring-utils"
) / Compile / productDirectories).value ++
(LocalProject(
"runtime-instrument-common"
) / Test / productDirectories).value
// Patch test-classes into the runtime module. This is standard way to deal with the
// split package problem in unit tests. For example, Maven's surefire plugin does this.
val testClassesDir = (Test / productDirectories).value.head
Map(
(`runtime-fat-jar` / javaModuleName).value -> (modulesToPatchIntoRuntime ++ Seq(
testClassesDir
))
)
},
Test / addReads := {
val runtimeModName = (`runtime-fat-jar` / javaModuleName).value
val testInstrumentsModName =
(`runtime-test-instruments` / javaModuleName).value
Map(
// We patched the test-classes into the runtime module. These classes access some stuff from
// unnamed module. Thus, let's add ALL-UNNAMED.
runtimeModName -> Seq(
"ALL-UNNAMED",
testInstrumentsModName,
"truffle.tck.tests",
"org.openide.util.lookup.RELEASE180"
),
testInstrumentsModName -> Seq(runtimeModName)
)
}
)
.dependsOn(`runtime-fat-jar`)
.dependsOn(`runtime-test-instruments`)
.dependsOn(`logging-service-logback` % "test->test")
.dependsOn(testkit % Test)
.dependsOn(`connected-lock-manager-server`)
/** A project that holds only benchmarks for `runtime`. Unlike `runtime-integration-tests`, its execution requires
* the whole `runtime-fat-jar` assembly, as we want to be as close to the enso distribution as possible.
*/
lazy val `runtime-benchmarks` =
(project in file("engine/runtime-benchmarks"))
.enablePlugins(JPMSPlugin)
.settings(
frgaalJavaCompilerSetting,
// Note that withDebug command only makes sense if you use `@Fork(0)` in your benchmarks.
commands += WithDebugCommand.withDebug,
libraryDependencies ++= GraalVM.modules ++ GraalVM.langsPkgs ++ GraalVM.toolsPkgs ++ Seq(
"org.openjdk.jmh" % "jmh-core" % jmhVersion,
"org.openjdk.jmh" % "jmh-generator-annprocess" % jmhVersion,
"jakarta.xml.bind" % "jakarta.xml.bind-api" % jaxbVersion,
"com.sun.xml.bind" % "jaxb-impl" % jaxbVersion,
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion,
"org.graalvm.truffle" % "truffle-dsl-processor" % graalMavenPackagesVersion % "provided",
"org.slf4j" % "slf4j-api" % slf4jVersion,
"org.slf4j" % "slf4j-nop" % slf4jVersion
),
mainClass :=
Some("org.enso.interpreter.bench.benchmarks.RuntimeBenchmarksRunner"),
Compile / logManager :=
sbt.internal.util.CustomLogManager.excludeMsg(
"Could not determine source for class ",
Level.Warn
),
javacOptions --= Seq(
"-source",
frgaalSourceLevel,
"--enable-preview"
),
javacOptions ++= Seq(
"-s",
(Compile / sourceManaged).value.getAbsolutePath,
"-Xlint:unchecked"
),
Compile / compile := (Compile / compile)
dry-run benchmarks exits when some benchmark fails (#9397) If some benchmark fails in dry-run (compileOnly) mode, the whole process exits with non-zero return code. Also fixes failing engine compiler benchmarks. # Important Notes Manually added failure: ```diff diff --git a/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java b/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java index c8d86cecc..f9f4d7cbc 100644 --- a/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java +++ b/engine/runtime-benchmarks/src/main/java/org/enso/interpreter/bench/benchmarks/semantic/ArrayProxyBenchmarks.java @@ -95,7 +95,8 @@ public class ArrayProxyBenchmarks { @Benchmark public void sumOverComputingProxy(Blackhole matter) { - performBenchmark(matter); + //performBenchmark(matter); + throw new AssertionError("My error"); } @Benchmark ``` Run with `sbt "-Dbench.compileOnly=true runtime-benchmarks/benchOnly org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy"` fails with: ``` [info] Running benchmarks [org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy] in compileOnly mode [info] # JMH version: 1.36 [info] # VM version: JDK 21.0.2, Java HotSpot(TM) 64-Bit Server VM, 21.0.2+13-LTS-jvmci-23.1-b30 [info] # VM invoker: /home/pavel/.sdkman/candidates/java/21.0.2-graal/bin/java [info] # VM options: -XX:ThreadPriorityPolicy=1 -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCIProduct -XX:-UnlockExperimentalVMOptions -Dslf4j.provider=org.slf4j.nop.NOPServiceProvider -Dbench.compileOnly=true --module-path=/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/nativeimage/23.1.2/nativeimage-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/word/23.1.2/word-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/jniutils/23.1.2/jniutils-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/sdk/collections/23.1.2/collections-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/polyglot/polyglot/23.1.2/polyglot-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-api/23.1.2/truffle-api-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-runtime/23.1.2/truffle-runtime-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-compiler/23.1.2/truffle-compiler-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/js/js-language/23.1.2/js-language-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/regex/regex/23.1.2/regex-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/shadowed/icu4j/23.1.2/icu4j-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/python/python-language/23.1.2/python-language-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/python/python-resources/23.1.2/python-resources-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcutil-jdk18on/1.76/bcutil-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcpkix-jdk18on/1.76/bcpkix-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/bouncycastle/bcprov-jdk18on/1.76/bcprov-jdk18on-1.76.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/llvm/llvm-api/23.1.2/llvm-api-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-nfi/23.1.2/truffle-nfi-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/truffle/truffle-nfi-libffi/23.1.2/truffle-nfi-libffi-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/tools/profiler-tool/23.1.2/profiler-tool-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/graalvm/shadowed/json/23.1.2/json-23.1.2.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/tukaani/xz/1.9/xz-1.9.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-api/2.0.9/slf4j-api-2.0.9.jar:/home/pavel/.cache/coursier/v1/https/repo1.maven.org/maven2/org/slf4j/slf4j-nop/2.0.9/slf4j-nop-2.0.9.jar:/home/pavel/dev/enso/runtime.jar --add-modules=org.enso.runtime --add-exports=org.slf4j.nop/org.slf4j.nop=org.slf4j [info] # Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable) [info] # Warmup: <none> [info] # Measurement: 1 iterations, 1 s each [info] # Timeout: 10 min per iteration [info] # Threads: 1 thread, will synchronize iterations [info] # Benchmark mode: Average time, time/op [info] # Benchmark: org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy [info] # Run progress: 0.00% complete, ETA 00:00:01 [info] # Fork: N/A, test runs in the host VM [info] # *** WARNING: Non-forked runs may silently omit JVM options, mess up profilers, disable compiler hints, etc. *** [info] # *** WARNING: Use non-forked runs only for debugging purposes, not for actual performance runs. *** [error] SLF4J: Attempting to load provider "org.slf4j.nop.NOPServiceProvider" specified via "slf4j.provider" system property [info] Iteration 1: <failure> [info] java.lang.AssertionError: My error [info] at org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy(ArrayProxyBenchmarks.java:99) [info] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_avgt_jmhStub(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:232) [info] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_AverageTime(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:173) [info] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) [info] at java.base/java.lang.reflect.Method.invoke(Method.java:580) [info] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:475) [info] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:458) [info] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) [info] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) [info] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) [info] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [info] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [error] Benchmark run failed: Benchmark caught the exception [info] at java.base/java.lang.Thread.run(Thread.java:1583) [error] org.openjdk.jmh.runner.RunnerException: Benchmark caught the exception [error] at org.openjdk.jmh.runner.Runner.runBenchmarks(Runner.java:575) [error] at org.openjdk.jmh.runner.Runner.internalRun(Runner.java:310) [error] at org.openjdk.jmh.runner.Runner.run(Runner.java:209) [error] at org.enso.interpreter.bench.BenchmarksRunner.runCompileOnly(BenchmarksRunner.java:93) [error] at org.enso.interpreter.bench.BenchmarksRunner.run(BenchmarksRunner.java:36) [error] at org.enso.interpreter.bench.benchmarks.RuntimeBenchmarksRunner.main(RuntimeBenchmarksRunner.java:8) [error] Caused by: org.openjdk.jmh.runner.BenchmarkException: Benchmark error during the run [error] at org.openjdk.jmh.runner.BenchmarkHandler.runIteration(BenchmarkHandler.java:424) [error] at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:281) [error] at org.openjdk.jmh.runner.BaseRunner.runBenchmark(BaseRunner.java:233) [error] at org.openjdk.jmh.runner.BaseRunner.doSingle(BaseRunner.java:138) [error] at org.openjdk.jmh.runner.BaseRunner.runBenchmarksEmbedded(BaseRunner.java:110) [error] at org.openjdk.jmh.runner.Runner.runBenchmarks(Runner.java:555) [error] ... 5 more [error] Suppressed: java.lang.AssertionError: My error [error] at org.enso.interpreter.bench.benchmarks.semantic.ArrayProxyBenchmarks.sumOverComputingProxy(ArrayProxyBenchmarks.java:99) [error] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_avgt_jmhStub(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:232) [error] at org.enso.interpreter.bench.benchmarks.semantic.jmh_generated.ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.sumOverComputingProxy_AverageTime(ArrayProxyBenchmarks_sumOverComputingProxy_jmhTest.java:173) [error] at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) [error] at java.base/java.lang.reflect.Method.invoke(Method.java:580) [error] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:475) [error] at org.openjdk.jmh.runner.BenchmarkHandler$BenchmarkTask.call(BenchmarkHandler.java:458) [error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) [error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) [error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317) [error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [error] at java.base/java.lang.Thread.run(Thread.java:1583) [error] Nonzero exit code returned from runner: 1 [error] (Compile / run) Nonzero exit code returned from runner: 1 [error] Total time: 5 s, completed Mar 13, 2024, 12:49:59 PM ```
2024-03-14 18:21:38 +03:00
.dependsOn(`runtime-fat-jar` / assembly)
.dependsOn(Def.task { (Compile / sourceManaged).value.mkdirs })
.value,
parallelExecution := false,
modulePath := {
val requiredModIds = GraalVM.modules ++ GraalVM.langsPkgs ++ Seq(
"org.slf4j" % "slf4j-api" % slf4jVersion,
"org.slf4j" % "slf4j-nop" % slf4jVersion
)
val requiredMods = JPMSUtils.filterModulesFromUpdate(
(Compile / update).value,
requiredModIds,
streams.value.log,
shouldContainAll = true
)
val runtimeMod =
(`runtime-fat-jar` / assembly / assemblyOutputPath).value
requiredMods ++ Seq(runtimeMod)
},
addModules := {
val runtimeModuleName = (`runtime-fat-jar` / javaModuleName).value
Seq(runtimeModuleName)
},
addExports := {
Map("org.slf4j.nop/org.slf4j.nop" -> Seq("org.slf4j"))
},
javaOptions ++= {
Seq(
// To enable logging in benchmarks, add ch.qos.logback module on the modulePath
"-Dslf4j.provider=org.slf4j.nop.NOPServiceProvider"
)
},
javaOptions ++= benchOnlyOptions,
run / fork := true,
run / connectInput := true,
bench := Def
.task {
(Compile / run).toTask("").tag(Exclusive).value
}
.dependsOn(
buildEngineDistribution
)
.value,
benchOnly := Def.inputTaskDyn {
import complete.Parsers.spaceDelimited
val name = spaceDelimited("<name>").parsed match {
case List(name) => name
case _ => throw new IllegalArgumentException("Expected one argument.")
}
Def.task {
(Compile / run).toTask(" " + name).value
}
}.evaluated
)
.dependsOn(`runtime-fat-jar`)
.dependsOn(`benchmarks-common`)
2019-11-18 16:36:03 +03:00
lazy val `runtime-parser` =
(project in file("engine/runtime-parser"))
.settings(
frgaalJavaCompilerSetting,
commands += WithDebugCommand.withDebug,
fork := true,
Test / javaOptions ++= Seq(
"-Dgraalvm.locatorDisabled=true",
s"--upgrade-module-path=${file("engine/runtime/build-cache/truffle-api.jar").absolutePath}"
),
libraryDependencies ++= Seq(
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided"
2023-12-19 17:43:48 +03:00
),
(Compile / logManager) :=
sbt.internal.util.CustomLogManager.excludeMsg(
"Could not determine source for class ",
Level.Warn
)
)
.dependsOn(syntax)
.dependsOn(`syntax-rust-definition`)
.dependsOn(`persistance`)
.dependsOn(`persistance-dsl` % "provided")
lazy val `runtime-compiler` =
(project in file("engine/runtime-compiler"))
.settings(
frgaalJavaCompilerSetting,
instrumentationSettings,
libraryDependencies ++= Seq(
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided"
)
)
.dependsOn(`runtime-parser`)
.dependsOn(pkg)
.dependsOn(`polyglot-api`)
.dependsOn(editions)
.dependsOn(`persistance-dsl` % "provided")
lazy val `runtime-instrument-common` =
(project in file("engine/runtime-instrument-common"))
.configs(Benchmark)
.settings(
frgaalJavaCompilerSetting,
inConfig(Compile)(truffleRunOptionsSettings),
inConfig(Benchmark)(Defaults.testSettings),
instrumentationSettings,
Test / javaOptions ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"-Dpolyglotimpl.DisableClassPathIsolation=true"
),
bench := (Benchmark / test).tag(Exclusive).value,
Benchmark / parallelExecution := false,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
(Benchmark / javaOptions) :=
(LocalProject("std-benchmarks") / Compile / javaOptions).value,
Test / fork := true,
Test / envVars ++= distributionEnvironmentOverrides ++ Map(
"ENSO_TEST_DISABLE_IR_CACHE" -> "false"
),
libraryDependencies ++= Seq(
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.dependsOn(`refactoring-utils`)
.dependsOn(
LocalProject(
"runtime"
) % "compile->compile;runtime->runtime;bench->bench"
)
2022-07-01 04:58:14 +03:00
lazy val `runtime-instrument-id-execution` =
(project in file("engine/runtime-instrument-id-execution"))
.settings(
frgaalJavaCompilerSetting,
2022-07-01 04:58:14 +03:00
inConfig(Compile)(truffleRunOptionsSettings),
instrumentationSettings
)
.dependsOn(LocalProject("runtime"))
.dependsOn(`runtime-instrument-common`)
2022-07-01 04:58:14 +03:00
lazy val `runtime-instrument-repl-debugger` =
(project in file("engine/runtime-instrument-repl-debugger"))
.settings(
inConfig(Compile)(truffleRunOptionsSettings),
instrumentationSettings
)
.dependsOn(LocalProject("runtime"))
.dependsOn(`runtime-instrument-common`)
2022-07-01 04:58:14 +03:00
lazy val `runtime-instrument-runtime-server` =
(project in file("engine/runtime-instrument-runtime-server"))
.settings(
inConfig(Compile)(truffleRunOptionsSettings),
instrumentationSettings
)
.dependsOn(LocalProject("runtime"))
.dependsOn(`runtime-instrument-common` % "test->test;compile->compile")
/** A "meta" project that exists solely to provide logic for assembling the `runtime.jar` fat Jar.
* We do not want to put this task into any other existing project, as it internally copies some
* classes from other projects into the `classes` directory, therefore, pollutes the build.
* There is only one Java source in this project - `module-info.java`. During the assembling of the
* fat jar, all the classes from the dependent projects are copied into the `classes` directory of
* this project and then, a custom task is invoked to compile the `module-info.java`.
*/
lazy val `runtime-fat-jar` =
(project in file("engine/runtime-fat-jar"))
.enablePlugins(JPMSPlugin)
.settings(
Compile / compileModuleInfo := {
JPMSUtils.compileModuleInfo(
copyDepsFilter = ScopeFilter(
inProjects(
LocalProject("runtime"),
LocalProject("runtime-language-epb"),
LocalProject("runtime-instrument-common"),
LocalProject("runtime-instrument-id-execution"),
LocalProject("runtime-instrument-repl-debugger"),
LocalProject("runtime-instrument-runtime-server")
),
inConfigurations(Compile)
),
modulePath = JPMSUtils.componentModules
)
}
.dependsOn(Compile / compile)
.value,
// Filter module-info.java from the compilation
excludeFilter := excludeFilter.value || "module-info.java",
javaModuleName := "org.enso.runtime",
compileOrder := CompileOrder.JavaThenScala
)
/** The following libraryDependencies are provided in Runtime scope.
* Later, we will collect them into --module-path option.
* We don't collect them in Compile scope as it does not even make sense
* to run `compile` task in this project.
*/
2022-07-01 04:58:14 +03:00
.settings(
libraryDependencies ++= {
val graalMods =
GraalVM.modules.map(_.withConfigurations(Some(Runtime.name)))
val langMods =
GraalVM.langsPkgs.map(_.withConfigurations(Some(Runtime.name)))
val logbackMods =
logbackPkg.map(_.withConfigurations(Some(Runtime.name)))
graalMods ++ langMods ++ logbackMods
}
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
)
/** Assembling Uber Jar */
.settings(
assembly := assembly
.dependsOn(Compile / compile)
.dependsOn(Compile / compileModuleInfo)
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
.value,
2022-07-01 04:58:14 +03:00
assembly / assemblyJarName := "runtime.jar",
assembly / test := {},
assembly / assemblyOutputPath := file("runtime.jar"),
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
assembly / assemblyExcludedJars := {
val pkgsToExclude = JPMSUtils.componentModules
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val ourFullCp = (Runtime / fullClasspath).value
JPMSUtils.filterModulesFromClasspath(
ourFullCp,
pkgsToExclude,
streams.value.log
)
},
2022-07-01 04:58:14 +03:00
assembly / assemblyMergeStrategy := {
case PathList("META-INF", file, xs @ _*) if file.endsWith(".DSA") =>
MergeStrategy.discard
case PathList("META-INF", file, xs @ _*) if file.endsWith(".SF") =>
MergeStrategy.discard
case PathList("META-INF", "MANIFEST.MF", xs @ _*) =>
MergeStrategy.discard
case PathList("META-INF", "services", xs @ _*) =>
MergeStrategy.concat
case PathList("module-info.class") =>
MergeStrategy.preferProject
case PathList(xs @ _*) if xs.last.contains("module-info.class") =>
MergeStrategy.discard
2022-07-01 04:58:14 +03:00
case _ => MergeStrategy.first
}
)
.dependsOn(`runtime-instrument-common`)
2022-07-01 04:58:14 +03:00
.dependsOn(`runtime-instrument-id-execution`)
.dependsOn(`runtime-instrument-repl-debugger`)
.dependsOn(`runtime-instrument-runtime-server`)
.dependsOn(`runtime-language-epb`)
.dependsOn(LocalProject("runtime"))
/* Note [Unmanaged Classpath]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~
* As the definition of the core primitives in `core_definition` is achieved
* entirely using the graph macros, this means that the IDE experience for those
* using these primitives is very poor.
*
* To get around this, we want to treat the core definition as a .jar dependency
* to force the IDE to depend on bytecode for its diagnostics, rather than the
* source code (as this means it sees the macros expanded). A standard workflow
* with local publishing would not recompile the definition automatically on
* changes, so the `unmanagedClasspath` route allows us to get automatic
* recompilation but still convince the IDE that it is a .jar dependency.
*/
lazy val `engine-runner` = project
.in(file("engine/runner"))
2019-11-18 16:36:03 +03:00
.settings(
frgaalJavaCompilerSetting,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
truffleDslSuppressWarnsSetting,
javaOptions ++= {
// Note [Classpath Separation]
val runtimeClasspath =
(runtime / Compile / fullClasspath).value
.map(_.data)
.mkString(File.pathSeparator)
Seq(s"-Dtruffle.class.path.append=$runtimeClasspath")
},
packageOptions := Seq(
// The `Multi-Release: true` comes from the `org.xerial/sqlite-jdbc` dependency.
// But the current version of sbt-assembly does not allow to merge MANIFEST.MF
// files this way.
Package.ManifestAttributes(("Multi-Release", "true"))
),
2021-05-14 15:08:39 +03:00
Compile / run / mainClass := Some("org.enso.runner.Main"),
assembly / mainClass := (Compile / run / mainClass).value,
assembly / assemblyJarName := "runner.jar",
assembly / test := {},
assembly / assemblyOutputPath := file("runner.jar"),
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
assembly / assemblyExcludedJars :=
JPMSUtils.filterTruffleAndGraalArtifacts(
(Compile / fullClasspath).value
),
2021-05-14 15:08:39 +03:00
assembly / assemblyMergeStrategy := {
2020-03-06 17:17:46 +03:00
case PathList("META-INF", file, xs @ _*) if file.endsWith(".DSA") =>
MergeStrategy.discard
case PathList("META-INF", file, xs @ _*) if file.endsWith(".SF") =>
MergeStrategy.discard
case PathList("META-INF", "MANIFEST.MF", xs @ _*) =>
MergeStrategy.discard
case "application.conf" =>
MergeStrategy.concat
case "reference.conf" =>
MergeStrategy.concat
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
case PathList(xs @ _*) if xs.last.contains("module-info") =>
// runner.jar must not be a JPMS module
MergeStrategy.discard
2020-03-06 17:17:46 +03:00
case x =>
MergeStrategy.first
},
2020-03-06 16:40:29 +03:00
commands += WithDebugCommand.withDebug,
2019-11-18 16:36:03 +03:00
inConfig(Compile)(truffleRunOptionsSettings),
libraryDependencies ++= Seq(
"org.graalvm.sdk" % "polyglot-tck" % graalMavenPackagesVersion % Provided,
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % Provided,
"commons-cli" % "commons-cli" % commonsCliVersion,
"com.monovore" %% "decline" % declineVersion,
"org.jline" % "jline" % jlineVersion,
"org.typelevel" %% "cats-core" % catsVersion,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
),
2021-05-14 15:08:39 +03:00
run / connectInput := true
2019-11-18 16:36:03 +03:00
)
.settings(
assembly := assembly
.dependsOn(`runtime-fat-jar` / assembly)
.value,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
rebuildNativeImage :=
NativeImage
.buildNativeImage(
"runner",
staticOnLinux = false,
additionalOptions = Seq(
"-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog",
"-H:IncludeResources=.*Main.enso$",
// "-g",
// "-H:+DashboardAll",
// "-H:DashboardDump=runner.bgv"
"-Dnic=nic"
),
mainClass = Some("org.enso.runner.Main"),
additionalCp = Seq(
"runtime.jar",
"runner.jar"
),
initializeAtRuntime = Seq(
"org.jline.nativ.JLineLibrary",
"org.jline.terminal.impl.jna",
"io.methvin.watchservice.jna.CarbonAPI",
"org.enso.syntax2.Parser",
"zio.internal.ZScheduler$$anon$4",
"org.enso.runner.Main$",
"sun.awt",
"sun.java2d",
"sun.font",
"java.awt",
"com.sun.imageio",
"com.sun.jna.internal.Cleaner",
"com.sun.jna.Structure$FFIType",
"akka.http"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
)
Generate native image for engine-runner (#3638) This PR adds a possibility to generate native-image for engine-runner. Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported (that will be resolved at a later point). The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily. To generate a native image simply execute: ``` sbt> engine-runner-native/buildNativeImage ... (wait a few minutes) ``` The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits see the timings difference between the non-native and native one: ``` >time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6 720 real 0m4.503s user 0m9.248s sys 0m1.494s > time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6 720 real 0m0.176s user 0m0.042s sys 0m0.038s ``` # Important Notes Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`. Will need to add the above command to CI.
2022-09-22 17:45:10 +03:00
)
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
.dependsOn(assembly)
.value,
Generate native image for engine-runner (#3638) This PR adds a possibility to generate native-image for engine-runner. Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported (that will be resolved at a later point). The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily. To generate a native image simply execute: ``` sbt> engine-runner-native/buildNativeImage ... (wait a few minutes) ``` The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits see the timings difference between the non-native and native one: ``` >time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6 720 real 0m4.503s user 0m9.248s sys 0m1.494s > time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6 720 real 0m0.176s user 0m0.042s sys 0m0.038s ``` # Important Notes Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`. Will need to add the above command to CI.
2022-09-22 17:45:10 +03:00
buildNativeImage := NativeImage
.incrementalNativeImageBuild(
rebuildNativeImage,
"runner"
)
.value
)
.dependsOn(`version-output`)
.dependsOn(pkg)
.dependsOn(cli)
.dependsOn(`library-manager`)
.dependsOn(`language-server`)
.dependsOn(`edition-updater`)
.dependsOn(`logging-service`)
.dependsOn(`logging-service-logback` % Runtime)
.dependsOn(`polyglot-api`)
Generate native image for engine-runner (#3638) This PR adds a possibility to generate native-image for engine-runner. Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported (that will be resolved at a later point). The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily. To generate a native image simply execute: ``` sbt> engine-runner-native/buildNativeImage ... (wait a few minutes) ``` The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits see the timings difference between the non-native and native one: ``` >time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6 720 real 0m4.503s user 0m9.248s sys 0m1.494s > time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6 720 real 0m0.176s user 0m0.042s sys 0m0.038s ``` # Important Notes Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`. Will need to add the above command to CI.
2022-09-22 17:45:10 +03:00
2020-07-10 13:57:42 +03:00
lazy val launcher = project
.in(file("engine/launcher"))
.configs(Test)
.settings(
resolvers += Resolver.bintrayRepo("gn0s1s", "releases"),
2020-07-10 13:57:42 +03:00
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.typelevel" %% "cats-core" % catsVersion,
"org.apache.commons" % "commons-compress" % commonsCompressVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
akkaSLF4J
)
2020-07-10 13:57:42 +03:00
)
.settings(
2020-12-09 16:58:11 +03:00
rebuildNativeImage := NativeImage
.buildNativeImage(
"enso",
staticOnLinux = true,
2020-12-09 16:58:11 +03:00
additionalOptions = Seq(
"-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog",
2020-12-09 16:58:11 +03:00
"-H:IncludeResources=.*Main.enso$"
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
),
includeRuntime = false,
mainClass = Some("org.enso.launcher.cli.Main")
)
2020-12-09 16:58:11 +03:00
.dependsOn(assembly)
.dependsOn(VerifyReflectionSetup.run)
2020-12-09 16:58:11 +03:00
.value,
buildNativeImage := NativeImage
.incrementalNativeImageBuild(
rebuildNativeImage,
"enso"
)
.value,
2021-05-14 15:08:39 +03:00
assembly / test := {},
assembly / assemblyOutputPath := file("launcher.jar"),
assembly / assemblyMergeStrategy := {
case PathList("META-INF", file, xs @ _*) if file.endsWith(".DSA") =>
MergeStrategy.discard
case PathList("META-INF", file, xs @ _*) if file.endsWith(".SF") =>
MergeStrategy.discard
case PathList("META-INF", "MANIFEST.MF", xs @ _*) =>
MergeStrategy.discard
case "application.conf" => MergeStrategy.concat
case "reference.conf" => MergeStrategy.concat
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// launcher.jar must not be an explicit Jar module
case PathList(xs @ _*) if xs.last.contains("module-info") =>
MergeStrategy.discard
case x =>
MergeStrategy.first
}
2020-07-10 13:57:42 +03:00
)
.settings(
Test / fork := true,
Test / javaOptions ++= testLogProviderOptions,
(Test / test) := (Test / test)
2020-12-09 16:58:11 +03:00
.dependsOn(buildNativeImage)
.dependsOn(LauncherShimsForTest.prepare())
.value,
(Test / testOnly) := (Test / testOnly)
.dependsOn(buildNativeImage)
.dependsOn(LauncherShimsForTest.prepare())
Always log to console and file (#7825) * Always log verbose to a file The change adds an option by default to always log to a file with verbose log level. The implementation is a bit tricky because in the most common use-case we have to always log in verbose mode to a socket and only later apply the desired log levels. Previously socket appender would respect the desired log level already before forwarding the log. If by default we log to a file, verbose mode is simply ignored and does not override user settings. To test run `project-manager` with `ENSO_LOGSERVER_APPENDER=console` env variable. That will output to the console with the default `INFO` level and `TRACE` log level for the file. * add docs * changelog * Address some PR requests 1. Log INFO level to CONSOLE by default 2. Change runner's default log level from ERROR to WARN Took a while to figure out why the correct log level wasn't being passed to the language server, therefore ignoring the (desired) verbose logs from the log file. * linter * 3rd party uses log4j for logging Getting rid of the warning by adding a log4j over slf4j bridge: ``` ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console... ``` * legal review update * Make sure tests use test resources Having `application.conf` in `src/main/resources` and `test/resources` does not guarantee that in Tests we will pick up the latter. Instead, by default it seems to do some kind of merge of different configurations, which is far from desired. * Ensure native launcher test log to console only Logging to console and (temporary) files is problematic for Windows. The CI also revealed a problem with the native configuration because it was not possible to modify the launcher via env variables as everything was initialized during build time. * Adapt to method changes * Potentially deal with Windows failures
2023-09-26 12:32:04 +03:00
.evaluated,
Test / fork := true
)
.dependsOn(cli)
.dependsOn(`runtime-version-manager`)
2020-07-10 13:57:42 +03:00
.dependsOn(`version-output`)
.dependsOn(pkg)
.dependsOn(`logging-utils` % "test->test")
.dependsOn(`logging-service`)
.dependsOn(`logging-service-logback` % "test->test;runtime->runtime")
2021-06-18 17:39:45 +03:00
.dependsOn(`distribution-manager` % Test)
.dependsOn(`runtime-version-manager-test` % Test)
2021-06-18 17:39:45 +03:00
lazy val `distribution-manager` = project
.in(file("lib/scala/distribution-manager"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
2021-06-18 17:39:45 +03:00
resolvers += Resolver.bintrayRepo("gn0s1s", "releases"),
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"io.circe" %% "circe-yaml" % circeYamlVersion,
"commons-io" % "commons-io" % commonsIoVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.dependsOn(editions)
2021-07-22 09:24:06 +03:00
.dependsOn(cli)
2021-06-18 17:39:45 +03:00
.dependsOn(pkg)
.dependsOn(`logging-utils`)
lazy val `benchmarks-common` =
(project in file("lib/java/benchmarks-common"))
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= GraalVM.modules ++ Seq(
"org.openjdk.jmh" % "jmh-core" % jmhVersion,
"org.openjdk.jmh" % "jmh-generator-annprocess" % jmhVersion,
"jakarta.xml.bind" % "jakarta.xml.bind-api" % jaxbVersion,
"com.sun.xml.bind" % "jaxb-impl" % jaxbVersion
)
)
.dependsOn(`polyglot-api`)
lazy val `bench-processor` = (project in file("lib/scala/bench-processor"))
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"jakarta.xml.bind" % "jakarta.xml.bind-api" % jaxbVersion,
"com.sun.xml.bind" % "jaxb-impl" % jaxbVersion,
"org.openjdk.jmh" % "jmh-core" % jmhVersion % "provided",
"org.openjdk.jmh" % "jmh-generator-annprocess" % jmhVersion % "provided",
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided",
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.graalvm.truffle" % "truffle-api" % graalMavenPackagesVersion % Test
),
Compile / javacOptions := ((Compile / javacOptions).value ++
// Only run ServiceProvider processor and ignore those defined in META-INF, thus
// fixing incremental compilation setup
Seq(
"-processor",
"org.netbeans.modules.openide.util.ServiceProviderProcessor"
)),
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
mainClass := Some("org.enso.benchmarks.libs.LibBenchRunner"),
commands += WithDebugCommand.withDebug,
(Test / fork) := true,
(Test / parallelExecution) := false,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
(Test / javaOptions) ++=
Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"-Dpolyglot.engine.WarnInterpreterOnly=false",
"-Dpolyglotimpl.DisableClassPathIsolation=true"
),
// Append enso language on the class-path
(Test / unmanagedClasspath) :=
(LocalProject("runtime-fat-jar") / Compile / fullClasspath).value
)
.dependsOn(`benchmarks-common`)
.dependsOn(`polyglot-api`)
.dependsOn(runtime)
lazy val `std-benchmarks` = (project in file("std-bits/benchmarks"))
.enablePlugins(JPMSPlugin)
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= GraalVM.modules ++ GraalVM.langsPkgs ++ Seq(
"org.openjdk.jmh" % "jmh-core" % jmhVersion,
"org.openjdk.jmh" % "jmh-generator-annprocess" % jmhVersion,
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion,
"org.slf4j" % "slf4j-api" % slf4jVersion,
"org.slf4j" % "slf4j-nop" % slf4jVersion
),
commands += WithDebugCommand.withDebug,
(Compile / logManager) :=
sbt.internal.util.CustomLogManager.excludeMsg(
"Could not determine source for class ",
Level.Warn
)
)
.settings(
parallelExecution := false,
run / fork := true,
run / connectInput := true,
mainClass :=
(LocalProject("bench-processor") / mainClass).value,
Compile / compile := (Compile / compile)
.dependsOn(`runtime-fat-jar` / assembly)
.value,
Compile / javacOptions ++= Seq(
"-s",
(Compile / sourceManaged).value.getAbsolutePath,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"-Xlint:unchecked",
"-J-Dpolyglotimpl.DisableClassPathIsolation=true",
"-J-Dpolyglot.engine.WarnInterpreterOnly=false"
),
modulePath := {
val requiredModIds = GraalVM.modules ++ GraalVM.langsPkgs ++ Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.slf4j" % "slf4j-api" % slf4jVersion,
"org.slf4j" % "slf4j-nop" % slf4jVersion
)
val requiredMods = JPMSUtils.filterModulesFromUpdate(
(Compile / update).value,
requiredModIds,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
streams.value.log,
shouldContainAll = true
)
val runtimeMod =
(`runtime-fat-jar` / assembly / assemblyOutputPath).value
requiredMods ++ Seq(runtimeMod)
},
addModules := {
val runtimeModuleName = (`runtime-fat-jar` / javaModuleName).value
Seq(runtimeModuleName)
},
addExports := {
Map("org.slf4j.nop/org.slf4j.nop" -> Seq("org.slf4j"))
},
javaOptions ++= {
Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// To enable logging in benchmarks, add ch.qos.logback module on the modulePath
"-Dslf4j.provider=org.slf4j.nop.NOPServiceProvider"
)
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
},
javaOptions ++= benchOnlyOptions
)
.settings(
bench := Def
.task {
(Compile / run).toTask("").tag(Exclusive).value
}
.dependsOn(
buildEngineDistribution
)
.value,
benchOnly := Def.inputTaskDyn {
import complete.Parsers.spaceDelimited
val name = spaceDelimited("<name>").parsed match {
case List(name) => name
case _ => throw new IllegalArgumentException("Expected one argument.")
}
Def.task {
(Compile / run).toTask(" " + name).value
}
}.evaluated
)
.dependsOn(`bench-processor`)
.dependsOn(`runtime-fat-jar`)
.dependsOn(`std-table`)
2021-06-18 17:39:45 +03:00
lazy val editions = project
.in(file("lib/scala/editions"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
2021-06-18 17:39:45 +03:00
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"io.circe" %% "circe-yaml" % circeYamlVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.settings(
(Compile / compile) := (Compile / compile)
.dependsOn(
Def.task {
Editions.writeEditionConfig(
editionsRoot = file("distribution") / "editions",
2021-08-13 19:14:20 +03:00
ensoVersion = ensoVersion,
editionName = currentEdition,
libraryVersion = stdLibVersion,
log = streams.value.log
)
}
)
.value,
cleanFiles += baseDirectory.value / ".." / ".." / "distribution" / "editions"
)
.dependsOn(semver)
.dependsOn(testkit % Test)
lazy val semver = project
.in(file("lib/scala/semver"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"io.circe" %% "circe-yaml" % circeYamlVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test
)
)
.settings(
(Compile / compile) := (Compile / compile)
.dependsOn(
Def.task {
Editions.writeEditionConfig(
editionsRoot = file("distribution") / "editions",
ensoVersion = ensoVersion,
editionName = currentEdition,
libraryVersion = stdLibVersion,
log = streams.value.log
)
}
)
.value,
cleanFiles += baseDirectory.value / ".." / ".." / "distribution" / "editions"
)
2021-06-22 14:35:15 +03:00
.dependsOn(testkit % Test)
2021-06-18 17:39:45 +03:00
2021-07-22 09:24:06 +03:00
lazy val downloader = (project in file("lib/scala/downloader"))
.settings(
frgaalJavaCompilerSetting,
// Fork the tests to make sure that the withDebug command works (we can
// attach debugger to the subprocess)
(Test / fork) := true,
commands += WithDebugCommand.withDebug,
2021-07-22 09:24:06 +03:00
version := "0.1",
libraryDependencies ++= circe ++ Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"commons-io" % "commons-io" % commonsIoVersion,
"org.apache.commons" % "commons-compress" % commonsCompressVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"junit" % "junit" % junitVersion % Test,
"com.github.sbt" % "junit-interface" % junitIfVersion % Test,
"org.hamcrest" % "hamcrest-all" % hamcrestVersion % Test
2021-07-22 09:24:06 +03:00
)
)
.dependsOn(cli)
.dependsOn(`http-test-helper`)
.dependsOn(testkit % Test)
2021-07-22 09:24:06 +03:00
lazy val `edition-updater` = project
.in(file("lib/scala/edition-updater"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
2022-07-01 04:58:14 +03:00
Test / test := (Test / test).tag(simpleLibraryServerTag).value,
2021-07-22 09:24:06 +03:00
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.dependsOn(editions)
.dependsOn(downloader)
2021-08-12 17:55:23 +03:00
.dependsOn(`distribution-manager`)
.dependsOn(`library-manager-test` % Test)
lazy val `edition-uploader` = project
.in(file("lib/scala/edition-uploader"))
.settings(
frgaalJavaCompilerSetting
)
2021-08-12 17:55:23 +03:00
.dependsOn(editions)
.dependsOn(`version-output`)
2021-07-22 09:24:06 +03:00
2021-06-18 17:39:45 +03:00
lazy val `library-manager` = project
.in(file("lib/scala/library-manager"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
2021-06-18 17:39:45 +03:00
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
2021-07-08 16:38:20 +03:00
.dependsOn(`version-output`) // Note [Default Editions]
2021-06-18 17:39:45 +03:00
.dependsOn(editions)
.dependsOn(cli)
.dependsOn(`distribution-manager`)
2021-07-22 09:24:06 +03:00
.dependsOn(downloader)
2021-06-18 17:39:45 +03:00
.dependsOn(testkit % Test)
lazy val `library-manager-test` = project
.in(file("lib/scala/library-manager-test"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
Test / fork := true,
commands += WithDebugCommand.withDebug,
Test / javaOptions ++= testLogProviderOptions,
2022-07-01 04:58:14 +03:00
Test / test := (Test / test).tag(simpleLibraryServerTag).value,
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.dependsOn(`library-manager`)
.dependsOn(`logging-utils` % "test->test")
.dependsOn(testkit)
.dependsOn(`logging-service-logback` % "test->test")
lazy val `connected-lock-manager` = project
.in(file("lib/scala/connected-lock-manager"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.dependsOn(`distribution-manager`)
.dependsOn(`connected-lock-manager-server` % "test->test")
.dependsOn(`polyglot-api`)
/** Unlike `connected-lock-manager` project, has a dependency on akka.
*/
lazy val `connected-lock-manager-server` = project
.in(file("lib/scala/connected-lock-manager-server"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
akkaActor,
akkaTestkit % Test,
"org.scalatest" %% "scalatest" % scalatestVersion % Test
)
)
.dependsOn(`distribution-manager`)
.dependsOn(`polyglot-api`)
.dependsOn(testkit % Test)
lazy val `runtime-version-manager` = project
.in(file("lib/scala/runtime-version-manager"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
resolvers += Resolver.bintrayRepo("gn0s1s", "releases"),
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.typelevel" %% "cats-core" % catsVersion,
"org.apache.commons" % "commons-compress" % commonsCompressVersion,
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
akkaHttp
)
)
.dependsOn(pkg)
2021-07-22 09:24:06 +03:00
.dependsOn(downloader)
.dependsOn(cli)
.dependsOn(`version-output`)
2021-08-12 17:55:23 +03:00
.dependsOn(`edition-updater`)
2021-06-18 17:39:45 +03:00
.dependsOn(`distribution-manager`)
lazy val `runtime-version-manager-test` = project
.in(file("lib/scala/runtime-version-manager-test"))
.configs(Test)
.settings(
frgaalJavaCompilerSetting,
libraryDependencies ++= Seq(
"com.typesafe.scala-logging" %% "scala-logging" % scalaLoggingVersion,
"org.scalatest" %% "scalatest" % scalatestVersion,
"commons-io" % "commons-io" % commonsIoVersion
)
)
2021-05-14 15:08:39 +03:00
.settings(Test / parallelExecution := false)
2020-11-06 15:53:45 +03:00
.settings(
(Test / test) := (Test / test)
.dependsOn(`locking-test-helper` / assembly)
.value
)
.dependsOn(`runtime-version-manager`)
2020-11-06 15:53:45 +03:00
.dependsOn(testkit)
2021-06-18 17:39:45 +03:00
.dependsOn(cli)
.dependsOn(`distribution-manager`)
2020-11-06 15:53:45 +03:00
lazy val `locking-test-helper` = project
.in(file("lib/scala/locking-test-helper"))
.settings(
frgaalJavaCompilerSetting,
2021-05-14 15:08:39 +03:00
assembly / test := {},
assembly / assemblyOutputPath := file("locking-test-helper.jar")
2020-11-06 15:53:45 +03:00
)
2021-07-08 16:38:20 +03:00
val `std-lib-root` = file("distribution/lib/Standard/")
def stdLibComponentRoot(name: String): File =
`std-lib-root` / name / stdLibVersion
val `base-polyglot-root` = stdLibComponentRoot("Base") / "polyglot" / "java"
val `table-polyglot-root` = stdLibComponentRoot("Table") / "polyglot" / "java"
val `image-polyglot-root` = stdLibComponentRoot("Image") / "polyglot" / "java"
2021-09-03 22:41:12 +03:00
val `google-api-polyglot-root` =
stdLibComponentRoot("Google_Api") / "polyglot" / "java"
2021-07-08 16:38:20 +03:00
val `database-polyglot-root` =
stdLibComponentRoot("Database") / "polyglot" / "java"
val `std-aws-polyglot-root` =
stdLibComponentRoot("AWS") / "polyglot" / "java"
lazy val `std-base` = project
.in(file("std-bits") / "base")
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / compile / compileInputs := (Compile / compile / compileInputs)
.dependsOn(SPIHelpers.ensureSPIConsistency)
.value,
Compile / packageBin / artifactPath :=
`base-polyglot-root` / "std-base.jar",
libraryDependencies ++= Seq(
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion,
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
"com.fasterxml.jackson.core" % "jackson-databind" % jacksonVersion
),
Compile / packageBin := Def.task {
val result = (Compile / packageBin).value
val _ensureCoreIsCompiled =
(`common-polyglot-core-utils` / Compile / packageBin).value
val _ = StdBits
.copyDependencies(
`base-polyglot-root`,
Seq("std-base.jar", "common-polyglot-core-utils.jar"),
ignoreScalaLibrary = true
)
.value
result
}.value
)
.dependsOn(`common-polyglot-core-utils`)
lazy val `common-polyglot-core-utils` = project
.in(file("lib/scala/common-polyglot-core-utils"))
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / packageBin / artifactPath :=
`base-polyglot-root` / "common-polyglot-core-utils.jar",
libraryDependencies ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"com.ibm.icu" % "icu4j" % icuVersion,
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided"
)
)
lazy val `enso-test-java-helpers` = project
.in(file("test/Base_Tests/polyglot-sources/enso-test-java-helpers"))
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / packageBin / artifactPath :=
file("test/Base_Tests/polyglot/java/helpers.jar"),
libraryDependencies ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided"
),
Compile / packageBin := Def.task {
val result = (Compile / packageBin).value
val primaryLocation = (Compile / packageBin / artifactPath).value
val secondaryLocations = Seq(
file("test/Table_Tests/polyglot/java/helpers.jar")
)
secondaryLocations.foreach { target =>
IO.copyFile(primaryLocation, target)
}
result
}.value
)
.dependsOn(`std-base` % "provided")
.dependsOn(`std-table` % "provided")
lazy val `exploratory-benchmark-java-helpers` = project
.in(
file(
"test/Exploratory_Benchmarks/polyglot-sources/exploratory-benchmark-java-helpers"
)
)
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / packageBin / artifactPath :=
file(
"test/Exploratory_Benchmarks/polyglot/java/exploratory-benchmark-java-helpers.jar"
),
libraryDependencies ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided"
)
)
.dependsOn(`std-base` % "provided")
.dependsOn(`std-table` % "provided")
lazy val `benchmark-java-helpers` = project
.in(
file(
"test/Benchmarks/polyglot-sources/benchmark-java-helpers"
)
)
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / packageBin / artifactPath :=
file(
"test/Benchmarks/polyglot/java/benchmark-java-helpers.jar"
),
libraryDependencies ++= Seq(
"org.graalvm.sdk" % "graal-sdk" % graalMavenPackagesVersion % "provided"
)
)
.dependsOn(`std-base` % "provided")
.dependsOn(`std-table` % "provided")
lazy val `std-table` = project
.in(file("std-bits") / "table")
.enablePlugins(Antlr4Plugin)
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / compile / compileInputs := (Compile / compile / compileInputs)
.dependsOn(SPIHelpers.ensureSPIConsistency)
.value,
Compile / packageBin / artifactPath :=
`table-polyglot-root` / "std-table.jar",
Antlr4 / antlr4PackageName := Some("org.enso.table.expressions"),
Antlr4 / antlr4Version := antlrVersion,
Antlr4 / antlr4GenVisitor := true,
Antlr4 / antlr4TreatWarningsAsErrors := true,
Compile / managedSourceDirectories += {
(Antlr4 / sourceManaged).value / "main" / "antlr4"
},
libraryDependencies ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided",
Always log to console and file (#7825) * Always log verbose to a file The change adds an option by default to always log to a file with verbose log level. The implementation is a bit tricky because in the most common use-case we have to always log in verbose mode to a socket and only later apply the desired log levels. Previously socket appender would respect the desired log level already before forwarding the log. If by default we log to a file, verbose mode is simply ignored and does not override user settings. To test run `project-manager` with `ENSO_LOGSERVER_APPENDER=console` env variable. That will output to the console with the default `INFO` level and `TRACE` log level for the file. * add docs * changelog * Address some PR requests 1. Log INFO level to CONSOLE by default 2. Change runner's default log level from ERROR to WARN Took a while to figure out why the correct log level wasn't being passed to the language server, therefore ignoring the (desired) verbose logs from the log file. * linter * 3rd party uses log4j for logging Getting rid of the warning by adding a log4j over slf4j bridge: ``` ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console... ``` * legal review update * Make sure tests use test resources Having `application.conf` in `src/main/resources` and `test/resources` does not guarantee that in Tests we will pick up the latter. Instead, by default it seems to do some kind of merge of different configurations, which is far from desired. * Ensure native launcher test log to console only Logging to console and (temporary) files is problematic for Windows. The CI also revealed a problem with the native configuration because it was not possible to modify the launcher via env variables as everything was initialized during build time. * Adapt to method changes * Potentially deal with Windows failures
2023-09-26 12:32:04 +03:00
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
"com.univocity" % "univocity-parsers" % univocityParsersVersion,
"org.apache.poi" % "poi-ooxml" % poiOoxmlVersion,
"org.apache.xmlbeans" % "xmlbeans" % xmlbeansVersion,
"org.antlr" % "antlr4-runtime" % antlrVersion,
"org.apache.logging.log4j" % "log4j-to-slf4j" % "2.18.0" // org.apache.poi uses log4j
),
Compile / packageBin := Def.task {
val result = (Compile / packageBin).value
val _ = StdBits
.copyDependencies(
`table-polyglot-root`,
Seq("std-table.jar"),
ignoreScalaLibrary = true
)
.value
result
}.value
)
.dependsOn(`std-base` % "provided")
lazy val `std-image` = project
.in(file("std-bits") / "image")
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / compile / compileInputs := (Compile / compile / compileInputs)
.dependsOn(SPIHelpers.ensureSPIConsistency)
.value,
Compile / packageBin / artifactPath :=
`image-polyglot-root` / "std-image.jar",
libraryDependencies ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided",
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
"org.openpnp" % "opencv" % "4.7.0-0"
),
Compile / packageBin := Def.task {
val result = (Compile / packageBin).value
val _ = StdBits
.copyDependencies(
`image-polyglot-root`,
Seq("std-image.jar"),
ignoreScalaLibrary = true
)
.value
result
}.value
)
.dependsOn(`std-base` % "provided")
2021-09-03 22:41:12 +03:00
lazy val `std-google-api` = project
.in(file("std-bits") / "google-api")
.settings(
frgaalJavaCompilerSetting,
2021-09-03 22:41:12 +03:00
autoScalaLibrary := false,
Compile / compile / compileInputs := (Compile / compile / compileInputs)
.dependsOn(SPIHelpers.ensureSPIConsistency)
.value,
2021-09-03 22:41:12 +03:00
Compile / packageBin / artifactPath :=
`google-api-polyglot-root` / "std-google-api.jar",
2021-09-03 22:41:12 +03:00
libraryDependencies ++= Seq(
"com.google.api-client" % "google-api-client" % "2.2.0" exclude ("com.google.code.findbugs", "jsr305"),
"com.google.apis" % "google-api-services-sheets" % "v4-rev612-1.25.0" exclude ("com.google.code.findbugs", "jsr305"),
"com.google.analytics" % "google-analytics-data" % "0.44.0" exclude ("com.google.code.findbugs", "jsr305")
2021-09-03 22:41:12 +03:00
),
Compile / packageBin := Def.task {
val result = (Compile / packageBin).value
val _ = StdBits
.copyDependencies(
`google-api-polyglot-root`,
Seq("std-google-api.jar"),
2021-09-03 22:41:12 +03:00
ignoreScalaLibrary = true
)
.value
result
}.value
)
lazy val `std-database` = project
.in(file("std-bits") / "database")
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / compile / compileInputs := (Compile / compile / compileInputs)
.dependsOn(SPIHelpers.ensureSPIConsistency)
.value,
Compile / packageBin / artifactPath :=
`database-polyglot-root` / "std-database.jar",
libraryDependencies ++= Seq(
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
"org.graalvm.polyglot" % "polyglot" % graalMavenPackagesVersion % "provided",
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
"org.xerial" % "sqlite-jdbc" % sqliteVersion,
"org.postgresql" % "postgresql" % "42.4.0"
),
Compile / packageBin := Def.task {
val result = (Compile / packageBin).value
val _ = StdBits
.copyDependencies(
`database-polyglot-root`,
Seq("std-database.jar"),
ignoreScalaLibrary = true
)
.value
result
}.value
)
.dependsOn(`std-base` % "provided")
.dependsOn(`std-table` % "provided")
lazy val `std-aws` = project
.in(file("std-bits") / "aws")
.settings(
frgaalJavaCompilerSetting,
autoScalaLibrary := false,
Compile / compile / compileInputs := (Compile / compile / compileInputs)
.dependsOn(SPIHelpers.ensureSPIConsistency)
.value,
Compile / packageBin / artifactPath :=
`std-aws-polyglot-root` / "std-aws.jar",
libraryDependencies ++= Seq(
"org.netbeans.api" % "org-openide-util-lookup" % netbeansApiVersion % "provided",
"com.amazon.redshift" % "redshift-jdbc42" % redshiftVersion,
"com.amazonaws" % "aws-java-sdk-core" % awsJavaSdkV1Version,
"com.amazonaws" % "aws-java-sdk-redshift" % awsJavaSdkV1Version,
"com.amazonaws" % "aws-java-sdk-sts" % awsJavaSdkV1Version,
"software.amazon.awssdk" % "auth" % awsJavaSdkV2Version,
"software.amazon.awssdk" % "s3" % awsJavaSdkV2Version
),
Compile / packageBin := Def.task {
val result = (Compile / packageBin).value
2021-02-22 16:32:55 +03:00
val _ = StdBits
.copyDependencies(
`std-aws-polyglot-root`,
Seq("std-aws.jar"),
ignoreScalaLibrary = true
)
.value
result
}.value
)
.dependsOn(`std-base` % "provided")
.dependsOn(`std-table` % "provided")
.dependsOn(`std-database` % "provided")
2020-08-28 14:03:09 +03:00
/* Note [Native Image Workaround for GraalVM 20.2]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* In GraalVM 20.2 the Native Image build of even simple Scala programs has
* started to fail on a call to `Statics.releaseFence`. It has been reported as
* a bug in the GraalVM repository: https://github.com/oracle/graal/issues/2770
*
* A proposed workaround for this bug is to substitute the original function
* with a different implementation that does not use the problematic
* MethodHandle. This is implemented in class
* `org.enso.launcher.workarounds.ReplacementStatics` using
* `org.enso.launcher.workarounds.Unsafe` which gives access to
* `sun.misc.Unsafe` which contains a low-level function corresponding to the
* required "release fence".
*
* To allow for that substitution, the launcher code requires annotations from
* the `svm` module and that is why this additional dependency is needed as long
* as that workaround is in-place. The dependency is marked as "provided"
* because it is included within the native-image build.
*/
/* Note [WSLoggerManager Shutdown Hook]
* ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* As the WSLoggerManager registers a shutdown hook when its initialized to
* ensure that logs are not lost in case of logging service initialization
* failure, it has to be initialized at runtime, as otherwise if the
* initialization was done at build time, the shutdown hook would actually also
* run at build time and have no effect at runtime.
*/
2020-12-09 16:58:11 +03:00
lazy val engineDistributionRoot =
settingKey[File]("Root of built engine distribution")
lazy val launcherDistributionRoot =
settingKey[File]("Root of built launcher distribution")
lazy val projectManagerDistributionRoot =
settingKey[File]("Root of built project manager distribution")
engineDistributionRoot :=
packageBuilder.localArtifact("engine") / s"enso-$ensoVersion"
launcherDistributionRoot := packageBuilder.localArtifact("launcher") / "enso"
projectManagerDistributionRoot :=
packageBuilder.localArtifact("project-manager") / "enso"
2020-12-09 16:58:11 +03:00
lazy val buildEngineDistribution =
taskKey[Unit]("Builds the engine distribution")
buildEngineDistribution := {
val _ = (`engine-runner` / assembly).value
updateLibraryManifests.value
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val modulesToCopy = componentModulesPaths.value.map(_.data)
val arrow = Seq((`runtime-language-arrow` / Compile / packageBin).value)
val engineModules = Seq(file("runtime.jar")) ++ arrow
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val root = engineDistributionRoot.value
val log = streams.value.log
val cacheFactory = streams.value.cacheStoreFactory
2020-12-09 16:58:11 +03:00
DistributionPackage.createEnginePackage(
distributionRoot = root,
cacheFactory = cacheFactory,
log = log,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
jarModulesToCopy = modulesToCopy ++ engineModules,
graalVersion = graalMavenPackagesVersion,
javaVersion = graalVersion,
ensoVersion = ensoVersion,
editionName = currentEdition,
sourceStdlibVersion = stdLibVersion,
targetStdlibVersion = targetStdlibVersion,
targetDir = (`syntax-rust-definition` / rustParserTargetDirectory).value,
generateIndex = true
)
log.info(s"Engine package created at $root")
}
// This makes the buildEngineDistribution task usable as a dependency
// of other tasks.
ThisBuild / buildEngineDistribution := {
buildEngineDistribution.result.value
}
lazy val buildEngineDistributionNoIndex =
taskKey[Unit]("Builds the engine distribution without generating indexes")
buildEngineDistributionNoIndex := {
val _ = (`engine-runner` / assembly).value
updateLibraryManifests.value
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val modulesToCopy = componentModulesPaths.value.map(_.data)
val engineModules = Seq(file("runtime.jar"))
val root = engineDistributionRoot.value
val log = streams.value.log
val cacheFactory = streams.value.cacheStoreFactory
DistributionPackage.createEnginePackage(
distributionRoot = root,
cacheFactory = cacheFactory,
log = log,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
jarModulesToCopy = modulesToCopy ++ engineModules,
graalVersion = graalMavenPackagesVersion,
javaVersion = graalVersion,
ensoVersion = ensoVersion,
editionName = currentEdition,
sourceStdlibVersion = stdLibVersion,
targetStdlibVersion = targetStdlibVersion,
targetDir = (`syntax-rust-definition` / rustParserTargetDirectory).value,
generateIndex = false
2020-12-09 16:58:11 +03:00
)
log.info(s"Engine package created at $root")
}
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
// This makes the buildEngineDistributionNoIndex task usable as a dependency
// of other tasks.
ThisBuild / buildEngineDistributionNoIndex := {
buildEngineDistributionNoIndex.result.value
}
lazy val runEngineDistribution =
inputKey[Unit]("Run or --debug the engine distribution with arguments")
runEngineDistribution := {
buildEngineDistribution.value
val args: Seq[String] = spaceDelimited("<arg>").parsed
DistributionPackage.runEnginePackage(
engineDistributionRoot.value,
args,
streams.value.log
)
}
lazy val runProjectManagerDistribution =
inputKey[Unit](
"Run or --debug the project manager distribution with arguments"
)
runProjectManagerDistribution := {
buildEngineDistribution.value
buildProjectManagerDistribution.value
val args: Seq[String] = spaceDelimited("<arg>").parsed
DistributionPackage.runProjectManagerPackage(
engineDistributionRoot.value,
projectManagerDistributionRoot.value,
args,
streams.value.log
)
}
val allStdBitsSuffix = List("All", "AllWithIndex")
val stdBitsProjects =
List(
"AWS",
"Base",
"Database",
"Google_Api",
"Image",
"Table"
) ++ allStdBitsSuffix
val allStdBits: Parser[String] =
stdBitsProjects.map(v => v: Parser[String]).reduce(_ | _)
lazy val `http-test-helper` = project
.in(file("tools") / "http-test-helper")
.settings(
customFrgaalJavaCompilerSettings(targetJdk = "21"),
autoScalaLibrary := false,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
Compile / javacOptions ++= Seq("-Xlint:all"),
Compile / run / mainClass := Some("org.enso.shttp.HTTPTestHelperServer"),
libraryDependencies ++= Seq(
"org.apache.commons" % "commons-text" % commonsTextVersion,
"org.apache.httpcomponents" % "httpclient" % httpComponentsVersion,
"com.fasterxml.jackson.core" % "jackson-databind" % jacksonVersion
),
assembly / assemblyMergeStrategy := {
case PathList("META-INF", "MANIFEST.MF", xs @ _*) =>
MergeStrategy.discard
case PathList(xs @ _*) if xs.last.contains("module-info") =>
MergeStrategy.discard
case _ => MergeStrategy.first
},
assembly / mainClass := (Compile / run / mainClass).value,
(Compile / run / fork) := true,
(Compile / run / connectInput) := true
)
.configs(Test)
lazy val buildStdLib =
inputKey[Unit]("Build an individual standard library package")
buildStdLib := Def.inputTaskDyn {
val cmd: String = allStdBits.parsed
val root: File = engineDistributionRoot.value
// Ensure that a complete distribution was built at least once.
// Because of `if` in the sbt task definition and usage of `streams.value` one has to
// delegate to another task definition (sbt restriction).
if ((root / "manifest.yaml").exists) {
pkgStdLibInternal.toTask(cmd)
} else buildEngineDistribution
}.evaluated
lazy val pkgStdLibInternal = inputKey[Unit]("Use `buildStdLib`")
pkgStdLibInternal := Def.inputTask {
val cmd = allStdBits.parsed
val root = engineDistributionRoot.value
val log: sbt.Logger = streams.value.log
val cacheFactory = streams.value.cacheStoreFactory
val standardNamespace = "Standard"
val buildAllCmd = allStdBitsSuffix.contains(cmd)
cmd match {
case "Base" =>
(`std-base` / Compile / packageBin).value
case "Database" =>
(`std-database` / Compile / packageBin).value
case "Google_Api" =>
(`std-google-api` / Compile / packageBin).value
case "Image" =>
(`std-image` / Compile / packageBin).value
case "Table" =>
(`std-table` / Compile / packageBin).value
case "TestHelpers" =>
(`enso-test-java-helpers` / Compile / packageBin).value
(`exploratory-benchmark-java-helpers` / Compile / packageBin).value
(`benchmark-java-helpers` / Compile / packageBin).value
case "AWS" =>
(`std-aws` / Compile / packageBin).value
case _ if buildAllCmd =>
(`std-base` / Compile / packageBin).value
(`enso-test-java-helpers` / Compile / packageBin).value
(`exploratory-benchmark-java-helpers` / Compile / packageBin).value
(`benchmark-java-helpers` / Compile / packageBin).value
(`std-table` / Compile / packageBin).value
(`std-database` / Compile / packageBin).value
(`std-image` / Compile / packageBin).value
(`std-google-api` / Compile / packageBin).value
(`std-aws` / Compile / packageBin).value
case _ =>
}
val libs =
if (!buildAllCmd) Seq(cmd)
else {
val prefix = s"$standardNamespace."
Editions.standardLibraries
.filter(_.startsWith(prefix))
.map(_.stripPrefix(prefix))
}
val generateIndex = cmd.endsWith("WithIndex")
libs.foreach { lib =>
StdBits.buildStdLibPackage(
lib,
root,
cacheFactory,
log,
defaultDevEnsoVersion
)
if (generateIndex) {
val stdlibStandardRoot = root / "lib" / standardNamespace
DistributionPackage.indexStdLib(
libMajor = stdlibStandardRoot,
libName = stdlibStandardRoot / lib,
stdLibVersion = defaultDevEnsoVersion,
ensoVersion = defaultDevEnsoVersion,
ensoExecutable = root / "bin" / "enso",
cacheFactory = cacheFactory.sub("stdlib"),
log = log
)
}
}
}.evaluated
2020-12-09 16:58:11 +03:00
lazy val buildLauncherDistribution =
taskKey[Unit]("Builds the launcher distribution")
buildLauncherDistribution := {
val _ = (launcher / buildNativeImage).value
val root = launcherDistributionRoot.value
val log = streams.value.log
val cacheFactory = streams.value.cacheStoreFactory
DistributionPackage.createLauncherPackage(root, cacheFactory)
log.info(s"Launcher package created at $root")
}
lazy val buildProjectManagerDistribution =
taskKey[Unit]("Builds the project manager distribution")
buildProjectManagerDistribution := {
val _ = (`project-manager` / buildNativeImage).value
val root = projectManagerDistributionRoot.value
val log = streams.value.log
val cacheFactory = streams.value.cacheStoreFactory
DistributionPackage.createProjectManagerPackage(root, cacheFactory)
log.info(s"Project Manager package created at $root")
}
lazy val buildGraalDistribution =
taskKey[Unit]("Builds the GraalVM distribution")
buildGraalDistribution := {
val log = streams.value.log
val distOs = "DIST_OS"
val distArch = "DIST_ARCH"
val osName = "os.name"
val archName = "os.arch"
val distName = sys.env.get(distOs).getOrElse {
val name = sys.props(osName).takeWhile(!_.isWhitespace)
if (sys.env.contains("CI")) {
log.warn(
s"$distOs env var is empty. Fallback to system property $osName=$name."
)
}
name
}
val arch = sys.env.get(distArch).orElse(sys.env.get(archName))
val os = DistributionPackage.OS(distName, arch).getOrElse {
throw new RuntimeException(s"Failed to determine OS: $distName.")
}
packageBuilder.createGraalPackage(
log,
os,
os.archs.head
)
}
lazy val updateLibraryManifests =
taskKey[Unit](
"Recomputes dependencies to update manifests bundled with libraries."
)
updateLibraryManifests := {
val _ = (`engine-runner` / assembly).value
val log = streams.value.log
val cacheFactory = streams.value.cacheStoreFactory
val libraries = Editions.standardLibraries.map(libName =>
BundledLibrary(libName, stdLibVersion)
)
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
val runnerCp = (LocalProject("engine-runner") / Runtime / fullClasspath).value
val runtimeCp = (LocalProject("runtime") / Runtime / fullClasspath).value
val fullCp = (runnerCp ++ runtimeCp).distinct
val modulesOnModulePath =
JPMSUtils
.filterModulesFromClasspath(
fullCp,
JPMSUtils.componentModules,
log,
shouldContainAll = true
)
.map(_.data)
val modulePath = modulesOnModulePath ++ Seq(file("runtime.jar"))
val runnerJar = (LocalProject("engine-runner") / assembly).value
val javaOpts = Seq(
"-Denso.runner=" + runnerJar.getAbsolutePath,
"--module-path",
modulePath.map(_.getAbsolutePath).mkString(File.pathSeparator),
"-m",
"org.enso.runtime/org.enso.EngineRunnerBootLoader"
)
LibraryManifestGenerator.generateManifests(
libraries,
file("distribution"),
log,
Upgrade enso to GraalVM for jdk 21 (#7991) Upgrade to GraalVM JDK 21. ``` > java -version openjdk version "21" 2023-09-19 OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15) OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing) ``` With SDKMan, download with `sdk install java 21-graalce`. # Important Notes - After this PR, one can theoretically run enso with any JRE with version at least 21. - Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution. - `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded. - sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368 - Which means that we can have `module-info.java` files only for Java-only projects. - Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar. - `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent. - Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory. - The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module. - The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725) - [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 21:02:36 +03:00
javaOpts,
cacheFactory
)
}