* Drop per artifact signing
Per artifact signing was an unnecessary parameter. It is sufficient to
simply have `--sign-artifacts` parameter to control the action.
Tested locally (previously hindered by my poor understanding of CI
build config).
* Empty commit
* Use moduleDependencies instead of modulePath
* Fix compilation of editions
* Fix compilation of distribution-manager
* polyglot-api needs to explicitly compile module-info
* Fix compilationOrder in library-manager and edition-updater
* engine-runner-common is module
* JPMSPlugin provides default implementation of compileModuleInfo
* Remove unused setting key from JPMSUtils.compileModuleInfo
* JPMSPlugin has internalModuleDependencies and exportedModule tasks.
* Use BuildVersion instead of buildInfo
* Manual compilation of module-info.java is reported as warning
* Define org.enso.scalalibs.wrapper meta project.
* Fix module check in JPMSPlugin.
This is a fix for projects that declare `Compile /exportJars := true`
* version-output is a module
* ydoc-server uses internalModuleDependencies
* persistance is module
* engine-common uses internalModuleDependencies
* polyglot-api does not override compileModuleInfo task
* runtime-parser uses internalModuleDependencies
* edition-updater is module
* Update moduleDependencies for distribution-manager
* editions is module
* Fix some dependencies of modules
* scala-yaml is a module
* Add scala-compiler to scala-libs-wrapper
* cli depends on scala-library module
* Add dependencies for distribution-manager module
* Add some scala-library dependencies in some modules
* engine-runner uses internalModuleDependencies
* Fix module dependencies of library-manager
* Rename org.enso.scalalibs.wrapper to org.enso.scala.wrapper
* Add jsoniter-scala-macros to org.enso.scala.wrapper fat module
* Fix dependencies of some projects
* polyglot-api does not depend on truffle-api
* Fix dependencies of some projects
* runtime does not use com.google.common
* runtime is a module
* text-buffer is a module
* refactoring-utils is a module
* runtime-compiler is a module
* runtime-instrument-common is a module
* connected-lock-manager is a module
* JPMSUtils reports project name in some error messages
* Modularize some instruments
* module-info compilation is cached
* runtime-instrument-runtime-server is module
* runtime-language-epb is module
* Remove runtime-fat-jar
* engine-runner is not a fat jar
* JPMSPlugin defines exportedModuleBin task
* Redefine componentModulesPaths task
* interpreter-dsl is module
* Redefine componentModulesPaths task
* fmt sbt
* scala-libs-wrapper is a modular fat jar
* Add some module deps to org.enso.runtime
* engine-runner is not a fat jar
* Rename package in logging-config
* Rename package in logging-service
* Rename package in logging-service-logback
* Fix dependencies of exportedModuleBin task
* Mixed projects have own compileJava task
this task does not compile only module-info.java but all the java sources. So that we can see errors more easily.
When only module-info.java is compiled, the only errors that we can see are that we did not include some modules on module-path.
* Fix definition of exportedModule task.
* Remove usages of non-existing buildInfo and replace it with BuildVersion
* Fix some dependencies of org.enso.runtime module
* module-info compilation is handled directly by FrgaalCompiler
* module-info compilation is forced for projects that has only Scala sources with single module-info.java
* Fix compilation of org.enso.runtime
* manual module-info compilation is not a warning
* Rename packages in logging-utils-akka
* Create org.enso.language.server.deps.wrapper module
* language-server is module
* Creat akka-wrapper modular fat jar
* fmt
* Define common settings for modularFatJarWrapper
* Fix compilation of json-rpc-server
* Use akka and zio wrappers
* language-server depends on org.eclipse.jgit
* Fix some dependencies - update library manifests works now!
* update library manifests invokes runner directly
* buildEngineDistribution does not copy runner.jar
* Remove EngineRunnerBootLoader
* Fix compilation of std libs
* --patch-module and --add-exports are also passed to javac
* Rename package in runtime-integration-tests.
The package name org.enso.compiler clashes with the package from the module
* Remove usage of buildInfo
* FrgaalJavaCompiler can deal with non-existing module-info.java when shouldCompileModuleInfo is true.
It just generates a warning in such case as it suggests that there is something wrong with the project configuration.
* Revert AliasAnalysisTest.scala
* Fix dependencies and java cmdline options for runtime-integration-tests
* Rename test package
* runtime-integration-test depends on logging-service-logback/Test/compile
* Rename package in logging-service-logback/Test
* Fix FrgaalJavaCompiler creation for projects
* Sanitize Test/javaOptions arguments
* Sanitize Test/javaOptions arguments
* All the JPMSPlugin settings are scoped
* Remove unused sbt tasks
* modularFatJarWrapperSettings do not override javacOptions
* Resolve issue "Cannot find TestLoggerProvider" in runtime-integration-tests
* org.enso.runtime module is open
* Test that test classes are unconditionally opened and exported
* polyglot-api-macros is a module
* JPMSPlugin handles --add-opens cmdline option
* RuntimeServerTest ensures instruments are initialized
* Add some exports to org.enso.runtime.compiler
* Add instruments on module-path to runtime-integration-tests
* Replace TestLogProviderOnClassPath with TestLogProviderOnModulePath
* Replace buildInfo with BuildVersion
* Add jpms-wrapper-scalatest
* ReportLogsOnFailure is in non-modular testkit project
* Add necessary dependencies to testkit project
* Revert "Add jpms-wrapper-scalatest"
This reverts commit 732b3427a2.
* modularize filewatcher and wrap its dependencies
* Initial fix for language-server/test
* frgaal compiler setting are scoped for Compile and Test
* Rename package in language-server/test
* Exclude com.sun.jna from wrapper jars
* Rename package in library-manager-test
* testkit is an automatic module
* process-utils is module
* akka-wrapper contains akka-http
* Some fixes for library-manager-test
* Fix dependencies for akka-wrapper
* scala-libs-wrapper exports shapeless
* lang server deps wrapper exports pureconfig
* json-rpc-server requires org.slf4j
* Add some dependencies
* lang server deps wrapper exports pureconfig.generic
* language server test requires bouncycastle provider
* language server depends on cli
* directory-watcher wrapper requires org.slf4j
* WatcherAdapter logs unsuccessful initialization errors
* Fix error reporting in WatcherAdapter
* Fix rest of the language-server tests
* language-server-deps-wrapper depends on scala-libs-wrapper
* Fix rest of the language-server tests
* Missing module-info.class in an internal project is a warning, not an error
* Rename jpms-methvin-directory-watcher-wrapper to a simpler name
* compileOrder has to be specified before libraryDependencies
* exclude module-info.java from polyglot-api-macros
* Remove temporary logging in customFrgaalCompilerSettings
* Fix compilation of logging-service-logback
* Fix compilation of runtime-benchmarks
* Fix runtime-benchmarks/run
* HostClassLoader delegates to org.graalvm.polyglot class loader if org.enso.runtime is not on boot layer
* org.enso.runtime.lnaguage.epb module must be opened to allow it to be used by annnotation processor
* fmt
* Fix afetr merge
* Add module deps after merge
* Print stack trace of the uncaught exception from the annotation processor
* Remove akka-actor-typed from akka-wrapper
* runtime-instrument-common depends on slf4j
* Fix module-path for runtime-instrument-repl-debugger
* runtime-benchmarks depends on runtime-language-arrow
* --module-path is passed directly to frgaal
* Fix some module-related cmd line options for std-benchmarks
* Revert "--module-path is passed directly to frgaal"
This reverts commit da63f66a0e.
* Avoid closing of System.err when closing Context
* Avoid processing altogether when requested annotations are empty
* Pass shouldNotLimitModules opt to frgaal
* Pass module-path and add-modules options with -J prefix to frgaal
* BenchProcessor annotation processor creates its own truffle module layer
* bench-processor and benchmarks-common are modules
* fmt
* Fix after mege
* Enable JMH annotation processor
* Fix compileOrder in some projects
* Insert TruffleBoundary to QualifiedName.
This is a revert
* Fix building of engine-runner native image
* Add more deps to the native image
* Force module-info compilation in instruments.
This fixes some weird sbt bug
* Don't run engine-runner/assembly from Rust build script
* Update docs of JPMSPlugin
* fmt
* runtime-benchmarks depends on benchmarks-common module
* Fix benchmark report writing
* std-benchmarks annot processing does not take settings from runtime-benchmarks
* Suppress interpreter only warning in annotation processor
* Runtime version manager does not expect runtime.jar fat jar
* fmt
* Fix module entry point
* Move some polyglot tests to runtime-integration-tests.
Also make their output silent
* pkg has no dependency on org.graalvm.truffle
* Fix compiler dependencies test
* Rename all runtime.jar in fake releases
* Add language-server with dependencies to component dir
* No module-info.class in target dir is warning not error
* language-server does not depend on netbeans lookup uitl
* Declare LanguageServerApi service provider in module-info
* connected-lock-manager-server is JPMS module
* task-progress-notifications is module
* Add fansi-wrapper module
* Fix compilation of connected-lock-manager-server
* Define correct Test/internalModuleDependencies for project-manager
* fmt
* Fix LauncherRunnerSpec - no runtime.jar
* Add fansi-wrapper to runtime-integration-tests and runtime-benchmarks
* Fix engine-runner native image build
* Use newer JNA version - fixes running of hyperd
* DRY
* scala-compiler DRY
* fmt
* More build.sbt refactoring
* Include runtime-instrument-id-execution in engine-runner native image
* TruffleBoundary for QualifiedName.toString
* Finding a needle in a haystack
🤦
* More scala-library DRY
* more mixed-java/scala goodies
* Fix compilation of syntax-rust-definition
* Test that engine-runner does not depend on language-server
* Append rather than assign `moduleDependencies`
`++=` is less error prone than `:=`. Also discovered some unnecessary
dependencies.
* Replace : with File.pathSeparator
* [WIP] Make logging in ProjectService more verbose
* language-server/test didn't start because of missing lookup and fansi modules
* Formatting
* org.enso.cli.task.notifications needs Akka and Circe to link
* project-manager/test depends on buildEngineDistribution
* [WIP] Even more verbose logging for creating projects
* [WIP] Even more verbose logging for creating projects
* Revert "[WIP] Even more verbose logging for creating projects"
This reverts commit a7067c8472.
* Revert "[WIP] Even more verbose logging for creating projects"
This reverts commit fc6f53d4f1.
* Revert "[WIP] Make logging in ProjectService more verbose"
This reverts commit 427428e142.
* All the project with JPMSPlugin has stripped artifact names
* Revert all placeholder fake release components to runtime.jar without version
* Eliminate a cross version hack
We shouldn't be specifying Scala dependencies with a Scala cross version
in the suffix.
* Address SBT lint warnings
* Revert "Eliminate a cross version hack"
This reverts commit 8861dab288.
* logging-service-logback is mixedJavaScalaProject
* fmt
* Stripped artifact name contains classifier.
This fixes tests as those were named like `artifact-tests.jar`.
* Don't use LocalProject unless really needed
* Add more logging when BenchProcessor fails
* logging-service-logback is not mixed project
* Work with java.io.File.getPath to avoid mixing slash and backslash on Windows
* Reapply "Eliminate a cross version hack"
This reverts commit edaa436ee8.
* Pass scalaBinaryVersion correctly
* Remove scala-compiler from the distribution
* Fix IllegalAccessErrors from serde
* typos
* License review
* fmt
* Move testLogProviderOnModulePath to TestJPMSConfiguration
* logging-service-logback is not a mixed project
---------
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
* More info when critical failure occurs
Log problematic module to help with debugging critical failure.
* One more exception
* s/System.err/Logger.error/
* maybe append slf4j deps
* fix what looks like a long standing typo
`GeneratedFormatTests.java` not `GeneratedFormatTests..java`
* one more typo
* Fix directory where to look for classpath
`./run java-gen test --skip-version-check` now works. At least locally.
* Local is fine, CI is not. More temporary debugging...
* Ensure project's managedClasspath is exported
Running java tests requires us knowing all additional dependencies as
they have to be added to the classpath manually. That can only be
ensured by invoking the right sbt target.
* Move sbt call after graalvm setup
* removing CI debugging
* Apply suggestions from code review
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
---------
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
* Disable signing artifacts for PRs and develop
Added `*-sign-artifacts` to disable/enable signing. By default it is
true, except for PRs and develop.
* nit
* typo
* lint
* linting
- Update all the templates to the newest versions.
- Fix any defects in the templates.
- Update the rust and scala code to package and download the templates.
- Update the front end to have images for every template (with some Gemini assists).
https://github.com/user-attachments/assets/59a1f845-f409-45bb-9c77-fe3a2d1262b1
- Followup of #10964
- I assumed `aws` CLI is available, because it is on GH runners. But we are running on self-hosted by default.
- Ideally we should make the CLI available there and switch back.
- But for now, trying to run on the GH runner.
Removed all `#![feature]` flags, except for `#![feature(test)]`. Once parser benchmarks are ported to something that is compatible with stable rust, we will be able to switch to it.
# Important Notes
The command to run the gui dev environment has been changed. Invoking the old command will print a message about that.
From now on, use `pnpm dev:gui2` in repository root.
- Close#10622
- Changes `project-manager` and `ensoup` launcher to run the engine/language-server with working directory set to the directory containing currently running project.
- If the working directory is _not_ "the directory containing currently running project", a warning is written to logs. This can happen if the raw `/bin/enso` engine runner is used in a different directory.
- In the Cloud, the `File.new` interprets relative paths as cloud paths relative to the Cloud directory containing the current project. Absolute paths are unaffected.
In a sequence of value-level operators, whitespace does not affect relative precedence. Functional operators still follow the space-precedence rules.
The "functional" operators are: `>> << |> |>> <| <<| : .`, application, and any operator containing `<-` or `->`. All other operators are considered value-level operators.
Asymmetric whitespace can still be used to form *operator sections* of value-level operators, e.g. `+2 * 3` is still equivalent to `x -> (x+2) * 3`.
Precedence of application is unchanged, so `f x+y` is still equivalent to `f (x + y)` and `f x+y * z` is still equivalent to `(f (x + y)) * z`.
Any attempt to use spacing to override value-level operator precedence will be caught by the new enso linter. Mixed spacing (for clarity) in value-operator expressions is allowed, as long as it is consistent with the precedences of the operators.
Closes#10366.
# Important Notes
Precedence warnings:
- The parser emits a warning if the whitespace in an expression is inconsistent with its effective precedence.
- A new enso linter can be run with `./run libraries lint`. It parses all `.enso` files in `distribution/lib` and `test`, and reports any errors or warnings. It can also be run on individual files: `cargo run --release --bin check_syntax -- file1 file2...` (the result may be easier to read than the `./run` output).
- The linter is also run as part of `./run lint`, so it is checked in CI.
Additional language change:
- The exponentiation operator (`^`) now has higher precedence than the multiplication class (`*`, `/`, `%`). This change did not affect any current enso files.
Library changes:
- The libraries have been updated. The new warnings were used to identify all affected code; the changes themselves have not been programmatically verified (in many cases their equivalence relies on the commutativity of string concatenation).
- Remove unnecessary modules
- Remove `ts-plugin-namespace-auto-import` as it was a workaround to use the non-conventional `import *` convention
- Remove `esbuild-plugin-copy-directories` as it is unuse
- Inline modules that are only ever used once
- Inline `project-manager-shim` into `gui2` - it is only used during `gui2`'s dev mode
- Inline `content-config` into `client`
- Flatten `app/ide-desktop/lib/` to `app/ide-desktop/`
- Flatten `app/ide-desktop/lib/dashboard/` to `app/dashboard/`
# Important Notes
- As mentioned above, all remaining modules have been moved up from `app/ide-desktop/lib/` to `app/ide-desktop/`. It's not ideal but I'd rather hold off on moving them anywhere else before we have a consensus on what should go where.
- (That is to say, this may not be the final directory structure - but I figure it's fine to get *something* done so that hopefully the rest of the restructuring is simpler.)
Addresses one of two concerns of #5298 - adds support for `--jvm` argument to allow us to switch from _native image_ built Enso binary (as developed by #10126) to regular JVM based Enso execution. This change _doesn't affect production builds_. The _native executable_ continues to be only built by `engine-runner/buildNativeImage` which is tested on CI, but not in the production jobs.
Fixes#10197
# Important Notes
From now on, package installation will be using `pnpm install`. Installing it globally is fine for convenience, but it can also be used as `corepack pnpm install` without having to install anything other than node. For now, all other scripts are still invoked using `npm`, so we can still invoke them with usual `--workspace` setting. As far as I can tell that doesn't really have any other side effects and is identical as running the script through `pnpm run` in respective workspace project subdirectory.
This PR fixes several issues that were appearing when running CI jobs on PRs created against the repository forks:
* electron-builder on Windows and macOS will properly recognize that the secrets are missing and will not attempt to sign the artifacts;
* similarly, fixed the S3 library tests;
* test reporter step will be now skipped, as it does not support forks.
This PR removes unused, commented-out or otherwise spurious code from build script. Also, dependencies were reviewed and cleaned.
No functional changes intended.
This PR updates the build script to retry uploading release assets if the upload fails.
Network issues can cause the upload to fail, failing in turn the whole release process. This PR adds a retry mechanism to the upload process, making the release process more robust.
Now that #9815 has landed, we can finally bump electron-builder to the latest release. As this brings in python3 support out-of-the-box, workaround of the runtime-bump on macOS runners can be removed.
This PR introduces a new installer and uninstaller for the Windows platform.
Both are written in Rust and compiled to a single executable. The executable has no dependencies (other than what is included in the Windows), links the C++ runtime statically if needed.
The change is motivated by numerous issues with with the `electron-builder`-generated installers. The new installer should behave better, not have issues with long paths and unblock the `electron-builder` upgrade (which will significantly simplify the workflow definitions).
To build an installer, one needs to provide the unpacked application (generated by `electron-builder`) and the `electron-builder` configuration (with a few minor extensions). Code signing is also supported.
GitHub made arm64 runners generally available and changed macos-latest label to point to them.
The runner architecture is coupled with GH-hosted runners OS version: macos-13 is the last one to run on x64.
This PR essentially brings back the previous behavior, by explicitly requesting that all our x64 macOS jobs are run on macos-12 (as was before).
We should eventually migrate to macos-13 for x64 macOS and macos-14/macos-latest for arm64 macOS. However, this leads to issues with `npm install` getting stuck, so it should be probably reattempted after the CI rework.
- #9779 introduced (incorrect) detection to determine when to inject the Google Analytics tag. Instead, it should be injected by CI, because sending Google Analytics events is undesirable in development mode.
# Important Notes
None
This PR bumps the FlatBuffers version used by the backend to `24.3.25` (the latest version as of now).
Since the newer FlatBuffers releases come with prebuilt binaries for all platforms we target, we can simplify the build process by simply downloading the required `flatc` binary from the official FlatBuffers GitHub release page. This allows us to remove the dependency on `conda`, which was the only reliable way to get the outdated `flatc`.
The `conda` setup has been removed from the CI steps and the relevant code has been removed from the build script.
The FlatBuffers version is no longer hard-coded in the Rust build script, it is inferred from the `build.sbt` definition (similar to GraalVM).
# Important Notes
This does not affect the GUI binary protocol implementation.
While I initially wanted to update it, it turned out farly non-trivial.
As there are multiple issues with the generated TS code, it was significantly refactored by hand and it is impossible to automatically update it. Work to address this problem is left as [a future task](https://github.com/enso-org/enso/issues/9658).
As the Flatbuffers binary protocol is guaranteed to be compatible between versions (unlike the generated sources), there should be no adverse effects from bumping `flatc` only on the backend side.
This PR adds a new file to the release: `assets.json`, that offers information about the assets available in the release.
The purpose is to have one persistent link `https://github.com/enso-org/enso/releases/latest/download/assets.json` that has the current download links that can be consumed by the website.
Also, the release template has been updated to use the same assets information source, rather than duplicate the information about artifact names.
Additionally, additional step for release validation was added, so the CI can alert if one of the expected assets is missing.
This PR introduces [a new workflow — nightly checks](https://github.com/enso-org/enso/actions/workflows/nightly-tests.yml). It consists of the whole array of Backend checks:
* build check, Scala tests and Standard Library tests;
* covers both Community and Oracle (Enterprise) GraalVM editions (Linux-only);
* includes checks for Aarch64 macOS runner.
We do not want to run these checks on each PR due to limited runners capacity. By running them nightly, we can still catch any issues that might arise on `develop` branch.
# Important Notes
* [ ] Before merging, this requires updating the GH required checks list.
The `sysinfo` crate returns now bytes, not kilobytes.
This was changed in `sysinfo`'s `0.26.0` version. I have missed this change previously while bumping CI code dependencies.
The effects were not drastic as both fast and slow paths were meant to be generally equivalent.
Removes a bulk of rust crates that we no longer need, but that added significant install, build and testing time to the Rust parser.
Most significantly, removed `enso-web` and `enso-shapely`, and got rid of many no longer necessary `#![feature]`s. Moved two still used proc-macros from shapely to prelude. The last remaining usage of `web-sys` is within the logger (`console.log`), but we may actually want to keep that one.
