`Meta.get_qualified_type_name` correctly returns fully qualified type name when running a single file from a project with `enso --run Proj/src/Main.enso`.
* Use moduleDependencies instead of modulePath
* Fix compilation of editions
* Fix compilation of distribution-manager
* polyglot-api needs to explicitly compile module-info
* Fix compilationOrder in library-manager and edition-updater
* engine-runner-common is module
* JPMSPlugin provides default implementation of compileModuleInfo
* Remove unused setting key from JPMSUtils.compileModuleInfo
* JPMSPlugin has internalModuleDependencies and exportedModule tasks.
* Use BuildVersion instead of buildInfo
* Manual compilation of module-info.java is reported as warning
* Define org.enso.scalalibs.wrapper meta project.
* Fix module check in JPMSPlugin.
This is a fix for projects that declare `Compile /exportJars := true`
* version-output is a module
* ydoc-server uses internalModuleDependencies
* persistance is module
* engine-common uses internalModuleDependencies
* polyglot-api does not override compileModuleInfo task
* runtime-parser uses internalModuleDependencies
* edition-updater is module
* Update moduleDependencies for distribution-manager
* editions is module
* Fix some dependencies of modules
* scala-yaml is a module
* Add scala-compiler to scala-libs-wrapper
* cli depends on scala-library module
* Add dependencies for distribution-manager module
* Add some scala-library dependencies in some modules
* engine-runner uses internalModuleDependencies
* Fix module dependencies of library-manager
* Rename org.enso.scalalibs.wrapper to org.enso.scala.wrapper
* Add jsoniter-scala-macros to org.enso.scala.wrapper fat module
* Fix dependencies of some projects
* polyglot-api does not depend on truffle-api
* Fix dependencies of some projects
* runtime does not use com.google.common
* runtime is a module
* text-buffer is a module
* refactoring-utils is a module
* runtime-compiler is a module
* runtime-instrument-common is a module
* connected-lock-manager is a module
* JPMSUtils reports project name in some error messages
* Modularize some instruments
* module-info compilation is cached
* runtime-instrument-runtime-server is module
* runtime-language-epb is module
* Remove runtime-fat-jar
* engine-runner is not a fat jar
* JPMSPlugin defines exportedModuleBin task
* Redefine componentModulesPaths task
* interpreter-dsl is module
* Redefine componentModulesPaths task
* fmt sbt
* scala-libs-wrapper is a modular fat jar
* Add some module deps to org.enso.runtime
* engine-runner is not a fat jar
* Rename package in logging-config
* Rename package in logging-service
* Rename package in logging-service-logback
* Fix dependencies of exportedModuleBin task
* Mixed projects have own compileJava task
this task does not compile only module-info.java but all the java sources. So that we can see errors more easily.
When only module-info.java is compiled, the only errors that we can see are that we did not include some modules on module-path.
* Fix definition of exportedModule task.
* Remove usages of non-existing buildInfo and replace it with BuildVersion
* Fix some dependencies of org.enso.runtime module
* module-info compilation is handled directly by FrgaalCompiler
* module-info compilation is forced for projects that has only Scala sources with single module-info.java
* Fix compilation of org.enso.runtime
* manual module-info compilation is not a warning
* Rename packages in logging-utils-akka
* Create org.enso.language.server.deps.wrapper module
* language-server is module
* Creat akka-wrapper modular fat jar
* fmt
* Define common settings for modularFatJarWrapper
* Fix compilation of json-rpc-server
* Use akka and zio wrappers
* language-server depends on org.eclipse.jgit
* Fix some dependencies - update library manifests works now!
* update library manifests invokes runner directly
* buildEngineDistribution does not copy runner.jar
* Remove EngineRunnerBootLoader
* Fix compilation of std libs
* --patch-module and --add-exports are also passed to javac
* Rename package in runtime-integration-tests.
The package name org.enso.compiler clashes with the package from the module
* Remove usage of buildInfo
* FrgaalJavaCompiler can deal with non-existing module-info.java when shouldCompileModuleInfo is true.
It just generates a warning in such case as it suggests that there is something wrong with the project configuration.
* Revert AliasAnalysisTest.scala
* Fix dependencies and java cmdline options for runtime-integration-tests
* Rename test package
* runtime-integration-test depends on logging-service-logback/Test/compile
* Rename package in logging-service-logback/Test
* Fix FrgaalJavaCompiler creation for projects
* Sanitize Test/javaOptions arguments
* Sanitize Test/javaOptions arguments
* All the JPMSPlugin settings are scoped
* Remove unused sbt tasks
* modularFatJarWrapperSettings do not override javacOptions
* Resolve issue "Cannot find TestLoggerProvider" in runtime-integration-tests
* org.enso.runtime module is open
* Test that test classes are unconditionally opened and exported
* polyglot-api-macros is a module
* JPMSPlugin handles --add-opens cmdline option
* RuntimeServerTest ensures instruments are initialized
* Add some exports to org.enso.runtime.compiler
* Add instruments on module-path to runtime-integration-tests
* Replace TestLogProviderOnClassPath with TestLogProviderOnModulePath
* Replace buildInfo with BuildVersion
* Add jpms-wrapper-scalatest
* ReportLogsOnFailure is in non-modular testkit project
* Add necessary dependencies to testkit project
* Revert "Add jpms-wrapper-scalatest"
This reverts commit 732b3427a2.
* modularize filewatcher and wrap its dependencies
* Initial fix for language-server/test
* frgaal compiler setting are scoped for Compile and Test
* Rename package in language-server/test
* Exclude com.sun.jna from wrapper jars
* Rename package in library-manager-test
* testkit is an automatic module
* process-utils is module
* akka-wrapper contains akka-http
* Some fixes for library-manager-test
* Fix dependencies for akka-wrapper
* scala-libs-wrapper exports shapeless
* lang server deps wrapper exports pureconfig
* json-rpc-server requires org.slf4j
* Add some dependencies
* lang server deps wrapper exports pureconfig.generic
* language server test requires bouncycastle provider
* language server depends on cli
* directory-watcher wrapper requires org.slf4j
* WatcherAdapter logs unsuccessful initialization errors
* Fix error reporting in WatcherAdapter
* Fix rest of the language-server tests
* language-server-deps-wrapper depends on scala-libs-wrapper
* Fix rest of the language-server tests
* Missing module-info.class in an internal project is a warning, not an error
* Rename jpms-methvin-directory-watcher-wrapper to a simpler name
* compileOrder has to be specified before libraryDependencies
* exclude module-info.java from polyglot-api-macros
* Remove temporary logging in customFrgaalCompilerSettings
* Fix compilation of logging-service-logback
* Fix compilation of runtime-benchmarks
* Fix runtime-benchmarks/run
* HostClassLoader delegates to org.graalvm.polyglot class loader if org.enso.runtime is not on boot layer
* org.enso.runtime.lnaguage.epb module must be opened to allow it to be used by annnotation processor
* fmt
* Fix afetr merge
* Add module deps after merge
* Print stack trace of the uncaught exception from the annotation processor
* Remove akka-actor-typed from akka-wrapper
* runtime-instrument-common depends on slf4j
* Fix module-path for runtime-instrument-repl-debugger
* runtime-benchmarks depends on runtime-language-arrow
* --module-path is passed directly to frgaal
* Fix some module-related cmd line options for std-benchmarks
* Revert "--module-path is passed directly to frgaal"
This reverts commit da63f66a0e.
* Avoid closing of System.err when closing Context
* Avoid processing altogether when requested annotations are empty
* Pass shouldNotLimitModules opt to frgaal
* Pass module-path and add-modules options with -J prefix to frgaal
* BenchProcessor annotation processor creates its own truffle module layer
* bench-processor and benchmarks-common are modules
* fmt
* Fix after mege
* Enable JMH annotation processor
* Fix compileOrder in some projects
* Insert TruffleBoundary to QualifiedName.
This is a revert
* Fix building of engine-runner native image
* Add more deps to the native image
* Force module-info compilation in instruments.
This fixes some weird sbt bug
* Don't run engine-runner/assembly from Rust build script
* Update docs of JPMSPlugin
* fmt
* runtime-benchmarks depends on benchmarks-common module
* Fix benchmark report writing
* std-benchmarks annot processing does not take settings from runtime-benchmarks
* Suppress interpreter only warning in annotation processor
* Runtime version manager does not expect runtime.jar fat jar
* fmt
* Fix module entry point
* Move some polyglot tests to runtime-integration-tests.
Also make their output silent
* pkg has no dependency on org.graalvm.truffle
* Fix compiler dependencies test
* Rename all runtime.jar in fake releases
* Add language-server with dependencies to component dir
* No module-info.class in target dir is warning not error
* language-server does not depend on netbeans lookup uitl
* Declare LanguageServerApi service provider in module-info
* connected-lock-manager-server is JPMS module
* task-progress-notifications is module
* Add fansi-wrapper module
* Fix compilation of connected-lock-manager-server
* Define correct Test/internalModuleDependencies for project-manager
* fmt
* Fix LauncherRunnerSpec - no runtime.jar
* Add fansi-wrapper to runtime-integration-tests and runtime-benchmarks
* Fix engine-runner native image build
* Use newer JNA version - fixes running of hyperd
* DRY
* scala-compiler DRY
* fmt
* More build.sbt refactoring
* Include runtime-instrument-id-execution in engine-runner native image
* TruffleBoundary for QualifiedName.toString
* Finding a needle in a haystack
🤦
* More scala-library DRY
* more mixed-java/scala goodies
* Fix compilation of syntax-rust-definition
* Test that engine-runner does not depend on language-server
* Append rather than assign `moduleDependencies`
`++=` is less error prone than `:=`. Also discovered some unnecessary
dependencies.
* Replace : with File.pathSeparator
* [WIP] Make logging in ProjectService more verbose
* language-server/test didn't start because of missing lookup and fansi modules
* Formatting
* org.enso.cli.task.notifications needs Akka and Circe to link
* project-manager/test depends on buildEngineDistribution
* [WIP] Even more verbose logging for creating projects
* [WIP] Even more verbose logging for creating projects
* Revert "[WIP] Even more verbose logging for creating projects"
This reverts commit a7067c8472.
* Revert "[WIP] Even more verbose logging for creating projects"
This reverts commit fc6f53d4f1.
* Revert "[WIP] Make logging in ProjectService more verbose"
This reverts commit 427428e142.
* All the project with JPMSPlugin has stripped artifact names
* Revert all placeholder fake release components to runtime.jar without version
* Eliminate a cross version hack
We shouldn't be specifying Scala dependencies with a Scala cross version
in the suffix.
* Address SBT lint warnings
* Revert "Eliminate a cross version hack"
This reverts commit 8861dab288.
* logging-service-logback is mixedJavaScalaProject
* fmt
* Stripped artifact name contains classifier.
This fixes tests as those were named like `artifact-tests.jar`.
* Don't use LocalProject unless really needed
* Add more logging when BenchProcessor fails
* logging-service-logback is not mixed project
* Work with java.io.File.getPath to avoid mixing slash and backslash on Windows
* Reapply "Eliminate a cross version hack"
This reverts commit edaa436ee8.
* Pass scalaBinaryVersion correctly
* Remove scala-compiler from the distribution
* Fix IllegalAccessErrors from serde
* typos
* License review
* fmt
* Move testLogProviderOnModulePath to TestJPMSConfiguration
* logging-service-logback is not a mixed project
---------
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
- Update all the templates to the newest versions.
- Fix any defects in the templates.
- Update the rust and scala code to package and download the templates.
- Update the front end to have images for every template (with some Gemini assists).
https://github.com/user-attachments/assets/59a1f845-f409-45bb-9c77-fe3a2d1262b1
* Rename packages in logging-utils-akka
* Migrate buildInfo to Java
* Rename packages in logging
* Rename package in scala-yaml
* No usage of CompilerDirectives inside pkg
* log errors of initialization of directory watcher
* HashCodeNode does not use com.google.common.base.Objects
* Rename rest of the packages
* fmt
* Fix dependencies on version-output
* Add necessary dependencies to testkit
* Rename instruments in runtime-fat-jar module-info
* Fix compilation errors because of BuildVersion
* Fix logger renames
* Use java.util.List directly
* Fixes after merge
* Improve error message in NativeLauncherSpec
* Fix logger renames
* Fix json version formatting
* Revert "No usage of CompilerDirectives inside pkg"
This reverts commit cc7e078416.
* fmt
- Linting updates.
- Add an `Examples.welcome` and adjust the start up project to use it.
- Merge all of Cass's work into the source code.
- Make example render in mono space font.
- Closes#9486
- All tests are succeeding or marked pending
- Created follow up tickets for things that still need to be addressed, including:
- Fixing upload / table update #10609
- Fixing `Count_Distinct` on Boolean columns #10611
- Running the tests on CI is not part of this PR - to be addressed separately
Closes#10564. Dashboard accepts only projects with non-empty namespace.
The change in project-manager ensures that we serialize `namespace` even if it is a default one.
The change in dashboard ensures that namespace can be empty.
* Eliminating circe-yaml
This change adds our very-own YAML parser on top of SnakeYAML. Compared
to Circe parser on top of SnakeYAML. The advantage? In some not-so-distant
future we might actually get rid of circe and the related performance
issues.
The logic is similar to what circe does i.e. analyzing SnakeYAML to
build our own structure.
This change is not complete, as there are still some tests failing, but
most common Configs are already parseable.
We _could_ auto-generate some of the code but still some of the logic
would have to be tweaked by hand; the current logic has a number of
special cases, as I found out the hard way.
* wip: more tests passing
* Fix remaining tests in ConfigSpec
* Fixing YAML decoder for editions
Dropping circe as a decoder for Editions revealed some problems. Turns
out the current implementation had even more special cases to deal with.
* nit
* Allow for empty exports
* Mostly complete encodin part
Replaced almost all `toYAML` locations with SnakeYAML equivalent.
The encoding has to use Java collections for which there exists a
built-in support. If we were to use Scala collections we would have to
deal with tagging, at the very least.
* Remove the last remaining Circe's YAML parser
* Bug fix + further loop optimization
* removal of some dependencies
* Remove circe-yaml
Added a custom SnakeYAML Node updater to mimick the JSON -> YAML -> JSON
conversion needed for updating fields. The algorithm recursively follows
the key-path and inserts the desired Node. This is not a performance
oriented code on purpose.
* Fix compilation issues
`circe-core` was marked as `provided` but no one eventually included it
in the final jar, hence `NoClassFoundException`.
* fix licensing
* Removing obsolete circe definitions
* fmt
* nits
* s/SnakeYamlDecoder/YamlDecoder
* fmt
* Partial revert, PM needs JSON decoders/encoders
* style
* incremental compilation gone wrong
The current implementation contains logic that should enable us to make some backward compatibility config changes.
At the same time, the logic is tightly integrated with circe's JSON library, which we want to eventually to get rid off.
Rather than trying to keep it somehow around and maintain via some hacks this PR proposes to ditch that logic completely as we currently have no use-case for such scenarios.
As a result, classes modelling YAML configs now don't have the extra fields and there is 1:1 correspondence.
Performance has also improved although that wasn't the main objective, yet. Follow up PR will attempt to replace `circe-yaml` with `snakeyaml` directly.
In preparation for #9113. Note that the dependency upgrade is necessary because it brings latest available `snakeyaml` (as part of `circe-yaml`).
`Bump` library uses parser combinators behind the scenes which are known to be good at expressing grammars but are not performance-oriented.
This change ditches the dependency in favour of an existing Java implementation. `jsemver` implements the full specification, which is probably an overkill in our case, but proved to be an almost drop-in replacement for the previous library.
Closes#8692
# Important Notes
Peformance improvements:
- roughly 50ms compared to the previous approach (from 80ms to 20-40ms)
I don't see any time spent in the new implementation during startup so it could be potentially aggressively inlined.
Further more, we could use a facade and offer our own strip down version of semver.
For our users "Nodes" are called "Components". Which the users selects using the component browser. This MR just sets the text in the default workflow.
Add a local clone of javaFormatter plugin. The upstream is not maintained anymore. And we need to update it to use the newest Google java formatter because the old one, that we use, cannot format sources with Java 8+ syntax.
# Important Notes
Update to Google java formatter 1.18.1 - https://github.com/google/google-java-format/releases/tag/v1.18.1
* Test illustrating problems with FQNs
Inline execution fails with `Compile error: The name `Standard` could
not be found.`.
* Ensure InlineContext carries Package Repos info
Previously, there was no requirement that inline execution should allow
for FQNs. This meant that the omission of Package Repository info went
unnoticed.
In order to be able to refer to `Standard.Visualization.Preprocessor` it
has to be exported as well.
It seems that Runtime Connector wasn't respecting the protocol it defined itself. The connector should be waiting on the `Api.InitializedNotification` message and only then start forwarding messages. So far it seems this hasn't been a problem, or at least wasn't reported as such, because initialization was fast enough.
Modified `Handler` so that we are certain that its fields hold initialized values when being accessed by different threads.
Should fix problems mentioned in #7898.
- Closes#7633
- Moves `Round_Spec.enso` from published `Standard.Test` into our `test/Tests` project; the `Table_Tests` that depend on it, simply `import enso_dev.Tests`.
- Changes the layout of the local libraries directory:
- It used to be `root/<namespace>/<name>`.
- Now it is `root/<dir>` - the namespace and name are now read from `package.yaml` instead.
- Adds the parent directory of the current project to the default `ENSO_LIBRARY_PATH`.
- It is treated as a secondary path, so the default `ENSO_HOME/lib` still takes precedence.
- This allows projects to reference and load 'sibling' projects easily - the only requirement is for the project to enable `prefer-local-libraries: true` or add the other local project to its edition. The edition resolution logic is **not changed**.
close#7604
After moving the rename action to the dashboard, IDE is unaware of the new project name. PR implements a new `refactoring/projectRenamed` notification that is sent from the server to clients and informs them about the changed project name.
# Important Notes
https://github.com/enso-org/enso/assets/357683/7c62726d-217e-4e69-8e48-568e0b7b8c34
This was meant to be a trivial change, but actually a dirty workaround needed to be applied. Because enter is used to both open searcher and accept input/entry, both actions were fired at once. I fixed it by debouncing opening searcher event (so the searcher will be opened only once key event handling is over)
close#7194
Changelog:
- add: `/projects/{project_id}/enso_project` HTTP endpoint returning an `.enso-project` archive structure
- update: archive enso project to a `.enso-project` `.tar.gz` archive
- update: make project `path` a required field
It is relatively easy to reach timeouts on weak systems for startup
operations on project manager. Once a timeout is reached, startup will
not proceed any further.
This PR is a bit of an experiment. It adds adds timeout retries to give a bit of a leeway to
under-powered machines and to log some progress on the way, so that we
know that certain actions are still in progress.
Package's config information, once loaded, never changed. While there is typically no need for it, this was problematic when the config became out-of-sync with the filesystem, like in the case of project rename action.
In rename, the config's properties would be updated in the FS, but that would never be reflected in module's package. Therefore further compilations would continue to ask for the old namespace.
Most of the changes are cosmetic (s/`.config`/`.getConfig()`) except for the new `reloadConfig` method on `Package` that is being called in `RenameProjectCmd` handler.
Closes#7062.
# Important Notes
The reported `ExecutionFailed` error should have been mostly fixed already via #7143. This change makes sure that all the related warnings are gone as well and the compiler uses the updated namespace.
Empty edition (null value) was parsed as NaN, which was confusing. This change correctly detects the case before trying different fallback mechanisms.
Addresses invalid warning mentioned in #6806.
close#5911
In interactive mode, perform writing IR caches in the background jobs queue. Background jobs execution is delayed until the first execution is complete.
This change downgrades hashing algorithm used in caching IR and library bindings to SHA-1. It is sufficient and significantly faster for the purpose of simple checksum we use it for.
Additionally, don't calculate the digest for serialized bytes - if we get the expected object type then we are confident about the integrity.
Don't initialize Jackson's ObjectMapper for every metadata serialization/de-serialization. Initialization is very costly.
Avoid unnecessary conversions between Scala and Java. Those back-and-forth `asScala` and `asJava` are pretty expensive.
Finally fix an SBT warning when generating library cache.
Closes https://github.com/enso-org/enso/issues/5763
# Important Notes
The change cuts roughly 0.8-1s from the overall startup.
This change will certainly lead to invalidation of existing caches. It is advised to simply start with a clean slate.
This change adds serialization and deserialization of library bindings.
In order to be functional, one needs to first generate IR and
serialize bindings using `--compiled <path-to-library>` command. The bindings
will be stored under the library with `.bindings` suffix.
Bindings are being generated during `buildEngineDistribution` task, thus not
requiring any extra steps.
When resolving import/exports the compiler will first try to load
module's bindings from cache. If successful, it will not schedule its
imports/exports for immediate compilation, as we always did, but use the
bindings info to infer the dependent modules.
The current change does not make any optimizations when it comes to
compiling the modules, yet. It only delays the actual
compilation/loading IR from cache so that it can be done in bulk.
Further optimizations will come from this opportunity such as parallel
loading of caches or lazily inferring only the necessary modules.
Part of https://github.com/enso-org/enso/issues/5568 work.
In cloud we want to allow users to create new project from the template. List of templates is a bit outdated and doesn't contain all from the https://github.com/enso-org/project-templates. This PR simply adds missing ones
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720
real 0m4.503s
user 0m9.248s
sys 0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720
real 0m0.176s
user 0m0.042s
sys 0m0.038s
```
# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.
Will need to add the above command to CI.
This change allows for importing modules using a qualified name and deals with any conflicts on the way.
Given a module C defined at `A/B/C.enso` with
```
type C
type C a
```
it is now possible to import it as
```
import project.A
...
val x = A.B.C 10
```
Given a module located at `A/B/C/D.enso`, we will generate
intermediate, synthetic, modules that only import and export the successor module along the path.
For example, the contents of a synthetic module B will look like
```
import <namespace>.<pkg-name>.A.B.C
export <namespace>.<pkg-name>.A.B.C
```
If module B is defined already by the developer, the compiler will _inject_ the above statements to the IR.
Also removed the last elements of some lowercase name resolution that managed to survive recent
changes (`Meta.Enso_Project` would now be ambiguous with `enso_project` method).
Finally, added a pass that detects shadowing of the synthetic module by the type defined along the path.
We print a warning in such a situation.
Related to https://www.pivotaltracker.com/n/projects/2539304
# Important Notes
There was an additional request to fix the annoying problem with `from` imports that would always bring
the module into the scope. The changes in stdlib demonstrate how it is now possible to avoid the workaround of
```
from X.Y.Z as Z_Module import A, B
```
(i.e. `as Z_Module` part is almost always unnecessary).
Result of automatic formatting with `scalafmtAll` and `javafmtAll`.
Prerequisite for https://github.com/enso-org/enso/pull/3394
### Important Notes
This touches a lot of files and might conflict with existing PRs that are in progress. If that's the case, just run
`scalafmtAll` and `javafmtAll` after merge and everything should be in order since formatters should be deterministic.
Changelog:
- add: component groups to package descriptions
- add: `executionContext/getComponentGroups` method that returns component groups of libraries that are currently loaded
- doc: cleanup unimplemented undo/redo commands
- refactor: internal component groups datatype
