Commit Graph

29 Commits

Author SHA1 Message Date
Jaroslav Tulach
fe45da98d7
Use enso.dev.insight property to turn Insight on (#11385) 2024-10-24 13:56:28 +02:00
Pavel Marek
9182f91e35
engine-runner and language-server are separate JPMS modules (#10823)
* Use moduleDependencies instead of modulePath

* Fix compilation of editions

* Fix compilation of distribution-manager

* polyglot-api needs to explicitly compile module-info

* Fix compilationOrder in library-manager and edition-updater

* engine-runner-common is module

* JPMSPlugin provides default implementation of compileModuleInfo

* Remove unused setting key from JPMSUtils.compileModuleInfo

* JPMSPlugin has internalModuleDependencies and exportedModule tasks.

* Use BuildVersion instead of buildInfo

* Manual compilation of module-info.java is reported as warning

* Define org.enso.scalalibs.wrapper meta project.

* Fix module check in JPMSPlugin.

This is a fix for projects that declare `Compile /exportJars := true`

* version-output is a module

* ydoc-server uses internalModuleDependencies

* persistance is module

* engine-common uses internalModuleDependencies

* polyglot-api does not override compileModuleInfo task

* runtime-parser uses internalModuleDependencies

* edition-updater is module

* Update moduleDependencies for distribution-manager

* editions is module

* Fix some dependencies of modules

* scala-yaml is a module

* Add scala-compiler to scala-libs-wrapper

* cli depends on scala-library module

* Add dependencies for distribution-manager module

* Add some scala-library dependencies in some modules

* engine-runner uses internalModuleDependencies

* Fix module dependencies of library-manager

* Rename org.enso.scalalibs.wrapper to org.enso.scala.wrapper

* Add jsoniter-scala-macros to org.enso.scala.wrapper fat module

* Fix dependencies of some projects

* polyglot-api does not depend on truffle-api

* Fix dependencies of some projects

* runtime does not use com.google.common

* runtime is a module

* text-buffer is a module

* refactoring-utils is a module

* runtime-compiler is a module

* runtime-instrument-common is a module

* connected-lock-manager is a module

* JPMSUtils reports project name in some error messages

* Modularize some instruments

* module-info compilation is cached

* runtime-instrument-runtime-server is module

* runtime-language-epb is module

* Remove runtime-fat-jar

* engine-runner is not a fat jar

* JPMSPlugin defines exportedModuleBin task

* Redefine componentModulesPaths task

* interpreter-dsl is module

* Redefine componentModulesPaths task

* fmt sbt

* scala-libs-wrapper is a modular fat jar

* Add some module deps to org.enso.runtime

* engine-runner is not a fat jar

* Rename package in logging-config

* Rename package in logging-service

* Rename package in logging-service-logback

* Fix dependencies of exportedModuleBin task

* Mixed projects have own compileJava task

this task does not compile only module-info.java but all the java sources. So that we can see errors more easily.

When only module-info.java is compiled, the only errors that we can see are that we did not include some modules on module-path.

* Fix definition of exportedModule task.

* Remove usages of non-existing buildInfo and replace it with BuildVersion

* Fix some dependencies of org.enso.runtime module

* module-info compilation is handled directly by FrgaalCompiler

* module-info compilation is forced for projects that has only Scala sources with single module-info.java

* Fix compilation of org.enso.runtime

* manual module-info compilation is not a warning

* Rename packages in logging-utils-akka

* Create org.enso.language.server.deps.wrapper module

* language-server is module

* Creat akka-wrapper modular fat jar

* fmt

* Define common settings for modularFatJarWrapper

* Fix compilation of json-rpc-server

* Use akka and zio wrappers

* language-server depends on org.eclipse.jgit

* Fix some dependencies - update library manifests works now!

* update library manifests invokes runner directly

* buildEngineDistribution does not copy runner.jar

* Remove EngineRunnerBootLoader

* Fix compilation of std libs

* --patch-module and --add-exports are also passed to javac

* Rename package in runtime-integration-tests.

The package name org.enso.compiler clashes with the package from the module

* Remove usage of buildInfo

* FrgaalJavaCompiler can deal with non-existing module-info.java when shouldCompileModuleInfo is true.

It just generates a warning in such case as it suggests that there is something wrong with the project configuration.

* Revert AliasAnalysisTest.scala

* Fix dependencies and java cmdline options for runtime-integration-tests

* Rename test package

* runtime-integration-test depends on logging-service-logback/Test/compile

* Rename package in logging-service-logback/Test

* Fix FrgaalJavaCompiler creation for projects

* Sanitize Test/javaOptions arguments

* Sanitize Test/javaOptions arguments

* All the JPMSPlugin settings are scoped

* Remove unused sbt tasks

* modularFatJarWrapperSettings do not override javacOptions

* Resolve issue "Cannot find TestLoggerProvider" in runtime-integration-tests

* org.enso.runtime module is open

* Test that test classes are unconditionally opened and exported

* polyglot-api-macros is a module

* JPMSPlugin handles --add-opens cmdline option

* RuntimeServerTest ensures instruments are initialized

* Add some exports to org.enso.runtime.compiler

* Add instruments on module-path to runtime-integration-tests

* Replace TestLogProviderOnClassPath with TestLogProviderOnModulePath

* Replace buildInfo with BuildVersion

* Add jpms-wrapper-scalatest

* ReportLogsOnFailure is in non-modular testkit project

* Add necessary dependencies to testkit project

* Revert "Add jpms-wrapper-scalatest"

This reverts commit 732b3427a2.

* modularize filewatcher and wrap its dependencies

* Initial fix for language-server/test

* frgaal compiler setting are scoped for Compile and Test

* Rename package in language-server/test

* Exclude com.sun.jna from wrapper jars

* Rename package in library-manager-test

* testkit is an automatic module

* process-utils is module

* akka-wrapper contains akka-http

* Some fixes for library-manager-test

* Fix dependencies for akka-wrapper

* scala-libs-wrapper exports shapeless

* lang server deps wrapper exports pureconfig

* json-rpc-server requires org.slf4j

* Add some dependencies

* lang server deps wrapper exports pureconfig.generic

* language server test requires bouncycastle provider

* language server depends on cli

* directory-watcher wrapper requires org.slf4j

* WatcherAdapter logs unsuccessful initialization errors

* Fix error reporting in WatcherAdapter

* Fix rest of the language-server tests

* language-server-deps-wrapper depends on scala-libs-wrapper

* Fix rest of the language-server tests

* Missing module-info.class in an internal project is a warning, not an error

* Rename jpms-methvin-directory-watcher-wrapper to a simpler name

* compileOrder has to be specified before libraryDependencies

* exclude module-info.java from polyglot-api-macros

* Remove temporary logging in customFrgaalCompilerSettings

* Fix compilation of logging-service-logback

* Fix compilation of runtime-benchmarks

* Fix runtime-benchmarks/run

* HostClassLoader delegates to org.graalvm.polyglot class loader if org.enso.runtime is not on boot layer

* org.enso.runtime.lnaguage.epb module must be opened to allow it to be used by annnotation processor

* fmt

* Fix afetr merge

* Add module deps after merge

* Print stack trace of the uncaught exception from the annotation processor

* Remove akka-actor-typed from akka-wrapper

* runtime-instrument-common depends on slf4j

* Fix module-path for runtime-instrument-repl-debugger

* runtime-benchmarks depends on runtime-language-arrow

* --module-path is passed directly to frgaal

* Fix some module-related cmd line options for std-benchmarks

* Revert "--module-path is passed directly to frgaal"

This reverts commit da63f66a0e.

* Avoid closing of System.err when closing Context

* Avoid processing altogether when requested annotations are empty

* Pass shouldNotLimitModules opt to frgaal

* Pass module-path and add-modules options with -J prefix to frgaal

* BenchProcessor annotation processor creates its own truffle module layer

* bench-processor and benchmarks-common are modules

* fmt

* Fix after mege

* Enable JMH annotation processor

* Fix compileOrder in some projects

* Insert TruffleBoundary to QualifiedName.

This is a revert

* Fix building of engine-runner native image

* Add more deps to the native image

* Force module-info compilation in instruments.

This fixes some weird sbt bug

* Don't run engine-runner/assembly from Rust build script

* Update docs of JPMSPlugin

* fmt

* runtime-benchmarks depends on benchmarks-common module

* Fix benchmark report writing

* std-benchmarks annot processing does not take settings from runtime-benchmarks

* Suppress interpreter only warning in annotation processor

* Runtime version manager does not expect runtime.jar fat jar

* fmt

* Fix module entry point

* Move some polyglot tests to runtime-integration-tests.

Also make their output silent

* pkg has no dependency on org.graalvm.truffle

* Fix compiler dependencies test

* Rename all runtime.jar in fake releases

* Add language-server with dependencies to component dir

* No module-info.class in target dir is warning not error

* language-server does not depend on netbeans lookup uitl

* Declare LanguageServerApi service provider in module-info

* connected-lock-manager-server is JPMS module

* task-progress-notifications is module

* Add fansi-wrapper module

* Fix compilation of connected-lock-manager-server

* Define correct Test/internalModuleDependencies for project-manager

* fmt

* Fix LauncherRunnerSpec - no runtime.jar

* Add fansi-wrapper to runtime-integration-tests and runtime-benchmarks

* Fix engine-runner native image build

* Use newer JNA version - fixes running of hyperd

* DRY

* scala-compiler DRY

* fmt

* More build.sbt refactoring

* Include runtime-instrument-id-execution in engine-runner native image

* TruffleBoundary for QualifiedName.toString

* Finding a needle in a haystack

🤦

* More scala-library DRY

* more mixed-java/scala goodies

* Fix compilation of syntax-rust-definition

* Test that engine-runner does not depend on language-server

* Append rather than assign `moduleDependencies`

`++=` is less error prone than `:=`. Also discovered some unnecessary
dependencies.

* Replace : with File.pathSeparator

* [WIP] Make logging in ProjectService more verbose

* language-server/test didn't start because of missing lookup and fansi modules

* Formatting

* org.enso.cli.task.notifications needs Akka and Circe to link

* project-manager/test depends on buildEngineDistribution

* [WIP] Even more verbose logging for creating projects

* [WIP] Even more verbose logging for creating projects

* Revert "[WIP] Even more verbose logging for creating projects"

This reverts commit a7067c8472.

* Revert "[WIP] Even more verbose logging for creating projects"

This reverts commit fc6f53d4f1.

* Revert "[WIP] Make logging in ProjectService more verbose"

This reverts commit 427428e142.

* All the project with JPMSPlugin has stripped artifact names

* Revert all placeholder fake release components to runtime.jar without version

* Eliminate a cross version hack

We shouldn't be specifying Scala dependencies with a Scala cross version
in the suffix.

* Address SBT lint warnings

* Revert "Eliminate a cross version hack"

This reverts commit 8861dab288.

* logging-service-logback  is mixedJavaScalaProject

* fmt

* Stripped artifact name contains classifier.

This fixes tests as those were named like `artifact-tests.jar`.

* Don't use LocalProject unless really needed

* Add more logging when BenchProcessor fails

* logging-service-logback is not mixed project

* Work with java.io.File.getPath to avoid mixing slash and backslash on Windows

* Reapply "Eliminate a cross version hack"

This reverts commit edaa436ee8.

* Pass scalaBinaryVersion correctly

* Remove scala-compiler from the distribution

* Fix IllegalAccessErrors from serde

* typos

* License review

* fmt

* Move testLogProviderOnModulePath to TestJPMSConfiguration

* logging-service-logback is not a mixed project

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
2024-09-25 21:33:13 +02:00
Pavel Marek
660c5e7a9d
Atom constructors can be private (#9692)
Closes #8836.

Atom constructors can be declared as private (project-private). project-private constructors can be called only from the same project. See the encapsulation.md docs for more info.

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
2024-04-29 14:43:18 +02:00
Jaroslav Tulach
8d9c25cdda
Syntax Color and Debug JavaScript/Python in Enso Source (#9440) 2024-03-15 10:28:13 +01:00
Pavel Marek
c1098865f2
Update java formatter sbt plugin (#8543)
Add a local clone of javaFormatter plugin. The upstream is not maintained anymore. And we need to update it to use the newest Google java formatter because the old one, that we use, cannot format sources with Java 8+ syntax.

# Important Notes
Update to Google java formatter 1.18.1 - https://github.com/google/google-java-format/releases/tag/v1.18.1
2023-12-15 14:45:23 +00:00
Pavel Marek
4b65e44ef3
EpbLanguage re-uses other TruffleContext support to run tests with assertions enabled (#7882) 2023-12-15 13:31:32 +01:00
Pavel Marek
a67297aebf
Add graalpy packages to the component directory (#8351)
Adds these JAR modules to the `component` directory inside Engine distribution:
- `graal-language-23.1.0`
- `org.bouncycastle.*` - these need to be added for graalpy language

# Important Notes
- Remove `org.bouncycastle.*` packages from `runtime.jar` fat jar.
- Make sure that the `./run` script preinstalls GraalPy standalone distribution before starting engine tests
- Note that using `python -m venv` is only possible from standalone distribution, we cannot distribute `graalpython-launcher`.
- Make sure that installation of `numpy` and its polyglot execution example works.
- Convert `Text` to `TruffleString` before passing to GraalPy - 8ee9a2816f
2023-12-04 11:50:59 +00:00
Hubert Plociniczak
0402e8bafb
Ensure compilation is run with a compilation lock (#8395)
Evaluating visualization expression may trigger a full compilation. A change in #7042 went a bit too far and led to a situation when there could be compilations running at the same time leading to a rather obscure `RedefinedMethodException` when the compilation on one thread already finished. This will make the logic correct again at the price of potentially slowing the processing of visualization.

Closes #8296.

# Important Notes
Should make visualizations a bit more stable as well.
2023-11-27 14:01:23 +00:00
Hubert Plociniczak
af5be19be8
Minor tweaks post jdk21 upgrade (#8373) 2023-11-23 11:58:49 +00:00
Jaroslav Tulach
198ab7f487
Support for Python libraries like numpy (#7678) 2023-08-30 06:10:18 +02:00
Jaroslav Tulach
6461e20870
Special support for Python Date/Time/Zone interop (#7617) 2023-08-25 10:27:16 +02:00
Jaroslav Tulach
e224eb350a
Eliminate compile time dependency on runtime-language-epb (#7611) 2023-08-21 05:31:39 +02:00
Jaroslav Tulach
7a272ec152
Encapsulating array-like data and operations into a single package (#7544) 2023-08-15 13:00:47 +02:00
Pavel Marek
cab6968340
Update GraalVM to 17.0.7 (23.0.0 JDK17) (#7176)
- Previous GraalVM update: https://github.com/enso-org/enso/pull/6750

Removed warnings:
- Remove deprecated `ConditionProfile.createCountingProfile()`.
- Add `@Shared` to some `@Cached` parameters (Truffle now emits warnings about potential `@Share` usage).
- Specialization method names should not start with execute
- Add limit attribute to some specialization methods
- Add `@NeverDefault` for some cached initializer expressions
- Add `@Idempotent` or `@NonIdempotent` where appropriate

BigInteger and potential Node inlining are tracked in follow-up issues.

# Important Notes
For `SDKMan` users:
```
sdk install java 17.0.7-graalce
sdk use java 17.0.7-graalce
```

For other users - download link can be found at https://github.com/graalvm/graalvm-ce-builds/releases/tag/jdk-17.0.7

Release notes: https://www.graalvm.org/release-notes/JDK_17/

R component was dropped from the release 23.0.0, only `python` is available to install via `gu install python`.
2023-07-20 15:11:30 +00:00
Dmitry Bushev
6249c79ffd
Update sbt-java-formatter plugin (#7011)
Update java formatter plugin. The new version can remove unused imports.
2023-06-12 14:18:48 +00:00
Jaroslav Tulach
cb3bcbb495
Schedule initialization of JS context in a separate thread (#5680)
Implements the #5643 idea. As soon as `MainModule` creates `Context` for GraalVM execution, it schedules a background task to initialize JavaScript. The initialization finishes sooner than Enso compiler is ready to work, saving time when it is actually needed.

# Important Notes
Only modifies boot sequence of `MainModule` (used in the IDE) and `VerifyJavaScriptIsAvailableTest` (to verify the _"context passing logic"_ works OK between threads). Regular CLI execution remains unchanged for now assuming batch execution may not need JavaScript in all the cases and if it does the initialization speed isn't that critical.
2023-02-17 15:54:36 +00:00
Pavel Marek
1f8511dab2
Add Comparator conversion for all types (#4067)
Add `Comparator` type class emulation for all types. Migrate all the types in stdlib to this new `Comparator` API. The main documentation is in `Ordering.enso`.

Fixes these pivotals:
- https://www.pivotaltracker.com/story/show/183945328
- https://www.pivotaltracker.com/story/show/183958734
- https://www.pivotaltracker.com/story/show/184380208

# Important Notes
- The new Comparator API forces users to specify both `equals` and `hash` methods on their custom comparators.
- All the `compare_to` overrides were replaced by definition of a custom _ordered_ comparator.
- All the call sites of `x.compare_to y` method were replaced with `Ordering.compare x y`.
- `Ordering.compare` is essentially a shortcut for `Comparable.from x . compare x y`.
- The default comparator for `Any` is `Default_Unordered_Comparator`, which just forwards to the builtin `EqualsNode` and `HashCodeNode` nodes.
- For `x`, one can get its hash with `Comparable.from x . hash x`.
- This makes `hash` as _hidden_ as possible. There are no other public methods to get a hash code of an object.
- Comparing `x` and `y` can be done either by `Ordering.compare x y` or `Comparable.from x . compare x y` instead of `x.compare_to y`.
2023-02-10 09:22:11 +00:00
Pavel Marek
fcc2163ae3
All Enso objects are hasheable (#3878)
* Hash codes prototype

* Remove Any.hash_code

* Improve caching of hashcode in atoms

* [WIP] Add Hash_Map type

* Implement Any.hash_code builtin for primitives and vectors

* Add some values to ValuesGenerator

* Fix example docs on Time_Zone.new

* [WIP] QuickFix for HashCodeTest before PR #3956 is merged

* Fix hash code contract in HashCodeTest

* Add times and dates values to HashCodeTest

* Fix docs

* Remove hashCodeForMetaInterop specialization

* Introduce snapshoting of HashMapBuilder

* Add unit tests for EnsoHashMap

* Remove duplicate test in Map_Spec.enso

* Hash_Map.to_vector caches result

* Hash_Map_Spec is a copy of Map_Spec

* Implement some methods in Hash_Map

* Add equalsHashMaps specialization to EqualsAnyNode

* get and insert operations are able to work with polyglot values

* Implement rest of Hash_Map API

* Add test that inserts elements with keys with same hash code

* EnsoHashMap.toDisplayString use builder storage directly

* Add separate specialization for host objects in EqualsAnyNode

* Fix specialization for host objects in EqualsAnyNode

* Add polyglot hash map tests

* EconomicMap keeps reference to EqualsNode and HashCodeNode.

Rather than passing these nodes to `get` and `insert` methods.

* HashMapTest run in polyglot context

* Fix containsKey index handling in snapshots

* Remove snapshots field from EnsoHashMapBuilder

* Prepare polyglot hash map handling.

- Hash_Map builtin methods are separate nodes

* Some bug fixes

* Remove ForeignMapWrapper.

We would have to wrap foreign maps in assignments for this to be efficient.

* Improve performance of Hash_Map.get_builtin

Also, if_nothing parameter is suspended

* Remove to_flat_vector.

Interop API requires nested vector (our previous to_vector implementation). Seems that I have misunderstood the docs  the first time I read it.

- to_vector does not sort the vector by keys by default

* Fix polyglot hash maps method dispatch

* Add tests that effectively test hash code implementation.

Via hash map that behaves like a hash set.

* Remove Hashcode_Spec

* Add some polyglot tests

* Add Text.== tests for NFD normalization

* Fix NFD normalization bug in Text.java

* Improve performance of EqualsAnyNode.equalsTexts specialization

* Properly compute hash code for Atom and cache it

* Fix Text specialization in HashCodeAnyNode

* Add Hash_Map_Spec as part of all tests

* Remove HashMapTest.java

Providing all the infrastructure for all the needed Truffle nodes is no longer manageable.

* Remove rest of identityHashCode message implementations

* Replace old Map with Hash_Map

* Add some docs

* Add TruffleBoundaries

* Formatting

* Fix some tests to accept unsorted vector from Map.to_vector

* Delete Map.first and Map.last methods

* Add specialization for big integer hash

* Introduce proper HashCodeTest and EqualsTest.

- Use jUnit theories.
- Call nodes directly

* Fix some specializations for primitives in HashCodeAnyNode

* Fix host object specialization

* Remove Any.hash_code

* Fix import in Map.enso

* Update changelog

* Reformat

* Add truffle boundary to BigInteger.hashCode

* Fix performance of HashCodeTest - initialize DataPoints just once

* Fix MetaIsATest

* Fix ValuesGenerator.textual - Java's char is not Text

* Fix indent in Map_Spec.enso

* Add maps to datapoints in HashCodeTest

* Add specialization for maps in HashCodeAnyNode

* Add multiLevelAtoms to ValuesGenerator

* Provide a workaround for non-linear key inserts

* Fix specializations for double and BigInteger

* Cosmetics

* Add truffle boundaries

* Add allowInlining=true to some truffle boundaries.

Increases performance a lot.

* Increase the size of vectors, and warmup time for Vector.Distinct benchmark

* Various small performance fixes.

* Fix Geo_Spec tests to accept unsorted Map.to_vector

* Implement Map.remove

* FIx Visualization tests to accept unsorted Map.to_vector

* Treat java.util.Properties as Map

* Add truffle boundaries

* Invoke polyglot methods on java.util.Properties

* Ignore python tests if python lang is missing
2023-01-19 10:33:25 +01:00
Hubert Plociniczak
49204e92cf
Simplify exception handling for polyglot exceptions (#3981)
This removes the special handling of polyglot exceptions and allows matching on Java exceptions in the same way as for any other types.

`Polyglot_Error`, `Panic.catch_java` and `Panic.catch_primitive` are gone

The change mostly deals with the backslash of removing `Polyglot_Error` and two `Panic` methods.
`Panic.catch` was implemented as a builtin instead of delegating to `Panic.catch_primitive` builtin that is now gone.

This fixes https://www.pivotaltracker.com/story/show/182844611
2022-12-19 19:16:43 +00:00
Jaroslav Tulach
402ebb2f8e
Upgrade to GraalVM 22.3.0 (#3663)
Upgrading to GraalVM 22.3.0.

# Important Notes
- Removed all deprecated `FrameSlot`, and replaced them with frame indexes - integers.
- Add more information to `AliasAnalysis` so that it also gathers these indexes.
- Add quick build mode option to `native-image` as default for non-release builds
- `graaljs` and `native-image` should now be downloaded via `gu` automatically, as dependencies.
- Remove `engine-runner-native` project - native image is now build straight from `engine-runner`.
- We used to have `engine-runner-native` without `sqldf` in classpath as a workaround for an internal native image bug.
- Fixed chrome inspector integration, such that it shows values of local variables both for current stack frame and caller stack frames.
- There are still many issues with the debugging in general, for example, when there is a polyglot value among local variables, a `NullPointerException` is thrown and no values are displayed.
- Removed some deprecated `native-image` options
- Remove some deprecated Truffle API method calls.
2022-11-23 14:30:48 +00:00
Pavel Marek
47148a2ff1
Missing foreign language generates proper Enso error. (#3798)
Trying to invoke a foreign method with non-installed language (either not enabled in the Truffle `Context`, or not installed in the GraalVM distribution) results in `Polyglot_Error`, rather than crashing the entire engine.
2022-10-17 09:59:31 +00:00
Pavel Marek
e9260227c4
Duration type is a builtin type (#3759)
- Reimplement the `Duration` type to a built-in type.
- `Duration` is an interop type.
- Allow Enso method dispatch on `Duration` interop coming from different languages.

# Important Notes
- The older `Duration` type should now be split into new `Duration` builtin type and a `Period` type.
- This PR does not implement `Period` type, so all the `Period`-related functionality is currently not working, e.g., `Date - Period`.
- This PR removes `Integer.milliseconds`, `Integer.seconds`, ..., `Integer.years` extension methods.
2022-10-14 18:08:08 +00:00
Hubert Plociniczak
096fcfee82
Generate native image for engine-runner (#3638)
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m4.503s
user	0m9.248s
sys	0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m0.176s
user	0m0.042s
sys	0m0.038s
```

# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.

Will need to add the above command to CI.
2022-09-22 14:45:10 +00:00
Hubert Plociniczak
fba5047acc
Improved Vector/Array interop (#3667)
`Vector` type is now a builtin type. This requires a bunch of additional builtin methods for its creation:
- Use `Vector.from_array` to convert any array-like structure into a `Vector` [by copy](f628b28f5f)
- Use (already existing) `Vector.from_polyglot_array` to convert any array-like structure into a `Vector` **without** copying
- Use (already existing) `Vector.fill 1 item` to create a singleton `Vector`

Additional, for pattern matching purposes, we had to implement a `VectorBranchNode`. Use following to match on `x` being an instance of `Vector` type:
```
import Standard.Base.Data.Vector

size = case x of
Vector.Vector -> x.length
_ -> 0
```

Finally, `VectorLiterals` pass that transforms `[1,2,3]` to (roughly)
```
a1 = 1
a2 = 2
a3 = 3
Vector (Array (a1,a2, a3))
```
had to be modified to generate
```
a1 = 1
a2 = 2
a3 = 3
Vector.from_array (Array (a1, a2, a3))
```
instead to accomodate to the API changes. As of 025acaa676 all the known CI checks passes. Let's start the review.

# Important Notes
Matching in `case` statement is currently done via `Vector_Data`. Use:
```
case x of
Vector.Vector_Data -> True
```
until a better alternative is found.
2022-09-13 03:07:17 +00:00
Hubert Plociniczak
d87a32d019
Builtin Date_Time, Time_Of_Day, Zone (#3658)
* Builtin Date_Time, Time_Of_Day, Zone

Improved polyglot support for Date_Time (formerly Time), Time_Of_Day and
Zone. This follows the pattern introduced for Enso Date.

Minor caveat - in tests for Date, had to bend a lot for JS Date to pass.
This is because JS Date is not really only a Date, but also a Time and
Timezone, previously we just didn't consider the latter.
Also, JS Date does not deal well with setting timezones so the trick I
used is to first call foreign function returning a polyglot JS Date,
which is converted to ZonedDateTime and only then set the correct
timezone. That way none of the existing tests had to be changes or
special cased.

Additionally, JS deals with milliseconds rather than nanoseconds so
there is loss in precision, as noted in Time_Spec.

* Add tests for Java's LocalTime

* changelog

* Make date formatters in table happy

* PR review, add more tests for zone

* More tests and fixed a bug in column reader

Column reader didn't take into account timezone but that was a mistake
since then it wouldn't map to Enso's Date_Time.
Added tests that check it now.

* remove redundant conversion

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Time.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* First round of addressing PR review

* don't leak java exceptions in Zone

* Move Date_Time to top-level module

* PR review

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2022-08-24 12:31:29 +02:00
Hubert Plociniczak
4b9c91626e
Use Vector.from_polyglot_array to make Vectors backed by polyglot arrays (#3628)
Use Proxy_Polyglot_Array as a proxy for polyglot arrays, thus unifying
the way the underlying array is accessed in Vector.

Used the opportunity to cleanup builtin lookup, which now actually
respects what is defined in the body of @Builtin_Method annotation.

Also discovered that polyglot null values (in JS, Python and R) were leaking to Enso.
Fixed that by doing explicit translation to `Nothing`.

https://www.pivotaltracker.com/story/show/181123986
2022-08-23 21:13:16 +00:00
Hubert Plociniczak
f63e40df1b
Explicit self (#3569)
This change modifies the current language by requiring explicit `self` parameter declaration
for methods. Methods without `self` parameter in the first position should be treated as statics
although that is not yet part of this PR. We add an implicit self to all methods
This obviously required updating the whole stdlib and its components, tests etc but the change
is pretty straightforward in the diff.

Notice that this change **does not** change method dispatch, which was removed in the last changes.
This was done on purpose to simplify the implementation for now. We will likely still remove all
those implicit selfs to bring true statics.
Minor caveat - since `main` doesn't actually need self, already removed that which simplified
a lot of code.
2022-07-27 17:45:36 +00:00
Jaroslav Tulach
4465d63dd8
Improved polyglot Date support (#3559)
Significantly improves the polyglot Date support (as introduced by #3374). It enhances the `Date_Spec` to run it in four flavors:
- with Enso Date (as of now)
- with JavaScript Date
- with JavaScript Date wrapped in (JavaScript) array
- with Java LocalDate allocated directly

The code is then improved by necessary modifications to make the `Date_Spec` pass.

# Important Notes
James has requested in [#181755990](https://www.pivotaltracker.com/n/projects/2539304/stories/181755990) - e.g. _Review and improve InMemory Table support for Dates, Times, DateTimes, BigIntegers_ the following program to work:
```
foreign js dateArr = """
return [1, new Date(), 7]

main =
IO.println <| (dateArr.at 1).week_of_year
```
the program works with here in provided changes and prints `27` as of today.

@jdunkerley has provided tests for proper behavior of date in `Table` and `Column`. Those tests are working as of [f16d07e](f16d07e640). One just needs to accept `List<Value>` and then query `Value` for `isDate()` when needed.

Last round of changes is related to **exception handling**. 8b686b12bd makes sure `makePolyglotError` accepts only polyglot values. Then it wraps plain Java exceptions into `WrapPlainException` with `has_type` method - 60da5e70ed - the remaining changes in the PR are only trying to get all tests working in the new setup.

The support for `Time` isn't part of this PR yet.
2022-07-21 06:32:40 +00:00
Jaroslav Tulach
dc30e44b60
Register instruments/language in their own compilation units to fix the sbt build issues (#3509)
New plan to [fix the `sbt` build](https://www.pivotaltracker.com/n/projects/2539304/stories/182209126) and its annoying:
```
log.error(
"Truffle Instrumentation is not up to date, " +
"which will lead to runtime errors\n" +
"Fixes have been applied to ensure consistent Instrumentation state, " +
"but compilation has to be triggered again.\n" +
"Please re-run the previous command.\n" +
"(If this for some reason fails, " +
s"please do a clean build of the $projectName project)"
)
```
When it is hard to fix `sbt` incremental compilation, let's restructure our project sources so that each `@TruffleInstrument` and `@TruffleLanguage` registration is in individual compilation unit. Each such unit is either going to be compiled or not going to be compiled as a batch - that will eliminate the `sbt` incremental compilation issues without addressing them in `sbt` itself.

fa2cf6a33ec4a5b2e3370e1b22c2b5f712286a75 is the first step - it introduces `IdExecutionService` and moves all the `IdExecutionInstrument` API up to that interface. The rest of the `runtime` project then depends only on `IdExecutionService`. Such refactoring allows us to move the `IdExecutionInstrument` out of `runtime` project into independent compilation unit.
2022-06-13 14:09:08 +00:00