Commit Graph

73 Commits

Author SHA1 Message Date
Hubert Plociniczak
d6f7afc624
Open Java modules for Snowflake setup (#9664)
# Important Notes
Some workarounds are being considered but we need this change sooner than later.
2024-04-16 21:01:32 +00:00
Hubert Plociniczak
0133e9ef4f
Drop inefficient SemVer implementation (#9089)
`Bump` library uses parser combinators behind the scenes which are known to be good at expressing grammars but are not performance-oriented.

This change ditches the dependency in favour of an existing Java implementation. `jsemver` implements the full specification, which is probably an overkill in our case, but proved to be an almost drop-in replacement for the previous library.

Closes #8692

# Important Notes
Peformance improvements:
- roughly 50ms compared to the previous approach (from 80ms to 20-40ms)

I don't see any time spent in the new implementation during startup so it could be potentially aggressively inlined.
Further more, we could use a facade and offer our own strip down version of semver.
2024-02-22 09:59:09 +00:00
Pavel Marek
edc98593b9
All unit tests use the same testing logging provider (#8593)
Make sure that the correct test logging provider is loaded in `project-manager/Test`, so that only WARN and ERROR log messages are displayed. Also, make sure that the test log provider parses the correct configuration file - Rename all the `application.conf` files in the test resources to `application-test.conf`.

The problem was introduced in #8467
2023-12-21 13:45:33 +00:00
Hubert Plociniczak
a14ebd6259
Increase timeout for running launcher command (#8486)
We've been experiencing consistently failures on MacOS due to timeouts.
Doubling the timeout, hoping this will be sufficient to eliminate such
false failures. Will seek alternative solutions if that does not rememdy
the problem on CI.
2023-12-07 16:34:55 +01:00
Pavel Marek
5a7ad6bfe4
Upgrade enso to GraalVM for jdk 21 (#7991)
Upgrade to GraalVM JDK 21.
```
> java -version
openjdk version "21" 2023-09-19
OpenJDK Runtime Environment GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15)
OpenJDK 64-Bit Server VM GraalVM CE 21+35.1 (build 21+35-jvmci-23.1-b15, mixed mode, sharing)
```

With SDKMan, download with `sdk install java 21-graalce`.

# Important Notes
- After this PR, one can theoretically run enso with any JRE with version at least 21.
- Removed `sbt bootstrap` hack and all the other build time related hacks related to the handling of GraalVM distribution.
- `project-manager` remains backward compatible - it can open older engines with runtimes. New engines now do no longer require a separate runtime to be downloaded.
- sbt does not support compilation of `module-info.java` files in mixed projects - https://github.com/sbt/sbt/issues/3368
- Which means that we can have `module-info.java` files only for Java-only projects.
- Anyway, we need just a single `module-info.class` in the resulting `runtime.jar` fat jar.
- `runtime.jar` is assembled in `runtime-with-instruments` with a custom merge strategy (`sbt-assembly` plugin). Caching is disabled for custom merge strategies, which means that re-assembly of `runtime.jar` will be more frequent.
- Engine distribution contains multiple JAR archives (modules) in `component` directory, along with `runner/runner.jar` that is hidden inside a nested directory.
- The new entry point to the engine runner is [EngineRunnerBootLoader](https://github.com/enso-org/enso/pull/7991/files#diff-9ab172d0566c18456472aeb95c4345f47e2db3965e77e29c11694d3a9333a2aa) that contains a custom ClassLoader - to make sure that everything that does not have to be loaded from a module is loaded from `runner.jar`, which is not a module.
- The new command line for launching the engine runner is in [distribution/bin/enso](https://github.com/enso-org/enso/pull/7991/files#diff-0b66983403b2c329febc7381cd23d45871d4d555ce98dd040d4d1e879c8f3725)
- [Newest version of Frgaal](https://repo1.maven.org/maven2/org/frgaal/compiler/20.0.1/) (20.0.1) does not recognize `--source 21` option, only `--source 20`.
2023-11-17 18:02:36 +00:00
Hubert Plociniczak
cfba3c6887
Add support for https and wss (#7937)
* Add support for https and wss

Preliminary support for https and wss. During language server startup we
will read the application config and search for the `https` config with
necessary env vars set.

The configuration supports two modes of creating ssl-context - via
PKCS12 format and certificat+private key.

Fixes #7839.

* Added tests, improved documentation

Generic improvements along with actual tests.

* lint

* more docs + wss support

* changelog

* Apply suggestions from code review

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>

* PR comment

* typo

* lint

* make windows line endings happy

---------

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
2023-10-12 00:03:34 +02:00
Hubert Plociniczak
63802544f7
Enable log-to-file configuration (#7918)
* Enable log-to-file configuration

PR #7825 enabled parallel logging to a file with a much more
fine-grained log level by default.
However, logging at `TRACE` level on Windows appears to be still
problematic.

This PR reduced the default log level to file from `DEBUG` to `TRACE`
and allows to control it via an environment variable if one wishes to
change the verbosity without making code changes.

* PR comments
2023-10-02 14:25:09 +02:00
Hubert Plociniczak
18b2491a41
Always log to console and file (#7825)
* Always log verbose to a file

The change adds an option by default to always log to a file with
verbose log level.
The implementation is a bit tricky because in the most common use-case
we have to always log in verbose mode to a socket and only later apply
the desired log levels. Previously socket appender would respect the
desired log level already before forwarding the log.

If by default we log to a file, verbose mode is simply ignored and does
not override user settings.

To test run `project-manager` with `ENSO_LOGSERVER_APPENDER=console` env
variable. That will output to the console with the default `INFO` level
and `TRACE` log level for the file.

* add docs

* changelog

* Address some PR requests

1. Log INFO level to CONSOLE by default
2. Change runner's default log level from ERROR to WARN

Took a while to figure out why the correct log level wasn't being passed
to the language server, therefore ignoring the (desired) verbose logs
from the log file.

* linter

* 3rd party uses log4j for logging

Getting rid of the warning by adding a log4j over slf4j bridge:
```
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
```

* legal review update

* Make sure tests use test resources

Having `application.conf` in `src/main/resources` and `test/resources`
does not guarantee that in Tests we will pick up the latter. Instead, by
default it seems to do some kind of merge of different configurations,
which is far from desired.

* Ensure native launcher test log to console only

Logging to console and (temporary) files is problematic for Windows.
The CI also revealed a problem with the native configuration because it
was not possible to modify the launcher via env variables as everything
was initialized during build time.

* Adapt to method changes

* Potentially deal with Windows failures
2023-09-26 11:32:04 +02:00
Hubert Plociniczak
ddbccd8764
Fix console output for native image (#7744)
* Add serialization config to fix native image

Discovered when switching to console output.

* Add more documentation
2023-09-06 12:12:52 +02:00
Hubert Plociniczak
8a60bc6dcd
Replace a custom logger with off the shelf implementation (#7559)
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.

Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.

Most important differences have been outlined in `docs/infrastructure/logging.md`.

# Important Notes
Addresses:
- #7253
- #6739
2023-09-04 09:40:16 +00:00
Dmitry Bushev
80c4b1ca06
Allow users to give a project other than Upper_Snake_Case name (#7397)
close #6356

Allow arbitrary names for user projects.

# Important Notes
https://github.com/enso-org/enso/assets/357683/55a3b660-af23-4b09-959b-eac515766788
2023-07-28 13:44:39 +00:00
Jaroslav Tulach
6a6d7dbff3
Simplify onboarding instructions for engine & project-manager developers (#7181)
The current instructions to _build, use and debug_ `project-manager` and its engine/ls process are complicated and require a lot of symlinks to properly point to each other. This pull requests simplifies all of that by introduction of `ENSO_ENGINE_PATH` and `ENSO_JVM_PATH` environment variables. Then it hides all the complexity behind a simple _sbt command_: `runProjectManagerDistribution --debug`.

# Important Notes
I decided to tackle this problem as I have three repositories with different branches of Enso and switching between them requires me to mangle the symlinks. I hope I will not need to do that anymore with the introduction of the `runProjectManagerDistribution` command.
2023-07-17 17:38:54 +00:00
Hubert Plociniczak
4e5cb9cca6
Consistent naming of visualization (#7124) 2023-06-26 17:04:35 +00:00
Hubert Plociniczak
a6487e5105
Don't log installed engines and runtimes in prod (#5900)
The `logAvailableComponentsForDebugging` will check and install all necessary components of GraalVM for every mentioned version. While not harmful, it adds up to startup time.
Additionally added an option in language server startup to skip installation of GraalVM components. The latter is already performed by project-manager when opening the project and it is unnecessary to do it twice. Due to LS' architecture this configuration has to be passed around via multiple configs.

Finally, skipped the attempt to install Python component on Windows - this is not supported by GraalVM atm.

Closes #5749.

# Important Notes
The impact of this problem could be really felt the more versions of Enso and GraalVM one had since it would go through all of them.
2023-03-16 10:36:55 +00:00
Dmitry Bushev
5e114acbb5
Update Scala to 2.13.8 (#3631)
Update Scala compiler and libraries.
2022-08-08 19:32:55 +00:00
Dmitry Bushev
f9d2964e83
Update profiling CLI arguments (#3461) 2022-05-24 16:01:26 +03:00
Jaroslav Tulach
ab1ca54acd
Profile the language server (#3389)
In order to analyse why the `runner.jar` is slow to start, let's _"self sample"_ it using the [sampler library](https://bits.netbeans.org/dev/javadoc/org-netbeans-modules-sampler/org/netbeans/modules/sampler/Sampler.html). As soon as the `Main.main` is launched, the sampling starts and once the server is up, it writes its data into `/tmp/language-server.npss`.

Open the `/tmp/language-server.npss` with [VisualVM](https://visualvm.github.io) - you should have one copy in your
GraalVM `bin/jvisualvm` directory and there has to be a GraalVM to run Enso.

#### Changelog

- add: the `MethodsSampler` that gathers information in `.npss` format
- add: `--profiling` flag that enables the sampler
- add: language server processes the updates in batches
2022-05-10 12:44:05 +00:00
Hubert Plociniczak
4bbabc00be
Move Builtin Types and Methods to stdlib (#3363)
This PR replaces hard-coded `@Builtin_Method` and `@Builtin_Type` nodes in Builtins with an automated solution
that a) collects metadata from such annotations b) generates `BuiltinTypes` c) registers builtin methods with corresponding
constructors.
The main differences are:
1) The owner of the builtin method does not necessarily have to be a builtin type
2) You can now mix regular methods and builtin ones in stdlib 
3) No need to keep track of builtin methods and types in various places and register them by hand (a source of many typos or omissions as it found during the process of this PR)

Related to #181497846
Benchmarks also execute within the margin of error.

### Important Notes

The PR got a bit large over time as I was moving various builtin types and finding various corner cases.
Most of the changes however are rather simple c&p from Builtins.enso to the corresponding stdlib module.
Here is the list of the most crucial updates:
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/builtin/Builtins.java` - the core of the changes. We no longer register individual builtin constructors and their methods by hand. Instead, the information about those is read from 2 metadata files generated by annotation processors. When the builtin method is encountered in stdlib, we do not ignore the method. Instead we lookup it up in the list of registered functions (see `getBuiltinFunction` and `IrToTruffle`)
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java` has now information whether it corresponds to the builtin type or not.
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/RuntimeStubsGenerator.scala` - when runtime stubs generator encounters a builtin type, based on the @Builtin_Type annotation, it looks up an existing constructor for it and registers it in the provided scope, rather than creating a new one. The scope of the constructor is also changed to the one coming from stdlib, while ensuring that synthetic methods (for fields) also get assigned correctly
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/IrToTruffle.scala` - when a builtin method is encountered in stdlib we don't generate a new function node for it, instead we look it up in the list of registered builtin methods. Note that Integer and Number present a bit of a challenge because they list a whole bunch of methods that don't have a corresponding method (instead delegating to small/big integer implementations).
During the translation new atom constructors get initialized but we don't want to do it for builtins which have gone through the process earlier, hence the exception
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor not only  generates the actual code fpr nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization 
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor no longer generates only (root) nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/TypeProcessor.java` - Similar to MethodProcessor but handles @Builtin_Type annotations. It doesn't, **yet**, generate any builtin objects.  It also collects the names, as present in stdlib, if any, so that we can generate the names automatically (see generated `types/ConstantsGen.java`)
- `engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin` - various classes annotated with @BuiltinType to ensure that the atom constructor is always properly registered for the builitn. Note that in order to support types fields in those, annotation takes optional `params` parameter (comma separated). 
- `engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala` - drop manual creation of test list which seemed to be a relict of the old design
2022-05-05 20:18:06 +02:00
Radosław Waśko
247b284316
Data analysts should be able to use Text.location_of to find indexes within string using various matchers (#3324)
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/181266029
2022-03-12 19:42:00 +00:00
Michał Wawrzyniec Urbańczyk
4baad5f146
Nightly proccess preparations: Setting Enso version through the environment (#3241)
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Radosław Waśko <wasko.radek@gmail.com>
2022-02-07 15:14:32 +01:00
Dmitry Bushev
0b9a5b5f65
Fix path in Windows tests (#3191) 2021-12-14 14:52:38 +03:00
Dmitry Bushev
51d1c941f5
Flaky upgrade spec (#3179) 2021-12-07 15:55:51 +03:00
Radosław Waśko
46c31bb9a5
Preinstalling With Dependencies (#1981) 2021-11-23 11:51:17 +03:00
Adam Obuchowicz
bb3b145af5
Refactor gui/src/rust/ide to two app/gui and app/ide-desktop (#3157) 2021-11-16 10:04:56 +01:00
Ara Adkins
7cced6a9de
Make the BindingsMap able to break links (#1980) 2021-08-31 15:50:33 +01:00
Radosław Waśko
826e28a149
Implement library/preinstall (Without Dependencies Yet) (#1972) 2021-08-27 14:01:13 +02:00
Radosław Waśko
0a60e5180a
Update Simple Library Server (#1952) 2021-08-18 10:01:28 +02:00
Radosław Waśko
fac0405fd0
Updating Editions (#1944) 2021-08-12 16:55:23 +02:00
Radosław Waśko
31167fd6ae
Revert "Revert "Library Publishing MVP (#1898)"" (#1933) 2021-08-09 16:00:04 +02:00
Dmitry Bushev
8336a97931
Add Support for Project Templates (#1902)
Add an ability to create a project from a template
2021-07-27 19:13:14 +03:00
Ara Adkins
a7478bc573
Revert "Library Publishing MVP (#1898)"
This reverts commit 1bd4e5824e.
2021-07-26 10:32:19 +01:00
Radosław Waśko
1bd4e5824e
Library Publishing MVP (#1898) 2021-07-23 18:16:34 +02:00
Radosław Waśko
f3bc88b91e
Library Downloads MVP (#1885) 2021-07-22 08:24:06 +02:00
Radosław Waśko
86fcd86055
New Language Server API Implementations / Mocks (#1875) 2021-07-17 16:49:51 +02:00
Radosław Waśko
b58e895b18
Editions Integration (#1832) 2021-07-08 15:38:20 +02:00
Radosław Waśko
e58b5eb81d
Dynamically Loading Libraries (#1826) 2021-07-06 00:27:14 +02:00
Radosław Waśko
44ef29a3b7
Fix the PM Not Checking To Preinstall the Graal Runtime if the Engine Was Already Installed (#1824) 2021-06-29 11:46:20 +02:00
Marcin Kostrzewa
334a022ffd
Import syntax including namespace (#1806) 2021-06-24 12:42:24 +02:00
Radosław Waśko
241a1e7d74
Define Editions (#1797) 2021-06-18 16:39:45 +02:00
Dmitry Bushev
6c084fc8ae
Add support for detailed logger configuration (#1745) 2021-05-19 09:02:32 +01:00
Dmitry Bushev
c202c92b68
Log akka messages (#1719) 2021-05-18 09:23:13 +01:00
Dmitry Bushev
1b6388702f
Implement masking of sensitive data in logs (#1732) 2021-05-12 16:31:53 +01:00
Radosław Waśko
c3e4901eb1
Add a workflow for building nightly releases (#1689) 2021-05-05 11:58:14 +01:00
Dmitry Bushev
cf8034683c
Add detailed logging to the tooling init (#1704) 2021-04-28 15:11:42 +01:00
Ara Adkins
9f60e22508
Update all references to dev.enso.org 2021-04-07 20:39:46 +01:00
Dmitry Bushev
ec66117cc6
Install Required GraalVM Components (#1651)
Project manager ensures that the required 
GraalVM components are installed.
2021-04-07 18:19:24 +03:00
Ara Adkins
2b4f0ef2ad
Bump scalac to 2.13.5 (#1531) 2021-03-01 16:35:57 +00:00
Radosław Waśko
6075c1e765
Implement LocalReleaseProvider (#1370) 2020-12-21 23:30:59 +01:00
Radosław Waśko
2e6a5af4fb
Split Installer Versions in Manifest (#1355) 2020-12-16 12:34:33 +01:00
Radosław Waśko
de817af655
Finish Logging Service Integration (#1346) 2020-12-15 09:49:58 +01:00