Commit Graph

793 Commits

Author SHA1 Message Date
Jaroslav Tulach
a85a57681d
Avoid null values in constant node (#8061)
Prevents [this NullPointerException](https://github.com/enso-org/enso/pull/8044#issuecomment-1763736570) by making sure `ConstantNode` rejects `null` during construction.

# Important Notes
```ruby
from Standard.Base import all
polyglot java import java.util.Random

main =
operator1 = Random.new
```
2023-10-16 15:50:41 +00:00
Hubert Plociniczak
352ad06d2f
Reduce extra output in compilation and tests (#7809)
* Reduce extra output in compilation and tests

I couldn't stand the amount of extra output that we got when compiling
a clean project and when executing regular tests. We should strive to
keep output clean and not print anything additional to stdout/stderr.

* Getting rid of explicit setup by service loading

In order for SL4J to use service loading correctly had to upgrade to
latest slf4j. Unfortunately `TestLogProvider` which essentially
delegates to `logback` provider will lead to spurious ambiguous warnings
on multiple providers. In order to dictate which one to use and
therefore eliminate the warnings we can use the `slf4j.provider` env
var, which is only available in slf4j 2.x.

Now, there is no need to explicitly call `LoggerSetup.get().setup()` as
that is being called during service setup.

* legal review

* linter

* Ensure ConsoleHandler uses the default level

ConsoleHandler's constructor uses `Level.INFO` which is unnecessary for
tests.

* report warnings
2023-10-16 10:57:44 +02:00
Jaroslav Tulach
7ae0971d11
Runtime checks of signatures with polyglot java classes (#8016) 2023-10-11 15:05:36 +02:00
Jaroslav Tulach
a234e82ee9
Instrumenter to observe behavior of nodes with UUID (#7833)
Exposing instrumentation capabilities to Enso. Fixes #7683.
2023-10-10 02:36:59 +00:00
Pavel Marek
4db61210c0
Exporting non-existing symbols fails with compiler error (#7960)
Adds a new compiler analysis pass that ensures that all the symbols exported via `export ...` or `from ... export ...` statements exist. If not, generates an IR Error.

# Important Notes
We already have such a compiler pass for the version with imports in [ImportSymbolAnalysis.scala](https://github.com/enso-org/enso/blob/develop/engine/runtime/src/main/scala/org/enso/compiler/pass/analyse/ImportSymbolAnalysis.scala)
2023-10-09 09:48:43 +00:00
Hubert Plociniczak
16c8d2e302
Remove context locks from visualization commands (#7953)
It looks like visualization commands had required context lock unnecessairly. Context manager methods are synchronized and therefore should not need the lock before submitting a correspodning background job.

Additionally, the presence of a context lock leads a deadlock:
1. Consider a long running execution of a program that does not finish within the 5 seconds
2. In the meantime there comes either an `AttachVisualization` or `DetachVisualization` request from the client

The latter will get stuck and eventually timeout out because it cannot acquire the lock withing the required time limits. It is still possible that a single long-running `ExecuteJob` might block other ones (including visualization ones) but that's a separate work.

Fixes some issues present in #7941.

# Important Notes
We need to still investigate `ExecuteJob`s need for context lock which might delay the execution of other background jobs that require it as well (mostly concerned about visualization).
2023-10-06 16:48:38 +00:00
Dmitry Bushev
f1356374e1
Restore execution environment before the execution complete notification (#7980)
Fixes flaky RuntimeServerTest https://github.com/enso-org/enso/actions/runs/6410804439/job/17420283503?pr=7953#step:10:4004
2023-10-05 14:25:17 +00:00
Jaroslav Tulach
997e0fc465
Using ENSO_JAVA env variable to turn on the Espresso support (#7951)
* Q: Is it normal for `--inspect` mode to print two debug urls?
* A: No, it should print just one.
* Q: Putting there a Java breakpoint to find out why it the chromeinspector gets initialized twice might reveal the culprit.
* A: The additional listener is happening [here](https://github.com/enso-org/enso/blob/develop/engine/runner/src/main/scala/org/enso/runner/ContextFactory.scala#L117).

# Important Notes
There is no easy check for a language being present without creating an `Engine`. It was thought creating an `Engine` is cheap operation, but it seems to have some downsides. Let's use `ENSO_JAVA` environment variable to decide whether _experimental Espresso_ support shall be enabled.
2023-10-04 11:55:31 +00:00
Pavel Marek
c22928ecc2
Implement private modules (#7840)
Adds the ability to declare a module as *private*. Modifies the parser to add the `private` keyword as a reserved keyword. All the checks for private modules are implemented as an independent *Compiler pass*. No checks are done at runtime.

# Important Notes
- Introduces new keyword - `private` - a reserved keyword.
- Modules that have `private` keyword as the first statement are declared as *private* (Project private)
- Public module cannot have private submodules and vice versa.
- This would require runtime access checks
- See #7088 for the specification.
2023-10-04 10:33:10 +00:00
Jaroslav Tulach
421e3f22a4
Avoid unnecessary conversions (#7952)
Fixes #7854 by checking all elements of a union type for direct match and only then, if direct match isn't available, applying conversions.
2023-10-04 12:31:06 +02:00
Jaroslav Tulach
6a59fa5e93
Meta.Type.find a type by FQN (#7885) 2023-10-02 17:02:00 +02:00
Jaroslav Tulach
515e66052f
Avoid classloading deadlock by delaying computations of other passes (#7943)
Fixes #7850 by computing other passes lazily.
2023-10-02 12:43:56 +00:00
Pavel Marek
9f15b90caa
Implement Enso-specific assert (#7883)
Implement Enso-specific assert - `Runtime.assert` that works like asserts in any other runtime.

# Important Notes
- Enso-specific assertions are enabled when JVM assertions are enabled, or when `ENSO_ENABLE_ASSERTIONS` env var is not empty (See 72cd8361cb/engine/runtime/src/main/java/org/enso/interpreter/runtime/EnsoContext.java (L139))
2023-09-29 14:46:58 +00:00
Jaroslav Tulach
3094dc74c5
Log nothing when class is found (#7897) 2023-09-26 17:55:58 +02:00
Hubert Plociniczak
18b2491a41
Always log to console and file (#7825)
* Always log verbose to a file

The change adds an option by default to always log to a file with
verbose log level.
The implementation is a bit tricky because in the most common use-case
we have to always log in verbose mode to a socket and only later apply
the desired log levels. Previously socket appender would respect the
desired log level already before forwarding the log.

If by default we log to a file, verbose mode is simply ignored and does
not override user settings.

To test run `project-manager` with `ENSO_LOGSERVER_APPENDER=console` env
variable. That will output to the console with the default `INFO` level
and `TRACE` log level for the file.

* add docs

* changelog

* Address some PR requests

1. Log INFO level to CONSOLE by default
2. Change runner's default log level from ERROR to WARN

Took a while to figure out why the correct log level wasn't being passed
to the language server, therefore ignoring the (desired) verbose logs
from the log file.

* linter

* 3rd party uses log4j for logging

Getting rid of the warning by adding a log4j over slf4j bridge:
```
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
```

* legal review update

* Make sure tests use test resources

Having `application.conf` in `src/main/resources` and `test/resources`
does not guarantee that in Tests we will pick up the latter. Instead, by
default it seems to do some kind of merge of different configurations,
which is far from desired.

* Ensure native launcher test log to console only

Logging to console and (temporary) files is problematic for Windows.
The CI also revealed a problem with the native configuration because it
was not possible to modify the launcher via env variables as everything
was initialized during build time.

* Adapt to method changes

* Potentially deal with Windows failures
2023-09-26 11:32:04 +02:00
Radosław Waśko
12c4f2981d
More robust Date/Time format patterns parsing (#7826)
- Closes #7461 by introducing a `Date_Time_Formatter` type and making parsing date time formats more robust and safer.
- The default ('simple') set of patterns is slightly simplified and made case insensitive (except for `M/m` and `H/h`) to avoid the `YYYY` vs `yyyy` issues and make it less error prone.
- The `YYYY` now has the same meaning as `yyyy` in simple mode. The old meaning (week-based year) is moved to a _separate mode_, triggered by `Date_Time_Formatter.from_iso_week_date_pattern`.
- Full Java syntax, as well as custom-built Java `DateTimeFormatter` can also be used by `Date_Time_Formatter.from_java`.
- Text-based constants (e.g. `ISO_ZONED_DATE_TIME`) have now become methods on `Date_Time_Formatter`, e.g. `Date_Time_Formatter.iso_zoned_date_time`).
2023-09-22 10:12:18 +00:00
Dmitry Bushev
c88765d259
Send conversion method suggestions (#7857)
close #7320

Changelog:
- update: enable conversion suggestions
- fix: conversion suggestion building
- fix: conversion suggestion types
- fix: conversion JSON-RPC representation

# Important Notes
For example, the [`Day_Of_Week_From`](5150c14afd/distribution/lib/Standard/Base/0.0.0-dev/src/Data/Time/Day_Of_Week_From.enso) conversion is sent as


```json
{
"type":"Add",
"id":32,
"suggestion":{
"type":"method",
"module":"Standard.Base.Data.Time.Day_Of_Week_From",
"name":"from",
"arguments":[
{
"name":"that",
"reprType":"Standard.Base.Data.Numbers.Integer",
"isSuspended":false,
"hasDefault":false,
"defaultValue":null,
"tagValues":null
},
{
"name":"first_day",
"reprType":"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week",
"isSuspended":false,
"hasDefault":true,
"defaultValue":"Day_Of_Week.Sunday",
"tagValues":[
"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week.Sunday",
"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week.Monday",
"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week.Tuesday",
"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week.Wednesday",
"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week.Thursday",
"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week.Friday",
"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week.Saturday"
]
},
{
"name":"start_at_zero",
"reprType":"Standard.Base.Data.Boolean.Boolean",
"isSuspended":false,
"hasDefault":true,
"defaultValue":"False",
"tagValues":[
"Standard.Base.Data.Boolean.Boolean.True",
"Standard.Base.Data.Boolean.Boolean.False"
]
}
],
"selfType":"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week",
"returnType":"Standard.Base.Data.Time.Day_Of_Week.Day_Of_Week",
"isStatic":false,
"documentation":" Convert from an integer to a Day_Of_Week\n\nArguments:\n- `that`: The first day of the week.\n- `first_day`: The first day of the week.\n- `start_at_zero`: If True, first day of the week is 0 otherwise is 1.",
"annotations":[

]
}
}
```
2023-09-21 09:00:31 +00:00
Jaroslav Tulach
5150c14afd
Avoid duplicated conversion & search target type scope (#7849) 2023-09-20 17:23:09 +02:00
James Dunkerley
74d1d0861c
S3 Read Access, Input Stream based reading (#7776)
- Added a `FileSystemSPI` allowing protocol resolution to a target type.
- Separated `Input_Stream` and `Output_Stream` from `File` to allow use in other spaces.
- `File_Format` types `read_web` changed to be `read_stream` working with `InputStream`.
- Added directory listing to `Auto_Detect` allowing for `Data.read` to list a folder.
- Adjusted HTTP to return an `InputStream` not a `byte[]`:
- `Response_Body` adjusted to wrap an `InputStream`.
- Added ability to materialize to either and in-memory vector (<4KB) or a temporary file.
- `Data.fetch` will materialize if not a recognized mime-type.
- Added `HTTP_Error` to handle IO exceptions from the stream.
- `Excel_Format` now supports mime-type and reading a stream.
- `Excel_Workbook` can now get a `Excel_Section` using `read_section`.
- Added S3 APIs:
- `parse_uri`: splits an S3 URI into bucket and key.
- `list_objects`: list the items in a S3 bucket with specified prefix.
- `read_bucket`: list prefixes and keys with a delimiter in a S3 bucket with specified prefix.
- `head`: either head_bucket (tests existance) or head_object API (reads object meta data).
- `get_object`: gets an object from S3 returning as a `Response_Body`.
- Added `S3_File` type acting like a `File`:
- No support for writing in this PR.
- **ToDo:** recursive listing, glob filtering, exists, size.
- Fixed a few invalid type signature line.
- Moved `create` methods for `Postgres_Connection` and `SQLite_Connection` into type instead of module.
- Renamed `Column_Fetcher.Builder` to `Column_Fetcher_Builder`.
- Fixed bug with `select_into` in Dry Run mode creating permanent tables.

**ToDo:** Unit tests.
2023-09-20 15:09:11 +00:00
Jaroslav Tulach
2e1791b63e
Accept type argument in instance method signatures (#7848) 2023-09-20 12:58:20 +02:00
Jaroslav Tulach
0efa1afc61
Encapsulate TruffleLanguage.Env in EnsoContext (#7842) 2023-09-20 11:30:44 +02:00
Jaroslav Tulach
3b790606e1
Avoid StackOverflow when comparing unknown foreign objects (#7780)
Closes #7677 by eliminating the _stackoverflow execption_. In general it seems _too adventurous_ to walk members of random foreign objects. There can be anything including cycles. Rather than trying to be too smart in these cases, let's just rely on `InteropLibrary.isIdentical` message.

# Important Notes
Calling `sort` on the `numpy` array no longer yields an error, but the array isn't sorted - that needs a fix on the Python side: https://github.com/oracle/graalpython/issues/354 - once it is in, the elements will be treated as numbers and the sorting happens automatically (without any changes in Enso code).
2023-09-19 15:39:04 +00:00
Jaroslav Tulach
6cbd111bad
Optional Espresso support with ENSO_JAVA=espresso env variable 2023-09-19 15:10:12 +02:00
Pavel Marek
a6335ac4b4
Do not suppress panic exception in runtime type check (#7793) 2023-09-18 11:48:59 +02:00
Jaroslav Tulach
af8a2c39d8
Throw UnsupportedMessageException when loosing BigInteger precision (#7817) 2023-09-15 17:00:09 +02:00
Dmitry Bushev
bbf96f0d16
expressionUpdates do not contain Method Pointer of operators (#7659)
close #7520

Changelog:
- update: SectionsToBinOp compiler pass produces function application for left sections
- refactor: simplify the registration of builtin methods
2023-09-15 08:01:21 +00:00
Hubert Plociniczak
1ee3d8f4f0
Rename Decimal to Float (#7807)
Implements #6889.
2023-09-14 15:01:30 +00:00
Jaroslav Tulach
30a62b97bb
Runtime checking of ascribed expression types (#7796) 2023-09-14 14:09:41 +02:00
Hubert Plociniczak
ea5096a288
Workaround crashes during regular execution (#7778)
This PR addresses two problems mentioned in #7766:
1. A random integer overflow, likely caused by a bug in Rust parser
2. A concurrent access to a methods' map

Re 1: Unable to reproduce but it doesn't mean it won't happen again. Added a try/catch to get in the logs source code that caused it **and** not crash hard when it occurs.

Re 2: Changing methods map from `HashMap` to `ConcurrentHashMap`. Due to a poor design we leaked the underlying structure in a number of places, unnecessairly. `ConcurrentHashMap` does not accept `null` keys therefore due to leaking implementation had to ensure that `methods` of `ModuleScope` never escapes as-is.

Both workarounds should ensure that we don't crash hard when they appear.

Closes #7766
2023-09-11 13:45:31 +00:00
Jaroslav Tulach
50124704c8
Support runtime checks of intersection types (#7769) 2023-09-11 14:05:20 +02:00
Jaroslav Tulach
a42850f980
Benchmark shall loop correctly (#7781) 2023-09-11 14:04:10 +02:00
Pavel Marek
1be9ba615a
Fix implicit conversion of foreign JS object to Text (#7774)
Fixes implicit conversion of foreign JS object to Text.
2023-09-11 10:21:33 +02:00
Pavel Marek
e0ee8fdda7
Merge Small_Integer and Big_Integer types (#7636)
Merge Small_Integer and Big_Integer types into a single Integer type
2023-09-08 16:04:39 +02:00
James Dunkerley
f0ae9bf9c5
Fixes issue writing to a dry run Excel File (#7763)
- Adds `size` to `File`.
- If file is empty, then create a new Excel file.
- Fixes dry run Excel write issue.
2023-09-08 08:52:00 +00:00
Pavel Marek
687686773e
Fix on argument for Vector.sort (#7751)
Fix treatment of the `on` argument of `Vector.sort`.
2023-09-07 17:32:08 +00:00
Jaroslav Tulach
4213bf9983
Display overriden non-default values (#7762) 2023-09-07 13:31:51 +02:00
Radosław Waśko
7d424bf8a2
Implement Table.delete_rows. (#7709)
- Closes #7238
- Aligns `update_database_table` to a more consistent and clearer API - `update_rows`.
- Adds a `truncate_table` helper function, to pair up with `drop_table`. Both are `PRIVATE` for now.
- Adds tests for NULLs in keys in `update_rows` and `delete_rows`.
- The behaviour is sometimes unexpected, so instead these fail with `Null_Values_In_Key_Columns`.
- Adds a workaround for https://github.com/oracle/graal/issues/7359
- Adds a workaround for a related bug where a stack frame has no name (its `rootNode.getName() == null`).
- I could not track down this bug to provide a neat repro.
2023-09-07 11:07:53 +00:00
Dmitry Bushev
a1f2560491
Fix re-exports of type constructors (#7724)
close #7674

Changelog:
- fix: setting reexport field when serializing the library
2023-09-06 11:02:21 +00:00
Jaroslav Tulach
ab1c1a4c12
Delay check of suspended arguments until they are about to be computed (#7727) 2023-09-06 09:34:12 +02:00
Hubert Plociniczak
8a60bc6dcd
Replace a custom logger with off the shelf implementation (#7559)
This change replaces Enso's custom logger with an existing, mostly off the shelf logging implementation. The change attempts to provide a 1:1 replacement for the existing solution while requiring only a minimal logic for the initialization.

Loggers are configured completely via `logging-server` section in `application.conf` HOCON file, all initial logback configuration has been removed. This opens up a lot of interesting opportunities because we can benefit from all the well maintained slf4j implementations without being to them in terms of functionality.

Most important differences have been outlined in `docs/infrastructure/logging.md`.

# Important Notes
Addresses:
- #7253
- #6739
2023-09-04 09:40:16 +00:00
Radosław Waśko
87ce78615a
Change layout of local library search path in order to be able to move Round_Spec.enso back to Tests (#7634)
- Closes #7633
- Moves `Round_Spec.enso` from published `Standard.Test` into our `test/Tests` project; the `Table_Tests` that depend on it, simply `import enso_dev.Tests`.
- Changes the layout of the local libraries directory:
- It used to be `root/<namespace>/<name>`.
- Now it is `root/<dir>` - the namespace and name are now read from `package.yaml` instead.
- Adds the parent directory of the current project to the default `ENSO_LIBRARY_PATH`.
- It is treated as a secondary path, so the default `ENSO_HOME/lib` still takes precedence.
- This allows projects to reference and load 'sibling' projects easily - the only requirement is for the project to enable `prefer-local-libraries: true` or add the other local project to its edition. The edition resolution logic is **not changed**.
2023-09-01 20:20:04 +00:00
Jaroslav Tulach
6deaf80a87
Standardization of Vector.sort array access (#7697)
Inspired by #6361 - let's use `ArrayLikeXyzNode` instead of `InteropLibrary`.
2023-09-01 08:07:41 +00:00
Jaroslav Tulach
1437a671e1
Introducing generic Any.to type conversion method (#7704) 2023-09-01 08:05:48 +02:00
Hubert Plociniczak
24f263b20b
Towards shallow nesting in IR (#7681)
Refactoring deeply nested IR classes to shallow nesting.
Fixes #7017

# Important Notes
It's a big PR but fairly consistent. Split multiple hierarchy levels into subpackages.
2023-08-31 15:42:53 +00:00
Dmitry Bushev
eefe74ed93
When opening the project for a second time suggestion database is not sent (#7699)
close #7413

Changelog:
- update: the language server listens for the client disconnection event and invalidates the suggestions index

# Important Notes
The component browser contains suggestion entries after the refresh


https://github.com/enso-org/enso/assets/357683/bcebb8bf-e09f-4fb0-86cf-95ced58413f3
2023-08-31 14:06:58 +00:00
Jaroslav Tulach
b8edf6a358
Removing references to Builtins from Compiler (#7666) 2023-08-27 17:24:51 +02:00
Jaroslav Tulach
dd7ab1da38
Avoid _ after . (#7657) 2023-08-26 08:02:20 +02:00
Jaroslav Tulach
6461e20870
Special support for Python Date/Time/Zone interop (#7617) 2023-08-25 10:27:16 +02:00
Jaroslav Tulach
59a83030ed
Turn abnormal CompilerError into regular Panic (#7651) 2023-08-25 08:29:18 +02:00
Jaroslav Tulach
20e18d22df
More descriptive function information (#7629)
Fixes #7359 by printing more information about the function including partially applied arguments and over-saturated arguments.
2023-08-24 18:04:08 +00:00