Commit Graph

3543 Commits

Author SHA1 Message Date
James Dunkerley
0dcfc3e9bf
Minor improvements from last couple of Book Clubs (#8034)
- Added some ALIASes.
- Added `sheet` to `Excel_Workbook` to give familiar API to read sheet.
- Added conversion from range to vector allowing easy use with Zip.
- Add `Map.from_keys_and_values.
2023-10-12 14:29:59 +00:00
Adam Obuchowicz
0783618276
Interpreting CB input using AST. (#8022)
Fixes #7784

Instead of some heuristic, we now parse the CB's input and get the pattern and qualified name. Also we keep the "context" structure which may be used to apply suggestions properly.

# Important Notes
I created several helpers for AST. One of them is very hacky: we traverse AST tree by reading all getters in JS. Instead, we could generate it basing on schema.
2023-10-12 14:00:48 +00:00
Pavel Marek
58e846e9d5
Remove dependency on jdk.compiler from builtin method processor (#8027) 2023-10-12 14:31:11 +02:00
Dmitry Bushev
1a7e83b80c
Fix warnings in SBT build (#8026)
Fixes warnings on SBT startup

```
$ sbt
[info] welcome to sbt 1.9.0 (GraalVM Community Java 17.0.7)
...
[warn] 5 feature warnings; re-run with -feature for details
[warn] one warning found
[info] loading settings for project enso from build.sbt ...
```
2023-10-12 08:19:45 +00:00
Radosław Waśko
cd84ac16ce
Restructure Table.from_objects to use conversions (#8020)
Closes #7957
2023-10-11 22:25:18 +00:00
Hubert Plociniczak
cfba3c6887
Add support for https and wss (#7937)
* Add support for https and wss

Preliminary support for https and wss. During language server startup we
will read the application config and search for the `https` config with
necessary env vars set.

The configuration supports two modes of creating ssl-context - via
PKCS12 format and certificat+private key.

Fixes #7839.

* Added tests, improved documentation

Generic improvements along with actual tests.

* lint

* more docs + wss support

* changelog

* Apply suggestions from code review

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>

* PR comment

* typo

* lint

* make windows line endings happy

---------

Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
2023-10-12 00:03:34 +02:00
Paweł Grabarz
bf76be6e6b
[GUI2] use DataView directly in generated parser code (#8028)
- Removed `Cursor` and moved all accessors to standalone functions, reducing the generated code size significantly (especially after minification).
- Removed unnecessary `seek`s from everywhere. Now the offseting is part of each `read` implementation.
- Removed unnecessary differences between `read`s on abstract and concrete types. Now abstract types do the pointer access internally in their `read`.
2023-10-11 21:28:29 +00:00
somebody1234
34f34cf0ad
Remove incorrect dependencies in the root package.json (#8023)
Removes dependencies that should not be in the root `package.json`
Basic testing done with `npm run ci-check`.

Introduced by the following PRs (in order of being merged):
See blame for more details: https://github.com/enso-org/enso/blame/develop/package.json
- #7813
- #7656 (oops)
- #7881

# Important Notes
None
2023-10-11 16:24:22 +00:00
somebody1234
62019f6334
Filter visualization types based on node type (#8004)
- Closes #7925

# Important Notes
The type -> matching visualization lookup is populated using a `Promise`, so there is a race condition where the selector will incorrectly show the full list of visualizations if it is opened before the `Promise` resolves.
2023-10-11 15:27:03 +00:00
somebody1234
b64a7d392f
Node icons and colors (#7988)
Adds node icons and colors.

# Important Notes
None
2023-10-11 13:25:38 +00:00
Jaroslav Tulach
7ae0971d11
Runtime checks of signatures with polyglot java classes (#8016) 2023-10-11 15:05:36 +02:00
Kaz Wesley
9fd1ab9092
Parser TS bindings (#7881)
Generate TS bindings and lazy deserialization for the parser types.

# Important Notes
- The new API is imported into `ffi.ts`, but not yet used.
- I have tested the generated code in isolation, but cannot commit tests as we are not currently able to load WASM modules when running in `vitest`.
2023-10-11 13:04:38 +00:00
GregoryTravis
a2c8940218
Add tests for Table.from_objects and Table.expand_column. (#8010) 2023-10-11 11:50:51 +00:00
Jaroslav Tulach
ef76df4c38
Define and enforce locking order during runtime (#8014) 2023-10-11 13:49:58 +02:00
somebody1234
3c31155fe9
Dashboard tests (#7656)
- Implements https://github.com/enso-org/cloud-v2/issues/631
- Tests for dashboard (`app/ide-desktop/lib/dashboard/`):
- End-to-end tests
- Unit tests
- Component tests

The purpose of this PR is to introduce the testing framework - more tests can be added later in separate PRs.

# Important Notes
To test, run `npm run test` in `app/ide-desktop`, or `app/ide-desktop/lib/dashboard/`. All tests should pass.

Individual test types can be run using `npm run test-unit`, `npm run test-component` and `npm run test-e2e` in `app/ide-desktop/lib/dashboard/`.
Individual end-to-end tests can be run using `npx playwright test -c playwright-e2e.config.ts test-e2e/<file name>.spec.ts` in `app/ide-desktop/lib/dashboard/`.

End-to-end tests require internet access to pass (for things like fonts).

This PR *does* check in screenshots to guard against visual regessions (and/or to make visual changes obvious)
2023-10-11 10:24:33 +00:00
somebody1234
94d3a05905
Check asset name uniqueness (#8018)
- Implements frontend component of https://github.com/enso-org/cloud-v2/issues/702
- Ensures that the new name is not already present in a sibling in the directory
- Only compares between directories when renaming a directory
- Only compares between files/projects/connectors when renaming one of those

# Important Notes
- This has not been implemented for connectors and files as currently there is no backend endpoint to rename those.
- Secrets are also not implemented yet, AFAIK, so there is no behavior related to secrets.
2023-10-11 10:17:33 +00:00
somebody1234
826127d8ff
Eliminate line feeds from XML.outer_xml on Windows (#8013)
- Closes #7999

# Important Notes
None
2023-10-10 23:21:34 +00:00
Radosław Waśko
6f78570115
Fix a DROP table bug, add SQL debug logging (#8007)
- Fixes a bug where creating a temporary table could accidentally issue a `DROP` statement of the table name that the user provided, risking destruction of user data.
- Fortunately, the bad scenario was almost impossible, because the `DROP` statement was only issued _if_ we previously checked that the mentioned table _does not exist_ - dropping a nonexistent table does not do any harm.
- It could have been dangerous in a very unlikely scenario that a table was created just between the _existence check_ and the _drop_.
- After the fix the existence check and any modifications are done within a transaction to avoid interference from concurrent modifications, and the DROP is correctly applied to a temporary Enso table instead of the original one.
- Replaced a temporary log with proper simple logging of SQL statements into a file, if an Environment variable is set.
- Used that feature to test that no unexpected statements occur.
2023-10-10 13:16:06 +00:00
Radosław Waśko
6e0bd86753
Implement Table.lookup_and_replace for in-memory (#7979)
- Closes #7749 implementing the in-memory logic.
- Additional complications have surfaced regarding the Database logic, so it has been split off into a separate ticket: #7981
2023-10-10 10:42:06 +00:00
Jaroslav Tulach
a234e82ee9
Instrumenter to observe behavior of nodes with UUID (#7833)
Exposing instrumentation capabilities to Enso. Fixes #7683.
2023-10-10 02:36:59 +00:00
Dmitry Bushev
11997e9344
Fix opening projects with arbitrary name (#8012)
related #7973

Fixes opening the projects with names that are not in Upper_Snake_Case.

# Important Notes
https://github.com/enso-org/enso/assets/357683/ea820658-5dee-474a-9dce-201d09796af7
2023-10-10 01:15:21 +00:00
somebody1234
bbd1652727
Fix Language Server documentation (#8003)
- Fix TypeScript errors (`String` -> `string`, `[T]` -> `T[]`, `A | B` -> `"A" | "B"`)
- Add highlighting for IDL (C# highlighting) and Enso (Rust highlighting) code blocks
- Fix broken links to `runtime/getComponentGroups` (`runtime-` -> `runtime`)
- Make formatting consistent

# Important Notes
None
2023-10-09 09:58:31 +00:00
Pavel Marek
4db61210c0
Exporting non-existing symbols fails with compiler error (#7960)
Adds a new compiler analysis pass that ensures that all the symbols exported via `export ...` or `from ... export ...` statements exist. If not, generates an IR Error.

# Important Notes
We already have such a compiler pass for the version with imports in [ImportSymbolAnalysis.scala](https://github.com/enso-org/enso/blob/develop/engine/runtime/src/main/scala/org/enso/compiler/pass/analyse/ImportSymbolAnalysis.scala)
2023-10-09 09:48:43 +00:00
somebody1234
6a127c501b
Make engine version optional (#7975)
Fixes this issue:
- #7917

This is caused by the dashboard expecting `engineVersion` to be present, whereas it is actually optional.

# Important Notes
This may have been fixed on the Project Manager side, so the bug may not be reproducible. To properly test that this fix works, an older `project-manager` (e.g `2023.2.1-nightly.2023.9.30`) should be used.
2023-10-09 08:21:47 +00:00
somebody1234
8125200571
Read groups from LS (#7986)
- Closes #7928

# Important Notes
Instead of omitting color, a fallback color is generated using `colorFromString`.
2023-10-09 07:10:21 +00:00
Michael Mauderer
10a95e43d6
Add functionality to handle Expression Updates to GUI2 (#7982)
Implements #7783. Adds functionality to handle and store expression updates, as well as show the output type of node.



https://github.com/enso-org/enso/assets/1428930/31ffff78-ff2c-4e0b-bcde-ddc507cc0226
2023-10-09 05:55:12 +00:00
somebody1234
44f2f425c0
[gui2] Language Server binary protocol, and loading of visualization data (#7873)
- Depends on #7773.

- Implements binary WebSocket protocol (data protocol)
- Performs some editor initialization (the bare minimum so that visualizations work)
- Adds event handlers to receive visualization data updates

# Important Notes
None
2023-10-07 20:57:47 +00:00
GregoryTravis
a90af9f844
Cache filtered children in XML (#7998)
We currently scan the entire child list for each call to children, num_children, and get_child_element.

Instead, we should cache the filtered child list on first access.
2023-10-06 19:44:01 +00:00
Dmitry Bushev
f348083dfb
Fix bundled GraalVM path (#7948)
close #7871
close #7698

Changelog:
- fix: the `run` script logic to place the GraalVM runtime in the expected directory when building the bundle
- fix: the `makeBundles` SBT logic to place the GraalVM runtime in the expected directory
2023-10-06 18:49:57 +00:00
GregoryTravis
9ba7be20af
Basic XML support (#7947)
This PR includes
* Reading XML from a file, stream, or string
* Reading XML via Data.fetch
* Accessing the root element, element children, and attributes
* Accessing tag text contents
* Get tags by name
* Inner / Outer XML string
2023-10-06 17:52:19 +00:00
Hubert Plociniczak
16c8d2e302
Remove context locks from visualization commands (#7953)
It looks like visualization commands had required context lock unnecessairly. Context manager methods are synchronized and therefore should not need the lock before submitting a correspodning background job.

Additionally, the presence of a context lock leads a deadlock:
1. Consider a long running execution of a program that does not finish within the 5 seconds
2. In the meantime there comes either an `AttachVisualization` or `DetachVisualization` request from the client

The latter will get stuck and eventually timeout out because it cannot acquire the lock withing the required time limits. It is still possible that a single long-running `ExecuteJob` might block other ones (including visualization ones) but that's a separate work.

Fixes some issues present in #7941.

# Important Notes
We need to still investigate `ExecuteJob`s need for context lock which might delay the execution of other background jobs that require it as well (mostly concerned about visualization).
2023-10-06 16:48:38 +00:00
Ilya Bogdanov
3b6f13c7b9
Create and edit nodes in temporary ast nodes with execution context disabled (#7779)
Closes #6417

Creating and editing nodes with a component browser does not edit the node in place but creates a temporary AST node with the execution context switch. When the node is committed, the original AST is replaced by the new one, and the temporary node is removed.

This solves several issues:
1. Previewing suggestions or editing nodes should no longer produce unwanted side effects, as the edited node expression is always evaluated with the output context being disabled.
2. Errors no longer propagate down the graph when you edit the node. The edited node has no outputs connected. Inputs behave normally, though.
3. Undo-redo and editing abort is much simpler. To abort editing, remove the temporary node.

https://github.com/enso-org/enso/assets/6566674/8d38e6bc-a4de-4e69-8d48-52d9b45b6554

# Important Notes
The code is a bit awkward in a few places. This is the consequence of previous architecture.
2023-10-06 14:56:09 +00:00
Michał Wawrzyniec Urbańczyk
4d9bc325ef
Fixed the ide/gui watch regression (#7997) 2023-10-06 16:47:29 +02:00
Dmitry Bushev
f1356374e1
Restore execution environment before the execution complete notification (#7980)
Fixes flaky RuntimeServerTest https://github.com/enso-org/enso/actions/runs/6410804439/job/17420283503?pr=7953#step:10:4004
2023-10-05 14:25:17 +00:00
Dmitry Bushev
5a1aaed952
Remove extra dot in error messages (#7974) 2023-10-05 15:06:07 +03:00
Ilya Bogdanov
022a5b0d1b
Make the code editor editable again (#7844)
Fixes #7474

Again, a quite complicated issue. The code editor was broken by https://github.com/enso-org/enso/pull/7167 for a few reasons:
1. The text component's shortcuts depend on the `focused` FRP endpoint, which is controlled by `deprecated_(de)focus` inputs. `focused` output was not changed by the code editor `model.focus()` call. (we still use `deprecated_focus` for all other components)
2. `focus()` FRP input was added to the text component but never triggered. It's too easy to confuse it with the `focus()` method of the `object::Instance` (Deref is evil!)
3. The graph editor handles clicks on the background by `blur_tree()`, effectively blurring any focused element in the application. But clicking on code editor (to focus it) also causes a click on the background! So we focus the code editor, and then handle the background click, defocusing it from the graph editor's point of view. It allows graph editor to open CB on <kbd>Enter</kbd> while the code editor is still active.

I solved (3) by disallowing CB opening while code editor is visible. Not ideal, but I guess it solves the issue.

Do you think the other solution is possible – to add some overlay for the code editor and avoid triggering `blur_tree` when clicking on it? That won't work because you still can't defocus the code editor using the new API, so it will always be focused.

So we have a situation when half of the app uses the new focus system, another half uses the old one, and none works properly.


https://github.com/enso-org/enso/assets/6566674/3a0d36ed-4d50-4606-9486-272fbd5d9cb7
2023-10-05 08:40:52 +00:00
Jaroslav Tulach
aada58e549
Removal of unused onExceptionalCallback (#7970) 2023-10-04 16:15:29 +02:00
Pavel Marek
c525698e5c
Enable Enso asserts on CI (#7929)
---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2023-10-04 15:52:44 +02:00
Jaroslav Tulach
997e0fc465
Using ENSO_JAVA env variable to turn on the Espresso support (#7951)
* Q: Is it normal for `--inspect` mode to print two debug urls?
* A: No, it should print just one.
* Q: Putting there a Java breakpoint to find out why it the chromeinspector gets initialized twice might reveal the culprit.
* A: The additional listener is happening [here](https://github.com/enso-org/enso/blob/develop/engine/runner/src/main/scala/org/enso/runner/ContextFactory.scala#L117).

# Important Notes
There is no easy check for a language being present without creating an `Engine`. It was thought creating an `Engine` is cheap operation, but it seems to have some downsides. Let's use `ENSO_JAVA` environment variable to decide whether _experimental Espresso_ support shall be enabled.
2023-10-04 11:55:31 +00:00
Ilya Bogdanov
2cd3912402
Add automatic generation of missing group colors (#7902)
Closes #7823

We set fallback color for components without an assigned group, use a color defined in the group, or otherwise derive the color from the group name.

All group colors are available in the graph editor as CSS variables so that they can be dynamically edited and debugged via dev tools.

For deriving the color, we use [murmurhash](https://www.npmjs.com/package/murmurhash), which quickly produces sufficiently uniformly distributed 32-bit values and [oklch](https://evilmartians.com/chronicles/oklch-in-css-why-quit-rgb-hsl) colorspace. We will still have *some* hash collisions between different groups, but it shouldn't matter that much. I have also tried a few other hashes (namely `cybr53` and Java's string hashing), and the results were more or less similar.

`oklch` is rather a new thing in CSS, but it is available starting with Chrome 111, which is exactly the version we're using in Electron.


https://github.com/enso-org/enso/assets/6566674/7177f041-93e9-4536-b19f-6b67a2da8b7e
2023-10-04 11:24:10 +00:00
Paweł Grabarz
ad0c1bc188
[GUI2] Fix and add tests for delta format translation. (#7968)
Fixes #7967

The text updates should no longer be rejected after applying edits containing newlines. The tricky update translation logic was also covered with property-based unit tests.

https://github.com/enso-org/enso/assets/919491/0bfb6181-7244-4eff-8d72-5b1a4630b9a6
2023-10-04 10:53:54 +00:00
Pavel Marek
c22928ecc2
Implement private modules (#7840)
Adds the ability to declare a module as *private*. Modifies the parser to add the `private` keyword as a reserved keyword. All the checks for private modules are implemented as an independent *Compiler pass*. No checks are done at runtime.

# Important Notes
- Introduces new keyword - `private` - a reserved keyword.
- Modules that have `private` keyword as the first statement are declared as *private* (Project private)
- Public module cannot have private submodules and vice versa.
- This would require runtime access checks
- See #7088 for the specification.
2023-10-04 10:33:10 +00:00
Jaroslav Tulach
421e3f22a4
Avoid unnecessary conversions (#7952)
Fixes #7854 by checking all elements of a union type for direct match and only then, if direct match isn't available, applying conversions.
2023-10-04 12:31:06 +02:00
somebody1234
0228022809
Fix lint error when running compile-server (#7969) 2023-10-04 05:11:17 +02:00
somebody1234
7e7b0f8c43
Hotfix code editor focus (#7965)
This fixes not being able to click the code editor.
This is a regression introduced by:
- #7893

# Important Notes
None
2023-10-03 20:56:40 +00:00
Hubert Plociniczak
30100605da
Native image should not be tied a particular hardware (#7956)
Added `-march=compatibility` argument to native image builds to ensure that we don't generate binaries that cannot be run on a target machine.
2023-10-03 20:34:00 +02:00
Radosław Waśko
0cd446432f
Fix inconsistency when building a Mixed column, fixes to Union (#7919)
- Fixes #7352 by remembering original value types in type inference mode to be able to reconstruct them for Mixed.
   - Added more benchmarks for comparing performance of constructing columns.
- Fixes missing implementations that caused `Table.union` crashing on some type pairs.
- Ensures that `Loss_Of_Integer_Precision` warning is not swallowed when numeric columns are unioned to create a `Float` column.
- Adds test for all of the above cases.
- Allow to output benchmark results to a CSV by setting an environment variable - useful for quickly comparing benchmarks, e.g. in Enso.
2023-10-03 20:33:34 +02:00
Radosław Waśko
08cd449a99
Fix NumberParser to avoid thousandSeparator==decimalPoint and prefer US decimal format (#7946)
Closes #7930
2023-10-03 20:07:54 +02:00
Michał Wawrzyniec Urbańczyk
2d39e644b8
New GUI/IDE build script support (#7832)
This PR adds support for the new Vue-based GUI (aka `gui2`).

The user-facing changes are primarily:
* support for `./run gui2` and `./run ide2` commands (that build just the new GUI and the whole IDE package with new GUI embedded — respectively);
* the top-level `test` and `lint` commands will now invoke the relevant commands on the new GUI

---------

Co-authored-by: Paweł Grabarz <frizi09@gmail.com>
2023-10-03 20:07:20 +02:00
Michał Wawrzyniec Urbańczyk
8d8c0b62d6
Disable broken lints (#7961) 2023-10-03 19:30:24 +02:00