Commit Graph

3979 Commits

Author SHA1 Message Date
James Dunkerley
eeaddbc434
Add parser for line by line processing (#8719)
- Linting fixes and groups.
- Add `File.from that:Text` and use `File` conversions instead of taking both `File` and `Text` and calling `File.new`.
- Align Unix Epoc with the UTC timezone and add converting from long value to `Date_Time` using it.
- Add simple first logging API allowing writing to log messages from Enso.
- Fix minor style issue where a test type had a empty constructor.
- Added a `long` based array builder.
- Added `File_By_Line` to read a file line by line.
- Added "fast" JSON parser based off Jackson.
- Altered range `to_vector` to be a proxy Vector.
- Added `at` and `get` to `Database.Column`.
- Added `get` to `Table.Column`.
- Added ability to expand `Vector`, `Array` `Range`, `Date_Range` to columns.
- Altered so `expand_to_column` default column name will be the same as the input column (i.e. no `Value` suffix).
- Added ability to expand `Map`, `JS_Object` and `Jackson_Object` to rows with two columns coming out (and extra key column).
-  Fixed bug where couldn't use integer index to expand to rows.
2024-02-01 07:29:50 +00:00
Pavel Marek
bb8ff8f89e
Refactor Base_Tests/src/Data to Test_New (#8890)
Refactor `Base_Tests` to `Test_New` testing framework. Mostly automatic text replacements.

# Important Notes
List of changes that were not done automatically (not via automatic text replacement):
- Fix indexes in Instrumentor_Spec - f590c4a398
- If group or spec is pending, its block is not evaluated - 8d797f1a4a
- Spec_Result is not private - 5767535af2

Tests marked as *pending*:
- #8913
- #8910
2024-01-31 14:25:16 +00:00
Michael Mauderer
1ba28a3c5c
New scenarios for E2E tests - Displaying Edges (#8904)
Adds e2e test for rendering edges.
2024-01-31 13:42:36 +00:00
somebody1234
cbf6d41e4c
More E2E tests; export default classes from modules (#8730)
This is a prerequisite for adding a CI action for E2E tests.

- Fix E2E tests
- Remove visual regression testing (VRT) and associated fixtures (screenshots) for now
- Switch dashboard almost fully to Vite, from ESBuild, to match GUI2's build tooling.
- Add some new E2E tests:
- Creating assets
- Deleting assets
- Creating assets from the samples on the home page
- Sort assets
- Includes fixes for sorting:
- Group sorted assets by type again (regression) (see https://github.com/enso-org/cloud-v2/issues/554)
- Make sorting by title, case insensitive. This is because it is more intuitive for non-programmers if all items with uppercase names *aren't* separated from those with lowercase names - especially since the Windows FS is case-insensitive.
- Normalization of Unicode letters is *not* currently being done. It can potentially be added later.
- Double-clicking *anywhere* on a directory row now expands it. Previously it was only being expanded when double clicking
- Add recursive label adding/removal to mirror backend
- Note: The current implementation is not exactly the same as the backend's implementation.
- Fix https://github.com/enso-org/cloud-v2/issues/872
- Unset "saved project details" (for opening the last open project) if fetching it produces an error.

# Important Notes
- All tests pass. (run `npm run test:e2e` in `app/ide-desktop/lib/dashboard`)
- All `npm` commands should be run in `app/ide-desktop/lib/dashboard`. `dashboard:*` npm scripts have been removed from `app/ide-desktop` to prevent a mess.
- `npm run dev` confirmed to still work. Note that it has not been changed as it was already using Vite.
- `npm run build` now uses `vite build`. This has been tested using a local HTTP server that supports `404.html`.
- Other cases have been tested:
- `npm run test:e2e` works (all tests pass)
- `./run ide build` works
- `./run ide watch` works
- `./run ide2 build` works
- `./run gui watch` works
2024-01-31 11:35:41 +00:00
Michał Wawrzyniec Urbańczyk
06f1c772d8
CI: Use GH Actions concurrency setting to manage cancelling. (#8901) 2024-01-31 12:12:08 +01:00
Hubert Plociniczak
68ce4ae1b7
Reduced resources for various Akka dispatchers (#8892)
The defaults picked up by Akka tend to make us of all resources which is unnecessary and overwhelming for tests.

Improves #8806, potentially.

Before
![Screenshot from 2024-01-28 22-34-42](https://github.com/enso-org/enso/assets/292128/f80eb66a-2f37-44d5-bcdb-f00a78fe72fd)
After
![Screenshot from 2024-01-31 00-12-10](https://github.com/enso-org/enso/assets/292128/c5223912-5f6e-413c-a0a4-050afa3ed463)

when running the problematic `LibrariesTest`.

Full `language-server` test suite.
Before
![Screenshot from 2024-01-31 00-20-50](https://github.com/enso-org/enso/assets/292128/f1c94a66-6905-4f57-8a7d-7df049714353)
After
![Screenshot from 2024-01-31 00-18-40](https://github.com/enso-org/enso/assets/292128/3a11125e-d593-43df-8d35-1a8915812b2b)

# Important Notes
Note that Executors assigned to Zio and initializers should also be improved. Unfortunately due to various blocking threadpools  it is easy to get timeouts when running the whole suite.
2024-01-31 10:25:17 +00:00
Ilya Bogdanov
057bf17ffb
Fix wrapping in doc panel (#8887)
Fixes soft-wrapping of lines in documentation panel.
2024-01-30 15:05:39 +01:00
Ilya Bogdanov
ad6348a12a
Text input widget (#8873)
Closes #8823

https://github.com/enso-org/enso/assets/6566674/966576ec-6507-401c-98d3-bd71c2ffc6b2

Adds a basic text widget for text literals. 

### Important Notes

Several known restrictions:
- Separators would always be replaced with single quotation marks. All types of separators in Enso are supported though, and they would be correctly escaped if needed.
- Logic for widget selection probably needs refinement (works for text literals and `Text` types, but does not work for `Text | Integer`, for example)
- **(!)** There is a very annoying issue when the input field suddenly loses focus, closing the editing mode and discarding any changes. Debugging shows that it happens when we receive an engine update (and probably recreate the node component/widget tree (???)). It requires a separate investigation.
2024-01-30 15:05:28 +01:00
Hubert Plociniczak
081c8c889c
Don't cancel pending visualization's upserts (#8853)
Uniqueness check of `UpsertVisualizationJob` only involved expressionId. Apparently now GUI sends mutliple visualizations for the same expressions and expects all of them to exist. Since previously we would cancel duplicate jobs, this was problematic.

This change makes sure that uniqueness also takes into account visualization id. Fixed a few logs that were not passing arguments properly.

Closes #8801

# Important Notes
I have not noticed any more problems with loading visualizations so the issue appears to be resolved with this change.
Added a unit test case that would previously fail due to cancellation of a job that upserts visualization.
2024-01-30 00:13:43 +00:00
GregoryTravis
7436848e90
Implement relational NULL/Nothing for join for in-memory tables (#8849)
Implements relational NULL for join, for all `Join_Kind`s.
2024-01-29 16:19:07 +00:00
Michał Wawrzyniec Urbańczyk
0b16db4399
CI: Unconditional cleaning for benchmark runs. (#8839)
Now the `clean` CI steps are run always for benchmarking jobs. We run the full `./run git-clean` before and after benchmarks. Benchmarks take long enough to make any savings by not cleaning negligible.

### Important Notes
This PR brings partial refactoring in the workflow generating code which was very dirty. I'll build on this further soon when adding proper aarch64 macOS support.

Also, some minor tweaks to the generation were made:
* not writing `always() &&` twice;
* run only the latter cleaning step for canceled jobs.
2024-01-29 13:02:02 +01:00
Adam Obuchowicz
2ca4fe94e2
Fix breadcrumbs navigation (#8874)
Fixes #8756

The items were added to stack in wrong order (I wonder how on earth it worked with the engine).
2024-01-29 10:53:44 +00:00
Michael Mauderer
abe0875072
New scenarios for E2E - Displaying proper types on nodes on hover (#8856)
Adds a new test scenario that checks that types are correctly set and displayed after an expression update. Includes checking the string set on the node as well as showing the type label when hovering the output port of a node.
2024-01-29 10:37:44 +00:00
Jaroslav Tulach
946f931d21
Log histograms with -Dorg.enso.persist.Logger.level=debug (#8881)
Fixes #8644 by using `slf4j` instead of `java.util.logging`.
2024-01-29 10:26:58 +00:00
Adam Obuchowicz
ad7fad42fa
Make run button execute in live mode (#8876)
What can I say? It's not quite a bug. Rather a wrong implementation

Anyway, fixes #8810
2024-01-29 09:50:52 +00:00
Jaroslav Tulach
084245868d
Canonicalize the temporary directory files (#8879)
Due to some Mac OS X symlinks trickery we need to canonicalize files created by `@Rule TemporaryFolder` to compare them for equality.
2024-01-29 09:06:43 +00:00
Jaroslav Tulach
74b808b00d
Argument block application has no place in type body (#8877)
Fixes #8832 by reporting a `SyntaxError` in `TreeToIr` conversion.
2024-01-28 20:29:14 +00:00
GregoryTravis
644c9af579
Attach a warning when Nothing is used in a Filter_Condition (#8865)
Attach a warning when Nothing is used as a value in a comparison or `is_in` Filter_Condition.
2024-01-27 08:45:45 +00:00
Jaroslav Tulach
9a37357247
Binary operator resolution based on that value (#8779) 2024-01-27 08:38:47 +01:00
Radosław Waśko
ca4f98c78e
Adding tests and missing methods for Enso_File. (#8815)
- Closes #8808 - adds tests for various scenarios.
- Implements `size` using HEAD.
- Updates existing functions to changes in Cloud API.
- Adds stubs for `*_time` methods, `parent`, `path`.
- [x] TODO: resolve the `Enso_File.current_working_directory` from an environment variable.
- ~~TODO: recursive directory deletion?~~ left for later

# Important Notes
- Currently, the Cloud API does not offer an easy way to extract metadata for a file, in particular to get the parent folder from the file `id`.
- We should be able to get the parent, and stuff like creation/modified time.
- We need a way to resolve paths to asset ids, for `path` to work as well as `current_working_directory`.
- What is the environment variable that will be used to feed the `current_working_directory` property?
2024-01-26 19:04:42 +00:00
Adam Obuchowicz
057245ea5e
Fix nested drop downs + test scenarios for drop-down and vector widgets. (#8859)
Part of #8518
Closes #8822

The drop downs were missing, because WidgetFunction had a higher score than them and took precedence. The fix was to adjust scores, so drop down is before both argument name and function. Generally, any widget expecting to "overlap" existing code (and possibly further widgets) should go before WidgetFunction.

# Important Notes
* Extended mockExpressionUpdate to accept a subexpression of a node. Also, I removed it from GraphDb - I want this function to have as little impact on app code as possible.
2024-01-26 17:18:27 +00:00
Pavel Marek
d0fdeca6df
Refactor Table_Tests to the builder API (#8622)
Refactor `test/Table_Test` to the builder API. The builder API is in a new library called `Test_New` that is alongside the old `Test` library. There will be follow-up PRs that will migrate the rest of the tests. Meanwhile, let's keep these two libraries, and merge them after the last PR.

# Important Notes
- For a brief introduction into the new API, see **Prototype 1** section in https://github.com/enso-org/enso/pull/8622#issuecomment-1889706168
- When executing all the tests, the behavior should be the same as with the old library. With the only exception that if `ENSO_TEST_ANSI_COLORS` env var is set, the output is more colorful than it used to be.
2024-01-26 12:08:24 +00:00
Kaz Wesley
a04385a6a4
New AST APIs (#8851)
Introduce new AST APIs. They will be used for synchronization in the next PR.

# Important Notes
- `Module.edit` now clones a YDoc containing all the module's data; the resulting `MutableModule` can be edited with mutation APIs.
- Every AST type now has a mutable subtype. These `MutableAst` types expose all editing operations that were previously in `MutableModule`, such as `replace`; they also provide field setters.
2024-01-26 11:32:35 +00:00
James Dunkerley
0b6db5797c
Refactor OrderMask to avoid memory copying (#8863)
Goal of this PR is to refactor the design of OrderMask and avoid copying arrays or lists wherever possible.
We have removed a few legacy functions which were not being used.

On a poor mans benchmark seems to be quicker (13s vs 16s) and memory usage should be lower.
2024-01-26 11:16:16 +00:00
Adam Obuchowicz
5dd2dc1c93
Fix gui2 typecheck (#8870) 2024-01-26 09:15:01 +01:00
Ilya Bogdanov
6f51894273
Align collapsed node to the leftmost selected one (#8844)
As discussed on the latest demo.


https://github.com/enso-org/enso/assets/6566674/c68f1df4-285e-413b-b05e-13394dffe3bd
2024-01-25 16:19:41 +01:00
somebody1234
6e7672a424
Add settings page; add endpoints for deleting user and uploading profile picture (#8720)
- Depends on unmerged backend features:
- https://github.com/enso-org/cloud-v2/issues/832
- https://github.com/enso-org/cloud-v2/issues/833
- Close https://github.com/enso-org/cloud-v2/issues/829
- Add scaffold for settings page
- Add endpoint for deleting user
- Add endpoint for uploading profile picture
- Modify API types to add profile picture to user info
- Add shortcuts:
- <kbd>Cmd</kbd>+<kbd>,</kbd> to open settings page (shortcut taken from VS Code)
- <kbd>Esc</kbd> to close settings page
- ℹ️ Note that, while the settings page is considered as a page by the logic, it is not saved for the next session, as the settings page should be a transitional page (it should only ever be open to do one specific task, and then immediately closed again)
- Partly implement https://github.com/enso-org/cloud-v2/issues/840
- Add "members" tab with member list
- Add "Invite Users" button to be able to invite multiple users
- Misc. QoL improvements
- Deselect selections when clicking away from the selection

# Important Notes
None
2024-01-25 15:10:51 +00:00
Adam Obuchowicz
3a849ea01b
Connection and visualization fixes (#8803)
Fixes several points from #8745
* Brought back cursor pointer in full screen visualization
* Changed behavior of edge disconnected from one side: Esc will bring the connection back, while click will remove it (and start creating new node if the source was connected).
* When hovering connection, the "active" part (i.e. the part which will be the hanging connection after click) is in normal color, and the rest is dimmed.

[Screencast from 2024-01-19 09-06-48.webm](https://github.com/enso-org/enso/assets/3919101/6df28a9c-51b3-4f98-be10-a35275eac800)
2024-01-25 14:18:26 +00:00
Ilya Bogdanov
a3873b9565
Numeric input widget (#8837)
Closes #8751
Closes #8752

- The numeric widget allows the use of the input field after clicking with LMB
- Slider is only visible if the engine provides widget configuration with set limits (see below for testing)
- Setting value outside limits is possible
- For now, to distinguish drag from click, we compare relative mouse movement on the mouse up event. We might benefit from using a timer instead, but let’s see how good it is now.
- Changes after demo
- No more input validation. You can enter literally anything and it would be accepted.
- Updates debouncing – the code is updated on defocus or when slider dragging has finished.


https://github.com/enso-org/enso/assets/6566674/b3580083-c678-4734-881c-97f8ac56176b
2024-01-25 09:41:37 +00:00
somebody1234
899f7e4e61
Fix offline detection (#8829)
- Closes https://github.com/enso-org/cloud-v2/issues/860

# Important Notes
None
2024-01-25 06:36:03 +00:00
Kaz Wesley
f88bd90ebb
Refactor for YJs AST (#8840)
Some refactoring separated from #8825 for easier review.

# Important Notes
**ID types**

The new *synchronization IDs* will replace `ExprId` for `Ast` references in frontend logic. `ExprId` (now called `ExternalId`) is now used only for module serialization and engine communication. The graph database will maintain an index that is used to translate at the boundaries. For now, this translation is implemented as a type cast, as the IDs have the same values until the next PR.

- `AstId`: Identifies an `Ast` node.
- `NodeId`: A subtype of `AstId`.
- `ExternalId`: UUID used for serialization and engine communication.

**Other changes**:

- Immediate validation of `Owned` usage.
- Eliminate `Ast.RawCode`.
- Prepare to remove `IdMap` from yjsModel.
2024-01-24 19:22:05 +00:00
AdRiley
1c6898b19d
Add Join Large Small benchmarks (#8845)
Adds new benchmark joining a large table to a small table in preparation for a coming optimisation that will only index the smaller of the 2 tables in #8342
2024-01-24 19:13:09 +00:00
GregoryTravis
5eb3f3bd1d
Implement relational NULL semantics for Nothing for in-memory Column operations (#8816)
Updates in-memory table column operations to treat Nothing as a relational NULL.
This PR does not include changes to Table.join.
2024-01-24 17:02:45 +00:00
AdRiley
23d6fcdd9c
Make max of Random methods inclusive rather than exclusive (#8768)
* Make max inclusive rather than exclusive

* Update names to min max. Remove end_inclusive

* Indicies should remain exclusive

* Make the tests pass
2024-01-24 16:58:33 +00:00
AdRiley
6eb00a7c5f
Remove Conditions Helper Class (#8842)
* Tests green checkpoint

* Remove ConditionsHelper

* javafmtAll

* Dedupe
2024-01-24 16:56:39 +00:00
Adam Obuchowicz
f5c02d36e2
Fix addRequiredImport so it is not fooled by existing module changes (#8813)
A follow-up of [this comment](https://github.com/enso-org/enso/pull/8740#discussion_r1452282468)

`addRequiredImport` now reads the existing import from the edit instead of taking them from the current module. This way, it will consider any imports added so far as part of this edit.
2024-01-24 12:03:25 +00:00
Radosław Waśko
edfcfde11c
Tests and improvements for secrets in cloud subdirectories (#8791)
- Closes #8723
- Adds some missing features that were needed to make this work:
- `Enso_File.create_directory` and `Enso_File.delete`, and basic tests for it
- Changes how `Enso_Secret.list` is obtained - using a different Cloud endpoint allows us to implement the desired logic, the default endpoint was giving us _all_ secrets which was not what we wanted here.
- Implements `Enso_Secret.update` and tests for it

# Important Notes
Notes describing any problems with the current Cloud API:
https://docs.google.com/document/d/1x8RUt3KkwyhlxGux7XUGfOdtFSAZV3fI9lSSqQ3XsXk/edit

Apparently, everything that was needed to make this feature work has already been implemented, although a few features needed workarounds on Enso side to work properly.
2024-01-24 10:17:22 +00:00
somebody1234
0a493de012
Update uploaded files (#8797)
- Close https://github.com/enso-org/cloud-v2/issues/856
- Implement upload for existing files
- This is triggered by uploading a file with the same name.
- Add a modal that allows the user to select whether to rename or update the file, for each conflicting file

# Important Notes
- There are a *lot* of buttons in the modal. Any ideas on how to make this screen clearer/simpler/easier to understand would be greatly appreciated.
- There are currently a few minor backend issues preventing all functionality from fully working:
- Projects currently cannot be updated due to a bug
- Neither files nor projects can be uploaded if their name contains `()` - e.g. `foo (2).txt`
2024-01-23 09:55:15 +00:00
Hubert Plociniczak
24007f1db4
Add persistance to Operator.Binary (#8826) 2024-01-23 10:43:26 +01:00
Hubert Plociniczak
dfe867a9cd
Edits are processed in the order of submission (#8787)
This is a quick fix to a long standing problem of
`org.enso.interpreter.service.error.FailedToApplyEditsException` which would prevent backend from processing any more changes, rendering GUI (and backend) virtually useless.
Edits are submitted for (background) processing in the order they are handled. However the order of execution of such tasks is not guaranteed. Most of the time edits are processed in the same order as their requests but when they don't, files get quickly out of sync.

Related to #8770.

# Important Notes
I'm not a fan of this change because it essentially blocks all open/file requests until all edits are processed and we already have logic to deal with that appropriately. Moreover those tasks can and should be processed independently. Since we already had the single thread executor present to ensure correct synchronization of open/file/push commands, we are simply adding edit commands to the list.

Ideally we want to have a specialized executor that executes tasks within the same group sequentially but groups of tasks can be executed in parallel, thus ensuring sufficient throughput. The latter will take much longer and will require significant rewrite of the command execution.

Added tests that would previously fail due to non-deterministic execution.
2024-01-22 23:05:41 +00:00
somebody1234
5f1333a519
Change dashboard to use 2 spaces for indentation (#8798)
This makes it consistent with the indentation style of GUI2.

# Important Notes
- This commit *will* need to be added to [`.git-blame-ignore-revs`](https://github.com/enso-org/enso/blob/develop/.git-blame-ignore-revs) *after* it is merged.
- This shouldn't need any particular QA (although QA doesn't hurt), as the only thing that was done is a `npx prettier -w .`, meaning that there should be zero logic changes.
2024-01-22 21:26:15 +00:00
GregoryTravis
659243316a
Mark a JDBC test as pending if cloud not set up (#8828)
Marking the test pending=cloud_setup.pending.
2024-01-22 20:05:22 +00:00
somebody1234
bd4f46499f
Remove underscores from default project and folder names (#8819)
- Closes https://github.com/enso-org/cloud-v2/issues/857
- Changes format for new folders from `New_Folder_1` to `New Folder 1`
- Changes format for new projects from `New_Project_1` to `New Project 1`
- Uses names for projects created from templates, rather than `Upper_Snake_Case`d IDs

# Important Notes
- Folders containing spaces work fine on cloud backend
- ⚠️ Unable to test opening a project with a space in its name on the cloud backend.
- Projects containing spaces work (open, edit, and execute fine on local backend
2024-01-22 11:38:15 +00:00
somebody1234
d16a645e8e
Prevent dragging rows when editing, and when on local backend (#8747)
- Fix https://github.com/enso-org/cloud-v2/issues/848
- Disables dragging rows when a row is being edited
- Also disables dragging rows on the local backend, as folders are not yet supported there
- Fix other issues found (not sure if they were introduced by this PR):
- Cut (Cmd+X) does not fade out assets
- Cancelling editing names actually still renames the asset
- Cut/Copy/Paste/Undo shortcuts when renaming *also* trigger "copy assets" (etc.) shortcuts
- Apply suggested QoL improvement:
- Disallow renaming an asset to a blank name (a name consisting only of whitespace). Current (new) behavior is to silently revert the asset name to its original name.

# Important Notes
None
2024-01-22 11:06:45 +00:00
Radosław Waśko
368e4867b4
Allow secrets in AWS_Credential (#8774)
- Closes #8722
2024-01-19 19:00:56 +00:00
Jaroslav Tulach
5d877abe74
Dump more details when two objects at the same locations are read (#8802) 2024-01-19 06:58:50 +01:00
Jaroslav Tulach
457eb2279f
VSCode JUnit classpath fix (#8795)
Recently a classpath misconfiguration appeared in `engine/runtime` project:

![org.junit not found](https://github.com/enso-org/enso/assets/26887752/215a074b-1fad-4d4d-a5a8-a573b0d53e76)

The problem was caused by the `XyzTest.java` files being incorrectly assigned to `engine/runtime/src/main/java` source root and its classpath (which obviously doesn't contain JUnit). This PR restricts the `srcCp` to `inputDir`, when it is known. That properly assignes the `XyzTest.java` files to `engine/runtime/src/test/java` source root. The IGV as well as VSCode support seems to recognize unit test classpath properly now.
2024-01-18 15:57:45 +00:00
Michał Wawrzyniec Urbańczyk
32d0459534
CI: Remove bench-report.xml in the whole repo subtree (#8762)
As requested in https://github.com/enso-org/enso/pull/8707#issuecomment-1882512361 and discussed on Discord.
2024-01-18 15:32:00 +00:00
Ilya Bogdanov
d5d4793e0e
Positioning for nodes created when collapsing (#8771)
Closes #8699


https://github.com/enso-org/enso/assets/6566674/82e1bdb3-3f37-4122-9e73-92d39acd0f13
2024-01-18 14:30:13 +00:00
Ilya Bogdanov
f96e5dddd6
e2e tests for collapsing and entering nodes (#8758)
Related to https://github.com/enso-org/enso/issues/8518

These tests already caught one regression in nav breadcrumbs: https://github.com/enso-org/enso/issues/8756

It also provides API for mocking expression updates for arbitrary nodes on the screen. The implementation is a bit convoluted and includes setting a callback on `window`, but it looks like the only possible solution given our architecture and playwright restrictions.
2024-01-18 13:45:18 +00:00