Completes #8342 . Creates a SimpleHashJoin and CompoundHashJoin.
# Important Notes
Creates SimpleHashJoin and CompoundHashJoin.
CompoundHashJoin is what was HashJoin.
SimpleHashJoin is a new implementation that only indexs the smaller of the 2 tables being joined together.
The rest is refactor and clean-up of the shared join code.
This PR adds a native aarch64 target to our release process.
It also includes refactoring of workflow generation and minor tweaks:
* removing some workarounds in the generated action code that are not needed anymore;
* some version bumps that are harmless;
* release builds have cleaning enabled unconditionally.
- Close#8911
- Add dashboard unit tests to GUI2 CI
- Add dashboard E2E tests to GUI2 CI
- Fix (minor) issues in dashboard unit tests
# Important Notes
None
- Closes https://github.com/enso-org/cloud-v2/issues/784
- Add version panel
# Important Notes
- The backend sends an `isLatest` field in the response, but this is currently *not* shown in the UI because I think it isn't particularly useful.
- Versions are displayed in descending order (most recent first)
- Versions have hover effects, but there is not yet any relevant behavior that can be triggered on click.
- The versions breadcrumbs control displays the version number in the Figma design, however it is not yet returned by the cloud backend (and should ideally not take an extra request)
- ✅Linting fixes and groups.
- ✅Add `File.from that:Text` and use `File` conversions instead of taking both `File` and `Text` and calling `File.new`.
- ✅Align Unix Epoc with the UTC timezone and add converting from long value to `Date_Time` using it.
- ❌Add simple first logging API allowing writing to log messages from Enso.
- ✅Fix minor style issue where a test type had a empty constructor.
- ❌Added a `long` based array builder.
- Added `File_By_Line` to read a file line by line.
- Added "fast" JSON parser based off Jackson.
- ✅Altered range `to_vector` to be a proxy Vector.
- ✅Added `at` and `get` to `Database.Column`.
- ✅Added `get` to `Table.Column`.
- ✅Added ability to expand `Vector`, `Array` `Range`, `Date_Range` to columns.
- ✅Altered so `expand_to_column` default column name will be the same as the input column (i.e. no `Value` suffix).
- ✅Added ability to expand `Map`, `JS_Object` and `Jackson_Object` to rows with two columns coming out (and extra key column).
- ✅ Fixed bug where couldn't use integer index to expand to rows.
Refactor `Base_Tests` to `Test_New` testing framework. Mostly automatic text replacements.
# Important Notes
List of changes that were not done automatically (not via automatic text replacement):
- Fix indexes in Instrumentor_Spec - f590c4a398
- If group or spec is pending, its block is not evaluated - 8d797f1a4a
- Spec_Result is not private - 5767535af2
Tests marked as *pending*:
- #8913
- #8910
This is a prerequisite for adding a CI action for E2E tests.
- Fix E2E tests
- Remove visual regression testing (VRT) and associated fixtures (screenshots) for now
- Switch dashboard almost fully to Vite, from ESBuild, to match GUI2's build tooling.
- Add some new E2E tests:
- Creating assets
- Deleting assets
- Creating assets from the samples on the home page
- Sort assets
- Includes fixes for sorting:
- Group sorted assets by type again (regression) (see https://github.com/enso-org/cloud-v2/issues/554)
- Make sorting by title, case insensitive. This is because it is more intuitive for non-programmers if all items with uppercase names *aren't* separated from those with lowercase names - especially since the Windows FS is case-insensitive.
- Normalization of Unicode letters is *not* currently being done. It can potentially be added later.
- Double-clicking *anywhere* on a directory row now expands it. Previously it was only being expanded when double clicking
- Add recursive label adding/removal to mirror backend
- Note: The current implementation is not exactly the same as the backend's implementation.
- Fix https://github.com/enso-org/cloud-v2/issues/872
- Unset "saved project details" (for opening the last open project) if fetching it produces an error.
# Important Notes
- All tests pass. (run `npm run test:e2e` in `app/ide-desktop/lib/dashboard`)
- All `npm` commands should be run in `app/ide-desktop/lib/dashboard`. `dashboard:*` npm scripts have been removed from `app/ide-desktop` to prevent a mess.
- `npm run dev` confirmed to still work. Note that it has not been changed as it was already using Vite.
- `npm run build` now uses `vite build`. This has been tested using a local HTTP server that supports `404.html`.
- Other cases have been tested:
- `npm run test:e2e` works (all tests pass)
- `./run ide build` works
- `./run ide watch` works
- `./run ide2 build` works
- `./run gui watch` works
Closes#8823https://github.com/enso-org/enso/assets/6566674/966576ec-6507-401c-98d3-bd71c2ffc6b2
Adds a basic text widget for text literals.
### Important Notes
Several known restrictions:
- Separators would always be replaced with single quotation marks. All types of separators in Enso are supported though, and they would be correctly escaped if needed.
- Logic for widget selection probably needs refinement (works for text literals and `Text` types, but does not work for `Text | Integer`, for example)
- **(!)** There is a very annoying issue when the input field suddenly loses focus, closing the editing mode and discarding any changes. Debugging shows that it happens when we receive an engine update (and probably recreate the node component/widget tree (???)). It requires a separate investigation.
Uniqueness check of `UpsertVisualizationJob` only involved expressionId. Apparently now GUI sends mutliple visualizations for the same expressions and expects all of them to exist. Since previously we would cancel duplicate jobs, this was problematic.
This change makes sure that uniqueness also takes into account visualization id. Fixed a few logs that were not passing arguments properly.
Closes#8801
# Important Notes
I have not noticed any more problems with loading visualizations so the issue appears to be resolved with this change.
Added a unit test case that would previously fail due to cancellation of a job that upserts visualization.
Now the `clean` CI steps are run always for benchmarking jobs. We run the full `./run git-clean` before and after benchmarks. Benchmarks take long enough to make any savings by not cleaning negligible.
### Important Notes
This PR brings partial refactoring in the workflow generating code which was very dirty. I'll build on this further soon when adding proper aarch64 macOS support.
Also, some minor tweaks to the generation were made:
* not writing `always() &&` twice;
* run only the latter cleaning step for canceled jobs.
Adds a new test scenario that checks that types are correctly set and displayed after an expression update. Includes checking the string set on the node as well as showing the type label when hovering the output port of a node.
- Closes#8808 - adds tests for various scenarios.
- Implements `size` using HEAD.
- Updates existing functions to changes in Cloud API.
- Adds stubs for `*_time` methods, `parent`, `path`.
- [x] TODO: resolve the `Enso_File.current_working_directory` from an environment variable.
- ~~TODO: recursive directory deletion?~~ left for later
# Important Notes
- Currently, the Cloud API does not offer an easy way to extract metadata for a file, in particular to get the parent folder from the file `id`.
- We should be able to get the parent, and stuff like creation/modified time.
- We need a way to resolve paths to asset ids, for `path` to work as well as `current_working_directory`.
- What is the environment variable that will be used to feed the `current_working_directory` property?
Part of #8518Closes#8822
The drop downs were missing, because WidgetFunction had a higher score than them and took precedence. The fix was to adjust scores, so drop down is before both argument name and function. Generally, any widget expecting to "overlap" existing code (and possibly further widgets) should go before WidgetFunction.
# Important Notes
* Extended mockExpressionUpdate to accept a subexpression of a node. Also, I removed it from GraphDb - I want this function to have as little impact on app code as possible.
Refactor `test/Table_Test` to the builder API. The builder API is in a new library called `Test_New` that is alongside the old `Test` library. There will be follow-up PRs that will migrate the rest of the tests. Meanwhile, let's keep these two libraries, and merge them after the last PR.
# Important Notes
- For a brief introduction into the new API, see **Prototype 1** section in https://github.com/enso-org/enso/pull/8622#issuecomment-1889706168
- When executing all the tests, the behavior should be the same as with the old library. With the only exception that if `ENSO_TEST_ANSI_COLORS` env var is set, the output is more colorful than it used to be.
Introduce new AST APIs. They will be used for synchronization in the next PR.
# Important Notes
- `Module.edit` now clones a YDoc containing all the module's data; the resulting `MutableModule` can be edited with mutation APIs.
- Every AST type now has a mutable subtype. These `MutableAst` types expose all editing operations that were previously in `MutableModule`, such as `replace`; they also provide field setters.
Goal of this PR is to refactor the design of OrderMask and avoid copying arrays or lists wherever possible.
We have removed a few legacy functions which were not being used.
On a poor mans benchmark seems to be quicker (13s vs 16s) and memory usage should be lower.
- Depends on unmerged backend features:
- https://github.com/enso-org/cloud-v2/issues/832
- https://github.com/enso-org/cloud-v2/issues/833
- Close https://github.com/enso-org/cloud-v2/issues/829
- Add scaffold for settings page
- Add endpoint for deleting user
- Add endpoint for uploading profile picture
- Modify API types to add profile picture to user info
- Add shortcuts:
- <kbd>Cmd</kbd>+<kbd>,</kbd> to open settings page (shortcut taken from VS Code)
- <kbd>Esc</kbd> to close settings page
- ℹ️ Note that, while the settings page is considered as a page by the logic, it is not saved for the next session, as the settings page should be a transitional page (it should only ever be open to do one specific task, and then immediately closed again)
- Partly implement https://github.com/enso-org/cloud-v2/issues/840
- Add "members" tab with member list
- Add "Invite Users" button to be able to invite multiple users
- Misc. QoL improvements
- Deselect selections when clicking away from the selection
# Important Notes
None
Fixes several points from #8745
* Brought back cursor pointer in full screen visualization
* Changed behavior of edge disconnected from one side: Esc will bring the connection back, while click will remove it (and start creating new node if the source was connected).
* When hovering connection, the "active" part (i.e. the part which will be the hanging connection after click) is in normal color, and the rest is dimmed.
[Screencast from 2024-01-19 09-06-48.webm](https://github.com/enso-org/enso/assets/3919101/6df28a9c-51b3-4f98-be10-a35275eac800)
Closes#8751Closes#8752
- The numeric widget allows the use of the input field after clicking with LMB
- Slider is only visible if the engine provides widget configuration with set limits (see below for testing)
- Setting value outside limits is possible
- For now, to distinguish drag from click, we compare relative mouse movement on the mouse up event. We might benefit from using a timer instead, but let’s see how good it is now.
- Changes after demo
- No more input validation. You can enter literally anything and it would be accepted.
- Updates debouncing – the code is updated on defocus or when slider dragging has finished.
https://github.com/enso-org/enso/assets/6566674/b3580083-c678-4734-881c-97f8ac56176b
Some refactoring separated from #8825 for easier review.
# Important Notes
**ID types**
The new *synchronization IDs* will replace `ExprId` for `Ast` references in frontend logic. `ExprId` (now called `ExternalId`) is now used only for module serialization and engine communication. The graph database will maintain an index that is used to translate at the boundaries. For now, this translation is implemented as a type cast, as the IDs have the same values until the next PR.
- `AstId`: Identifies an `Ast` node.
- `NodeId`: A subtype of `AstId`.
- `ExternalId`: UUID used for serialization and engine communication.
**Other changes**:
- Immediate validation of `Owned` usage.
- Eliminate `Ast.RawCode`.
- Prepare to remove `IdMap` from yjsModel.
Adds new benchmark joining a large table to a small table in preparation for a coming optimisation that will only index the smaller of the 2 tables in #8342
A follow-up of [this comment](https://github.com/enso-org/enso/pull/8740#discussion_r1452282468)
`addRequiredImport` now reads the existing import from the edit instead of taking them from the current module. This way, it will consider any imports added so far as part of this edit.
- Closes#8723
- Adds some missing features that were needed to make this work:
- `Enso_File.create_directory` and `Enso_File.delete`, and basic tests for it
- Changes how `Enso_Secret.list` is obtained - using a different Cloud endpoint allows us to implement the desired logic, the default endpoint was giving us _all_ secrets which was not what we wanted here.
- Implements `Enso_Secret.update` and tests for it
# Important Notes
Notes describing any problems with the current Cloud API:
https://docs.google.com/document/d/1x8RUt3KkwyhlxGux7XUGfOdtFSAZV3fI9lSSqQ3XsXk/edit
Apparently, everything that was needed to make this feature work has already been implemented, although a few features needed workarounds on Enso side to work properly.
- Close https://github.com/enso-org/cloud-v2/issues/856
- Implement upload for existing files
- This is triggered by uploading a file with the same name.
- Add a modal that allows the user to select whether to rename or update the file, for each conflicting file
# Important Notes
- There are a *lot* of buttons in the modal. Any ideas on how to make this screen clearer/simpler/easier to understand would be greatly appreciated.
- There are currently a few minor backend issues preventing all functionality from fully working:
- Projects currently cannot be updated due to a bug
- Neither files nor projects can be uploaded if their name contains `()` - e.g. `foo (2).txt`
This is a quick fix to a long standing problem of
`org.enso.interpreter.service.error.FailedToApplyEditsException` which would prevent backend from processing any more changes, rendering GUI (and backend) virtually useless.
Edits are submitted for (background) processing in the order they are handled. However the order of execution of such tasks is not guaranteed. Most of the time edits are processed in the same order as their requests but when they don't, files get quickly out of sync.
Related to #8770.
# Important Notes
I'm not a fan of this change because it essentially blocks all open/file requests until all edits are processed and we already have logic to deal with that appropriately. Moreover those tasks can and should be processed independently. Since we already had the single thread executor present to ensure correct synchronization of open/file/push commands, we are simply adding edit commands to the list.
Ideally we want to have a specialized executor that executes tasks within the same group sequentially but groups of tasks can be executed in parallel, thus ensuring sufficient throughput. The latter will take much longer and will require significant rewrite of the command execution.
Added tests that would previously fail due to non-deterministic execution.
This makes it consistent with the indentation style of GUI2.
# Important Notes
- This commit *will* need to be added to [`.git-blame-ignore-revs`](https://github.com/enso-org/enso/blob/develop/.git-blame-ignore-revs) *after* it is merged.
- This shouldn't need any particular QA (although QA doesn't hurt), as the only thing that was done is a `npx prettier -w .`, meaning that there should be zero logic changes.
- Closes https://github.com/enso-org/cloud-v2/issues/857
- Changes format for new folders from `New_Folder_1` to `New Folder 1`
- Changes format for new projects from `New_Project_1` to `New Project 1`
- Uses names for projects created from templates, rather than `Upper_Snake_Case`d IDs
# Important Notes
- Folders containing spaces work fine on cloud backend
- ⚠️ Unable to test opening a project with a space in its name on the cloud backend.
- Projects containing spaces work (open, edit, and execute fine on local backend
- Fix https://github.com/enso-org/cloud-v2/issues/848
- Disables dragging rows when a row is being edited
- Also disables dragging rows on the local backend, as folders are not yet supported there
- Fix other issues found (not sure if they were introduced by this PR):
- Cut (Cmd+X) does not fade out assets
- Cancelling editing names actually still renames the asset
- Cut/Copy/Paste/Undo shortcuts when renaming *also* trigger "copy assets" (etc.) shortcuts
- Apply suggested QoL improvement:
- Disallow renaming an asset to a blank name (a name consisting only of whitespace). Current (new) behavior is to silently revert the asset name to its original name.
# Important Notes
None
