Commit Graph

5406 Commits

Author SHA1 Message Date
Gregory Michael Travis
978a009133
Create Regex benchmarks (#11504) 2024-11-12 22:43:28 +00:00
Dmitry Bushev
67f075b10d
Optimize runtime Docker image size (#11529)
Reduce the size of the runtime Docker image from `3.04GB` to `1.41GB`.

```
$ docker images
REPOSITORY                   TAG       IMAGE ID       CREATED         SIZE
runtime                      latest    ab76231c78c5   9 minutes ago   1.41GB
<none>                       <none>    c8f70807b04f   4 hours ago     3.04GB
```

The image was bloated because of `RUN chmod` and `RUN chown` commands. When the `RUN` command modifies the file, it copies it to a new layer, resulting in a final image with multiple layers containing the same files.

# Important Notes
Note that copying directories with `COPY --chmod=5xx` sets the executable flag to both files and directories. While it is suboptimal (we only need the executable attribute for directories), having executable files in `/opt/enso` should not be an issue. Permissions can be narrowed further when the new Dockerfile `COPY` syntax is stabilized.
2024-11-12 22:04:46 +00:00
Paweł Grabarz
a83297b7db
Drop git dependency on enso-bot repository (#11539) 2024-11-12 18:25:19 +00:00
Adam Obuchowicz
d02f7e21df
Restore pnpm-lock (#11514)
Bypassing issue with CI
2024-11-12 10:05:07 +00:00
Radosław Waśko
e76fe907d3
Initial implementation of Data.read_many (#11490)
- Part of #11311
- Adds ability to read a list of files (Vector, Column, Table) into a Vector.
- Reading into a Table of objects or merged will come in a next PR.
2024-11-08 19:03:47 +00:00
Jaroslav Tulach
67db825587
MetadataStorage is mutable - structural equals in copy isn't enough (#11513)
`MetadataStorage` is mutable. Using `Object.equals` in `IR.copy` methods to check whether a _copy is needed_ isn't enough. The fact that two storages are `Object.equal` may be just temporary. Replacing the checks in 69 `IR.copy` methods with identity check - e.g. `ne` in Scala which is `==` in Java.

Using proper structural check inside of `MetadataStorage` fixes #11171.

# Important Notes
I [used this regex](https://github.com/enso-org/enso/issues/11171#issuecomment-2463908932) to find out 69 instances of `IR.copy`:

![69 copy methods](https://github.com/user-attachments/assets/257580b9-54fc-4199-88ad-a22103b0041f)

and I modified all 69 of them.
2024-11-08 17:29:41 +00:00
marthasharkey
f2037ee9d0
Add data quality indicators to table Viz (#11307) 2024-11-08 16:59:09 +00:00
marthasharkey
8b5cd9bbfc
Extend Selection Menu to allow text selection (#11352) 2024-11-08 16:36:45 +00:00
AdRiley
29faf201fc
Make SQLServer join work (#11511)
* Make join work

* Code Review Changes
2024-11-08 15:48:55 +00:00
Jaroslav Tulach
a6460c17b2
Implement proper hash and equals for FrameVariableNames (#11512) 2024-11-08 16:25:28 +01:00
Adam Obuchowicz
86c59469d0
Increase timeout and vite version (#11492)
1. The electron test hit timeouts from time to time, probably because of slow response from the engine: increased timeout.
2. Unify vite version across packages.
2024-11-08 08:43:02 +00:00
AdRiley
014a2a7e6b
Refactor (#11506) 2024-11-07 16:02:54 +00:00
AdRiley
676a7d4256
SQLServer enable filter (#11471)
* checkpoint

* Passing by.integer.comparisons

* More passing tests

* by empty text

* Add flag

* 2 more green

* Back to 6 red

* 5 red

* 2 red

* 1 red

* Green

* Refactor

* refactor

* Refactor

* refactor

* Refactor

* clean up

* Green

* Refactor

* Cleanup

* Refactor

* Clean up

* Cleanup

* Clean up

* Fix tests

* Fix

* Fix

* Code review

* Refactor

* Code review changes

* Add literals

* Code review changes

* Code review changes

* Checkpoint

* checkpoint

* checkpoint

* Cleanup

* Refactor

* Refactor

* refactor

* Fix type name

* Fix
2024-11-07 15:42:55 +00:00
Hubert Plociniczak
a5ebdf4e79
Fix logging in CLI (#11472)
* Fix logging in CLI

Previously, unless `--logger-connect` was used, CLI would always
fallback to console logging. In addition it would be misconfigured if
`application.conf` was provided with logging configuration.

This change makes sure that CLI uses the same logging infrastructure as
the rest of the system.
As a result, CLI will now by default not only log to the console and
respect the provided configuration but also log to the file system.

* Update docs

* c&p error

* More examples
2024-11-07 16:03:19 +01:00
Kaz Wesley
867c77d5cc
New markdown editor (#11469)
Implements #11240.

https://github.com/user-attachments/assets/4d2f8021-3e0f-4d39-95df-bcd72bf7545b

# Important Notes
- Fix a Yjs document corruption bug caused by `DeepReadonly` being replaced by a stub; introduce a `DeepReadonly` implementation without Vue dependency.
- Fix right panel sizing when code editor is open.
- Fix right panel slide-in animation.
- `Ast.Function` renamed to `Ast.FunctionDef`.
2024-11-06 16:54:32 +00:00
Pavel Marek
701bba6504
Convert Array_Like_Helpers.map to a builtin to reduce stack size (#11363)
The ultimate goal is to reduce the method calls necessary for `Vector.map`.

# Important Notes
- I managed to reduce the number of Java stack frames needed for each `Vector.map` call from **150** to **22** (See https://github.com/enso-org/enso/pull/11363#issuecomment-2432996902)
- Introduced `Stack_Size_Spec` regression test that will ensure that Java stack frames needed for `Vector.map` method call does not exceed **40**.
2024-11-06 11:14:48 +00:00
Paweł Grabarz
9703cb1dd0
Align esbuild versions across packages (#11459)
Part of #11458

Should get rid of the "magical" CI failure: https://github.com/enso-org/enso/actions/runs/11576387008/job/32225209090#step:7:254
2024-11-05 23:43:27 +00:00
James Dunkerley
86c1cd9953
Support for 1904 date format. (#11496)
- Adds support for reading Excel workbooks in 1904 date format.
- When writing to a workbook in 1904 format, will write dates correctly.

![image](https://github.com/user-attachments/assets/c17cd65d-1a09-4aa8-a946-8d427a2b7c22)

![image](https://github.com/user-attachments/assets/66796dac-4271-4bd1-acb3-1127afb5ec0b)
2024-11-05 23:10:34 +00:00
Sergei Garin
7ffd5b5ee6
Fix TOS modal (#11473) 2024-11-05 22:36:38 +00:00
Pavel Marek
3df57557ca
Only single native image is built at a time (#11497)
#10783 introduced another definition of a buildnativeimage task. Since that time, our CI is transiently failing on out of memory error. This PR ensures that there can be just a single `buildNativeImage` task running at a time.

# Important Notes
Manually tested by running
```
sbt:enso> all engine-runner/buildNativeImage project-manager/buildNativeImage
```
And looking at spawn subprocesses. On develop, I have two `native-image` processes, on this PR, there is just a single one.
2024-11-05 18:33:58 +00:00
Jaroslav Tulach
73abe909ef
Use GraphBuilder to construct an alias Graph (#11491) 2024-11-05 18:56:05 +01:00
Sergei Garin
0f8f6da273
Add transitions support for zustand 'useStore' (#11474) 2024-11-05 17:08:08 +00:00
Sergei Garin
0e434cd318
Replace special characters while uploading a file (#11356)
Closes: https://github.com/enso-org/cloud-v2/issues/1489

This PR replaces all special characters with `:`
2024-11-05 15:36:28 +00:00
Dmitry Bushev
47943a2e62
Add compression to the metadata code snapshot (#11470)
close #11420

Changelog:
- update: add zlib compression to the `snapshot` metadata field
- add: implement nodejs `zlib` for polyglot ydoc-server
- add: implement nodejs `Buffer` for polyglot ydoc-server
2024-11-05 11:57:43 +00:00
James Dunkerley
c5734a8fc8
Improved Google Analytics integration (#11484)
- Enhanced Google Analytics API.
- Now published as a type with static methods not a module.
- Bump version and add Admin API.
- Moved the reading logic to Java from Enso.
- Add dependency on Standard Table allowing report to be built into a Java Table directly.
- New `Google_Credential.new` method.
![image](https://github.com/user-attachments/assets/54e3ad87-045f-4e40-b609-337d827c5d02)
- Ability to list accounts for a credential (`Google_Analytics.list_accounts`).
![image](https://github.com/user-attachments/assets/296c6dcc-3b24-43fa-b909-5e74c40d77a1)
- Ability to list properties (either for an account or for all) (`Google_Analytics.list_properties`).
![image](https://github.com/user-attachments/assets/e420c824-d08e-48d0-b21c-560b4c7c4809)
- Simple object structure of `Google_Analytics_Account`, `Google_Analytics_Property` and `Google_Analytics_Field` with some helper methods.
- Widget for `account`, `property` and `credentials`.
![image](https://github.com/user-attachments/assets/221c1450-964d-4fce-af8b-2273aa8739a1)
![image](https://github.com/user-attachments/assets/e1daf1dd-2ade-4c33-875c-4e3cb1544fe6)
![image](https://github.com/user-attachments/assets/cd37b018-4fad-4771-9f48-1448f0076ef9)
- Widget for `dimensions` and `metrics` with defaults and then reading from Admin API.
![image](https://github.com/user-attachments/assets/3a4b1d42-9555-499d-90da-04d7586ab4c1)
![image](https://github.com/user-attachments/assets/16efcb11-3547-4eaf-9f28-944fa21c4aa2)
- Added widget for `start_date` and `end_date` on `Google_Analytics.read`.
- Bug fix for `parse` with auto type by reordering to allow numeric dates to be parsed.
- **ToDo**: better exception handling.
2024-11-05 10:11:42 +00:00
Hubert Plociniczak
35e5ed53d2
Don't cancel aborted jobs immediately (#11375)
* Don't cancel aborted jobs immediately

Rather than cancelling Futures that capture jobs' logic,
this change introduces a two-level system:

- interrupt all jobs softly via ThreadInterrupted at safepoints
- if safepoint is not executed within some time period or it is
  but the job is still not cancelled, trigger a hard-interrupt
  by cancelling the job explicitly, if possible

Closes #11084.

* Only cancel Future when you mean it

Soft-cancelling a future only to later call it with `mayInterrupt` set
to `true` has no effect in the latter case.
Changed the logic so that interrupting a Future will really enforce it.

Ocassionally some commands should not attempt to run soft cancellations
- we know they will re-execute the program.

* Replace Thread.sleep with Future.get

No while loops etc, it's much easier to reason about what is soft and
hard interrupt supposed to do.

* Better comments/logs

* nit

* PR review

* Make test more robust
2024-11-05 10:33:02 +01:00
Jaroslav Tulach
988316f910
Make Graph.nextId() private (#11486) 2024-11-05 05:21:10 +01:00
Kaz Wesley
42fc2af5cd
Fix tests after semantic merge conflict (#11488)
Force merging to repair CI.
2024-11-04 22:24:14 +00:00
Jaroslav Tulach
dd107e0ab1
Ensure EnsoMultiValue returns some Meta.type_of (#11480) 2024-11-04 20:47:25 +01:00
Kaz Wesley
2b3bd2cc90
Move documentation into documentable types (#11441)
Move documentation into documentable types (implements #11302).

# Important Notes
GUI:
- Distinguish expression and statement
- `Ast.Ast` is still present, as the base class for AST objects. Most references to `Ast.Ast` are now references to `Ast.Expression`. Operations on blocks use `Ast.Statement`.
- `Ast.parse` has been replaced with: `Ast.parseExpression`, `Ast.parseStatement`, and `Ast.parseBlock`
- `syncToCode` is internally context-aware; it parses the provided code appropriately depending on whether its AST is an expression, a statement, or the top level of a module.
- Remove `wrappingExpression` / `innerExpression` APIs: Wrapper types have been eliminated; modifier lines are now fields inside parent types.
- Simplify AST printing:
- Fully implemented autospacing in `concreteChildren` implementations; the type returned by `concreteChildren` now ensures that spacing has been fully resolved.
- Eliminate `printBlock` / `printDocs`: `concreteChildren` is now aware of indentation context, and responsible for indentation of its child lines.
- The `Pattern` type is now parameterized to identify the AST type it constructs. The `Pattern.parseExpression` function helps create a `Pattern<Expression>`.
- Refactor `performCollape` for testability.
- e2e tests: Improve table viz test: It still doesn't pass on my Mac, but these changes are necessary if not sufficient.

Compiler (TreeToIr):
- An expression in statement context is now found in an `ExpressionStatement` wrapper.
- Documentation for a `Function` is now found inside the function node.
- Deduplicate some polyglot-function logic.
2024-11-04 15:33:53 +00:00
Ilya Bogdanov
d3beac3a90
Consider additional self types (#11451)
Fixes #11427

Before:

https://github.com/user-attachments/assets/fc16cefd-f264-4410-bd30-1747c580da1a

After:

https://github.com/user-attachments/assets/e85b8fb3-35c4-4d18-a9a0-2aeb69201b6f
2024-11-04 15:08:59 +00:00
Adam Obuchowicz
2bbd909705
Implement the limit of 256 cells in the Table Editor (#11448)
Fixes #10864

[Screencast From 2024-10-30 13-45-48.webm](https://github.com/user-attachments/assets/9ed2384a-de85-4059-a0fe-c0a45b2fe07c)
2024-11-04 09:10:59 +00:00
Jaroslav Tulach
4cb943b5ed
Occurences in Java. Don't expose setters for Scope vars. (#11464)
Another change motivated by work on #11365. Continuation of #11419.
2024-11-01 17:14:10 +00:00
Sergei Garin
7487a3b4d2
Addtional tiny performance follow-up (#11460)
This PR mostly improve performance of the assets table:
1. It removes calc of the `clipPath` on scroll which triggers position recalculation in `Navigator2D`
2. Adds caching for parsing a category (we do this a lot across components but we have only a few categories) and runtime validatation has relatively large perf penalty
3. Adds dom-based virtualization for rows (we still need to add proper react based virtualization though)
2024-11-01 13:03:21 +00:00
Dmitry Bushev
5f5254772b
Ydoc native image (#10783)
close #10757

Changelog:
- add: native-image configuration to the `ydoc-server` project
- add: native-image overrides for loom executors replacing them with platform threads
- update: Helidon `4.1.2`
- fix: issues related to the native-image build
2024-10-31 19:57:50 +00:00
Pavel Marek
536a49f35d
Fix Meta.get_qualified_type_name when run as single file (#11401)
`Meta.get_qualified_type_name` correctly returns fully qualified type name when running a single file from a project with `enso --run Proj/src/Main.enso`.
2024-10-31 15:25:45 +00:00
Radosław Waśko
aad1107a8e
Add widget for name_filter (#11455)
- Closes #11310
2024-10-31 14:05:08 +00:00
Radosław Waśko
2619399799
Enso_File integration update: Multi-part upload and presigned URL for download (#11440)
- Closes #11330
- Closes #11331
2024-10-31 13:11:42 +00:00
Radosław Waśko
cf5326fbd1
Fixing small bugs uncovered by type checker (#11422)
Once our libraries and tests are compiled with basic inference of method types, some warnings were reported:
```
X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Database\0.0.0-dev\src\DB_Column.enso:1003:19: warning: Calling member method `div` on type Number will result in a No_Such_Method error in runtime.
1003 |         halfway = scale.div 2
|                   ^~~~~~~~~~~
X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Image\0.0.0-dev\src\Matrix.enso:381:21: warning: Invoking a value that has a non-function type (type Image) will result in a Not_Invokable error in runtime.
381 |     to_image self = Image (Image.from_vector self.normalize.to_vector self.rows self.channels)
|                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Table\0.0.0-dev\src\Internal\Multi_Value_Key.enso:94:22: warning: Calling static method `new` on (type Illegal_State) will result in a No_Such_Method error in runtime.
94 |         Error.throw (Illegal_State.new "Ordered_Multi_Value_Key is not intended for usage in unordered collections.")
|                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

This PR attempts to fix them.

There was also an expected error in the Examples:
```
X:\NBO\enso\built-distribution\enso-engine-0.0.0-dev-windows-amd64\enso-0.0.0-dev\lib\Standard\Examples\0.0.0-dev\src\Main.enso:139:36: warning: Calling static method `frobnicate` on (type No_Methods) will result in a No_Such_Method error in runtime.
139 | no_such_method = Panic.recover Any No_Methods.frobnicate . catch
|                                    ^~~~~~~~~~~~~~~~~~~~~
```
To avoid getting a warning, I wrapped this in `(_ : Any)` to make the warning go away. The behaviour of the function is unchanged.
2024-10-31 12:49:50 +00:00
Sergei Garin
fdab2233ac
Add React compiler (#11405)
This PR adds React Compiler to the dashboard
Read more about the compiler: https://github.com/reactwg/react-compiler/discussions/5

- This PR expects that these PRs are merged:
- #11380
- #11382

## Reasoning

On our codebase adding React compiler led to significant perf improvements across the whole app. But the most impact was on the AssetsTable (see video attachment). Adding React compiler reduced the rerender scope of the AssetsTable from the whole table to the certain elements that changed, and made the table much more usable.

Without compiler (page freezed):

https://github.com/user-attachments/assets/16505b40-123b-4153-9c22-880f5dfa211c


Compiler solves the issue:

https://github.com/user-attachments/assets/6a298876-f28d-4b3d-8247-50b0c7ecb7f8

This PR requires thorough testing of the dashboard for potential bugs introduced by the compiler
2024-10-31 12:10:22 +00:00
somebody1234
c6e87c2a17
Optimize asset table rendering (#11382)
- Depends on:
- #11380
- Some optimizations for re-rendering assets table:
- Omit `visibilities` from `state` in favor of passing each `AssetRow`'s `visibility` directly to the row. This minimizes spurious `state` updates.
- Pass `id` and `parentId` to `AssetRow` instead of the entire object. This ensures that re-fetches do not force a re-render of the `AssetRow` - we are no longer passing a reference to the object, so we are now comparing by string comparison (which is almost always stable).

# Important Notes
None
2024-10-31 10:36:10 +00:00
James Dunkerley
610ee5fdec
Expand and improve pretty for core data types, vector and table. (#11438)
-  Alter default `Any.pretty` so constructor is prefixed with type name (as needed now).
![image](https://github.com/user-attachments/assets/72d5ff2f-b567-47e2-becf-2e4acd4d089d)
-  Tests for `pretty` on `Date`.
- `pretty` for  `Date_Time` and  `Time_Of_Day` improved to not have as much noise.
- `pretty` for  `Period`,  `Date_Range` and  `Range`.
- Added custom `pretty` for  `Vector` and  `Array` as built-in method doesn't call through to overrides.
- Added custom `pretty` for  `Column` and  `Table`.
- Bug fix for `pretty` in `Time_Zone` so calls through to `pretty` of the zone_id to ensure safely escaped.
- Initial `default_widget` for `Date` and `Time_Of_Day`.
- Improve widget for `Date.to_date_time`.
![image](https://github.com/user-attachments/assets/18bc1d88-8ea9-42d0-8a9c-bc873e5d6835)
- `to_text`, `to_display_text` and `pretty` for `Enso_Secret`
![image](https://github.com/user-attachments/assets/d850c109-d1af-4b6f-a450-013c4d137805)
- private constructor for `Enso_Secret` as can't be correctly built directly.
- Use `_` for the testing methods in `HTTP` to clarify they shouldn't be used in general code.
2024-10-31 10:04:52 +00:00
Paweł Grabarz
950a93ddeb
Display error message when viewed graph doesn't exist. (#11452)
Fixes #11211

<img width="258" alt="image" src="https://github.com/user-attachments/assets/7adea4f6-774a-49ac-8c86-5e785df2acff">

When the "go back" button is clicked, the user is sent back to the `main` function graph. The breadcrumbs are also present on that view and can be used for navigation.

Current design of the error state is improvised, likely to be changed once we have some design guidance about it.
2024-10-31 07:48:48 +00:00
Ilya Bogdanov
0d731adeff
Add undo/redo buttons to the top bar (#11433)
Closes #11222

https://github.com/user-attachments/assets/66f8dfd1-a7c4-497a-8ff6-a22f56dbb1dd
2024-10-30 17:01:35 +00:00
Paweł Grabarz
0cf1c021ce
Add enter node button to the dropdown menu (#11447)
Fixes #11179

<img width="223" alt="image" src="https://github.com/user-attachments/assets/9b5ba054-717f-4124-98df-3d076bc73b32">
2024-10-30 16:55:12 +00:00
Ilya Bogdanov
6566b2da2f
Fix help panel (#11421)
Fixes #11392

The issue was caused by somewhat cumbersome logic of updating/overriding displayed suggestion. I’m not sure if it even was working correctly at any time.


https://github.com/user-attachments/assets/51e6fcd4-2198-40a2-86e7-8fcfa8b8b8d5
2024-10-30 14:14:28 +00:00
Paweł Grabarz
29f1f0d4e1
Add IDE version to window title. (#11446)
Fixes #10966

![image](https://github.com/user-attachments/assets/a46e38e9-1ff3-4eb4-9a88-901d684dc205)
2024-10-30 13:48:59 +00:00
Gregory Michael Travis
dc50a7e369
HTTP response caching, with TTL and LRU logic (#11342) 2024-10-30 12:50:35 +00:00
Nikita Pekin
10d76ca614
chore(flake.nix): Add missing macOS-specific dependencies of enso-formatter (#11430)
When compiling the `enso-formatter` binary for use in `~/.cargo/bin` (for the `cloud-v2` repo), the formatter requires some additional dependencies that are not currently provided in the nix build environment. This PR adds those dependencies to `flake.nix` so that `enso-formatter` compiles successfully.

cc @somebody1234
2024-10-30 11:15:37 +00:00
Adam Obuchowicz
39c44e7adb
Table Input Widget: Size persistence (#11435)
Fixes #10861

Every widget may set metadata on its AST. Because once widget picker will be implemented a single AST node may have many possible widgets, their settings are kept in a map keyed by their name/key.
2024-10-30 09:34:58 +00:00