Commit Graph

5552 Commits

Author SHA1 Message Date
Radosław Waśko
069e2362dc
Revert "Add Meta.engine_version" (#11336) 2024-10-16 12:09:32 +00:00
Pavel Marek
b36fd1c01b
Migrate some passes to Mini passes (#11191)
Gets ready for avoiding IR traversal by introducing _mini passes_ as proposed by #10981:
- creates [MiniPassFactory](762045a357) (that extends common `IRProcessingPass`) to transform an `IR` element to another `IR` element
- modifies `PassManager` to recognize such _mini passes_ and treat them in a special way - by using `MiniIRPass.compile`
- `MiniIRPass.compile` is using `IR.mapExpressions` to traverse the `IR` - alternative approach [withNewChildren](1abc70d33c) rejected for now, see _future work_ for details
- unlike _mega passes_ `IRMiniPass.compile` **does not  recursively** traverse, but with 0964711ba9 it invokes each _mini pass_ at constant stack depth - way better for profiling
- `MiniIRPass.prepare` _works on edges_ since ffd27dfe9b - there is `IRMiniPass prepare(parent, child)` to collect information while pre-order traversing from a particular `IR` parent to a particular `IR` child
- `PassManager` rewritten to group _subsequent mini passes_ together by `MiniIRPass.combine` and really and traverse the `IR` just once - done in 2736a76
- converted to _mini pass_: `LambdaShorthandToLambda`, `OperatorToFunction`, `SectionsToBinOp` and `TailCall`
- tested for 1:1 compatibility by [converting original code to test code](f54ba6d162) and _comparing `IR` produced by old and new_ implementations
2024-10-15 19:26:08 +00:00
AdRiley
07d0015f2f
Set stack size to 16M (#11327)
* Set stack size to 16M

* Add -Xss16M
2024-10-15 18:11:35 +03:00
Radosław Waśko
21bd05f318
Add Meta.engine_version (#11320)
Numerous times I wasn't sure when running the IDE if I'm running the bundled engine or a development build. Usually this depends on if I launch the standalone IDE or use a development build of project-manager.

Still it's not always obvious, and making sure that your IDE is running the right engine version is very often the first step when debugging issues with e.g. engine changes not showing up properly.

Thus I thought it may be worth to add this method (currently hidden to users in component browser by marking as `PRIVATE`, one has to type it in manually):
![image](https://github.com/user-attachments/assets/13af3df4-49ff-49bb-9b19-601258a8ca02)

I think it should be a helpful tool for debugging.
2024-10-15 13:22:55 +00:00
Pavel Marek
2285b7d752
Fix invocation of single benchmarks in std-benchmarks (#11321)
# Important Notes
- `std-benchmarks/bench`, which is invoked on the CI, is not affected. It still works as expected.
- Updated list of all projects in `enso` aggregate in c1d43b3ebf
2024-10-15 12:17:17 +00:00
Sergei Garin
3711b25fa7
Allow updating local assets (#11314)
* Allow updating local assets

* Update shim

* Fix duplicate upload

* Manually invalidate once we upload file

* Fix queryKey

* upd prettier ignore

* revert prettier ignore
2024-10-15 14:32:52 +03:00
Pavel Marek
4a2e522935
Show only fields from current atom constructor in the debugger (#11217)
Debugger shows only fields of the current atom constructor: (internal members shown in gray, "public" members shown in bold purple)
![image](https://github.com/user-attachments/assets/21815296-c8aa-4ea2-ae7b-6feac78a221f)

(Note that `static_method` is not displayed as a member of `My_Type` - not in scope of this PR)

# Important Notes
The *interop* contract for `Atom` is changed as follows:
- Members are all methods and fields of the current constructor.
- All methods are internal members.
- If the constructor is project-private, fields are internal members.
- All the members are both readable, and invocable.
- Fields are field getters, that is, they are just methods.
- Fields are not invocable
- Constructors and static methods are **not** members of an atom. They should be members of the type.
- Note that methods used to be atom members before #9692
2024-10-15 10:04:07 +00:00
Radosław Waśko
6eeeb95125
Suggest running Extra Tests if relevant changes are made in a PR (#11315) 2024-10-15 09:28:46 +00:00
Hubert Plociniczak
e1da31b2a8
Ensure wrapper projects are listed in aggregates (#11323)
* Ensure wrapper projects are listed in aggregates

Previously, doing clean to deal with weird modularization failures would
have no effect.

* And updated formatting

* More projects

As cherry-picked from https://github.com/enso-org/enso/pull/11321
2024-10-14 20:46:08 +02:00
Adam Obuchowicz
d1ee7fadce
Remove download font scripts (#11317)
The fonts were added (by mistake) anyway, but they are rarely changed, and the scripts add unnecessary complexity.

This also fixes some post- #11287 problems.
2024-10-14 16:11:42 +00:00
Radosław Waśko
244effde0c
Enable audit logs for Snowflake backend (#11306)
- Closes #11292
- Tries to fix #11300
2024-10-14 14:30:42 +00:00
AdRiley
03369b9273
Refactor Dialect_Flag (#11273)
* Add property flags

* Update post initial design review

* Add more tests

* Fix tests

* Remove .LCK

* Fix
2024-10-14 10:32:19 +03:00
Adam Obuchowicz
4a249688e8
Unify Frontend App (#11287)
Fixes #10668
Fixes #8484

Summary of changes:
* `gui2` and `dashboard` are merged to `gui` directory. Various configs were merged (package.json, playwrigth, TS...). The src and e2e directories are split to `dashboard` and `project-view` for now.
* E2E tests run two servers on different ports. The tests are organized in projects. This is also to be changed soon, as we plan to [use better mocking in GUI/ProjectView](#9726)
* ESlint configs were merged to central `eslint.config.mjs`, and that file was moved to repository root. We kept the dashboard lints, but they can be relaxed. The dashboard code was changed to meet GUI lints.
* Also, the versions of linter plugins were bumped, and code fixed.
* The ide-desktop/client no longer has `dashboard` dependency - the only type used there was moved to common package.
* `common` package moved to `app`.
2024-10-11 18:23:02 +00:00
Jaroslav Tulach
204b37c6c3
Special handling of last signature in a block (#11289) 2024-10-11 15:38:47 +02:00
Pavel Marek
d9b3c75c5d
Remove TailCall.TailPosition.NotTail metadata. (#11303)
`TailCall.TailPosition.NotTail` metadata is attached to every `IR` element that is not in tail position. Which is true for most IR elements. This is inefficient and unnecessary. This PR removes this metadata and keeps only `TailCall.TailPosition.Tail`. This removes few bytes from `MetadataStorage` from most of IR elements

# Important Notes
To check whether `ir` element is not in tail position, we used to check it with something like this `ir.getMetadata(TailCall) == Some(TailCall.TailPosition.NotTail)`, now, we simply can do it with `ir.getMetadata(TailCall) == None`
2024-10-11 11:17:12 +00:00
Jaroslav Tulach
eccc3a0d1e
Avoid JavaScript and Python in Enso native launcher (#11279) 2024-10-11 06:22:28 +02:00
Radosław Waśko
2843dcbf4a
When connecting to a Postgres database through a datalink stored on Enso Cloud, its asset ID is included in the audit logs (#11291)
- Closes #9869
2024-10-10 15:18:47 +00:00
Sergei Garin
239a6803bf
Fix updating conflicting project (#11111)
This PR fixes a bug, when uploading a duplicate project to local backend threw a crash on windows.


https://github.com/user-attachments/assets/fe1bd7a7-d840-4bb1-b3f6-e0beec70fcd4
I found a bug when we upload a project to a remote backend, it throws a 500 error. Notified @PabloBuchu about that.

Also, @hubertp proved that the ProjectManager always duplicates a project no matter which option we choose(either "update" or "rename“).  So I removed the "update" button if we upload a project locally
2024-10-10 14:51:59 +00:00
Radosław Waśko
3458fe4fe1
Accessing and modifying description and labels of Enso Cloud assets (#11255)
- Closes #11227
- Additionally, it should fix #11278 by ensuring that every scheduled message goes to the desired endpoint, by splitting each batch by endpoint.
2024-10-10 12:11:10 +00:00
Hubert Plociniczak
468b643aad
Add retries to Data.read (#11269)
* Add retries to HTTP Get requests

A quick solution to random network failures for GET HTTP requests.
Should reduce the number of IOExceptions that users see while fetching
data.

* Use homemade retry logic for http requests

* Add retries to whole Data.read

Previously, we added retries only to fetch HTTP_Request. That was
insufficient as intermittent errors might happen while reading body's
stream.

Enhanced our simple server's crash endpoint to allow for different kind
of failures as well as simulate random failures.

* Remove retries from Java

Increased the scope of retries in the previous commit.

* nit

* Address PR comments

* PR comments

* Remove builtin
2024-10-10 10:25:16 +02:00
Radosław Waśko
669ac97d9e
Clean up credentials file after E2E job. (#11285)
- While #11255 fixes the root cause of #11278, this PR fixes what triggered it - since #11198 our runners were keeping `~/.enso/credentials` file between runs, meaning that some tests were unexpectedly running in 'authenticated' mode. This PR cleans up this file to avoid that.
2024-10-09 15:02:54 +00:00
Hubert Plociniczak
7f5c0e6ef2
Improve CI feedback when directory is missing (#10608)
* Improve CI feedback when directory is missing

As motivated by https://github.com/enso-org/enso/actions/runs/10006095519/job/27658100782?pr=10577#step:7:2277

* update codeowners

* nit
2024-10-09 16:58:42 +02:00
Adam Obuchowicz
7c5124094e
Add lint for jsdocs in GUI2 (#11234)
Part of #10526 (without TS structures: interfaces, type aliases etc.)

Added lints checking for jsdocs presence (and their format) and generated stub with TODO for missing ones. Now only 344 docs are missing 🙃

# Important Notes
Many changes are just automated, but they are in the middle commit. So recommended way of reviewing is review `Add jsdoc to eslint config` for checking config changes, and then `Some of the missing documentation and fixes`.
2024-10-09 12:26:56 +00:00
Dmitry Bushev
78993a0d1a
Run node in a different execution environment (#11173)
close #10719

Changelog:
- add: optional `expressionConfigs` parameter to the `executionContext/recompute` request
- update: `IdExecutionInstrument` allowing to run a single node in a specified execution environment
- refactor: move tests related to the recompute request to a separate test suite. Otherwise the `RuntimeServerTest` is becoming too bloated

# Important Notes
The updated `executionContext/recompute` request.

```typescript
interface ExecutionContextRecomputeParameters {
/** The execution context identifier. */
contextId: ContextId;

/** The expressions that will be invalidated before the execution.
*
*  Only the provided expression ids are invalidated excluding the dependencies.
*/
invalidatedExpressions?: "all" | ExpressionId[];

/** The execution environment that will be used in the execution. */
executionEnvironment?: ExecutionEnvironment;

/** The execution configurations for particular expressions.
*
*  The provided expressions will be invalidated from the cache with the
*  dependencies. The result of the execution will stay in the cache until the
*  cache is invalidated by editing the node or other means.
*/
expressionConfigs?: ExpressionConfig[];
}

interface ExpressionConfig {
/** The expression identifier. */
expressionId: ExpressionId;
/** The execution environment that should be used to run this expression. */
executionEnvironment?: ExecutionEnvironment;
}
```

#### Use cases

- to re-run a single node without re-running the dependent nodes (subtree), put the node id in the `invalidatedExpressions` parameter.
- to re-run a node with dependent nodes (subtree), put the node id in the `expressionConfigs` parameter with empty `executionEnvironment`
- to re-run a node in a different execution environment, put the node  id in the `expressionConfigs` and specify the `executionEnvieronment`
2024-10-09 12:09:45 +00:00
James Dunkerley
9d9c03eeaa
Some Additional Error checks for Enso_File (#11275)
- Check the validity of `Enso_Path` when making an `Enso_File`.
- Check that not an error when fetching asset reference.
- Added display text for `Enso_User`.

![image](https://github.com/user-attachments/assets/77e4bfe8-d4f4-4de5-9ccc-a7c6519d4759)

Check for `to_text` use within Enso_Cloud.
2024-10-09 11:12:25 +00:00
somebody1234
7a00e6ef26
Dashboard improvements (from 24 September 2024 + 30 September 2024) (#11219)
- Close https://github.com/enso-org/cloud-v2/issues/1508
- ⚠️ Labels modal - add selection indicator where user can (de)select multiple labels
- Checkboxes currently still cause the dialog to (incorrectly) close
- Edit datalink -> select enso secret -> options are too narrow for enso path. Strip enso://Users(Teams), if possible make the options list longer on the left side
- Added (or rather, re-enabled via CSS) horizontal scroll instead
- Make Versions, Sessions, Settings as tabs similar to documentation panel in graph editor
- Edit description in context menu should open asset panel with description input active
- Edit secret should be moved to asset panel (same like datalink)
- Dim background when "edit description", "edit secret", or "edit datalink" are clicked/triggered via shortcut to highlight input
- Hide unused (= no backend support) columns and icons: accessed by projects and accessed data
- *Partial* frontend fixes for https://github.com/enso-org/cloud-v2/issues/1529
- (1) Fix settings title being horizontally centered and split on multiple lines
- (2)  backend issue
- (3)  out of scope
- (4)  backend issue
- (5)  out of scope
- (6)  out of scope
- (7)  backend issue
- (8)  already fixed in #11126
- (9)  out of scope (potentially requires a way to trigger a tooltip on a disabled button)
- (10)  out of scope
- (11)  out of scope
- (12)  out of scope
- (13) URL encode `enso://` URLs in "copy as path"
- (14) Double click on datalink make asset open and close (not sure if this has already been fixed)
- (15) Clicking anywhere on Asset Panel no longer deselects assets (not sure if this has already been fixed)
- (16)  addressed in #11268
- (17) Make list of labels in Asset Panel (right sidebar) horizontal instead of (incorrectly) vertical
- (18) Only show "Billing" settings tab for organization admins
- Use "workspace" instead of "network" icon for project tabs
- Other fixes:
- Fix Asset Panel (right sidebar) not being able to be toggled off if it is temporarily open (when triggered from editing description, or editing secret, or editing datalink)
- Make "cancel" and "reset" buttons default to outline variant, instead of ghost variant
- Fix style of dropdown
- Change Datalink editor dialog so that object keys are above inputs, not beside them. This gives inputs much more horizontal space for children of deeply nested objects.

Issues left to fix:
- Checkboxes currently still cause the dialog to (incorrectly) close
- "Edit description" actions etc. do not properly focus inputs

Issues left to do (out of scope):
- Show username of user currently using a project (possibly as tooltip?) if the project is currently disabled.
- Dropdown and autocomplete entries should be in their own dialog, so that they can escape the parent dialog if they are too long

# Important Notes
None
2024-10-09 09:10:49 +00:00
Dmitry Bushev
80317dc950
Eliminate scala.Some allocations (#11259)
close #10973

Changelog:
- update: `IR` interface add nullable `identifiedLocation()`, and create a `location()` getter
- update: IR nodes contain nullable `identifiedLocation` field
- update: persistence of `IdentifiedLocation` supports nullable values

# Important Notes
80% less allocations of `scala.Some`

#### Before

![2024-10-07-131421_1165x185_scrot](https://github.com/user-attachments/assets/1d84ef19-652c-4bef-8d7f-eed63a7b6b50)

#### After

![2024-10-07-131513_1163x186_scrot](https://github.com/user-attachments/assets/3d71a932-7861-4a97-bc84-eafc530dcce3)
2024-10-09 08:54:37 +00:00
AdRiley
26c711ab14
Enable SQLServer select columns (#11270)
* Enable SQLServer select columns

* Code review feedback

* Code Review feedback
2024-10-08 22:57:36 +03:00
marthasharkey
1ae646419f
add method for xml doc drilldown (#11258) 2024-10-08 15:23:41 +00:00
Nikita Pekin
6e11e6ab9a
Fix invalid ${rust-jni.out} in Nix flake on non-Linux systems (#11264)
`fenix.packages.${system}.minimal` is a `toolchain` not a `derivation`, so Nix `devShell`s are broken on non-linux systems because the `devShell` expects `${rust-jni.out}` to evaluate to a path. Replacing `fenix.packages.${system}.minimal` with `fenix.packages.${system}.minimal.toolchain` fixes this because `.toolchain` is a `derivation` which has an `.out` field.

Also formats the `flake.nix` since it contains a mixture of tabs and spaces. The two commits are separated for ease-of-review.

cc @somebody1234
2024-10-07 15:50:13 +00:00
Gregory Michael Travis
47bf591b5a
Add format parameter to Float.parse (#11229) 2024-10-07 15:43:57 +00:00
Gregory Michael Travis
cce50fab3a
Extend the range of int/float arguments to outside the range of Java long, in ceil, floor, and truncate (#11135) 2024-10-07 11:36:17 -04:00
AdRiley
d0f0770ef4
Add DB_Table feature flags (#11163)
* Refactor

* Fix

* More fixes

* Fixes

* Fix

* Fix imports

* Fix

* Fix

* Fix Test

* Fix

* Fix

* is_feature_supported API

* Add SQLServer Tests

* SQLServer Select_columns functional

* SQLServer Filter functional

* Aggregate

* order feature

* Remove old order flag

* Join

* Union

* Distinct

* Text_cleanse

* Remove Nothing tests

* add_row_number and integration

* Make_table_from

* Feature flag failing tests

* Column_operations

* Set

* date_time off

* Conversion

* More flags

* Merge

* Expressions

* TakeDrop

* Missing

* Cross_join

* More flags

* Temporaily disable

* Fix merge issue

* Update other dbs

* Code review changes

* Code review changes

* Code revoew changes

* Fix

* Fix

* Fix

* Fix

* Fix

* Code review fix

* Typo

* Code review changes

* Casing

* Add comments

* Add comments

* Update

* Fix

* Fix imports
2024-10-07 16:05:09 +01:00
Jaroslav Tulach
c85363f425
Compute the exception message sooner than context is closed (#11241) 2024-10-07 14:26:40 +02:00
Hubert Plociniczak
9429a4540a
Do not run visualizations on InterruptedException (#11250)
* Do not run visualizations on InterruptException

There is no point in running visualization for the expression value that
is InterruptedException. The latter is likely to bubble up the exception
or create one that will be confusing to the user.

Closes #11243 and partially addresses some of the symptomes of #11084.

* Add a test for confusing visualization failures

Previously a visualization failure would be reported:
```
Method `+` of type sleep interrupted could not be found.
```

* PR review

Nit
2024-10-07 12:29:17 +02:00
AdRiley
a4f800dbf0
Add Coalesce tests and fix In-Memory version (#11242)
Add tests for coalesce and fix the In-memory version

# Important Notes
The distribution/lib/Standard/Database/0.0.0-dev/src/Internal/IR/From_Spec.enso change isn't actually needed for this MR as I switched away from using literal tables for these tests as trying to mix 2 literal tables doesn't look to be supported. The change I added will let any future developers know that.
2024-10-07 09:02:11 +00:00
Radosław Waśko
f1c2015970
Log failure to refresh the Cloud tokens (#11261)
- Trying to debug #11145, I'm adding logging of when token is refreshed and when that fails, with possibly some details that are included in the response.
- Also, the expired case is now narrowed down from 400-499 to just 401 as that's what [OAuth spec](https://www.rfc-editor.org/rfc/rfc6750.html#section-3.1) says is the correct status code for expired token. Other errors are reported as unexpected error, with more details.
2024-10-07 08:30:41 +00:00
James Dunkerley
fd72ab7052
Remove some catch alls (#11254)
- Allow Interrupted Exceptions to float out of the web requests.
- Use `Type_Error` rather than Any when catching auto scoping resolving.
- Rename `Java_Exception` to `JException`
2024-10-04 21:32:56 +00:00
Radosław Waśko
35e86e42b9
Run only relevant tests in the Cloud workflow (#11249)
- Yet another part of #11220.
- We are only running relevant tests that have something to do with cloud: Base, Table and AWS. We can add more if needed.
- Also we try to ensure multiple line message in the failure report annotation (followup of #10821).
2024-10-04 11:01:23 +00:00
marthasharkey
37621a9f90
Full white space rendering and links fix (#11238) 2024-10-03 16:01:26 +00:00
Adam Obuchowicz
c7cc5f0be0
Run integration in CI. (#11198)
Added a step for packaging IDE which runs integration tests.

This is an additional step, not a job, because here I'm sure the package will exist and won't be built unnecessarily twice. Technically I could make a job downloading it from GH action, but didn't want to invest time to rust scripts. Once Bazel will be fully introduced, the integration test will be improved.

# Important Notes
Fixes #8487
2024-10-03 13:41:22 +00:00
James Dunkerley
a2c853c6e3
Small Fixes (#11236)
- Left over from the last PR.
- Widget for `headers` in `URL.fetch` extension.
- Fix `Left` and Right` to allow columns or expressions in it.
2024-10-02 18:57:02 +00:00
Radosław Waśko
0399a4570d
Resolving ~ in Enso asset paths to user home (#11235)
- Closes #11226
2024-10-02 18:49:43 +00:00
Radosław Waśko
0302670092
Reporting Enso test failures as annotations on the PR (#10821) 2024-10-02 16:01:28 +00:00
marthasharkey
5f1d32f1aa
Scatterplot: Add filter down node functionality (#11162) 2024-10-02 13:35:11 +00:00
marthasharkey
7b63b66453
use new icon for adding new nodes (#11213) 2024-10-02 09:49:14 +00:00
Jaroslav Tulach
837a4d53b5
Avoid NPE on missing @anno name (#11224) 2024-10-02 11:15:41 +02:00
Radosław Waśko
a23b66925d
Fix cloud tests by using correct config from the repo (#11223)
- Part of #11220
2024-10-02 08:05:08 +00:00
Hubert Plociniczak
ad53c824ab
Don't send expression updates about interrupts (#11218)
There is no value in sending expression updates involving interrupts to the user:
![Screenshot from 2024-09-30 14-47-17-2](https://github.com/user-attachments/assets/78fca5bf-085d-4c1c-99fb-0acb5f0a31a3)

Adding more logging information to see how aborts affect execution.

Related to #11084.
2024-10-01 19:43:06 +00:00
Gregory Michael Travis
3a22147eaf
Add format parameter to Decimal.parse (#11205) 2024-10-01 18:55:40 +00:00