Commit Graph

506 Commits

Author SHA1 Message Date
AdRiley
06327f8fde
Add statistic product (#10122)
Add Statistic.Product

![image](https://github.com/enso-org/enso/assets/1720119/f7fc7bb5-9efe-4dbe-9150-cd9e5101c553)
2024-05-31 09:29:52 +00:00
AdRiley
28bd5c522b
Add named_pattern and make it usable in Text_Replace (#10040)
Creates a new type Named_Expression and allows it to be used in text_replace

![image](https://github.com/enso-org/enso/assets/1720119/673a62a1-1ce5-4e1f-8289-0fa10e87b9de)

![image](https://github.com/enso-org/enso/assets/1720119/2f555e9b-d8c5-41e3-8000-d959d5818666)

![image](https://github.com/enso-org/enso/assets/1720119/d2ae2b0f-5bc4-4e9e-a391-402b58ee72d5)

![image](https://github.com/enso-org/enso/assets/1720119/3bf0547f-c2c7-4987-a45e-d922be2b0bae)
2024-05-27 13:01:26 +00:00
James Dunkerley
200da1ad50
Remove old GUI1 ICONs and mapping function (#10086)
* Remove old GUI1 ICONs and mapping:

```
  dataframe_clean: 'table_clean',
  dataframe_map_row: 'map_row',
  dataframe_map_column: 'column_add',
  dataframes_join: 'join2-1',
  dataframes_union: 'union',
  sigma: 'transform4',
  io: 'in_out',
  date_and_time: 'time',
  spatial: 'location',
  predictive: 'predict',
  machine_learning: 'robot',
  split_text: 'split',
```

* Linter.

* Linter (2).

* Sort out some Returns:

* Sort out some ? Example
2024-05-27 11:17:21 +01:00
James Dunkerley
d8059fd22c
Some tweaks following Steve's testings (#10042)
- Add ranged number widget to `at` and `get`.
- Add defaults to `at` and `get` picking the first item.
- PRIVATE on various Excel_Workbook methods. It still works like a connection but not shown in CB.
2024-05-27 09:04:29 +00:00
AdRiley
084be279c5
Fix input_number icons (#10066)
icon is called input_number not number_input

![image](https://github.com/enso-org/enso/assets/1720119/c68790d3-05ad-4d17-8423-df3634fff29a)
2024-05-24 11:30:06 +00:00
Adam Obuchowicz
8ac0ee41c6
Allow picking non-existing files in write component (#10047)
Fixes #9984

Added a flag to `File_Browse` widget configuration specifying if we require an existing file. Also, the FileBrowserWidget will match against all "Writable_File", also displaying Save dialog.
2024-05-24 09:21:32 +00:00
Radosław Waśko
9d75f79ff9
Cloud Integration updates: renames in file metadata structure, remove path resolver workaround, partial fix for datalink resolver issue (#10050)
- Supersedes #9966 as I wanted to test these changes in one go.
- Fixes #10037 caused by lack of CI check and my oversight (forgot to run full tests after a minor change).
- Fixes a regression after [file metadata fields were renamed](c09d856ac8 (diff-9f59b6a0ee3155efecdc70c1ea0c90ab5cde00b5623d84363118b1793f941c46R2037)).
- Fixes handling of creating new datalinks and using them after cache was cleared (e.g. workflow restart).
- This was caused by troubles with path resolver.
- The fix addresses the most common issue and adds a test for it (test flushes the caches to ensure path resolver is used instead of the cached value).
- Some related issues were discovered on the cloud side, tracked by https://github.com/enso-org/cloud-v2/issues/1252
2024-05-23 16:05:48 +00:00
Jaroslav Tulach
16c1b74218
Enso Library Feature to execute (a bit of) Base_Tests (#9997) 2024-05-23 08:20:19 +02:00
marthasharkey
517299bb09
Add new aliases for take and drop methods (#10022)
Adds the aliases 'keep' and 'remove' to take/drop methods
2024-05-22 08:02:48 +00:00
James Dunkerley
2677e556e8
Align headers in doc comments... (#10025)
- Always use `Arguments:` for parameters or fields.
- Always use `> Example` for examples.
- Align sections behind `!` or `?` headers.
- Whitespace fixes.
2024-05-21 19:44:58 +00:00
James Dunkerley
31dd5944d7
Some fixes to make linter happy (#10004)
- A few import orderings.
- Missing doc comments on Text_Cleanse.
- Indent correction.
2024-05-20 16:15:11 +00:00
AdRiley
c7476c10f3
Add data cleanse component (#9879)
* Signature

* More API work

* Red

* Still red

* Green

* Red

* Green

* Red

* Green

* Refactor

* Refactor

* Refactor

* Red

* Green

* Red

* Green

* Red

* Green

* More tests

* non-ascii

* Ordering tests

* Remove tabs

* Numbers and Letters

* Changelog.md

* Add documentation

* Tests for non-text columns

* Add punctuation

* Add symbols

* Fix

* Refactor

* Refactor

* Move to base

* Fix

* Start of in-db tests. Not working

* DB versions

* Update widget

* Fix widgets

* Move tests to base

* Code review changes
2024-05-20 09:33:18 +01:00
James Dunkerley
3b29de313d
Update ICONs, linting fixes and add select_columns ALIASes. (#9985)
- Linting fixes.
- Add ICONs to all functions missing them.
- Add ALIASes for select columns.
2024-05-17 19:24:17 +00:00
GregoryTravis
4d49b00375
Combine builders for Vector.build and Vector.new_builder (#9922)
We have decided to keep the old new_builder, and to combine the new and old builders into one builder.
2024-05-17 16:18:47 +00:00
AdRiley
5c9ee94812
Make parse_fast_json private (#9978)
Micro MR to make deprecated method private
2024-05-17 15:47:11 +00:00
Radosław Waśko
38ad7b0afa
Creating datalinks from code (#9957)
- Closes #9673
- Adds the ability to save an existing Postgres connection as a datalink into the Enso Cloud, automatically promoting plain-text passwords into a Secret.
- Fixes dataflow error propagation in `JS_Object.from_pairs`.
2024-05-16 13:29:41 +00:00
Radosław Waśko
a2481036e6
Prepare for the Path resolver change to GET (#9958)
- Prepares for the Cloud API change, together with a fallback for the old API to avoid problems during migration.
- This PR should be merged before the https://github.com/enso-org/cloud-v2/pull/1236 PR is _deployed_.
2024-05-15 13:00:32 +00:00
Radosław Waśko
5f0a16c87c
Audit Logs for Postgres connections opened through a data link (#9873)
- Closes #9599
- Implemented API for sending audit logs to the cloud on a background thread.
- If the Postgres connection is opened through a datalink, its internal JDBC connection is replaced by a wrapper that reports executed queries to the audit log.
- Also introduces `EnsoMeta` - a helper Java class that can be used in our helper libraries to access Enso types.
- I have replaced the common pattern scattered throughout the codebase with calls to this 'library' to avoid repetitive code.
- Refactored `Table.display` to share code between in-memory and DB - it was needed as the function stopped working for `DB_Table` after adding making the `Table` constructor `private`.
- Clearer error when reading a SQLite database from a remote file (tells the user to download it first).
- Follow up - correlate asset id of the data link:
#9869
- Follow up - include project name (once bug is fixed):
#9875
- Some problems/improvements of the audit log:
- The audit log system is not yet ready for high throughput of logs
#9870
- The logs may be lost if `System.exit` is used
#9871
2024-05-11 08:54:33 +00:00
James Dunkerley
d97754da17
Error messages for rename_columns and Vector.duplicates (#9917)
- Improve error message for `rename_columns`.
- Add `length` to `Set` and `Map`.
- Add `duplicates` to `Vector` (and `Array`).

![image](https://github.com/enso-org/enso/assets/4699705/623df253-52e8-4bdc-a69c-ac8dc3ca594e)
2024-05-10 17:43:50 +00:00
Radosław Waśko
1d8fc1bf6e
Update endpoint name after it was updated in the Cloud (#9912)
Micro PR updating our API integration to the Cloud-side changes.
2024-05-09 15:17:19 +00:00
AdRiley
e25ec96aaa
Add table running variance skew sd and kurtosis (#9854)
Adds support for Variance, Skew, Standard Deviation and Kurtosis to Table.Running.
2024-05-09 08:45:29 +00:00
James Dunkerley
ce360f9002
Fix the group colours for consistent colouring. (#9893)
Fix the group colour to a fixed set.

![image](https://github.com/enso-org/enso/assets/4699705/476046ce-0ce3-4f98-9a75-a51f2e55f4e6)
2024-05-08 16:52:42 +00:00
GregoryTravis
a2c36def26
Add Vector.build_multiple and handle dataflow error and warning propagation in build and build_multiple (#9766) 2024-05-07 19:30:21 +00:00
James Dunkerley
c0fd6eed2d
Small tweaks from Ned, Mark, Cass and Steve's feedback. (#9859)
- Rename `.info` to `.column_info`.
- Rename `.select_by_type` to `.select_columns_by_type`. (to be merged into `select_columns` functionality)
- Rename `.remove_by_type` to `.remove_columns_by_type`. (to be merged into `remove_columns` functionality)
- Adding "field" `ALIAS`es.
- Making `Table.Value` private (and hence hiding `java_table`).
- Making `Column.Value` private (and hence hiding `java_column`, added method to `Storage` to allow tests to work).
- Making `DB_Table.Value` private (and hence hiding `connection`, `context` and `internal_columns`).
- Making `DB_Column.Value` private (and hence hiding `connection`, `sql_type_reference`, `expression` and `context`).
- Add widgets for everywhere `Filter_Condition` used (fix missing entry for lower, can't use auto-scoping).
![image](https://github.com/enso-org/enso/assets/4699705/bad7d5c8-cd01-4384-a1c8-dbcd7a5ad92b)
- Fix for widgets in `Simple_Calculation`: `date_add` and `format`.
![image](https://github.com/enso-org/enso/assets/4699705/14eeef2e-5069-4a88-8b68-eb675497addf)
![image](https://github.com/enso-org/enso/assets/4699705/75d4e097-f171-4bf6-a27e-f9d396c44a92)
- Fix for nested `Row`, `Column` and `DB_Column` rendering in Table Viz.
![image](https://github.com/enso-org/enso/assets/4699705/8c50c519-7c34-4228-bab7-be938a6ce1bf)
![image](https://github.com/enso-org/enso/assets/4699705/c83ad301-f31e-451d-a8be-b022e1456b4e)

# Important Notes
- Can't use `.` to access a private constructor field when mapping, have to use a lambda.
- We should agree convention for private backing fields which are wrapped by a public getter (went with `internal_` for this pass).
2024-05-06 13:49:12 +00:00
James Dunkerley
d2e6ff260e
Restructure SQLite_Details. (#9832)
```
type SQLite_Details
SQLite location:File|In_Memory

type In_Memory
```
to
```
type SQLite
From_File location:File

In_Memory
```

# Important Notes
Splits the In-Memory entry for Database Connect but still works nicely.

![image](https://github.com/enso-org/enso/assets/4699705/ec798ce0-9f41-4903-a2fd-722a9e37743c)

![image](https://github.com/enso-org/enso/assets/4699705/f233b055-893e-4c56-a23d-562e982560f6)
2024-05-01 22:15:41 +00:00
James Dunkerley
4d6d6f239c
Handle URL encoding automatically in query string. (#9823)
A small fix to automatically encode the query string.
Attaches a warning if needed.

![image](https://github.com/enso-org/enso/assets/4699705/032bdb59-6896-46c0-b970-f5a542cc6adf)

![image](https://github.com/enso-org/enso/assets/4699705/6b2075b9-3c98-4de2-8a34-c860ecd65d0c)
2024-04-30 22:03:46 +00:00
Radosław Waśko
f87c75a3da
Prepare for Cloud Environment variable name change (#9821)
Changing `ENSO_PROJECT_DIRECTORY_PATH` to `ENSO_CLOUD_PROJECT_DIRECTORY_PATH`.
2024-04-30 16:02:12 +00:00
Pavel Marek
660c5e7a9d
Atom constructors can be private (#9692)
Closes #8836.

Atom constructors can be declared as private (project-private). project-private constructors can be called only from the same project. See the encapsulation.md docs for more info.

---------

Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
Co-authored-by: Kaz Wesley <kaz@lambdaverse.org>
2024-04-29 14:43:18 +02:00
Radosław Waśko
16e1999040
Fix copy_to/move_to edge case when source and target are the same (#9798)
- Adds tests for an edge case where the source and target of copy/move operation are the same.
- And fixes the behaviour to be as expected.
2024-04-29 12:29:07 +00:00
Radosław Waśko
0c989c6938
Writing to Data Links (#9750)
- Closes #9292
- and also closes #9522
2024-04-25 17:55:49 +00:00
GregoryTravis
b150b091b0
Convert Vector.new_builder to Vector.build in Standard.Base (#9754)
* Vector.build

* new_builder deprecated

* convert one

* convert vector_spec

* limit param

* convert vector

* docs

* changelog

* wip

* rename limit to capacity

* wip

* review

* append comments

* review

* wip

* wip

* fix tests

* docs

* wip

* wip

* fix filter warning propagation

* doc weird dependencies

* fix tests

* cleanup

* check builder result
2024-04-25 10:38:29 -04:00
Jaroslav Tulach
0d495ffd97
Make conversion of double to BigDecimal exact (#9740)
Resolves #9607 by computing `Number.hash` by converting given number to `Float` first and then computing the hash. Also the conversion from `Float.to Decimal` is exact - done via `new BigDecimal(double)`. There is `Decimal.new` that handles the user-friendly conversion. However as a result `Decimal.from 2.1 != Decimal.new 2.1` - that's the only way to ensure consistency between hash code and conversions.
2024-04-25 11:22:50 +00:00
James Dunkerley
fb9cf38914
Excel_Workbook.read_many (#9759)
- Some minor linting fixes.
- Adjust `headers` parameter so a dedicated type.
![image](https://github.com/enso-org/enso/assets/4699705/989f464d-df95-410e-a03b-36661f1c4a37)
- Fix bug with `read` on an `Excel_Workbook` so error handled more gracefully and not panicking to UI.
![image](https://github.com/enso-org/enso/assets/4699705/23b4575f-daad-4719-a5cc-30d064bd7f7a)
- Fix bug when writing to a file with an `Excel_Format` with an invalid extension which was causing a panic.
![image](https://github.com/enso-org/enso/assets/4699705/dc0e055c-c1b6-482f-b129-eb69f6554d72)
- Add `read_many` to `Excel_Workbook` allowing reading more than one sheet at a time.
2024-04-24 13:16:44 +00:00
Jaroslav Tulach
ff62c1e25f
Access non-self arguments when widget annotations are computed (#9410) 2024-04-24 03:58:32 +02:00
AdRiley
6f3e649e7a
Make errors propagate out of each (#9742)
Make errors propagate out of each
2024-04-23 17:14:54 +00:00
AdRiley
4a97bfa31f
Add table running functionality for Sum, Mean, Min, Max. (#9577)
* Add Table.Running

* Code Review fixes

* Code Review changes

* Change null handling
2024-04-23 09:45:43 +01:00
GregoryTravis
bc61d686c1
Integer.to_decimal and Float.to_decimal (#9716) 2024-04-23 08:06:31 +00:00
GregoryTravis
681df82697
Implement Vector.build: implementation and some tests (#9725)
* Vector.build

* new_builder deprecated

* convert one

* convert vector_spec

* limit param

* convert vector

* docs

* changelog

* rename limit to capacity

* review

* append comments

* review
2024-04-18 17:52:48 -04:00
James Dunkerley
16126a0c71
Working through various issues from OKR / blog use. (#9739)
- Fixed `list_directory` so that it works with defaults and uses the `Folder_Browse`.
![image](https://github.com/enso-org/enso/assets/4699705/cf8eb187-a1fe-4eb7-a017-e7bb40e30e12)
- Adjusted types of `order_by` in `Aggregate_Column.First` and `Aggregate_Column.Last` so supports single `Sort_Column`.
- Adjusted types of `order_by` in `Aggregate_Column.First` and `Aggregate_Column.Last` so an empty Vector is valid (treated as Nothing).
- Adjusted `Count_Distinct` so if an empty set of columns then equivalent to `Count`.
- Use the multi-select widget for `columns` in `Count_Distinct`.
![image](https://github.com/enso-org/enso/assets/4699705/86e8f3c1-9cbe-4fb3-b6fd-43b937346082)
- Sort the widget for `order_by` in `First` / `Last`.
![image](https://github.com/enso-org/enso/assets/4699705/d624544c-6f4d-42ef-bb97-79b68aa60569)
- Bug fix for widgets with both tagValues and dynamic data. (Thanks @farmaazon).
- Use the multi-select for various things.
2024-04-18 21:50:20 +00:00
GregoryTravis
86ecd3e027
Add Decimal.floor, .ceil, and .trunc (#9694) 2024-04-17 18:42:38 +00:00
Radosław Waśko
34f3331561
Implement recursive argument for File.delete (#9719)
- Closes #8993
2024-04-17 16:53:45 +00:00
Radosław Waśko
41a39eb019
Fix Json.parse "" edge case and add a test for it (#9722)
- Fixes #9720
2024-04-17 08:48:48 +00:00
Radosław Waśko
fda41cbfd1
Writing Cloud files (#9686)
- Closes #9291
2024-04-16 14:01:03 +00:00
Pavel Marek
0d915d0344
Update to GraalVM 24.0.0 (#9647)
Update the GraalVM-related Maven packages from **23.1.2** to **24.0.0**.

# Important Notes
- Reverted workarounds mentioned in https://github.com/enso-org/enso/issues/9477

The language home search functionality in Truffle changed. It used to find the home for Enso from the class loader of `EnsoLanguage`:
https://github.com/oracle/graal/blob/graal-23.1.2/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/LanguageCache.java#L362

In 24.0.0, the language home is no longer found via class loader, but via system properties:
https://github.com/oracle/graal/blob/graal-24.0.0/truffle/src/com.oracle.truffle.polyglot/src/com/oracle/truffle/polyglot/LanguageCache.java#L353

I provided a "fix" for this by an explicit specification of the `org.graalvm.language.enso.home` system property in our launchers.
2024-04-12 17:01:49 +00:00
Radosław Waśko
bdda1830b7
Integrate Cloud path resolver (#9662)
- Closes #9363
- Cleans up the Cloud mock as it got a bit messy. It still implements the bare minimum to be able to test basic secret and auth handling logic 'offline' (added very simple path resolution, only handling the minimum set of cases for the tests to work).
- Adds first implementation of caching Cloud replies.
- Currently only caching the `Enso_User.current`. This is a simple one to cache because we do not expect it to ever change, so it can be safely cached for a long period of time (I chose 2h to make it still refresh from time to time while not being noticeable).
- We may try using this for caching other values in future PRs.
2024-04-12 13:03:09 +00:00
Pavel Marek
4dc7992ab5
Export works without import (#9683) 2024-04-12 14:23:34 +02:00
GregoryTravis
e3afa5561d
Add Decimal.round (#9672) 2024-04-11 15:47:50 +00:00
GregoryTravis
bf90e2e8a8
Attach warning when converting from Decimal to Integer (#9660) 2024-04-11 15:44:58 +00:00
GregoryTravis
676e989f7f
Add Decimal.min and .max (#9663)
* min and max

* changelog

* wip

* scale approach mostly works

* Revert "scale approach mostly works"

This reverts commit 88e6073f7a.

* review

* review

* return value type
2024-04-11 09:58:10 -04:00
GregoryTravis
9ac85b93b7
Decimal.parse and .format (#9637)
* to_text + tests

* locale note

* format

* parse

* parse examples

* wip

* type

* changelog
2024-04-11 09:29:59 -04:00