Commit Graph

982 Commits

Author SHA1 Message Date
AdRiley
a4f800dbf0
Add Coalesce tests and fix In-Memory version (#11242)
Add tests for coalesce and fix the In-memory version

# Important Notes
The distribution/lib/Standard/Database/0.0.0-dev/src/Internal/IR/From_Spec.enso change isn't actually needed for this MR as I switched away from using literal tables for these tests as trying to mix 2 literal tables doesn't look to be supported. The change I added will let any future developers know that.
2024-10-07 09:02:11 +00:00
James Dunkerley
fd72ab7052
Remove some catch alls (#11254)
- Allow Interrupted Exceptions to float out of the web requests.
- Use `Type_Error` rather than Any when catching auto scoping resolving.
- Rename `Java_Exception` to `JException`
2024-10-04 21:32:56 +00:00
Radosław Waśko
0399a4570d
Resolving ~ in Enso asset paths to user home (#11235)
- Closes #11226
2024-10-02 18:49:43 +00:00
Radosław Waśko
a23b66925d
Fix cloud tests by using correct config from the repo (#11223)
- Part of #11220
2024-10-02 08:05:08 +00:00
Gregory Michael Travis
3a22147eaf
Add format parameter to Decimal.parse (#11205) 2024-10-01 18:55:40 +00:00
James Dunkerley
6ea716f1b3
Widget for Database.connect (#11216)
- Use auto-scoping for Redshift, SQLServer and Snowflake.
![image](https://github.com/user-attachments/assets/2f5ff24a-44f4-4e87-909a-e064b8653511)
- Fix for widgets on Header functions.
![image](https://github.com/user-attachments/assets/4384efcf-a4da-48b1-b571-1167ca8d0134)
- Move `Snowflake_Details` to `Standard.Snowflake.Connection` namespace to make widgets work.
- Add widgets to `Snowflake_Details`.
![image](https://github.com/user-attachments/assets/b51d0126-a768-4f4a-9d87-c42c8e91e26b)
- Typo fix for SQLServer SPI.
- Change SQLServer port to be an Integer and added default.
- Reordered parameters on SQLServer **(potentially breaking change)**.
- Added widgets to SQLServer.
![image](https://github.com/user-attachments/assets/1c744da4-7913-4a87-9e64-fc10442a06eb)
- Added widget for JDBC options (as well as conversion from Vector to options).
![image](https://github.com/user-attachments/assets/4958b1e4-4cbc-43e3-8381-64e5ce7ea8ff)
- Added header alias to `use_first_row_as_names`.
- Added various aliases to `read` and `write`.
2024-10-01 08:43:03 +00:00
James Dunkerley
28bbc34257
Widget support for Data.fetch and Data.post. (#11199)
- Add `pretty` for `Date` and `Time`.
- Add constructors for `JS_Object` and `Dictionary` to the component browser.
- Add widgets to `Dictionary` methods.
![image](https://github.com/user-attachments/assets/4f6c58d5-9eb5-40e5-96c1-2e06e23051d0)
- Add conversion from Vector to Dictionary.
- Add `pair` method shorthand for `Pair.Value`.
- Created widget for `Header`.
- Added widgets for `Data.fetch` and `Data.post`.
- Added widgets for `Request_Body` constructors.
- Update the Forbidden Operation message to be friendlier.
![image](https://github.com/user-attachments/assets/eaac5def-a91f-450f-b814-d776311962e3)

Video before fixing Forbidden Message:

https://github.com/user-attachments/assets/f9c4bde4-3f0a-49f1-a3ca-a0aaa3219286
2024-09-27 18:08:12 +00:00
AdRiley
0f4fa42eea
Refactor unsupported db error messages (#11156)
* Refactor

* Fix

* More fixes

* Fixes

* Fix

* Fix imports

* Fix

* Fix

* Fix Test

* Fix

* Fix
2024-09-27 11:05:14 +03:00
Gregory Michael Travis
a0c3b901ac
Do not default the primary key in select_into_database_table (#11120) 2024-09-25 14:34:36 -04:00
Jaroslav Tulach
f37e50e87b
Propagate comparator warnings via Any.== (#11009)
Fixes #10679 by changing the return type of `EqualsXyzNode`s to `EqualsAndInfo`. This class holds the result of the comparation as well as any attached warnings. `EqualsBuiltinNode` then re-attaches the warnings, if there are any.
2024-09-25 09:48:34 +00:00
marthasharkey
a04ae1c9ad
scatterplot- on point click (#11142) 2024-09-24 09:14:39 +00:00
marthasharkey
b53d7b0521
scatterplot- time series support (#11085) 2024-09-20 15:48:28 +00:00
Gregory Michael Travis
859b572242
Better error message for NULLs in primary key columns (#11055) 2024-09-18 11:56:21 -04:00
Pavel Marek
3dd0fb95ae
Better error message for unresolved symbol (#11008)
* Add failing tests

* ExtensiomMethod resolution in the current module might be an error in GlobalNames

* Add another successful test

* Fix compilation error test

* Fix compilation of RedShift_Spec - `setup` is provided in compile scope
2024-09-17 15:38:31 +02:00
James Dunkerley
0e9821519d
Widget work for AWS (#11095)
- Added `to_display_text` for `S3_File` and `Enso_File`.
- Improved widget for `AWS_Credential` allowing use of Enso secrets.
- Adjust `S3.list_objects` to return `S3_File` objects, allowing easier drill down.
- Fix for merging inherited config with direct config in widgets.
- Add missing constant types to Date.Diff widget.

![image](https://github.com/user-attachments/assets/ea125a09-5067-4dee-bef2-3d7c8d551260)
2024-09-16 18:56:14 +00:00
AdRiley
b8516f7b4a
Make all Common_Spec tests pass for SQLSerever (#10810)
* More SQL tests

* Remove duplicate

* Fix another test

* Temp fix

* More temporay fixes

* More green

* Another green

* Another one

* Fix counts

* Temporary change

* Fix tests

* Auto-commit work in progress before clean build on 2024-09-06 10:53:46

* Another fix

* Green

* Specialise ensure_query_has_no_holes

* cleanup

* Cleanup

* Cleanup

* Clean

* Code review changes
2024-09-12 10:08:55 +01:00
GregoryTravis
f409cb8aac
fix test (#11049)
Repairs a test not run in CI
2024-09-11 16:34:07 +01:00
AdRiley
a666585afe
Add update mode to Running (#11045) 2024-09-11 17:41:33 +03:00
GregoryTravis
97a1628017
Add a banker's rounding primitive for Snowflake (#11037)
* log table_builder create

* fix log, 38,12

* fix scale in rounding tests

* shorten some decimals

* fix test

* dialect flags

* revert native bankers

* missing import

* sf builtin

* runs

* works on +/-2/4.5

* remove extreme precision limit tests

* remove another extreme precision test, and restore batching for bankers rounding tests

* cleanup

* cleanup

* cleanup, comment
2024-09-11 10:29:41 -04:00
James Dunkerley
7b4b635fa8
Add Row Number starts at 0 and other tweaks. (#11017)
- Change default to be from 0.
- Alter widgets so always shown.
- Tweaks to the Welcome screen following review with Product Team.
- Use 0 for default seed option in `Index_Sub_Range.Sample`.
- Add ALIASES for aggregate and running.
- Add more imports for `Widgets.enso` to fix extension methods not working.
2024-09-09 20:26:59 +00:00
GregoryTravis
2adb8bb86d
Use NUMBER(38, 12) as default range for Snowflake DECIMALs (#10971) 2024-09-09 18:31:09 +00:00
GregoryTravis
6f97e8041b
Upgrade SQLite to 3.46 (#10911)
Before 3.46, the SQLite parser had a limited stack, which could overflow for certain complex queries.

CTE optimizations make some of our queries much smaller, but also a little bit more deeply nested, causing the parser stack to overflow. 3.46 removes this stack limitation.

Closes #10910.
2024-09-07 17:58:59 +00:00
marthasharkey
836953289e
Wip/mk/scatter plot visual improvements (#10959)
* add mutliseries support to d3

* fix types

* fix scales when multiseries

* fix tests

* fix all tests

* basic hardcoded legend working, on only one plot at a time

* basic legend

* legend populating correctly with wrong label names

* legend with tooltip for full length names

* remove unused variale

* key on all nodes

* tooltips on points and labels

* turn on/off d3 'brush' so tool tip is availible to the user

* use colourScale

* fix type problems

* use default colour

* rename variables and remove unneeded code

* fix label

* sort labels in legend

* is multiseries show legend and dont show y label

* address comments

* fix type problem

* Update distribution/lib/Standard/Visualization/0.0.0-dev/src/Scatter_Plot.enso

Co-authored-by: James Dunkerley <jdunkerley@users.noreply.github.com>

---------

Co-authored-by: James Dunkerley <jdunkerley@users.noreply.github.com>
2024-09-06 10:12:55 +01:00
James Dunkerley
e6ea3b2607
Default Project, Templates and Docs (#10980)
- Linting updates.
- Add an `Examples.welcome` and adjust the start up project to use it.
- Merge all of Cass's work into the source code.
- Make example render in mono space font.
2024-09-05 14:36:08 +00:00
Jaroslav Tulach
2fc553065f
Get ready for python and js being optional when running test/Base_Tests (#10975)
Let's modify the `Base_Tests` to be able to succeed without JavaScript being installed - **done** in 3298b07dca
2024-09-05 13:00:44 +00:00
Radosław Waśko
0543a69594
Run Enso Cloud tests on the CI (#10964)
- Closes #9523
2024-09-04 11:04:54 +00:00
GregoryTravis
ad9fa4b8b6
Add vectorized rounding operation to Decimal columns (#10912) 2024-08-31 07:06:12 +00:00
James Dunkerley
91226be378
Small tweaks from QA (#10941) 2024-08-31 09:04:52 +02:00
Radosław Waśko
50325b6a1d
Pending Audit Logs are sent in batches (#10918) 2024-08-30 15:10:54 +02:00
Dmitry Bushev
6f895a5648
Add CI checks for aarch64 (#10881) 2024-08-30 09:16:01 +01:00
Jaroslav Tulach
339c275338
Benchmark and speed processing of polyglot java imports up (#10899) 2024-08-29 17:20:53 +02:00
GregoryTravis
5fba57206a
Add Text.to_decimal (#10874)
Add `Text.to_decimal`.
Also makes renames `Decimal.with_scale` to `set_scale` and makes it public.
2024-08-28 19:33:20 +00:00
GregoryTravis
1804f317b2
Implement .floor, .ceil, .trunc for the in-memory Decimal column (#10887)
* wip

* wip

* test

* round pending

* changelog

* fix test

* fully enable tests

* fix test
2024-08-28 14:27:26 -04:00
GregoryTravis
8260a9587f
Column-level lexically-scoped CTE expressions (#10826)
This implements `DB_Column.with`, which uses `WITH ... AS` SQL clauses to remove duplicates in the generated SQL.

After a discussion with @radeusgd, we concluded that we will probably want a more complete CTE implementation, so this one is useful for now to deal with big queries (like `round`).

# Important Notes
Still to do in this PR:

- [x] Rename `with` to `let` (or something similar)
- [x] tests
- [x] documentation
- [x] remove `State` hack by moving query generation into a class and using a `Ref` field for scoping

Results on `round_float`:

| --- | SQL length in characters (unprettified) | SQL length in lines (prettified) |
| --- | --- | --- |
| Without CTEs | 13193 | 851 |
| With CTEs | 3644 | 187 |

Compare the SQL:

[without-ctes.sql.txt](https://github.com/user-attachments/files/16629356/without-ctes.sql.txt)
[with-ctes.sql.txt](https://github.com/user-attachments/files/16629357/with-ctes.sql.txt)

Update, with name shortening:

| --- | SQL length in characters (unprettified) | SQL length in lines (prettified) |
| --- | --- | --- |
| Without CTEs | 13193 | 853 |
| With CTEs | 2427 | 176 |

[without-cte.txt](https://github.com/user-attachments/files/16694328/without-cte.txt)
[with-cte.txt](https://github.com/user-attachments/files/16694327/with-cte.txt)
2024-08-28 18:23:51 +00:00
marthasharkey
069cf61b95
Initial enso changes to ScatterPlot (#10871) 2024-08-28 13:39:55 +00:00
Radosław Waśko
80922dcd01
First/Last support in Snowflake, and partially in SQLite (#10841)
- Closes #10411
- Also a similar technique allowed to add partial support for SQLite as well.
- Only `include_nothing=True` case is still not supported as the technique used for Snowflake did not port well to SQLite and I did not find yet a satisfactory workaround (it is doable, but too ugly to justify it for now IMHO).
2024-08-28 10:00:07 +00:00
Radosław Waśko
2e5f011f3a
Move Connection classes out of Internal (#10886)
- Closes #10848
2024-08-28 08:15:07 +00:00
Radosław Waśko
ff5e4c4e0a
Include projectName in audit logs (#10892)
- Closes #9875
2024-08-27 13:13:28 +00:00
Dmitry Bushev
4d286e02b6
Fix S3 spec (#10893)
Fixes the issue with the S3 test on CI:
```
- [FAILED] should fail to write a file if permissions are lacking [1108ms]
Reason: The value (S3 Error: User: arn:aws:iam::539646847106:user/enso_ci_s3_tester is not authorized to perform: s3:PutObject on resource: "arn:aws:s3:::enso-data-samples/examples/no-write-permissions-here.txt.bak" because no identity-based policy allows the s3:PutObject action (AccessDenied)) did not contain the element (Access Denied) (at /runner/_work/enso/enso/test/AWS_Tests/src/S3_Spec.enso:488:13-68).
```
2024-08-26 17:51:50 +00:00
Jaroslav Tulach
b6f0675563
IO.print without new line (#10858) 2024-08-24 07:00:16 +02:00
Radosław Waśko
835aebdcde
Updating Cloud integration to new directory structure (#10762)
- Closes #10749
2024-08-21 18:11:27 +00:00
marthasharkey
b2237f77c7
Add drilldown to XML (#10824)
This adds the drilldown behaviour to XML_Elements.

![10557-xml-drilldown](https://github.com/user-attachments/assets/062d7de9-c157-4ea2-8a9b-a39f91c26581)
2024-08-19 10:12:42 +00:00
James Dunkerley
2dbdcb29a1
Tuning S3_File doc comments. (#10832)
- Review and update the doc comments of public functions in the AWS library.
- Reorder the functions to make the order in component browser (and online docs better).
- Align some error handling.
- Fix bug with `list` on root S3.
- Hide `S3.get_object` as it's single read makes it bad for GUI use.
2024-08-16 17:38:43 +00:00
Radosław Waśko
09137f7088
Fix Count_Distinct in Snowflake (#10818)
- Closes #10611
2024-08-16 12:43:19 +00:00
James Dunkerley
d6ca3ea47b
Add Regex link to all regex functions. (#10825)
Adds links to Regex documentation for all regex functions.
2024-08-15 16:21:19 +00:00
James Dunkerley
422fa8c16b
Adding support for creating Atoms in expressions (#10820)
- Enables the `..` autoscoping style for creating Atoms in expressions.
- Add type checking to methods in columns.
- Auto wrap returns from method in expressions into a column as needed.
- Remove `Time_Period.Day` to remove confusion..
2024-08-15 15:52:30 +00:00
GregoryTravis
e836373d9b
Mixed Decimal/Float operations throw error or attach warning (#10725) 2024-08-14 16:45:28 +00:00
Radosław Waśko
09f3a0b5cc
Add retries to tests depending on HTTP (#10815)
- Continuation of #9652 - apparently some tests that depend on HTTP requests were missed then.
- Closes #10742
2024-08-14 11:55:54 +00:00
AdRiley
9fc6af12c5
Clean up Text option from read and query (#10790)
* Revert

* from

* Fix

* fix

* Last few details

* Fix

* Fix test

* Fix

* Cleanup

* Fix error message

* fix

* Fix

* Improve error message
2024-08-13 13:25:21 +03:00
Radosław Waśko
b1958f8aa3
Adding vectorized implementations to some Column operations (#10795)
- Part of #6256 - implements operations that could have been vectorized without changes to the overall infrastructure
2024-08-13 08:53:39 +00:00