Commit Graph

969 Commits

Author SHA1 Message Date
Pavel Marek
3dd0fb95ae
Better error message for unresolved symbol (#11008)
* Add failing tests

* ExtensiomMethod resolution in the current module might be an error in GlobalNames

* Add another successful test

* Fix compilation error test

* Fix compilation of RedShift_Spec - `setup` is provided in compile scope
2024-09-17 15:38:31 +02:00
James Dunkerley
0e9821519d
Widget work for AWS (#11095)
- Added `to_display_text` for `S3_File` and `Enso_File`.
- Improved widget for `AWS_Credential` allowing use of Enso secrets.
- Adjust `S3.list_objects` to return `S3_File` objects, allowing easier drill down.
- Fix for merging inherited config with direct config in widgets.
- Add missing constant types to Date.Diff widget.

![image](https://github.com/user-attachments/assets/ea125a09-5067-4dee-bef2-3d7c8d551260)
2024-09-16 18:56:14 +00:00
AdRiley
b8516f7b4a
Make all Common_Spec tests pass for SQLSerever (#10810)
* More SQL tests

* Remove duplicate

* Fix another test

* Temp fix

* More temporay fixes

* More green

* Another green

* Another one

* Fix counts

* Temporary change

* Fix tests

* Auto-commit work in progress before clean build on 2024-09-06 10:53:46

* Another fix

* Green

* Specialise ensure_query_has_no_holes

* cleanup

* Cleanup

* Cleanup

* Clean

* Code review changes
2024-09-12 10:08:55 +01:00
GregoryTravis
f409cb8aac
fix test (#11049)
Repairs a test not run in CI
2024-09-11 16:34:07 +01:00
AdRiley
a666585afe
Add update mode to Running (#11045) 2024-09-11 17:41:33 +03:00
GregoryTravis
97a1628017
Add a banker's rounding primitive for Snowflake (#11037)
* log table_builder create

* fix log, 38,12

* fix scale in rounding tests

* shorten some decimals

* fix test

* dialect flags

* revert native bankers

* missing import

* sf builtin

* runs

* works on +/-2/4.5

* remove extreme precision limit tests

* remove another extreme precision test, and restore batching for bankers rounding tests

* cleanup

* cleanup

* cleanup, comment
2024-09-11 10:29:41 -04:00
James Dunkerley
7b4b635fa8
Add Row Number starts at 0 and other tweaks. (#11017)
- Change default to be from 0.
- Alter widgets so always shown.
- Tweaks to the Welcome screen following review with Product Team.
- Use 0 for default seed option in `Index_Sub_Range.Sample`.
- Add ALIASES for aggregate and running.
- Add more imports for `Widgets.enso` to fix extension methods not working.
2024-09-09 20:26:59 +00:00
GregoryTravis
2adb8bb86d
Use NUMBER(38, 12) as default range for Snowflake DECIMALs (#10971) 2024-09-09 18:31:09 +00:00
GregoryTravis
6f97e8041b
Upgrade SQLite to 3.46 (#10911)
Before 3.46, the SQLite parser had a limited stack, which could overflow for certain complex queries.

CTE optimizations make some of our queries much smaller, but also a little bit more deeply nested, causing the parser stack to overflow. 3.46 removes this stack limitation.

Closes #10910.
2024-09-07 17:58:59 +00:00
marthasharkey
836953289e
Wip/mk/scatter plot visual improvements (#10959)
* add mutliseries support to d3

* fix types

* fix scales when multiseries

* fix tests

* fix all tests

* basic hardcoded legend working, on only one plot at a time

* basic legend

* legend populating correctly with wrong label names

* legend with tooltip for full length names

* remove unused variale

* key on all nodes

* tooltips on points and labels

* turn on/off d3 'brush' so tool tip is availible to the user

* use colourScale

* fix type problems

* use default colour

* rename variables and remove unneeded code

* fix label

* sort labels in legend

* is multiseries show legend and dont show y label

* address comments

* fix type problem

* Update distribution/lib/Standard/Visualization/0.0.0-dev/src/Scatter_Plot.enso

Co-authored-by: James Dunkerley <jdunkerley@users.noreply.github.com>

---------

Co-authored-by: James Dunkerley <jdunkerley@users.noreply.github.com>
2024-09-06 10:12:55 +01:00
James Dunkerley
e6ea3b2607
Default Project, Templates and Docs (#10980)
- Linting updates.
- Add an `Examples.welcome` and adjust the start up project to use it.
- Merge all of Cass's work into the source code.
- Make example render in mono space font.
2024-09-05 14:36:08 +00:00
Jaroslav Tulach
2fc553065f
Get ready for python and js being optional when running test/Base_Tests (#10975)
Let's modify the `Base_Tests` to be able to succeed without JavaScript being installed - **done** in 3298b07dca
2024-09-05 13:00:44 +00:00
Radosław Waśko
0543a69594
Run Enso Cloud tests on the CI (#10964)
- Closes #9523
2024-09-04 11:04:54 +00:00
GregoryTravis
ad9fa4b8b6
Add vectorized rounding operation to Decimal columns (#10912) 2024-08-31 07:06:12 +00:00
James Dunkerley
91226be378
Small tweaks from QA (#10941) 2024-08-31 09:04:52 +02:00
Radosław Waśko
50325b6a1d
Pending Audit Logs are sent in batches (#10918) 2024-08-30 15:10:54 +02:00
Dmitry Bushev
6f895a5648
Add CI checks for aarch64 (#10881) 2024-08-30 09:16:01 +01:00
Jaroslav Tulach
339c275338
Benchmark and speed processing of polyglot java imports up (#10899) 2024-08-29 17:20:53 +02:00
GregoryTravis
5fba57206a
Add Text.to_decimal (#10874)
Add `Text.to_decimal`.
Also makes renames `Decimal.with_scale` to `set_scale` and makes it public.
2024-08-28 19:33:20 +00:00
GregoryTravis
1804f317b2
Implement .floor, .ceil, .trunc for the in-memory Decimal column (#10887)
* wip

* wip

* test

* round pending

* changelog

* fix test

* fully enable tests

* fix test
2024-08-28 14:27:26 -04:00
GregoryTravis
8260a9587f
Column-level lexically-scoped CTE expressions (#10826)
This implements `DB_Column.with`, which uses `WITH ... AS` SQL clauses to remove duplicates in the generated SQL.

After a discussion with @radeusgd, we concluded that we will probably want a more complete CTE implementation, so this one is useful for now to deal with big queries (like `round`).

# Important Notes
Still to do in this PR:

- [x] Rename `with` to `let` (or something similar)
- [x] tests
- [x] documentation
- [x] remove `State` hack by moving query generation into a class and using a `Ref` field for scoping

Results on `round_float`:

| --- | SQL length in characters (unprettified) | SQL length in lines (prettified) |
| --- | --- | --- |
| Without CTEs | 13193 | 851 |
| With CTEs | 3644 | 187 |

Compare the SQL:

[without-ctes.sql.txt](https://github.com/user-attachments/files/16629356/without-ctes.sql.txt)
[with-ctes.sql.txt](https://github.com/user-attachments/files/16629357/with-ctes.sql.txt)

Update, with name shortening:

| --- | SQL length in characters (unprettified) | SQL length in lines (prettified) |
| --- | --- | --- |
| Without CTEs | 13193 | 853 |
| With CTEs | 2427 | 176 |

[without-cte.txt](https://github.com/user-attachments/files/16694328/without-cte.txt)
[with-cte.txt](https://github.com/user-attachments/files/16694327/with-cte.txt)
2024-08-28 18:23:51 +00:00
marthasharkey
069cf61b95
Initial enso changes to ScatterPlot (#10871) 2024-08-28 13:39:55 +00:00
Radosław Waśko
80922dcd01
First/Last support in Snowflake, and partially in SQLite (#10841)
- Closes #10411
- Also a similar technique allowed to add partial support for SQLite as well.
- Only `include_nothing=True` case is still not supported as the technique used for Snowflake did not port well to SQLite and I did not find yet a satisfactory workaround (it is doable, but too ugly to justify it for now IMHO).
2024-08-28 10:00:07 +00:00
Radosław Waśko
2e5f011f3a
Move Connection classes out of Internal (#10886)
- Closes #10848
2024-08-28 08:15:07 +00:00
Radosław Waśko
ff5e4c4e0a
Include projectName in audit logs (#10892)
- Closes #9875
2024-08-27 13:13:28 +00:00
Dmitry Bushev
4d286e02b6
Fix S3 spec (#10893)
Fixes the issue with the S3 test on CI:
```
- [FAILED] should fail to write a file if permissions are lacking [1108ms]
Reason: The value (S3 Error: User: arn:aws:iam::539646847106:user/enso_ci_s3_tester is not authorized to perform: s3:PutObject on resource: "arn:aws:s3:::enso-data-samples/examples/no-write-permissions-here.txt.bak" because no identity-based policy allows the s3:PutObject action (AccessDenied)) did not contain the element (Access Denied) (at /runner/_work/enso/enso/test/AWS_Tests/src/S3_Spec.enso:488:13-68).
```
2024-08-26 17:51:50 +00:00
Jaroslav Tulach
b6f0675563
IO.print without new line (#10858) 2024-08-24 07:00:16 +02:00
Radosław Waśko
835aebdcde
Updating Cloud integration to new directory structure (#10762)
- Closes #10749
2024-08-21 18:11:27 +00:00
marthasharkey
b2237f77c7
Add drilldown to XML (#10824)
This adds the drilldown behaviour to XML_Elements.

![10557-xml-drilldown](https://github.com/user-attachments/assets/062d7de9-c157-4ea2-8a9b-a39f91c26581)
2024-08-19 10:12:42 +00:00
James Dunkerley
2dbdcb29a1
Tuning S3_File doc comments. (#10832)
- Review and update the doc comments of public functions in the AWS library.
- Reorder the functions to make the order in component browser (and online docs better).
- Align some error handling.
- Fix bug with `list` on root S3.
- Hide `S3.get_object` as it's single read makes it bad for GUI use.
2024-08-16 17:38:43 +00:00
Radosław Waśko
09137f7088
Fix Count_Distinct in Snowflake (#10818)
- Closes #10611
2024-08-16 12:43:19 +00:00
James Dunkerley
d6ca3ea47b
Add Regex link to all regex functions. (#10825)
Adds links to Regex documentation for all regex functions.
2024-08-15 16:21:19 +00:00
James Dunkerley
422fa8c16b
Adding support for creating Atoms in expressions (#10820)
- Enables the `..` autoscoping style for creating Atoms in expressions.
- Add type checking to methods in columns.
- Auto wrap returns from method in expressions into a column as needed.
- Remove `Time_Period.Day` to remove confusion..
2024-08-15 15:52:30 +00:00
GregoryTravis
e836373d9b
Mixed Decimal/Float operations throw error or attach warning (#10725) 2024-08-14 16:45:28 +00:00
Radosław Waśko
09f3a0b5cc
Add retries to tests depending on HTTP (#10815)
- Continuation of #9652 - apparently some tests that depend on HTTP requests were missed then.
- Closes #10742
2024-08-14 11:55:54 +00:00
AdRiley
9fc6af12c5
Clean up Text option from read and query (#10790)
* Revert

* from

* Fix

* fix

* Last few details

* Fix

* Fix test

* Fix

* Cleanup

* Fix error message

* fix

* Fix

* Improve error message
2024-08-13 13:25:21 +03:00
Radosław Waśko
b1958f8aa3
Adding vectorized implementations to some Column operations (#10795)
- Part of #6256 - implements operations that could have been vectorized without changes to the overall infrastructure
2024-08-13 08:53:39 +00:00
marthasharkey
0d5d01a294
Wip/mk/drive drilldown from enso (#10724)
This moves some of the drilldown logic to the enso code and utilises the new 'get_rows' method for Table
2024-08-12 12:51:51 +00:00
AdRiley
3581fd8e21
Make sqlserver temp tables work (#10781)
* Temp table support

* Fix

* Add query

* Clean up

* Clean up

* Fix

* Code review changes

* Added comment

* Code review changes

* Add execute
2024-08-09 13:06:16 +03:00
Pavel Marek
c2c6712b77
WithWarnings uses EnsoHashMap to speed things up (#10555)
Majority of warnings handling is now done via newly introduced nodes. Moreover, the underlying representation of warnings storage in `WithWarnings` was changed from `Warning[]` to `EnsoHashMap`.

# Important Notes
- Remove `ArrayRope`.
2024-08-07 15:29:57 +00:00
James Dunkerley
b8c036c476
Initial Tableau Reading Support (#10733)
- Adds `Hyper_File` allowing reading a Tableau hyper file.
- Can read the schema and table list.
- Can read the structure of a table.
- Can read data into an Enso Table.
2024-08-07 09:23:05 +00:00
AdRiley
0f688d0a25
Add newlines option to text cleanse/replace (#10761)
* Auto-commit work in progress before clean build on 2024-08-06 11:32:46

* Fixed Regex and additional test

* changelog

* .

* Make non-capturing
2024-08-06 18:59:54 +03:00
Radosław Waśko
3fd14642d9
Fix upload/delete transactions in Snowflake backend (#10738)
Fixes #10609 by rewriting all our upload-related operations to rely on `DDL_Transaction` - an abstraction that handles 'transactionality' of `CREATE TABLE` statements dependent on if a given backend allows DDLs within transactions or not (if not it emulates transactionality by creating the tables outside of transaction and then dropping them on rollback).
2024-08-06 08:14:44 +00:00
Kaz Wesley
aafdef1aeb
Improve parser contextualization (#10734) 2024-08-05 15:46:58 +00:00
Jaroslav Tulach
e94974a0a8
Internal tests run with --disable-private-check (#10743) 2024-08-05 15:22:22 +02:00
Radosław Waśko
6ad3faf03b
Refactor Upload_Table to be more readable: split into separate smaller submodules (#10713)
- First step of #10609 - before I actually modify it, I decided I need to refactor the `Upload_Table` logic as it was quite convoluted. Doing this as a separate PR for easier review. A big 600+ line file was replaced by several smaller ones grouped by topics.
- Practically no changes apart from moving stuff into separate modules.
- One small change - added `Missing_Argument` to `SQL_Query` as I noticed that lack of defaults was giving rise to confusing errors when working with `query` in the GUI.

Before:
![image](https://github.com/user-attachments/assets/b586caec-f25c-406e-be5a-d402f10feb86)
After:
![image](https://github.com/user-attachments/assets/6b1d4206-05b1-4587-b3e1-43ca95ea7c2e)
![image](https://github.com/user-attachments/assets/58c098bd-db0c-4ee2-823c-bf5c9e758ce4)
2024-07-31 09:54:17 +00:00
Radosław Waśko
9b2f611402
Relative paths are relative to current project locally and in Cloud (#10660)
- Close #10622
- Changes `project-manager` and `ensoup` launcher to run the engine/language-server with working directory set to the directory containing currently running project.
- If the working directory is _not_ "the directory containing currently running project", a warning is written to logs. This can happen if the raw `/bin/enso` engine runner is used in a different directory.
- In the Cloud, the `File.new` interprets relative paths as cloud paths relative to the Cloud directory containing the current project. Absolute paths are unaffected.
2024-07-31 09:43:17 +00:00
AdRiley
0c552489e3
Add Initial SQL Server support (#10624)
* Squash all commits to resolve merge conflicts

* Fix merge problems

* Merge fix

* Fix port

* Fix warning

* cargo fmt

* legal review

* Small fixes

* Update instructions

* Code review feedback

* Cleanup

* typo

* Fix

* Remove leftover snowflake code

* Remove comment

* Add underscore

* Type cleanup

* Code review fix

* Cleanup

* Add datetime roundtrip test

* add comment

* drop

* Refactor

* Refactor

* Fix merge

* Fix

* Fix

* fix

* Add comment
2024-07-30 11:13:08 +01:00
Jaroslav Tulach
b9214f052c
Must specify --repl to enable debug server (#10709) 2024-07-29 18:29:55 +02:00
Jaroslav Tulach
07bc728b6d
Avoid race condition in GC_Example test (#10665)
Re-enables https://github.com/enso-org/enso/pull/10602/files#r1690919173 - uses `IO.println` to allow us to see what the CI actually does.
2024-07-29 09:37:21 +00:00