Add tests for coalesce and fix the In-memory version
# Important Notes
The distribution/lib/Standard/Database/0.0.0-dev/src/Internal/IR/From_Spec.enso change isn't actually needed for this MR as I switched away from using literal tables for these tests as trying to mix 2 literal tables doesn't look to be supported. The change I added will let any future developers know that.
(cherry picked from commit a4f800dbf0)
- Allow Interrupted Exceptions to float out of the web requests.
- Use `Type_Error` rather than Any when catching auto scoping resolving.
- Rename `Java_Exception` to `JException`
(cherry picked from commit fd72ab7052)
- Left over from the last PR.
- Widget for `headers` in `URL.fetch` extension.
- Fix `Left` and Right` to allow columns or expressions in it.
(cherry picked from commit a2c853c6e3)
* Once more unto the lint dear friends.
* Default the arguments for process run.
Allow delimiter to be used to convert Text to Table.
(cherry picked from commit b2545b8b96)
- Added `to_display_text` for `S3_File` and `Enso_File`.
- Improved widget for `AWS_Credential` allowing use of Enso secrets.
- Adjust `S3.list_objects` to return `S3_File` objects, allowing easier drill down.
- Fix for merging inherited config with direct config in widgets.
- Add missing constant types to Date.Diff widget.
(cherry picked from commit 0e9821519d)
* More SQL tests
* Remove duplicate
* Fix another test
* Temp fix
* More temporay fixes
* More green
* Another green
* Another one
* Fix counts
* Temporary change
* Fix tests
* Auto-commit work in progress before clean build on 2024-09-06 10:53:46
* Another fix
* Green
* Specialise ensure_query_has_no_holes
* cleanup
* Cleanup
* Cleanup
* Clean
* Code review changes
(cherry picked from commit b8516f7b4a)
- Change default to be from 0.
- Alter widgets so always shown.
- Tweaks to the Welcome screen following review with Product Team.
- Use 0 for default seed option in `Index_Sub_Range.Sample`.
- Add ALIASES for aggregate and running.
- Add more imports for `Widgets.enso` to fix extension methods not working.
(cherry picked from commit 7b4b635fa8)
Before 3.46, the SQLite parser had a limited stack, which could overflow for certain complex queries.
CTE optimizations make some of our queries much smaller, but also a little bit more deeply nested, causing the parser stack to overflow. 3.46 removes this stack limitation.
Closes#10910.
(cherry picked from commit 6f97e8041b)
* add mutliseries support to d3
* fix types
* fix scales when multiseries
* fix tests
* fix all tests
* basic hardcoded legend working, on only one plot at a time
* basic legend
* legend populating correctly with wrong label names
* legend with tooltip for full length names
* remove unused variale
* key on all nodes
* tooltips on points and labels
* turn on/off d3 'brush' so tool tip is availible to the user
* use colourScale
* fix type problems
* use default colour
* rename variables and remove unneeded code
* fix label
* sort labels in legend
* is multiseries show legend and dont show y label
* address comments
* fix type problem
* Update distribution/lib/Standard/Visualization/0.0.0-dev/src/Scatter_Plot.enso
Co-authored-by: James Dunkerley <jdunkerley@users.noreply.github.com>
---------
Co-authored-by: James Dunkerley <jdunkerley@users.noreply.github.com>
(cherry picked from commit 836953289e)
- Linting updates.
- Add an `Examples.welcome` and adjust the start up project to use it.
- Merge all of Cass's work into the source code.
- Make example render in mono space font.
(cherry picked from commit e6ea3b2607)
* Reduce akka logs in PM and LS
After #10905 we suddenly got a lot of (useless) logs from akka in
project-manager and language-server. After some investigation it turned
out that akka-typed had a special configuration to enable logging in
akka using slf4j.
As we don't need the whole library for that I only stole the
configuration.
* nit
* licensing
* missing license files
(cherry picked from commit 8b30998afb)
* Add today, now and time to Expression.
* Move running and compute into Column as that allows them to be used in expressions.
* Fix bug.
* Fix exports.
* Java fmt.
* Akka-related startup improvements
Realized that Akka Typed and Spray were not used **at all** yet there
were in the list of dependencies.
Additionally, the former would also show up in the list of Akka's
library-extensions, initialized on startup.
* Hardcode list of library extensions
Hardcoding a list of library extensions is not recommended but I don't
see any other way of disabling expensive Serialization extension, which
we don't seem to use anyway.
* fmt
* licensing update
This implements `DB_Column.with`, which uses `WITH ... AS` SQL clauses to remove duplicates in the generated SQL.
After a discussion with @radeusgd, we concluded that we will probably want a more complete CTE implementation, so this one is useful for now to deal with big queries (like `round`).
# Important Notes
Still to do in this PR:
- [x] Rename `with` to `let` (or something similar)
- [x] tests
- [x] documentation
- [x] remove `State` hack by moving query generation into a class and using a `Ref` field for scoping
Results on `round_float`:
| --- | SQL length in characters (unprettified) | SQL length in lines (prettified) |
| --- | --- | --- |
| Without CTEs | 13193 | 851 |
| With CTEs | 3644 | 187 |
Compare the SQL:
[without-ctes.sql.txt](https://github.com/user-attachments/files/16629356/without-ctes.sql.txt)
[with-ctes.sql.txt](https://github.com/user-attachments/files/16629357/with-ctes.sql.txt)
Update, with name shortening:
| --- | SQL length in characters (unprettified) | SQL length in lines (prettified) |
| --- | --- | --- |
| Without CTEs | 13193 | 853 |
| With CTEs | 2427 | 176 |
[without-cte.txt](https://github.com/user-attachments/files/16694328/without-cte.txt)
[with-cte.txt](https://github.com/user-attachments/files/16694327/with-cte.txt)
- Closes#10411
- Also a similar technique allowed to add partial support for SQLite as well.
- Only `include_nothing=True` case is still not supported as the technique used for Snowflake did not port well to SQLite and I did not find yet a satisfactory workaround (it is doable, but too ugly to justify it for now IMHO).
- Review and update the doc comments of public functions in the AWS library.
- Reorder the functions to make the order in component browser (and online docs better).
- Align some error handling.
- Fix bug with `list` on root S3.
- Hide `S3.get_object` as it's single read makes it bad for GUI use.
- Enables the `..` autoscoping style for creating Atoms in expressions.
- Add type checking to methods in columns.
- Auto wrap returns from method in expressions into a column as needed.
- Remove `Time_Period.Day` to remove confusion..
- Closes#10716 by un-marking the `Snowflake_Connection` as `private`.
- Does the same to `SQLServer_Connection`.
- Makes sure that `Snowflake_Connection` has correct visualization and enables links for it.
- Adds a fallback in `to_default_visualization_data` and in Table's `prepare_visualization` so that if `to_js_object` of a given type is malformed, we still get some kind of visualization + debug info about the underlying error.
Majority of warnings handling is now done via newly introduced nodes. Moreover, the underlying representation of warnings storage in `WithWarnings` was changed from `Warning[]` to `EnsoHashMap`.
# Important Notes
- Remove `ArrayRope`.
- Adds `Hyper_File` allowing reading a Tableau hyper file.
- Can read the schema and table list.
- Can read the structure of a table.
- Can read data into an Enso Table.
Working on compiler IR is a daunting task. I have therefore added a new system property `enso.compiler.dumpIr` that will help with that. It dumps the encountered IRs to `ir-dumps` directory in the [GraphViz](www.graphviz.org) format. More info in updated docs.
Note that all the functionality to dump IRs to `dot` files was already implemented. This PR just adds the command line option and updates docs.
# Important Notes
- `--dump-graphs` cmd line option is removed as per [Jaroslav's request](https://github.com/enso-org/enso/pull/10740#pullrequestreview-2216676140).
- To dump graphs, use `-Dgraal.Dump=Truffle:2` system property passed via `JAVA_OPTS` env var.
If you run `env JAVA_OPTS='-Denso.compiler.dumpIr=true' enso --run tmp.enso` where `tmp.enso` is, e.g.:
```
from Standard.Base import all
main = 42
```
You will then have something like:
```
$ ls ir-dumps
Standard.Base.Data.Filter_Condition.dot Standard.Base.Data.Time.dot Standard.Base.System.Advanced.dot Standard.Base.Warning.dot
Standard.Base.Data.Locale.dot Standard.Base.Enso_Cloud.Enso_File.dot Standard.Base.System.File.Advanced.dot tmp.dot
Standard.Base.Data.Numeric.dot Standard.Base.Errors.dot Standard.Base.System.File.dot
Standard.Base.Data.Numeric.Internal.dot Standard.Base.Network.HTTP.Internal.dot Standard.Base.System.File.Generic.dot
Standard.Base.Data.Text.Regex.Internal.dot Standard.Base.Runtime.dot Standard.Base.System.Internal.dot
```
You can then visualize any of these with `dot -Tsvg -O ir-dumps/tmp.dot`.
An example how that could look like is
Fixes#10609 by rewriting all our upload-related operations to rely on `DDL_Transaction` - an abstraction that handles 'transactionality' of `CREATE TABLE` statements dependent on if a given backend allows DDLs within transactions or not (if not it emulates transactionality by creating the tables outside of transaction and then dropping them on rollback).
