Commit Graph

2214 Commits

Author SHA1 Message Date
James Dunkerley
4c82b657de
Tidy up type signatures and error types (#3693)
Small clean up PR.

- Aligns a few type signatures with their functions.
- Some formatting fixes.
- Remove a few unused types.
- Make error extension functions be standard methods.
2022-09-09 11:11:46 +00:00
James Dunkerley
2b425f8e08
Restructuring Database.Connection to allow for database specific types. (#3632)
- Added `databases`, `database`, `set_database`.
- Added `schemas`, `schema`, `set_schema`.
- Added `table_types`,
- Added `tables`.
- Moved the vast majority of the connection work into a lower level `JDBC_Connection` object.
- `Connection` represents the standard API for database connections and provides a base JDBC implementation.
- `SQLite_Connection` has the `Connection` API but with custom `databases` and `schemas` methods for SQLite.
- `Postgres_Connection` has the `Connection` API but with custom `set_database`, `databases`, `set_schema` and `schemas` methods for Postgres.
- Updated `Redshift` - no public API change.
2022-09-07 17:32:28 +00:00
Radosław Waśko
551100af3b
Add Table.distinct function to In-Memory table (#3684)
Implements https://www.pivotaltracker.com/story/show/182307143

# Important Notes
- Modified standard library Java helpers dependencies so that `std-table` module depends on `std-base`, as a provided dependency. This is allowed, because `std-table` is used by the `Standard.Table` Enso module which depends on `Standard.Base` which ensures that the `std-base` is loaded onto the classpath, thus whenever `std-table` is loaded by `Standard.Table`, so is `std-base`. Thus we can rely on classes from `std-base` and its dependencies being _provided_ on the classpath. Thanks to that we can use utilities like `Text_Utils` also in `std-table`, avoiding code duplication. Additional advantage of that is that we don't need to specify ICU4J as a separate dependency for `std-table`, since it is 'taken' from `std-base` already - so we avoid including it in our build packages twice.
2022-09-07 12:28:41 +00:00
Jaroslav Tulach
9967dd3da1
Check methods of a type are present on its Atoms (#3689)
Found a bug when accessing keys via `get(constructor)`. Providing a test and a fix.

# Important Notes
Marcin, is it correct that the whole set of members of `End` is: `[head, tail, Int, is_empty, IntList]`? What does `Int` and `IntList` do there? Shall test test check for their presence? **Answer**: rename `Int` and `IntList` to lowercase and yes, then the members shall be there. Done in [ca9f42a](ca9f42a2b8).
2022-09-07 11:03:09 +00:00
Dmitry Bushev
05a6415ec0
create inmem database (#3690) 2022-09-07 12:59:18 +03:00
Ilya Bogdanov
0f168abbfa
Add a note about the need of Python 2 on MacOS to CONTRIBUTING.md (#3682)
[ci no changelog needed]

Python 2 is required for our version of the `electron-builder` when building Enso IDE on modern Mac OS versions. Unfortunately, Apple removed the `/usr/bin/python` executable in the macOS Monterey release, so we are forced to give instructions on reinstalling it. I took the most popular solution from https://stackoverflow.com/questions/60298514/how-to-reinstall-python2-from-homebrew
2022-09-07 09:24:37 +00:00
Audrius Molis
a4f533300e
Fix a link in Readme.md and some punctuation. (#3685) 2022-09-06 16:25:45 +02:00
Radosław Waśko
eafba079d9
Make In Memory Table Aggregator types more specific where possible (#3679)
Many aggregation types fell back to the general `Any` type where they could have used the type of input column - for example `First` of a column of integers is guaranteed to fit the `Integer` storage type, so it doesn't have to fall back to `Any`. This PR fixes that and adds a test that checks this.
2022-09-05 09:17:41 +00:00
Kaz Wesley
1e3b9a3624
Parse text literals (#3681)
Parse text literals. See: https://www.pivotaltracker.com/story/show/182496940

# Important Notes
- The left-trimming algorithm (https://github.com/enso-org/design/blob/wip/wd/enso-spec/epics/enso-spec-1.0/04.%20Expressions.md#inline-and-block-text-literals) requires two passes over the sequence of text segments. This implementation performs one pass while parsing (identifying the correct amount of trim). The other pass (applying the trim) can be done when building the value of the quoted string: Trim the amount of whitespace identified by the `trim` field off of the whitespace of each `TextSection` (the value will not exceed the amount of whitespace found in the tokens' offsets, except for tokens with 0 offset, in which case no trimming is necessary/possible).
2022-09-03 06:38:06 +00:00
Kaz Wesley
c3f758e0dc
Parser: Parse UUIDs; implement comments in AST; implement type annotations and signatures; fix field names (#3653)
Implements:
- UUIDs: https://www.pivotaltracker.com/story/show/182931137
- Comments: https://www.pivotaltracker.com/story/show/182981779
- Type annotations and signatures: https://www.pivotaltracker.com/story/show/182497454
- Fix getter names (https://github.com/enso-org/enso/pull/3627#discussion_r940887460).

# Important Notes
- I can't fully test UUIDs; I have tested that the data obtained in Rust matches my understanding of how the format is supposed to work. What remains to be tested is that the data in Java matches the way the old parser handles the format. So @JaroslavTulach, let me know if you see any cases where I'm not returning the same values.
- This implementation of type annotations and signatures accepts any expression in type context. It would probably be nice to narrow this down at some point, but for now I have no design info on what specifically should be allowed in type expressions; this implementation should be at least an incremental improvement.
2022-09-03 03:15:27 +00:00
Dmitry Bushev
a769efe538
Remove GitHub discussions link (#3680)
GitHub Discussions forum is disabled for the repo and the link does not work.
2022-09-02 21:17:27 +00:00
Michał Wawrzyniec Urbańczyk
b8dd92e9dc
CI and build script update (#3677)
* added polyfill globals plugin to fix issue with missing types like Buffer that was affecting nightly releases;
* fixed exit code propagation for Windows build script wrapper;
* bumped the build script and refreshed the generated workflows.

Includes https://github.com/enso-org/ci-build/pull/8
2022-09-02 22:02:44 +02:00
Dmitry Bushev
de0a231417
IDE uses new visualization API (#3661) 2022-09-01 15:33:46 +03:00
Radosław Waśko
65140f48ca
Add storage support for Date, Time and DateTime to InMemory table (#3673)
Implements https://www.pivotaltracker.com/story/show/183080911
2022-08-31 22:06:29 +00:00
Marcin Kostrzewa
2a8201eb15
Remove a leftover debug print (#3676) 2022-08-31 12:28:41 +00:00
Marcin Kostrzewa
4fc6dcced0
Get rid of free-floating atoms. Everything has a type now! (#3671)
This is a step towards the new language spec. The `type` keyword now means something. So we now have
```
type Maybe a
Some (from_some : a)
None
```
as a thing one may write. Also `Some` and `None` are not standalone types now – only `Maybe` is.
This halfway to static methods – we still allow for things like `Number + Number` for backwards compatibility. It will disappear in the next PR.

The concept of a type is now used for method dispatch – with great impact on interpreter code density.

Some APIs in the STDLIB may require re-thinking. I take this is going to be up to the libraries team – some choices are not as good with a semantically different language. I've strived to update stdlib with minimal changes – to make sure it still works as it did.

It is worth mentioning the conflicting constructor name convention I've used: if `Foo` only has one constructor, previously named `Foo`, we now have:
```
type Foo
Foo_Data f1 f2 f3
```

This is now necessary, because we still don't have proper statics. When they arrive, this can be changed (quite easily, with SED) to use them, and figure out the actual convention then.

I have also reworked large parts of the builtins system, because it did not work at all with the new concepts.

It also exposes the type variants in SuggestionBuilder, that was the original tiny PR this was based on.

PS I'm so sorry for the size of this. No idea how this could have been smaller. It's a breaking language change after all.
2022-08-30 22:54:53 +00:00
Michał Wawrzyniec Urbańczyk
f00da24d0a
[CI] Cleaning runtime before building engine. (#3675) 2022-08-29 22:27:43 +02:00
Jaroslav Tulach
d4b4a31d89
Split long lines to 1024 long chunks (#3665)
Avoid long lines when using `dom().set_inner_text` - rather split the long lines to 1024 chunks and insert them as individual `<div>` elements.

# Important Notes
I was testing the behavior on following program:
```
from Standard.Base import all
import Standard.Base.Data.Statistics
import Standard.Visualization

main =
number1 = 200000
operator1 = 0.up_to number1 . to_vector . map .noise
operator2 = operator1.sort
operator3 = operator2.to_text
```
before my change the visualization of `operator3` was blank. With my change it gets filled with data.
2022-08-29 08:02:33 +00:00
Michał Wawrzyniec Urbańczyk
10e50c5d4a
CI Updates (ECR push and disabling integration tests) (#3672) 2022-08-28 14:44:10 +02:00
Wojciech Daniło
4b96b4887c
Better fonts support. (#3616) 2022-08-27 00:25:34 +02:00
Radosław Waśko
e6e4692692
DataFormatter should infer datetime from values without seconds (#3668)
Fixes https://www.pivotaltracker.com/story/show/183033133
2022-08-26 21:10:52 +00:00
Radosław Waśko
d7ebc4a338
Add Table.take and Table.drop functions to In-Memory table (#3647)
Implements https://www.pivotaltracker.com/story/show/182307347
2022-08-26 19:41:36 +00:00
Hubert Plociniczak
63fecec3b1
Add Jaroslav as codeowner of interpreter dsl (#3666)
Requesting to add @JaroslavTulach as codeowner of our interpreter dsl.
2022-08-26 16:47:53 +00:00
James Dunkerley
a20d43390e
Adding DateTime part functions (#3669)
- Added `Zone`, `Date_Time` and `Time_Of_Day` to `Standard.Base`.
- Renamed `Zone` to `Time_Zone`.
- Added `century`.
- Added `is_leap_year`.
- Added `length_of_year`.
- Added `length_of_month`.
- Added `quarter`.
- Added `day_of_year`.
- Added `Day_Of_Week` type and `day_of_week` function.
- Updated `week_of_year` to support ISO.

# Important Notes
- Had to pass locale to formatter for date/time tests to work on my PC.
- Changed default of `week_of_year` to use ISO.
2022-08-26 15:47:58 +00:00
Radosław Waśko
fd318cfa96
Remove Array.set_at (#3634)
Implements https://www.pivotaltracker.com/story/show/182879865

# Important Notes
Note that removing `set_at` still does not make our arrays fully immutable - `Array.copy` can still be used to mutate them.
2022-08-26 09:34:33 +00:00
Michael Mauderer
14c516aff3
Add api for updating searcher with selected component (#3659)
Implements [#182593882](https://www.pivotaltracker.com/story/show/182633582).

Adds API for updating Searcher with the selected suggestion. To-dos are left in the places where this will be implemented in a subsequent PR for the next task.

**Example of the debug output generated by typing into the searcher.**

![ShowSearcherInputLog](https://user-images.githubusercontent.com/1428930/186120851-7292824c-7c6e-444e-8453-5f2d234f3244.png)

**Example of the debug output generated by selecting something in the searcher.**

![ShowSearcherSelectionLog](https://user-images.githubusercontent.com/1428930/186120857-d9610eba-04fd-4e65-bb9e-71ec7bfe361d.png)

# Important Notes
For testing/QA set the `app/gui/src/lib.rs:145` logging initialization to `DEBUG` to see the console output.

[ci no changelog needed]
2022-08-26 08:33:19 +00:00
Michał Wawrzyniec Urbańczyk
020d0807ec
Bumped the build script. (#3670)
Updated script will remove benchmarking reports on CI.
2022-08-26 07:34:44 +02:00
Michał Wawrzyniec Urbańczyk
4537a4b57b
Adjusting CI / build configuration. (#3664)
* Downgrading the electron-builder to avoid https://github.com/electron-userland/electron-builder/issues/6865
* Removed "git clean" workaround from CI.
2022-08-25 00:46:36 +02:00
Adam Obuchowicz
610b128bfa
Component Panel List Layout structure. (#3662)
This PR does not add anything visual for the user: it's a first part of making efficient Component Browser using fresh Grid View implementation.

The Layout is an intermediate model, which keeps the information how the groups are laid out in the Component browser, and allows querying for group entry by location and location of specific group entry (or header).

The layout is meant for the new design, where there are no section separators and Local Scope section is below Favorites.

# Important Notes
The new structure is not used in action, but tested with unit tests.
2022-08-24 19:00:31 +00:00
Adam Obuchowicz
d4142cfee3
Grid View Headers (#3656) 2022-08-24 16:36:23 +02:00
Hubert Plociniczak
d87a32d019
Builtin Date_Time, Time_Of_Day, Zone (#3658)
* Builtin Date_Time, Time_Of_Day, Zone

Improved polyglot support for Date_Time (formerly Time), Time_Of_Day and
Zone. This follows the pattern introduced for Enso Date.

Minor caveat - in tests for Date, had to bend a lot for JS Date to pass.
This is because JS Date is not really only a Date, but also a Time and
Timezone, previously we just didn't consider the latter.
Also, JS Date does not deal well with setting timezones so the trick I
used is to first call foreign function returning a polyglot JS Date,
which is converted to ZonedDateTime and only then set the correct
timezone. That way none of the existing tests had to be changes or
special cased.

Additionally, JS deals with milliseconds rather than nanoseconds so
there is loss in precision, as noted in Time_Spec.

* Add tests for Java's LocalTime

* changelog

* Make date formatters in table happy

* PR review, add more tests for zone

* More tests and fixed a bug in column reader

Column reader didn't take into account timezone but that was a mistake
since then it wouldn't map to Enso's Date_Time.
Added tests that check it now.

* remove redundant conversion

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Time.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* First round of addressing PR review

* don't leak java exceptions in Zone

* Move Date_Time to top-level module

* PR review

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2022-08-24 12:31:29 +02:00
Hubert Plociniczak
4b9c91626e
Use Vector.from_polyglot_array to make Vectors backed by polyglot arrays (#3628)
Use Proxy_Polyglot_Array as a proxy for polyglot arrays, thus unifying
the way the underlying array is accessed in Vector.

Used the opportunity to cleanup builtin lookup, which now actually
respects what is defined in the body of @Builtin_Method annotation.

Also discovered that polyglot null values (in JS, Python and R) were leaking to Enso.
Fixed that by doing explicit translation to `Nothing`.

https://www.pivotaltracker.com/story/show/181123986
2022-08-23 21:13:16 +00:00
Ilya Bogdanov
8c504bd4b0
Grid View with variable column widths (#3651)
[ci no changelog needed]
[Task link](https://www.pivotaltracker.com/story/show/182955595)

This PR implements variable column widths in the new Grid View component. We need this feature to quickly implement various parts of the UI, including the breadcrumbs panel of the component browser.

There are two ways to change the width of the specific column:
1. "From the outside", using the `set_column_width` endpoint of the Grid View
2. "From the inside", using the `override_column_width` endpoint of the EntryFrp.

Both ways work similarly, but the latter is helpful for our breadcrumbs implementation, as it allows for entry to decide on the width of the column by its content.

See the screencast with three grid views. The top-left one has every even column shrunk by GridView API. Every grid view has a second column extended by EntryFrp API.


https://user-images.githubusercontent.com/6566674/185060985-7b7df076-c659-41fa-977a-22875493f8d4.mp4
2022-08-23 13:28:00 +00:00
Jaroslav Tulach
2b9352d2fc
Lazy scatterplot for Vector & Table (#3655)
First of all this PR demonstrates how to implement _lazy visualization_:
- one needs to write/enhance Enso visualization libraries - this PR adds two optional parameters (`bounds` and `limit`) to `process_to_json_text` function.
- the `process_to_json_text` can be tested by standard Enso test harness which this PR also does
- then one has to modify JavaScript on the IDE side to construct `setPreprocessor` expression using the optional parameters

The idea of _scatter plot lazy visualization_ is to limit the amount of points the IDE requests. Initially the limit is set to `limit=1024`. The `Scatter_Plot.enso` then processes the data and selects/generates the `limit` subset. Right now it includes `min`, `max` in both `x`, `y` axis plus randomly chosen points up to the `limit`.

![Zooming In](https://user-images.githubusercontent.com/26887752/185336126-f4fbd914-7fd8-4f0b-8377-178095401f46.png)

The D3 visualization widget is capable of _zooming in_. When that happens the JavaScript widget composes new expression with `bounds` set to the newly visible area. By calling `setPreprocessor` the engine recomputes the visualization data, filters out any data outside of the `bounds` and selects another `limit` points from the new data. The IDE visualization then updates itself to display these more detailed data. Users can zoom-in to see the smallest detail where the number of points gets bellow `limit` or they can select _Fit all_ to see all the data without any `bounds`.

# Important Notes
Randomly selecting `limit` samples from the dataset may be misleading. Probably implementing _k-means clustering_ (where `k=limit`) would generate more representative approximation.
2022-08-23 12:12:22 +00:00
Michał Wawrzyniec Urbańczyk
2507a2049b
Code Signing on Windows (#3654)
This PR reenables code signing on Windows.
Each Windows package built on CI should be now signed.
Additionally, some refactorings were done around electron-builder config, so it is easier to use outside the build script and offers more configuration options.
2022-08-23 00:49:23 +02:00
Michał Wawrzyniec Urbańczyk
183d2ba35d
Follow-up update to exclude removed workflow from required checks. 2022-08-23 00:48:29 +02:00
Michał Wawrzyniec Urbańczyk
d5b16e45dc
Removed the outdated Legal Review workflow.
The legal review is already covered as part of the Engine CI check.
2022-08-22 22:02:05 +02:00
James Dunkerley
684adcb7fb
Tidy up the default imports for Standard.Table (#3660)
- Removed various unnecessary `Standard.Base` imports still left behind.
- Added `Regex` to default `Standard.Base`.
- Removed aliasing from the examples as no longer needed (case coercion no long occurs).
- Remove `import Standard.Table` from within the Table library (directly importing types).
- Reviewed what was in `Standard.Database` - a few tweaks and removals.
- Removed various un-needed aliasing following Hubert's import work.
2022-08-22 19:21:54 +00:00
Michael Mauderer
58fb383791
Create new node when opening searcher and no node is selected. (#3645)
So far, when opening the searcher with no node selected, an empty input without an associated AST node was created. This input was manipulated by the user and the final expression from the input was used to create a new node when the user confirmed their input. This PR changes this, so that when the searcher is opened without a selected node, a new AST node is created right away with some placeholder content, and this node is updated when the user confirms their input.

The only change visible to the user, is that if the text editor is opened during editing, a new node will appear in the source when the searcher is opened to create a new node. All other behaviour should stay the same.
2022-08-22 15:32:44 +00:00
Radosław Waśko
bcca7f10d9
Add key functions to Table to make it act as [Column] (#3644)
Implements https://www.pivotaltracker.com/story/show/181370836
2022-08-18 12:33:02 +00:00
Mateusz Czapliński
fc089857d0
Filtering of Virtual Entries in Component Browser by input & return types. (#3652)
Filter the Virtual Entries displayed in the Component Browser based on the input type and the return type of the edited expression. Only the Virtual Entries with matching types are displayed.

https://www.pivotaltracker.com/story/show/182842444

#### Visuals

See below for two screenshots, showing the Component Browser with an empty expression and with the input port not connected. Two virtual entries are visible, with documentation.

<img width="871" alt="Screenshot 2022-08-16 at 15 19 55" src="https://user-images.githubusercontent.com/273837/184894042-0f64986d-2c89-4c94-90cf-215467e297c7.png">

<img width="866" alt="Screenshot 2022-08-16 at 15 20 04" src="https://user-images.githubusercontent.com/273837/184894059-5089414c-c1f3-407c-a6da-38fd9c3e7c81.png">

See below for a screenshot showing the Component Browser with `5.div ` expression entered. The `div` method expects an `Integer` argument. Only the `number input` virtual component is visible.

<img width="504" alt="Screenshot 2022-08-16 at 15 23 07" src="https://user-images.githubusercontent.com/273837/184894081-358ae780-cfca-4002-b7da-4be57f0357c1.png">

See below for a screenshot showing the Component Browser with `Geo.point ` expression entered. The `point` method expects a `Decimal` argument. Only the `number input` virtual component is visible.

<img width="500" alt="Screenshot 2022-08-16 at 15 24 37" src="https://user-images.githubusercontent.com/273837/184894119-d4f5e7e1-5948-4d8f-adc4-f94a2f4c7245.png">

See below for a screenshot showing the Component Browser with `5.format ` expression entered. The `format` method expects a `Text` argument. Only the `text input` virtual component is visible.


<img width="480" alt="Screenshot 2022-08-16 at 15 55 27" src="https://user-images.githubusercontent.com/273837/184897598-465569c8-8319-43ac-9589-4384ffdc33cc.png">

See below for a video showing that the Component Browser opened with the input port connected to a typed node. The Component Browser does not show virtual entries.

https://user-images.githubusercontent.com/273837/184914619-957a7369-1019-4636-989d-96d65954642f.mov

# Important Notes
- Unused code from the old `enso_gui::controller::searcher::action::hardcoded` module is removed. The `enso_gui::controller::searcher::component::hardcoded::Snippet` type is redefined to contain only the used fields from the old `action::hardcoded::Suggestion` type.
2022-08-18 09:40:41 +00:00
Mateusz Czapliński
3c12a8c0a2
Virtual Entries in Component Browser (#3621)
Add "text input" and "number input" virtual entries in the "Input" virtual component group in the Component Browser. The entries provide an easy way to put strings and numbers into a graph.

https://www.pivotaltracker.com/story/show/181870589

#### Visuals

See below for a video showing the "text input" and "number input" virtual entries in the "Input" component group in the "Favorites Data Science Tools" section of the Component Browser. Please note that the video also displays a few known issues that are present in the existing code and not introduced by this PR:

- "Opening the Component Browser 2nd or later time flashes its last contents from the previous time" - reported as [issue 15 in PR 3530](https://github.com/enso-org/enso/pull/3530#pullrequestreview-1035698205) (which is [expected](https://github.com/enso-org/enso/pull/3530#issuecomment-1187676313) to be fixed by https://www.pivotaltracker.com/story/show/182610422).
- The text of all the entries in the Component Browser does not show immediately, but the entries appear one by one instead (this is related to the performance of the current implementation of Component Browser and Text Area).
- Selection in the Component Browser can show half-way between entries - reported as https://www.pivotaltracker.com/story/show/182713338.

https://user-images.githubusercontent.com/273837/183472391-c14eeded-481f-492e-a1b8-b86f42faf0cd.mov

# Important Notes
- The virtual entries are not filtered by input type or return type. The filtering is expected to be implemented in https://www.pivotaltracker.com/story/show/182842444.
2022-08-17 13:28:07 +00:00
Hubert Plociniczak
68f9fce21a
Use Java's LocalDate for parsing date in tests (#3650)
Rather than using `Date.parse`, which is already being tested in other
tests, we use `LocalDate.parse`. Making use of a helper class to
mitigate API differences.
2022-08-17 09:34:31 +00:00
Radosław Waśko
fbf6c800f1
Short hand version for order_by (#3643)
Implements https://www.pivotaltracker.com/story/show/182868310
2022-08-16 15:41:37 +00:00
Michał W. Urbańczyk
146297a0a0 CR feedback 2022-08-16 15:38:10 +02:00
Michał Wawrzyniec Urbańczyk
6c82588972
Handling exceptions in IDE main to avoid entering "zombie process" state. (#3648)
This is meant to address https://www.pivotaltracker.com/story/show/182691027.
2022-08-16 12:56:08 +00:00
Edward Kmett
0dbbbaad00
Add type system documentation (#3376) 2022-08-15 18:36:03 +03:00
Dmitry Bushev
f10081d905
Increase library upload timeout in tests (#3649) 2022-08-15 12:40:46 +03:00
Dmitry Bushev
c2a5db0063
Update apply-edit benchmark to use lazy visualization (#3642)
This PR adds an ability to check that new API works. There is a separate task to integrate it in IDE
2022-08-15 07:11:51 +00:00
Michał W. Urbańczyk
60be3d2486 Bump build script to avoid issue with nightly builds. 2022-08-15 04:40:37 +02:00