Commit Graph

612 Commits

Author SHA1 Message Date
Jaroslav Tulach
835ac05218
Engine should send notification about node status (#3729)
When nodes get invalidated in the cache, they have to be recomputed. Let the IDE know which of the nodes are pending by sending `Api.ExpressionUpdate.Payload.Pending` message.

# Important Notes
This PR introduces new `Api.ExpressionUpdate.Payload.Pending` message. This message is delivered before re-computation of nodes. Later `Api.ExpressionUpdate.Payload.Value` or other is sent to notify the IDE that a value for given node is available.

Trivial implementation of of the `Api.ExpressionUpdate.Payload.Pending` message in the IDE is provided by this PR to (improperly) visualize pending node status - further improvements needed in follow up PRs.
2022-09-28 12:35:12 +00:00
Marcin Kostrzewa
726bfeb883
Wip/mk/statics (#3740)
Makes statics static. A type and its instances have different methods defined on them, as it should be. Constructors are now scoped in types, and can be imported/exported.

# Important Notes
The method of fixing stdlib chosen here is to just not. All the conses are exported to make all old code work. All such instances are marked with `TODO Dubious constructor export` so that it can be found and fixed.
2022-09-27 14:23:06 +00:00
Hubert Plociniczak
7a6ee0c200
Implement type_of (#3722)
This change implements a simple `type_of` method that returns a type of a given value, including for polyglot objects.

The change also allows for pattern matching on various time-related instances. It is a nice-to-have on its own, but it was primarily needed here to write some tests. For equality checks on types we currently can't use `==` due to a known _feature_ which essentially does wrong dispatching. This will be improved in the upcoming statics PR so we agreed that there is no point in duplicating that work and we can replace it later.

Also, note that this PR changes `Meta.is_same_object`. Comparing types revealed that it was wrong when comparing polyglot wrappers over the same value.
2022-09-26 16:01:39 +00:00
Radosław Waśko
cd10b5d34d
Add Date_Period.Week to start_of and end_of methods (#3733)
Implements https://www.pivotaltracker.com/story/show/183349732
2022-09-23 22:14:35 +00:00
James Dunkerley
6f54e80970
Adjust Database connection to use query/read to access data. (#3727)
Adjust Database connection API to align with new [design](https://github.com/enso-org/design/blob/wip/jd/database-read/epics/basic-libraries/database-read/design.md#querying-tables).
- `query` replaces the old `access_table` and is expanded to support raw SQL queries.
- `read` replaces `execute_query` and matches the API of `query`.
- `to_dataframe` is renamed to `read`.

# Important Notes
Added support for `++` to concatenate a Text without wrapping in a `SQL.Code`.
2022-09-23 07:35:08 +00:00
Hubert Plociniczak
096fcfee82
Generate native image for engine-runner (#3638)
This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m4.503s
user	0m9.248s
sys	0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m0.176s
user	0m0.042s
sys	0m0.038s
```

# Important Notes
Notice that due to a [bug in GraalVM](https://github.com/oracle/graal/issues/4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.

Will need to add the above command to CI.
2022-09-22 14:45:10 +00:00
Radosław Waśko
e9ebc663c1
Add business days functions to Date and Date_Time (#3726)
Implements https://www.pivotaltracker.com/story/show/183082087

# Important Notes
- Removed unnecessary invocations of `Error.throw` improving performance of `Vector.distinct`. The time of the `add_work_days and work_days_until should be consistent with each other` test suite came down from 15s to 3s after the changes.
2022-09-22 08:31:15 +00:00
Michael Mauderer
545e1d7ce9
Show Visualisation Preview when Selecting Item on Searcher (#3691)
Implements [#182634050](https://www.pivotaltracker.com/story/show/182634050).

Enables visualization previews for selections in the searcher.

Works for the old Searcher and the new Component Browser, but it was easier to show the examples for the old searcher, as the suggestions seem to be better/faster available currently (this will improve with the new implementation using better strings and the new GridView).

https://user-images.githubusercontent.com/1428930/188898560-fc4f412a-1529-49f7-9958-28bf5e01c001.mp4

Aborting an edit now also correctly reverts a node.


https://user-images.githubusercontent.com/1428930/188898549-ddd41294-2571-4e2e-b6d5-909cbf71de04.mp4
2022-09-16 16:05:40 +00:00
Hubert Plociniczak
0e5df935d3
Don't rename imported Main module that only imports names (#3710)
Turns that if you import a two-part import we had special code that would a) add Main submodule b) add an explicit rename.

b) is problematic because sometimes we only want to import specific names.
E.g.,
```
from Bar.Foo import Bar, Baz
```
would be translated to
```
from Bar.Foo.Main as Foo import Bar, Baz
```
and it should only be translated to
```
from Bar.Foo.Main import Bar, Baz
```

This change detects this scenario and does not add renames in that case.

Fixes [183276486](https://www.pivotaltracker.com/story/show/183276486).
2022-09-16 13:01:06 +00:00
Hubert Plociniczak
a04425576a
Force recompilation if imported module has changed (#3703)
IR cache never really took into account a situation when a binding from the imported module has changed. In other words, it would continue to happily use the serialized metadata without noticing that it changed.

This change forces cache invalidation when any of the imported modules was invalidated (or rather not loaded from cache).

# Important Notes
Added simple test infrastructure that simulates file modifications that would trigger the initial cache invalidation.
If they succeed, cache invalidation is propagated thus causing an error.
2022-09-15 13:41:58 +00:00
Radosław Waśko
b304402d8e
Add Period Start and End functions to Date and DateTime (#3695)
Implements https://www.pivotaltracker.com/story/show/183081152
2022-09-13 09:51:08 +00:00
Hubert Plociniczak
fba5047acc
Improved Vector/Array interop (#3667)
`Vector` type is now a builtin type. This requires a bunch of additional builtin methods for its creation:
- Use `Vector.from_array` to convert any array-like structure into a `Vector` [by copy](f628b28f5f)
- Use (already existing) `Vector.from_polyglot_array` to convert any array-like structure into a `Vector` **without** copying
- Use (already existing) `Vector.fill 1 item` to create a singleton `Vector`

Additional, for pattern matching purposes, we had to implement a `VectorBranchNode`. Use following to match on `x` being an instance of `Vector` type:
```
import Standard.Base.Data.Vector

size = case x of
Vector.Vector -> x.length
_ -> 0
```

Finally, `VectorLiterals` pass that transforms `[1,2,3]` to (roughly)
```
a1 = 1
a2 = 2
a3 = 3
Vector (Array (a1,a2, a3))
```
had to be modified to generate
```
a1 = 1
a2 = 2
a3 = 3
Vector.from_array (Array (a1, a2, a3))
```
instead to accomodate to the API changes. As of 025acaa676 all the known CI checks passes. Let's start the review.

# Important Notes
Matching in `case` statement is currently done via `Vector_Data`. Use:
```
case x of
Vector.Vector_Data -> True
```
until a better alternative is found.
2022-09-13 03:07:17 +00:00
Jaroslav Tulach
40ef4a01f1
Print result of main (if not Nothing) (#3696)
The goal of this request is to simplify hello world and other trivial Enso programs. No need to learn any standard library functions, enough to write:
```
main = "Hello World!"
```
and the result is going to be printed:
```bash
enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run hello.enso
'Hello World!'
```
the result is only printed, if it is not `Nothing`. E.g. if the last statement is `IO.print ...` (which returns `Nothing`), then no value is printed at the end by the launcher.
2022-09-12 15:37:46 +00:00
James Dunkerley
2b425f8e08
Restructuring Database.Connection to allow for database specific types. (#3632)
- Added `databases`, `database`, `set_database`.
- Added `schemas`, `schema`, `set_schema`.
- Added `table_types`,
- Added `tables`.
- Moved the vast majority of the connection work into a lower level `JDBC_Connection` object.
- `Connection` represents the standard API for database connections and provides a base JDBC implementation.
- `SQLite_Connection` has the `Connection` API but with custom `databases` and `schemas` methods for SQLite.
- `Postgres_Connection` has the `Connection` API but with custom `set_database`, `databases`, `set_schema` and `schemas` methods for Postgres.
- Updated `Redshift` - no public API change.
2022-09-07 17:32:28 +00:00
Radosław Waśko
551100af3b
Add Table.distinct function to In-Memory table (#3684)
Implements https://www.pivotaltracker.com/story/show/182307143

# Important Notes
- Modified standard library Java helpers dependencies so that `std-table` module depends on `std-base`, as a provided dependency. This is allowed, because `std-table` is used by the `Standard.Table` Enso module which depends on `Standard.Base` which ensures that the `std-base` is loaded onto the classpath, thus whenever `std-table` is loaded by `Standard.Table`, so is `std-base`. Thus we can rely on classes from `std-base` and its dependencies being _provided_ on the classpath. Thanks to that we can use utilities like `Text_Utils` also in `std-table`, avoiding code duplication. Additional advantage of that is that we don't need to specify ICU4J as a separate dependency for `std-table`, since it is 'taken' from `std-base` already - so we avoid including it in our build packages twice.
2022-09-07 12:28:41 +00:00
Dmitry Bushev
de0a231417
IDE uses new visualization API (#3661) 2022-09-01 15:33:46 +03:00
Radosław Waśko
65140f48ca
Add storage support for Date, Time and DateTime to InMemory table (#3673)
Implements https://www.pivotaltracker.com/story/show/183080911
2022-08-31 22:06:29 +00:00
Marcin Kostrzewa
4fc6dcced0
Get rid of free-floating atoms. Everything has a type now! (#3671)
This is a step towards the new language spec. The `type` keyword now means something. So we now have
```
type Maybe a
Some (from_some : a)
None
```
as a thing one may write. Also `Some` and `None` are not standalone types now – only `Maybe` is.
This halfway to static methods – we still allow for things like `Number + Number` for backwards compatibility. It will disappear in the next PR.

The concept of a type is now used for method dispatch – with great impact on interpreter code density.

Some APIs in the STDLIB may require re-thinking. I take this is going to be up to the libraries team – some choices are not as good with a semantically different language. I've strived to update stdlib with minimal changes – to make sure it still works as it did.

It is worth mentioning the conflicting constructor name convention I've used: if `Foo` only has one constructor, previously named `Foo`, we now have:
```
type Foo
Foo_Data f1 f2 f3
```

This is now necessary, because we still don't have proper statics. When they arrive, this can be changed (quite easily, with SED) to use them, and figure out the actual convention then.

I have also reworked large parts of the builtins system, because it did not work at all with the new concepts.

It also exposes the type variants in SuggestionBuilder, that was the original tiny PR this was based on.

PS I'm so sorry for the size of this. No idea how this could have been smaller. It's a breaking language change after all.
2022-08-30 22:54:53 +00:00
Jaroslav Tulach
d4b4a31d89
Split long lines to 1024 long chunks (#3665)
Avoid long lines when using `dom().set_inner_text` - rather split the long lines to 1024 chunks and insert them as individual `<div>` elements.

# Important Notes
I was testing the behavior on following program:
```
from Standard.Base import all
import Standard.Base.Data.Statistics
import Standard.Visualization

main =
number1 = 200000
operator1 = 0.up_to number1 . to_vector . map .noise
operator2 = operator1.sort
operator3 = operator2.to_text
```
before my change the visualization of `operator3` was blank. With my change it gets filled with data.
2022-08-29 08:02:33 +00:00
Radosław Waśko
d7ebc4a338
Add Table.take and Table.drop functions to In-Memory table (#3647)
Implements https://www.pivotaltracker.com/story/show/182307347
2022-08-26 19:41:36 +00:00
James Dunkerley
a20d43390e
Adding DateTime part functions (#3669)
- Added `Zone`, `Date_Time` and `Time_Of_Day` to `Standard.Base`.
- Renamed `Zone` to `Time_Zone`.
- Added `century`.
- Added `is_leap_year`.
- Added `length_of_year`.
- Added `length_of_month`.
- Added `quarter`.
- Added `day_of_year`.
- Added `Day_Of_Week` type and `day_of_week` function.
- Updated `week_of_year` to support ISO.

# Important Notes
- Had to pass locale to formatter for date/time tests to work on my PC.
- Changed default of `week_of_year` to use ISO.
2022-08-26 15:47:58 +00:00
Radosław Waśko
fd318cfa96
Remove Array.set_at (#3634)
Implements https://www.pivotaltracker.com/story/show/182879865

# Important Notes
Note that removing `set_at` still does not make our arrays fully immutable - `Array.copy` can still be used to mutate them.
2022-08-26 09:34:33 +00:00
Hubert Plociniczak
d87a32d019
Builtin Date_Time, Time_Of_Day, Zone (#3658)
* Builtin Date_Time, Time_Of_Day, Zone

Improved polyglot support for Date_Time (formerly Time), Time_Of_Day and
Zone. This follows the pattern introduced for Enso Date.

Minor caveat - in tests for Date, had to bend a lot for JS Date to pass.
This is because JS Date is not really only a Date, but also a Time and
Timezone, previously we just didn't consider the latter.
Also, JS Date does not deal well with setting timezones so the trick I
used is to first call foreign function returning a polyglot JS Date,
which is converted to ZonedDateTime and only then set the correct
timezone. That way none of the existing tests had to be changes or
special cased.

Additionally, JS deals with milliseconds rather than nanoseconds so
there is loss in precision, as noted in Time_Spec.

* Add tests for Java's LocalTime

* changelog

* Make date formatters in table happy

* PR review, add more tests for zone

* More tests and fixed a bug in column reader

Column reader didn't take into account timezone but that was a mistake
since then it wouldn't map to Enso's Date_Time.
Added tests that check it now.

* remove redundant conversion

* Update distribution/lib/Standard/Base/0.0.0-dev/src/Data/Time.enso

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>

* First round of addressing PR review

* don't leak java exceptions in Zone

* Move Date_Time to top-level module

* PR review

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Jaroslav Tulach <jaroslav.tulach@enso.org>
2022-08-24 12:31:29 +02:00
Hubert Plociniczak
4b9c91626e
Use Vector.from_polyglot_array to make Vectors backed by polyglot arrays (#3628)
Use Proxy_Polyglot_Array as a proxy for polyglot arrays, thus unifying
the way the underlying array is accessed in Vector.

Used the opportunity to cleanup builtin lookup, which now actually
respects what is defined in the body of @Builtin_Method annotation.

Also discovered that polyglot null values (in JS, Python and R) were leaking to Enso.
Fixed that by doing explicit translation to `Nothing`.

https://www.pivotaltracker.com/story/show/181123986
2022-08-23 21:13:16 +00:00
Jaroslav Tulach
2b9352d2fc
Lazy scatterplot for Vector & Table (#3655)
First of all this PR demonstrates how to implement _lazy visualization_:
- one needs to write/enhance Enso visualization libraries - this PR adds two optional parameters (`bounds` and `limit`) to `process_to_json_text` function.
- the `process_to_json_text` can be tested by standard Enso test harness which this PR also does
- then one has to modify JavaScript on the IDE side to construct `setPreprocessor` expression using the optional parameters

The idea of _scatter plot lazy visualization_ is to limit the amount of points the IDE requests. Initially the limit is set to `limit=1024`. The `Scatter_Plot.enso` then processes the data and selects/generates the `limit` subset. Right now it includes `min`, `max` in both `x`, `y` axis plus randomly chosen points up to the `limit`.

![Zooming In](https://user-images.githubusercontent.com/26887752/185336126-f4fbd914-7fd8-4f0b-8377-178095401f46.png)

The D3 visualization widget is capable of _zooming in_. When that happens the JavaScript widget composes new expression with `bounds` set to the newly visible area. By calling `setPreprocessor` the engine recomputes the visualization data, filters out any data outside of the `bounds` and selects another `limit` points from the new data. The IDE visualization then updates itself to display these more detailed data. Users can zoom-in to see the smallest detail where the number of points gets bellow `limit` or they can select _Fit all_ to see all the data without any `bounds`.

# Important Notes
Randomly selecting `limit` samples from the dataset may be misleading. Probably implementing _k-means clustering_ (where `k=limit`) would generate more representative approximation.
2022-08-23 12:12:22 +00:00
Michael Mauderer
58fb383791
Create new node when opening searcher and no node is selected. (#3645)
So far, when opening the searcher with no node selected, an empty input without an associated AST node was created. This input was manipulated by the user and the final expression from the input was used to create a new node when the user confirmed their input. This PR changes this, so that when the searcher is opened without a selected node, a new AST node is created right away with some placeholder content, and this node is updated when the user confirms their input.

The only change visible to the user, is that if the text editor is opened during editing, a new node will appear in the source when the searcher is opened to create a new node. All other behaviour should stay the same.
2022-08-22 15:32:44 +00:00
Radosław Waśko
bcca7f10d9
Add key functions to Table to make it act as [Column] (#3644)
Implements https://www.pivotaltracker.com/story/show/181370836
2022-08-18 12:33:02 +00:00
Radosław Waśko
fbf6c800f1
Short hand version for order_by (#3643)
Implements https://www.pivotaltracker.com/story/show/182868310
2022-08-16 15:41:37 +00:00
Michał Wawrzyniec Urbańczyk
6c82588972
Handling exceptions in IDE main to avoid entering "zombie process" state. (#3648)
This is meant to address https://www.pivotaltracker.com/story/show/182691027.
2022-08-16 12:56:08 +00:00
Hubert Plociniczak
8575b76b0a
Support pattern matching on constants (#3641)
This change adds support for matching on constants by:
1) extending parser to allow literals in patterns
2) generate branch node for literals

Related to https://www.pivotaltracker.com/story/show/182743559
2022-08-12 13:18:58 +00:00
Hubert Plociniczak
3fa78afd10
Support Autosave for open buffers (#3637)
This change adds Autosave action for open buffers. The action is scheduled
after every edit request and is cancelled by every explicit save file request, if
necessary. Successful autosave also notifies any active clients of the buffer.

Related to https://www.pivotaltracker.com/story/show/182721656

# Important Notes
WIP
2022-08-11 11:45:12 +00:00
Radosław Waśko
3dca738cf7
Add Vector.take and Vector.drop functions (#3629)
Implements https://www.pivotaltracker.com/story/show/182307048
2022-08-10 16:02:02 +00:00
Dmitry Bushev
98d30bccf3
Enable caching in visualization functions (#3618)
PR allows to attach metod pointers as a visualization expressions. This way it allows to attach a runtime instrument that enables caching of intermediate expressions.

# Important Notes
ℹ️ API is backward compatible.

To attach the visualization with caching support, the same `executionContext/attachVisualisation` method is used, but `VisualisationConfig` message should contain the message pointer.
While `VisualisationConfiguration` message has changed, the language server accepts both new and old formats to keep visualisations working in IDE.

#### Old format

```json
{
"executionContextId": "UUID",
"visualisationModule": "local.Unnamed.Main",
"expression": "x -> x.to_text"
}
```

#### New format

```json
{
"executionContextId": "UUID",
"expression": {
"module": "local.Unnamed.Main",
"definedOnType": "local.Unnamed.Main",
"name": "encode"
}
}
```
2022-08-10 12:01:33 +00:00
Dmitry Bushev
5e114acbb5
Update Scala to 2.13.8 (#3631)
Update Scala compiler and libraries.
2022-08-08 19:32:55 +00:00
Hubert Plociniczak
42dbd8bb59
Allow for importing methods (#3633)
Importing individual methods didn't work as advertised because parser
would allow them but later drop that information. This slipped by because we never had mixed atoms and methods in stdlib.

# Important Notes
Added some basic tests but we need to ensure that the new parser allows for this.
@jdunkerley will be adding some changes to stdlib that will be testing this functionality as well.
2022-08-05 16:25:51 +00:00
Radosław Waśko
0a2fea925c
Create Index_Sub_Range type and update Text.take and Text.drop (#3617) 2022-08-03 11:41:34 +00:00
Adam Obuchowicz
7f8190e663
Grid view selection and hover (#3622)
This PR adds a new variant of selection, where the mouse-hovered entry is highlighted and may be selected by clicking.

In the video below, we have three grid views with slightly different settings:
* In the left-top corner, both hover and selection highlight is just a shape under the label. Such a grid view does not require additional layers (when compared to non-selectable grid view).
* In the left-bottom corner the hover is normal shape, but selection is a _masked layer_ which allows us to have different text color. This setting requires three more layers to render.
* In the right-top corner, both hover and selection are displayed in the masked layer, creating 6 additional layers.

https://user-images.githubusercontent.com/3919101/181514178-f243bfeb-f2dd-4507-adc3-5344ae0579b7.mp4
2022-08-01 10:54:42 +00:00
Hubert Plociniczak
d59714a29d
Support module imports using a qualified name (#3608)
This change allows for importing modules using a qualified name and deals with any conflicts on the way.
Given a module C defined at `A/B/C.enso` with
```
type C
type C a
```
it is now possible to import it as
```
import project.A
...
val x = A.B.C 10
```

Given a module located at `A/B/C/D.enso`, we will generate
intermediate, synthetic, modules that only import and export the successor module along the path.
For example, the contents of a synthetic module B will look like
```
import <namespace>.<pkg-name>.A.B.C
export <namespace>.<pkg-name>.A.B.C
```
If module B is defined already by the developer, the compiler will _inject_ the above statements to the IR.

Also removed the last elements of some lowercase name resolution that managed to survive recent
changes (`Meta.Enso_Project` would now be ambiguous with `enso_project` method).

Finally, added a pass that detects shadowing of the synthetic module by the type defined along the path.
We print a warning in such a situation.

Related to https://www.pivotaltracker.com/n/projects/2539304

# Important Notes
There was an additional request to fix the annoying problem with `from` imports that would always bring
the module into the scope. The changes in stdlib demonstrate how it is now possible to avoid the workaround of
```
from X.Y.Z as Z_Module import A, B
```
(i.e. `as Z_Module` part is almost always unnecessary).
2022-07-29 14:19:07 +00:00
Hubert Plociniczak
f63e40df1b
Explicit self (#3569)
This change modifies the current language by requiring explicit `self` parameter declaration
for methods. Methods without `self` parameter in the first position should be treated as statics
although that is not yet part of this PR. We add an implicit self to all methods
This obviously required updating the whole stdlib and its components, tests etc but the change
is pretty straightforward in the diff.

Notice that this change **does not** change method dispatch, which was removed in the last changes.
This was done on purpose to simplify the implementation for now. We will likely still remove all
those implicit selfs to bring true statics.
Minor caveat - since `main` doesn't actually need self, already removed that which simplified
a lot of code.
2022-07-27 17:45:36 +00:00
Dmitry Bushev
0701e762ea
Add Language Server benchmarking tool (#3578) 2022-07-22 14:12:52 +00:00
James Dunkerley
be311457bd
Add Linear Regression support for Vectors. (#3601)
Adds least squares regression APIs. Covers the basic 4 trend line types from Excel (doesn't cover Polynomial or Moving Average).
Removes the old `Model` from the `Standard.Table`.
2022-07-22 08:41:17 +00:00
Radosław Waśko
16fd038c1a
Add support for .pgpass to PostgreSQL (#3593)
Implements https://www.pivotaltracker.com/story/show/182582924
2022-07-21 13:32:37 +00:00
Jaroslav Tulach
4465d63dd8
Improved polyglot Date support (#3559)
Significantly improves the polyglot Date support (as introduced by #3374). It enhances the `Date_Spec` to run it in four flavors:
- with Enso Date (as of now)
- with JavaScript Date
- with JavaScript Date wrapped in (JavaScript) array
- with Java LocalDate allocated directly

The code is then improved by necessary modifications to make the `Date_Spec` pass.

# Important Notes
James has requested in [#181755990](https://www.pivotaltracker.com/n/projects/2539304/stories/181755990) - e.g. _Review and improve InMemory Table support for Dates, Times, DateTimes, BigIntegers_ the following program to work:
```
foreign js dateArr = """
return [1, new Date(), 7]

main =
IO.println <| (dateArr.at 1).week_of_year
```
the program works with here in provided changes and prints `27` as of today.

@jdunkerley has provided tests for proper behavior of date in `Table` and `Column`. Those tests are working as of [f16d07e](f16d07e640). One just needs to accept `List<Value>` and then query `Value` for `isDate()` when needed.

Last round of changes is related to **exception handling**. 8b686b12bd makes sure `makePolyglotError` accepts only polyglot values. Then it wraps plain Java exceptions into `WrapPlainException` with `has_type` method - 60da5e70ed - the remaining changes in the PR are only trying to get all tests working in the new setup.

The support for `Time` isn't part of this PR yet.
2022-07-21 06:32:40 +00:00
James Dunkerley
5e4083978f
Type name case fixes: (#3590)
- MacOS => Mac_OS
- PostgreSQL => Postgres
- SQLite => SQLite (align a few)
- InMemory => In_Memory
- PointData => Point_Data
- Io_Error => IO_Error
- Standard.Table.Io => Standard.Table.IO

In Tests:
- MyError => My_Error
- NotFoo => Not_Foo
2022-07-19 14:09:09 +00:00
Adam Obuchowicz
7fa4e5e369
Grid View with Scrolling (#3588)
**Note**: This PR also contains content of previous Grid View PR. We decided to discard the previous, because this one did some refactoring of old one, and it's not a big addition.

Added a scrollable::GridView component, which just embeds the GridView in ScrollArea. Also, re-worked the idea of text layers.

https://user-images.githubusercontent.com/3919101/179020359-512ee127-c333-4f86-bff5-f1cb4154e03c.mp4
2022-07-19 08:39:23 +00:00
Radosław Waśko
35ddd2a89e
Add new options to the Delimited format (#3581)
Implements https://www.pivotaltracker.com/story/show/182662195 and https://www.pivotaltracker.com/story/show/182651884
2022-07-14 15:01:26 +00:00
Adam Obuchowicz
b0d627a797
Component list panel integration (#3530)
This PR contains all work for finishing integration of first Component List Panel in the IDE:
* It adds a stub for the whole Component Browser View. The documentation panel is re-used from the old searcher.
* It has the presenter implementation, integrating the view with Hierarchical Component List from the controller.
* It extends the View API, so the integration is possible, making use of Component Group Set wrapper.
* The selection integration was also merged into this PR, because it depended on the API extension mentioned above. However, we should avoid such practice in the future.

https://user-images.githubusercontent.com/3919101/177816427-8c4285b4-8941-4048-a400-52f4acf77a9f.mp4

# Important Notes
There are some known issues, to-be-fixed in the future.
* The performance is bad. It should be improved with new text::Area, and the decent one shall come with [GridView inside component browser](https://www.pivotaltracker.com/story/show/182561072)
* There is no keyboard navigation. It should also be delivered with [GridView](https://www.pivotaltracker.com/story/show/182561072).
* The Favorites section is not [filtered out by node source type](https://www.pivotaltracker.com/story/show/182661634).
2022-07-14 12:00:52 +00:00
James Dunkerley
9578dc1e43
Move write_bytes to be part of Vector. (#3583)
Updates `write_bytes` API to be part of `Vector` and to conform to `write` APIs.

# Important Notes
Ensures doesn't touch the file if an invalid byte array.
2022-07-14 11:30:40 +00:00
Dmitry Bushev
2c780ef6ba
Add command for changing expression value (#3562)
Implement `text/applyExpressionValue` command.
2022-07-13 15:59:47 +00:00
Radosław Waśko
df10e4ba7c
Add appending support for Delimited files (#3573)
Implements https://www.pivotaltracker.com/story/show/182309839
2022-07-11 12:36:01 +00:00
Jaroslav Tulach
735053c218
Implementing basic functions (#3554)
The language specification suggests to add [five basic functions into the standard library](https://github.com/enso-org/design/blob/wip/wd/enso-spec/epics/enso-spec-1.0/05.%20Functions.md#useful-functions-in-the-standard-library). `identity`, `flip`, `const`, `curry` & `uncurry`.

# Important Notes
The new functions are being added into existing `Function.enso` file. That may not be the best place, but it is not clear from the [design spec](https://github.com/enso-org/design/blob/wip/wd/enso-spec/epics/enso-spec-1.0/05.%20Functions.md#useful-functions-in-the-standard-library) how they are supposed to be imported. I can move them wherever needed.

There is a documentation provided for each of the functions, but I am not sure how to verify it is correct. Do we generate the documentation for stdlib somehow?
2022-07-11 10:30:44 +00:00
Radosław Waśko
28513a3389
Allow filtering caught error type in Error.catch (#3574)
More and more often I need a way to only recover a specific type of a dataflow error (in a similar manner as with panics). So the API for `Error.catch` has been amended to more closely resemble `Panic.catch`, allowing to handle only specific types of dataflow errors, passing others through unchanged. The default is `Any`, meaning all errors are caught by default, and the behaviour of `x.catch` remains unchanged.
2022-07-11 08:26:44 +00:00
Hubert Plociniczak
96e50648dd
Remove 'here' and make method name resolution case-sensitive (#3538)
Modified UppercaseNames to now resolve methods without an explicit `here` to point to the current module.
`here` was also often used instead of `self` which was allowed by the compiler.
Therefore UppercaseNames pass is now GlobalNames and does some extra work -
it translated method calls without an explicit target into proper applications.

# Important Notes
There was a long-standing bug in scopes usage when compiling standalone expressions.
This resulted in AliasAnalysis generating incorrect graphs and manifested itself only in unit tests
and when running `eval`, thus being a bit hard to locate.
See `runExpression` for details.

Additionally, method name resolution is now case-sensitive.

Obsolete passes like UndefinedVariables and ModuleThisToHere were removed. All tests have been adapted.
2022-07-07 10:31:06 +00:00
James Dunkerley
16e6f2fa08
Adding Append support to Excel.Write (#3558)
Adds support for appending to an existing Excel table.

# Important Notes
- Renamed `Column_Mapping` to `Column_Name_Mapping`
- Changed new type name to `Map_Column`
- Added last modified time and creation time to `File`.
2022-07-07 06:41:33 +00:00
Radosław Waśko
7c94fa6a77
Custom Encoding support when writing Delimited files (#3564)
Implements https://www.pivotaltracker.com/story/show/182545847
2022-07-07 00:20:00 +00:00
Mateusz Czapliński
d950499a90
Pan camera to created nodes. (#3552)
If a node created by the user gets placed off-screen, the screen's camera is panned to make the node visible.

https://www.pivotaltracker.com/story/show/181188687

#### Visuals

A screencast showing a number of node creation scenarios when the camera is panned to the newly created node, including when zoomed out.



https://user-images.githubusercontent.com/273837/177169716-50a12b0a-c742-4b01-9766-56206e7938b9.mov

# Important Notes
- Camera is panned also if the node is only partially visible, or if there's not enough free space visible around the node. The specific amount of free space that needs to be visible around a newly created node is configured in the theme.
- If the screen area is so small that the node cannot be fully fit in it (either horizontally or vertically), showing the left and top boundaries of the node's area takes priority over showing the corresponding opposite edges.
2022-07-06 10:33:35 +00:00
James Dunkerley
5174cc6ece
Update Database.connect to match new API (#3542)
Initial work restructuring the `Database.connect` API
- New SQLite API with support for InMemory.
- Updated PostgreSQL API with SSL and Client Certificate Support.
- Updated Redshift API.

# Important Notes
Follow up tasks:
- PostgreSQL SSL additional testing.
- Driver version updating.
- `.pgpass` support.
2022-07-04 20:26:44 +00:00
James Dunkerley
4ca2097488
Adding write support to File_Format.Excel (#3551)
Support for writing tables to Excel.

# Important Notes
Has custom support for Error mode as will allow appending a new table in this mode to the file.
2022-07-04 18:32:16 +00:00
Radosław Waśko
972b34d1a9
Implement value formatting and writing new files in Delimited format. (#3528)
Implements https://www.pivotaltracker.com/story/show/182309429 and https://www.pivotaltracker.com/story/show/182309573
2022-06-23 16:51:52 +00:00
Jaroslav Tulach
479f74cc84
Enso Integration with Ideal Graph Visualizer (#3533)
This PR adds sources for Enso language support in IGV (and NetBeans). The support is based on TextMate grammar shown in the editor and registration of the Enso language so IGV can find it. Then this PR adds new GitHub Actions workflow file to build the project using Maven.
2022-06-23 04:43:49 +00:00
James Dunkerley
7a2d304fa0
Update Excel reading API (#3523)
- Remove `from_xls` and `from_xlsx`.
- Add `headers` support to `File_Format.Excel`.
- Altered default read for Excel to be the first sheet.
- Altered behavior so that single cells grow down and right when reading sheet.
- Altered `Excel_Range` so knows if single cell or 1x1 range address.

# Important Notes
- Renamed `Range` to `Cell_Range` to avoid name clash.
2022-06-21 13:39:32 +00:00
Hubert Plociniczak
22a371a9c6
Substitute this with self (#3524)
A semi-manual s/this/self appied to the whole standard library.
Related to https://www.pivotaltracker.com/story/show/182328601

In the compiler promoted to use constants instead of hardcoded
`this`/`self` whenever possible.

# Important Notes
The PR **does not** require explicit `self` parameter declaration for methods as this part
of the design is still under consideration.
2022-06-21 10:53:52 +00:00
Jaroslav Tulach
c72a6582bc
Avoid whole source reparsing when the IDE performs a simple edit (#3508)
Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
Co-authored-by: Hubert Plociniczak <hubert.plociniczak@gmail.com>
2022-06-16 16:02:57 +02:00
Marcin Kostrzewa
ec3fa32fec
introduce a micro stdlib for testing (#3531)
This introduces a tiny alternative to our stdlib, that can be used for testing the interpreter. There are 2 main advantages of such a solution:
1. Performance: on my machine, `runtime-with-intstruments/test` drops from 146s to 65s, while `runtime/test` drops from 165s to 51s. >6 mins total becoming <2 mins total is awesome. This alone means I'll drink less coffee in these breaks and will be healthier.
2. Better separation of concepts – currently working on a feature that breaks _all_ enso code. The dependency of interpreter tests on the stdlib means I have no means of incremental testing – ALL of stdlib must compile. This is horrible, rendered my work impossible, and resulted in this PR.
2022-06-16 10:25:24 +00:00
James Dunkerley
a0c6fa9c96
Removing old functions and tidy up of Table types (#3519)
- Removed `select` method.
- Removed `group` method.
- Removed `Aggregate_Table` type.
- Removed `Order_Rule` type.
- Removed `sort` method from Table.
- Expanded comments on `order_by`.
- Update comment on `aggregate` on Database.
- Update Visualisation to use new APIs.
- Updated Data Science examples to use new APIs.
- Moved Examples test out of Tests to own test.

# Important Notes
Need to get Examples_Tests added to CI.
2022-06-14 13:37:20 +00:00
Hubert Plociniczak
fd46e84e8d
Towards a full-blown builtins DSL (part 3) (#3471)
Auto-generate all builtin methods for builtin `File` type from method signatures.
Similarly, for `ManagedResource` and `Warning`.
Additionally, support for specializations for overloaded and non-overloaded methods is added.
Coverage can be tracked by the number of hard-coded builtin classes that are now deleted.

## Important notes

Notice how `type File` now lacks `prim_file` field and we were able to get rid off all of those
propagating method calls without writing a single builtin node class.
Similarly `ManagedResource` and `Warning` are now builtins and `Prim_Warnings` stub is now gone.
2022-06-13 11:48:34 +00:00
Radosław Waśko
a04825a5ce
Add Text.write Function (#3518)
Implements https://www.pivotaltracker.com/story/show/182309026
2022-06-13 09:11:46 +00:00
James Dunkerley
e97d27e1e0
Adjusting First and Last order_by to use Sort_Column_Selector (#3517) 2022-06-10 09:59:03 +00:00
James Dunkerley
8afba43add
Implement In-Memory Table order_by (#3515)
Implemented the `order_by` function with support for all modes of operation.
Added support for case insensitive natural order.

# Important Notes
- Improved MultiValueIndex/Key to not create loads of arrays.
- Adjusted HashCode for MultiValueKey to have a simple algorithm.
- Added Text_Utils.compare_normalized_ignoring_case to allow for case insensitive comparisons.
- Fixed issues with ObjectComparator and added some unit tests for it.
2022-06-08 12:30:50 +00:00
Radosław Waśko
2af970fe52
Basic changes to File_Format (#3516)
Implements https://www.pivotaltracker.com/story/show/182308987
2022-06-08 09:53:18 +00:00
Hubert Plociniczak
b1db359f19
Minor compilation improvements (#3512)
Drop `Core` implementation (replacement for IR) as it (sadly) looks increasingly
unlikely this effort will be continued. Also, it heavily relies
on implicits which increases some compilation time (~1sec from `clean`)

Related to https://www.pivotaltracker.com/story/show/182359029
2022-06-07 14:48:50 +00:00
Radosław Waśko
a382e0c15e
Improve database Table.order_by (#3514)
Implements https://www.pivotaltracker.com/story/show/182195405

Adds support for the Postgres dialect and simple case insensitive collation for SQLite.
2022-06-07 12:31:55 +00:00
Radosław Waśko
7d94efa6f2
Implement Table.order_by for SQLite and the common scaffolding for all backends (#3502)
Implements the common and SQLite parts of https://www.pivotaltracker.com/story/show/182195405
2022-06-06 10:56:52 +00:00
Dmitry Bushev
0e867c663e
Keep the components order (#3505)
PR ensures that the component groups are returned in the same order that the packages were loaded.
2022-06-03 18:45:56 +00:00
Hubert Plociniczak
e43325bfe1
Short-circuiting || and && (#3492)
Short-circuiting || and && is typically taken for granted
by users of other PLs. This change makes it happen for Enso.

Related to https://www.pivotaltracker.com/story/show/182261401
2022-06-02 16:58:38 +00:00
Dmitry Bushev
218712301d
Add background suggestions processing (#3493)
PR implements background job processing.

#### Before
![2022-05-30-084652_1158x123_scrot](https://user-images.githubusercontent.com/357683/170936632-06413b0f-fb6c-4c31-89ca-a3fbd6f7293e.png)

#### After
![2022-05-30-084634_1118x117_scrot](https://user-images.githubusercontent.com/357683/170936626-9ed21b5c-f44e-4878-bdd8-4866dfeba060.png)

After moving the suggestions building out of the compilation job, the initial compilation time has been reduced by 25% (before ~11 seconds, after ~8 seconds).

[language-server.npss.zip](https://github.com/enso-org/enso/files/8796281/language-server.npss.zip)

Changelog:
- Implement background jobs executor
- Implement jobs for suggestion processing
- Move suggestion processing out of the compilation job
- Update handling of expected messages in various runtime tests. The runtime tests execution time was improved by a couple of minutes
2022-06-01 09:39:48 +00:00
James Dunkerley
1aa0bb3552
Rank Data, Correlation, Covariance, R Squared (#3484)
- Added new `Statistic`s: Covariance, Pearson, Spearman, R Squared
- Added `covariance_matrix` function
- Added `pearson_correlation` function to compute correlation matrix
- Added `rank_data` and Rank_Method type to create rankings of a Vector
- Added `spearman_correlation` function to compute Spearman Rank correlation matrix

# Important Notes
- Added `Panic.throw_wrapped_if_error` and `Panic.handle_wrapped_dataflow_error` to help with errors within a loop.
- Removed `Array.set_at` use from `Table.Vector_Builder`
2022-05-30 17:13:06 +00:00
Adam Obuchowicz
d07246d84c
Fix depth-ordering update (#3486)
There are two dirty flags in layers: depth_order_dirty and element_depth_order_dirty - one marking changed in Layer, second marking change in one of sublayers. The depth_order_dirty has a proper callback for setting element_depth_order_dirty of its parent. However, the latter did not propagate up.
I fixed it by adding callback for element_depth_order_dirty which sets the depth_order_dirty of the parent.

# Important Notes
* The question to @wdanilo : is it possible, that I can propagate dirty directly to element_depth_order_dirty, without setting depth_order_dirty? As far as I understand the code, it would also work (and we would omit some unnecessary updates).
* I tried to leave some logs, but I don't feel how to do that: the tooling I used was very specific, only the concrete ids of symbols and layers were logged, and I don't know how to generalize it.
2022-05-30 10:32:43 +00:00
Radosław Waśko
8828d801ea
Implement Table from Text conversion (#3478)
Implements https://www.pivotaltracker.com/story/show/181824168
2022-05-26 12:04:25 +00:00
Radosław Waśko
7f572bf3e4
The user should be able to have the headers Inferred when reading a Delimited file (#3472)
Implements https://www.pivotaltracker.com/story/show/181986831
2022-05-25 13:29:17 +00:00
Radosław Waśko
ec1b072824
Integrate value parsing with Delimited file reading (#3463)
Implements https://www.pivotaltracker.com/story/show/182200028
2022-05-24 17:59:00 +02:00
Dmitry Bushev
f9d2964e83
Update profiling CLI arguments (#3461) 2022-05-24 16:01:26 +03:00
Jaroslav Tulach
2973cd2db2
Avoid needless concatenations of warning/error messages (#3465)
While investigating [#182234656](https://www.pivotaltracker.com/n/projects/2539304/stories/182234656) I've noticed that  `GatherDiagnostics` calls `distinctBy` and uses `message` string. However such string has to be composed and that takes time:

![image](https://user-images.githubusercontent.com/26887752/169638644-cf7dd826-1016-402b-842a-fc27a77a64e1.png)

This PR eliminates the need for concatenation by associating each `Diagnostic` with an array of `keys` used to compute `equals` and `hashCode`.
2022-05-24 07:16:36 +02:00
Radosław Waśko
ff7700ebb1
Automatic inference of value types when parsing table columns (#3462)
Implements https://www.pivotaltracker.com/story/show/182199966
2022-05-20 15:08:36 +00:00
Hubert Plociniczak
688df9825c
Part 2 of system for builtin objects (#3454)
This is the 2nd part of DSL improvements that allow us to generate a lot of
builtins-related boilerplate code.
- [x] generate multiple method nodes for methods/constructors with varargs
- [x] expanded processing to allow for @Builtin to be added to classes and
and generate @BuiltinType classes
- [x] generate code that wraps exceptions to panic via `wrapException`
annotation element (see @Builtin.WrapException`

Also rewrote @Builtin annotations to be more structured and introduced some nesting, such as
@Builtin.Method or @Builtin.WrapException.

This is part of incremental work and a follow up on https://github.com/enso-org/enso/pull/3444.

# Important Notes
Notice the number of boilerplate classes removed to see the impact.
For now only applied to `Array` but should be applicable to other types.
2022-05-19 10:43:47 +00:00
Hubert Plociniczak
12d6ef799f
Refactor methods of Managed_Resource (#3460)
Promoted `with`, `take`, `finalize` to be methods of Managed_Resource
rather than static methods always taking `resource`, for consistency
reasons.

This required function dispatch boilerplate, similarly to `Ref`.
In future iterations we will address this boilerplate code.

Related to https://www.pivotaltracker.com/story/show/182212217
2022-05-18 17:27:42 +00:00
Radosław Waśko
8430ce2625
Parsing values with known types (#3455)
Implements https://www.pivotaltracker.com/story/show/181824146
2022-05-18 15:27:48 +00:00
Hubert Plociniczak
6b6b1430bc
Cleanup Ref - get/put (#3457)
The change promotes static methods of `Ref`, `get` and `put`, to be
methods of `Ref` type.
The change also removes `Ref` module from the default namespace.
Had to mostly c&p functional dispatch for now, in order for the methods
to be found. Will auto-generate that code as part of builtins system.

Related to https://www.pivotaltracker.com/story/show/182138899
2022-05-17 10:26:36 +00:00
Dmitry Bushev
d74fffd550
Replace Rope with CharSequence when building suggestions (#3453)
PR addresses an issue when during the initial compilation the `EnsureCompiledJob` depends too much time building the `LineView` of ropes.
Replacing the `Rope` with `CharSequence` in the compilation job reduces total time building the suggestions from ~600 ms to ~200 ms.

#### Before
![2022-05-13-160549_1310x120_scrot](https://user-images.githubusercontent.com/357683/168289736-fe8983ef-9bcc-4df0-bcd2-881bc2949773.png)
#### After
![2022-05-13-160609_1303x123_scrot](https://user-images.githubusercontent.com/357683/168289741-3326603d-8183-4925-b995-435655d6c8be.png)

[context-registry-npss.zip](https://github.com/enso-org/enso/files/8687294/context-registry-npss.zip)
2022-05-16 06:47:35 +00:00
Hubert Plociniczak
a2dae60aa9
Generate BuiltinMethods from simple method and constructor signatures (#3444)
A low-hanging fruit where we can automate the generation of many
@BuiltinMethod nodes simply from the runtime's methods signatures.
This change introduces another annotation, @Builtin, to distinguish from
@BuiltinType and @BuiltinMethod processing. @Builtin processing will
always be the first stage of processing and its output will be fed to
the latter.

Note that the return type of Array.length() is changed from `int` to
`long` because we probably don't want to add a ton of specializations
for the former (see comparator nodes for details) and it is fine to cast
it in a small number of places.

Progress is visible in the number of deleted hardcoded classes.

This is an incremental step towards #181499077.

# Important Notes
This process does not attempt to cover all cases. Not yet, at least.
We only handle simple methods and constructors (see removed `Array` boilerplate methods).
2022-05-12 08:42:00 +00:00
James Dunkerley
4f3a76817c
Statistics on a Vector (#3442)
- Implements various statistics on Vector

# Important Notes
Some minor codebase improvements:
- Some tweaks to Any/Nothing to improve performance
- Fixed bug in ObjectComparator
- Added if_nothing
- Removed Group_By_Key
2022-05-11 13:25:06 +00:00
Radosław Waśko
64f178f7a8
Delimited File Encoding (#3430)
Implements https://www.pivotaltracker.com/story/show/181998375
2022-05-10 22:44:05 +00:00
Jaroslav Tulach
21c46901b7
Debug Enso language in ChromeDev tools with --inspect option (#3432)
Finally this pull request proposes `--inspect` option to allow [debugging of `.enso`](e948f2535f/docs/debugger/README.md) in Chrome Developer Tools:

```bash
enso$ ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --inspect --run ./test/Tests/src/Data/Numbers_Spec.enso
Debugger listening on ws://127.0.0.1:9229/Wugyrg9Nm4OUL9YhzdcElmLft71ayZW3LMUPCdPyNAY
For help, see: https://www.graalvm.org/tools/chrome-debugger
E.g. in Chrome open: devtools://devtools/bundled/js_app.html?ws=127.0.0.1:9229/Wugyrg9Nm4OUL9YhzdcElmLft71ayZW3LMUPCdPyNAY
```
copy the printed URL into chrome browser and you should see:

![obrazek](https://user-images.githubusercontent.com/26887752/167235327-8ad15fb2-96d4-4a0c-9e31-ed67ab46578b.png)

One can also debug the `.enso` files in NetBeans or [VS Code with Apache Language Server extension](https://cwiki.apache.org/confluence/display/NETBEANS/Apache+NetBeans+Extension+for+Visual+Studio+Code) just pass in special JVM arguments:
```bash
enso$ JAVA_OPTS=-agentlib:jdwp=transport=dt_socket,server=y,address=8000 ./built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --run ./test/Tests/src/Data/Numbers_Spec.enso
Listening for transport dt_socket at address: 8000
```
and then _Debug/Attach Debugger_. Once connected choose the _Toggle Pause in GraalVM Script_ button in the toolbar (the "G" button):

![obrazek](https://user-images.githubusercontent.com/26887752/167235598-98266c7e-beb5-406b-adc6-8167b3d1b453.png)

and your execution shall stop on the next `.enso` line of code. This mode allows to debug both - the Enso code as well as Java code.

Originally started as an attempt to write test in Java:

* test written in Java
* support for JUnit in `build.sbt`
* compile Java with `-g` - so it can be debugged
* Implementation of `StatementNode` - only gets created when `materialize` request gets to `BlockNode`
2022-05-10 08:55:08 +00:00
James Dunkerley
078c665a60
File_Format.Excel work (#3425)
- Read in Excel files following the specification.
- Support for XLSX and XLS formats.
- Ability to select ranges and sheets.
- Skip Rows and Row Limits.

# Important Notes
- Minor fix to DelimitedReader for Windows
2022-05-06 13:21:10 +00:00
Hubert Plociniczak
4bbabc00be
Move Builtin Types and Methods to stdlib (#3363)
This PR replaces hard-coded `@Builtin_Method` and `@Builtin_Type` nodes in Builtins with an automated solution
that a) collects metadata from such annotations b) generates `BuiltinTypes` c) registers builtin methods with corresponding
constructors.
The main differences are:
1) The owner of the builtin method does not necessarily have to be a builtin type
2) You can now mix regular methods and builtin ones in stdlib 
3) No need to keep track of builtin methods and types in various places and register them by hand (a source of many typos or omissions as it found during the process of this PR)

Related to #181497846
Benchmarks also execute within the margin of error.

### Important Notes

The PR got a bit large over time as I was moving various builtin types and finding various corner cases.
Most of the changes however are rather simple c&p from Builtins.enso to the corresponding stdlib module.
Here is the list of the most crucial updates:
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/builtin/Builtins.java` - the core of the changes. We no longer register individual builtin constructors and their methods by hand. Instead, the information about those is read from 2 metadata files generated by annotation processors. When the builtin method is encountered in stdlib, we do not ignore the method. Instead we lookup it up in the list of registered functions (see `getBuiltinFunction` and `IrToTruffle`)
- `engine/runtime/src/main/java/org/enso/interpreter/runtime/callable/atom/AtomConstructor.java` has now information whether it corresponds to the builtin type or not.
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/RuntimeStubsGenerator.scala` - when runtime stubs generator encounters a builtin type, based on the @Builtin_Type annotation, it looks up an existing constructor for it and registers it in the provided scope, rather than creating a new one. The scope of the constructor is also changed to the one coming from stdlib, while ensuring that synthetic methods (for fields) also get assigned correctly
- `engine/runtime/src/main/scala/org/enso/compiler/codegen/IrToTruffle.scala` - when a builtin method is encountered in stdlib we don't generate a new function node for it, instead we look it up in the list of registered builtin methods. Note that Integer and Number present a bit of a challenge because they list a whole bunch of methods that don't have a corresponding method (instead delegating to small/big integer implementations).
During the translation new atom constructors get initialized but we don't want to do it for builtins which have gone through the process earlier, hence the exception
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor not only  generates the actual code fpr nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization 
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/MethodProcessor.java` - @Builtin_Method processor no longer generates only (root) nodes but also collects and writes the info about them (name, class, params) to a metadata file that is read during builtins initialization
- `lib/scala/interpreter-dsl/src/main/java/org/enso/interpreter/dsl/TypeProcessor.java` - Similar to MethodProcessor but handles @Builtin_Type annotations. It doesn't, **yet**, generate any builtin objects.  It also collects the names, as present in stdlib, if any, so that we can generate the names automatically (see generated `types/ConstantsGen.java`)
- `engine/runtime/src/main/java/org/enso/interpreter/node/expression/builtin` - various classes annotated with @BuiltinType to ensure that the atom constructor is always properly registered for the builitn. Note that in order to support types fields in those, annotation takes optional `params` parameter (comma separated). 
- `engine/runtime/src/bench/scala/org/enso/interpreter/bench/fixtures/semantic/AtomFixtures.scala` - drop manual creation of test list which seemed to be a relict of the old design
2022-05-05 20:18:06 +02:00
Marcin Kostrzewa
ce6a97e977
Sum type info in suggestions DB (#3422)
A draft of simple changes to the compiler to expose sum type information. Doesn't break the stdlib & at the same time allows for dropdowns. This is still broken, for example it doesn't handle exporting/importing types, only ones defined in the same module as the signature. Still, seems like a step in the right direction – please provide feedback.

# Important Notes
I've decided to make the variant info part of the type, not the argument – it is a property of the type logically.

Also, I've pushed it as far as I'm comfortable – i.e. to the `SuggestionHandler` – I have no idea if this is enough to show in IDE? cc @4e6
2022-05-05 16:05:22 +00:00
Jaroslav Tulach
3ab2c8f8a0
Delay conversion of Truffle function body nodes until the function is invoked (#3429)
Most of the functions in the standard library aren't gonna be invoked during particular program execution. It makes no sense to build their Truffle AST for the functions that are not executing. Let's delay the construction of the tree until a function is first executed.
2022-05-05 12:53:49 +02:00
Hubert Plociniczak
79c82da21c
Frgaal integration in sbt (#3421)
* Initial integration with Frgaal in sbt

Half-working since it chokes on generated classes from annotation
processor.

* Replace AutoService with ServiceProvider

For reasons unknown AutoService would fail to initialize and fail to
generate required builtin method classes.
Hidden error message is not particularly revealing on the reason for
that:
```
[error] error: Bad service configuration file, or exception thrown while constructing Processor object: javax.annotation.processing.Processor: Provider com.google.auto.service.processor.AutoServiceProcessor could not be instantiated
```

The sample records is only to demonstrate that we can now use newer Java
features.

* Cleanup + fix benchmark compilation

Bench requires jmh classes which are not available because we obviously
had to limit `java.base` modules to get Frgaal to work nicely.
For now, we default to good ol' javac for Benchmarks.
Limiting Frgaal to runtime for now, if it plays nicely, we can expand it
to other projects.

* Update CHANGELOG

* Remove dummy record class

* Update licenses

* New line

* PR review

* Update legal review

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2022-05-04 21:18:40 +02:00
Radosław Waśko
8219dca400
Improve support for reading Delimited files (#3424)
Implements https://www.pivotaltracker.com/story/show/181823957
2022-04-29 17:12:19 +00:00
Marcin Kostrzewa
96a0c92c8b
Automatically force all-defaulted functions (#3414)
This changes the interpreter to treat functions with all-defaulted args as thunks. Seems to have no performance impact in compiled code.
2022-04-27 17:57:00 +00:00
Radosław Waśko
14257d07aa
Data analysts should be able to use Text.split, Text.lines and Text.words to break up strings (#3415)
Implements https://www.pivotaltracker.com/story/show/181266184

### Important Notes

Changed example image download to only proceed if the file did not exist before - thus cutting on the build time (the build used to download it _every_ time - which completely failed the build if network is down). A redownload can be forced by performing a fresh repository checkout.
2022-04-26 17:22:53 +02:00
Dmitry Bushev
69b5e2ab8a
Fix search of local suggestions (#3417)
Changelog:
- fix: `search/completion` request with the position parameter.
- fix: `refactoring/renameProject` request. Previously it did not take into account the library namespace (e.g. `local.`)
2022-04-26 12:06:48 +00:00
Dmitry Bushev
01d308de74
Load component group modules (#3412) 2022-04-25 18:09:09 +03:00
Radosław Waśko
fecaa81551
Review Range and Interval, resolve infinite loop issue (#3408)
Implements: https://www.pivotaltracker.com/story/show/181652841
2022-04-20 16:22:01 +00:00
James Dunkerley
5a6b6749cc
Restructuring for File.read (#3390)
- Added Encoding type
- Added `Text.bytes`, `Text.from_bytes` with Encoding support
- Renamed `File.read` to `File.read_text`
- Renamed `File.write` to `File.write_text`
- Added Encoding support to `File.read_text` and `File.write_text`
- Added warnings to invalid encodings
2022-04-19 16:50:03 +00:00
Jaroslav Tulach
ab692b3b74
Enso Date shall be converted to java.time.LocalDate when passed to Java (#3374) 2022-04-15 06:02:05 +02:00
Dmitry Bushev
998d078b9a
Fill component groups of standard libraries (#3391)
Changelog:
- add: component groups to package descriptions
- add: `executionContext/getComponentGroups` method that returns component groups of libraries that are currently loaded
- doc: cleanup unimplemented undo/redo commands
- refactor: internal component groups datatype
2022-04-14 13:12:18 +00:00
Mateusz Czapliński
e75df61b2c
Component Group View with static header and without icons (#3373)
Add an initial version of the visual component for displaying the Component Group View. The component contains a header (for displaying the Group Name) and a list of labels (for displaying the component names).

https://www.pivotaltracker.com/story/show/181724889


#### Visuals

A screenshot from a debug scene demonstrating the component:

<img width="251" alt="Screenshot 2022-04-13 at 20 07 56" src="https://user-images.githubusercontent.com/273837/163243304-21c3ad78-4813-4368-b3bb-844d979da699.png">



Screenshots from other debug scenes (`list_view` and `text_area`), demonstrating that the other components still display correctly:

<img width="202" alt="Screenshot 2022-04-13 at 20 08 56" src="https://user-images.githubusercontent.com/273837/163243428-de9dc1c7-5a9f-45e0-9325-db60cece9768.png">


<img width="403" alt="Screenshot 2022-04-13 at 20 08 48" src="https://user-images.githubusercontent.com/273837/163243432-895061d9-5bd9-4349-8679-eb63b0f6724d.png">


A screenshot of the Node Searcher's list, showing that long entries in a ListView are now truncated, and an ellipsis character is added in place of removed characters:


<img width="651" alt="Screenshot 2022-04-13 at 20 10 16" src="https://user-images.githubusercontent.com/273837/163243664-5b671969-7aa0-4bef-8fd2-825602d85848.png">

# Important Notes
- Adding support for the text truncation feature in `ListView` required some changes in the`list_view::Entry`-related APIs.
- An embedded font was added (DejaVuSans-Bold) for use in the Component Group View debug scene, and 5 unused embedded fonts were removed.
2022-04-14 10:37:40 +00:00
Radosław Waśko
0ea5dc2a6f
Data analysts should be able to use Text.replace to substitute parts of the text (#3393)
Implements https://www.pivotaltracker.com/story/show/181266274
2022-04-13 19:21:47 +00:00
Adam Obuchowicz
8ca4e2b003
SDF-based mechanism for making letters bold (#3385)
Fixes [181641027](https://www.pivotaltracker.com/story/show/181641027). There are new methods of text::Area for making some letters bolder, without moving them nor any other letters in the line.

Tested on DejaVuSans text:
![image](https://user-images.githubusercontent.com/3919101/162199723-524f7560-535f-4c98-bcb4-6b760ad8cb04.png)
![image](https://user-images.githubusercontent.com/3919101/162199806-e6582c56-3071-4653-8ad2-114df86cd6b7.png)
![image](https://user-images.githubusercontent.com/3919101/162199880-1836e902-8663-4696-bcb7-2b15c3c243c6.png)

On MacOS:
<img width="824" alt="Screenshot 2022-04-07 at 14 54 40" src="https://user-images.githubusercontent.com/3919101/162203730-ffda3096-7aab-443f-a45a-bfa18784ba36.png">
<img width="379" alt="Screenshot 2022-04-07 at 14 54 52" src="https://user-images.githubusercontent.com/3919101/162203751-f5ca2e26-fdb6-4521-9821-999111b80ac0.png">
<img width="160" alt="Screenshot 2022-04-07 at 14 54 58" src="https://user-images.githubusercontent.com/3919101/162203769-3c367f81-67bb-48aa-994c-42eeb131017e.png">
2022-04-12 17:39:08 +00:00
Dmitry Bushev
c402d9a900
Implement Events Monitor (#3392)
PR adds a monitor that handles messages between the language server and the runtime and dumps them as a CSV file `/tmp/enso-api-events-*********.csv`

```
UTC timestamp,Direction,Request Id,Message class
```

# Important Notes
⚠️ Monitor is enabled when the log level is set to trace. You should pass `-vv` (very verbose) option to the backend when starting IDE
```
enso -- -vv
```
2022-04-12 15:17:47 +00:00
Radosław Waśko
891f064a6a
Extend Aggregate_Spec test suite with tests for missed edge-cases to ensure the feature is well-tested on all backends (#3383)
Implements https://www.pivotaltracker.com/story/show/181805693 and finishes the basic set of features of the Aggregate component.

Still not all aggregations are supported everywhere, because for example SQLite has quite limited support for aggregations. Currently the workaround is to bring the table into memory (if possible) and perform the computation locally. Later on, we may add more complex generator features to emulate the missing aggregations with complex sub-queries.
2022-04-12 11:02:01 +00:00
Marcin Kostrzewa
4e51f31eb7
Always call defaulted atom arguments (#3358)
Solves the issue of defaulted args not being called in atoms. Doesn't solve the more general function issue.
2022-04-08 08:21:59 +00:00
Dmitry Bushev
29e3f05f27
Fix multiline code docparser (#3379)
Changelog:
- fix: docparser handles multiline code sections correctly
- feat: split paragraphs into keyed sections
2022-04-06 04:39:58 +00:00
Nikita Pekin
42ac28d0de
Add benchmark for Text.reverse with strings of varying length (#3381)
This pull request adds a benchmark for the `Text.reverse` function added in #3377 as part of https://www.pivotaltracker.com/n/projects/2539304/stories/181265419.

Per discussion with @jdunkerley on Discord it is useful to have this benchmark as this is a low-level item we want to track.
2022-04-05 17:59:21 +00:00
Nikita Pekin
22e3941371
Data analysts should be able to reverse strings using Text.reverse (#3377)
This commit implements `Text.reverse` as an extension on `Text`.
`Text.reverse` reverses strings. For example: `"Hello World!".reverse`
results in `"!dlroW olleH"`.

Strings are reversed by their Extended Grapheme Clusters not by their
characters. This has some performance implications because we need to
find these grapheme cluster boundaries when iterating. To do so,
`BreakIterator.getCharacterInstance` is used.

Implements: https://www.pivotaltracker.com/n/projects/2539304/stories/181265419
2022-04-05 16:45:56 +00:00
Ilya Bogdanov
fb52c1a9a2
Applying Magnet Alignment Algorithm to newly opened Component Browser (#3366)
[Task link](https://www.pivotaltracker.com/story/show/181076594)

This PR enables new node position adjustment using the Magnet Alignment algorithm for the following cases:

- When creating node with (+) button without nodes selected
- When creating node with "Mouse pointer dictated placement." not under the source node
- When the node is pushed left due to lack of space - only horizontally

The size of the `alignment area` around node is slightly enlarged, so that it's impossible to create a node that is being too close to other nodes.

Videos with AC demonstration:

https://user-images.githubusercontent.com/6566674/160565491-818b1705-43f8-41ab-abad-047031b2f9e5.mp4


https://user-images.githubusercontent.com/6566674/160565501-d2d4ea85-68c7-4385-9681-701ab86fdd98.mp4


https://user-images.githubusercontent.com/6566674/160565535-9cbeadfe-397b-4cf8-802a-1ec4148a70a2.mp4
2022-04-05 11:22:29 +00:00
James Dunkerley
a4dbc9a37b
Moving Aggregation to Java (#3364) 2022-04-04 09:12:48 +00:00
Mateusz Czapliński
3c5f8d714a
Placement of newly opened Component Browser when nodes are selected (#3361)
When a new node is created with the <kbd>TAB</kbd> key or by clicking the `(+)` on-screen button while multiple nodes are selected, place the new node below all the selected nodes. (Previously, the new node was placed below the node that was selected earliest.)

Additionally, when placing a new node below an existing non-error node with a visualization enabled, place the new node below the visualization. (Previously, the new node was placed to the left of the visualization.)

https://www.pivotaltracker.com/story/show/180887079

#### Visuals

The following screencast demonstrates the feature on various arrangements of selected nodes, with visualization enabled and disabled.

https://user-images.githubusercontent.com/273837/159971452-148aa4d7-c0f3-4b48-871a-a2783989f403.mov

The following screencast demonstrates that new nodes created by double-clicking an output port of a node with visualization enabled are now placed below the visualization:

https://user-images.githubusercontent.com/273837/160107733-e3f7d0f9-0161-49d1-8cbd-06e18c843a20.mov

# Important Notes
- Some refactorings that were needed for this PR were ported from the #3301 PR:
- the code responsible for calculating the positions of new nodes was moved to a separate module (`new_node_position`);
- the `free_place_finder` module was made a submodule of the `new_node_position` module, due to the latter being its only user.
2022-03-31 20:17:04 +00:00
Mateusz Czapliński
b8a5e22ef8
Placement of newly opened Component Browser dictated by the mouse pointer. (#3301)
Use a new algorithm for placement of new nodes in cases when:

- a) there is no selected node, and the `TAB` key is pressed while the mouse pointer is near an existing node (especially in an area below an existing node);
- b) a connection is dragged out from an existing node and dropped near the node (especially in an area below the node).

In both cases mentioned above, the new node will now be placed in a location suggested by an internal algorithm, aligned to existing nodes. Specifically, the placement algorithm used is similar to when pressing `TAB` with a node selected.

For more details, see: https://www.pivotaltracker.com/story/show/181076066

# Important Notes
- Visible visualizations enabled with the "eye icon" button are treated as part of a node. (In case of nodes with errors, visualizations are not visible, and are not treated as part of a node.)
2022-03-31 14:16:28 +00:00
Dmitry Bushev
435d7bb47b
Fix resource management in ModuleCache (#3367)
PR fixes an issue when compiler crashes while trying to load an incompatible `IR` cache.
2022-03-29 13:47:03 +00:00
Dmitry Bushev
a34c2bcfe6
Implement Documentation Sections API (#3360) 2022-03-25 10:49:37 +03:00
James Dunkerley
02bcfbb2a8
Refactor Aggregate Column (#3349)
- Make it easier to understand the computations.
- Fix issue with First.
- Improve quote handling in Concatenate
- Added validation and warnings to input
2022-03-22 18:18:46 +00:00
Mateusz Czapliński
b117a7ddae
Opening Component Browser by clicking on the output port (#3346)
Double-clicking a node's output port or clicking the port with a right mouse button (RMB) creates a new node aligned to the clicked node.

#### Visuals

The screencast below demonstrates the following features:
- double-clicking the left mouse button on a node's output port;
- clicking the right mouse button on a node's output port;
- alignment of the nodes created as a result of the actions described above;
- corner case: double-clicking (and RMB-clicking) on output ports of a "collapsed" ("enterable") node;
- double-clicking on a "collapsed" ("enterable") node still allows entering the node when done over an area of the node that is not the node's output port;
- basic support for nodes with multiple output ports (shown on the `interface` demo scene).

https://user-images.githubusercontent.com/273837/158991856-e0faa5f0-9d2f-44bd-bddd-ba314977db6e.mov

The supplementary screencast below demonstrates that double-clicking or RMB-clicking a node's output port cancels the action of dragging a new connection from a node.

https://user-images.githubusercontent.com/273837/158998097-100aed42-37ff-4467-939f-2b755ef0d3dc.mov




https://www.pivotaltracker.com/story/show/181076145

# Important Notes
- The "double-clicking a node" shortcut was previously used to allow entering a "collapsed" node (for example, a node created by pressing the `cmd+g` keyboard shortcut after selecting a group of nodes). This PR keeps that functionality when the user double-clicks on a node, as long as the mouse is not positioned over the node's output ports.
- The support for nodes with multiple output ports is currently very basic. The information about a port (`Crumb`) is passed into the `create_node` function, but it is not passed further to `NodeSource`. The Node Searcher currently does not support passing port information through `NodeSource`.
2022-03-21 15:08:17 +00:00
Hubert Plociniczak
66e2135b0d
Initialize AtomConstructor's fields via local vars (#3330)
The mechanism follows a similar approach to what is being in functions
with default arguments.
Additionally since InstantiateAtomNode wasn't a subtype of EnsoRootNode it
couldn't be used in the application, which was the primary reason for
issue #181449213.
Alternatively InstantiateAtomNode could have been enhanced to extend
EnsoRootNode rather than RootNode to carry scope info but the former
seemed simpler.

See test cases for previously crashing and invalid cases.
2022-03-21 09:15:14 +00:00
Radosław Waśko
cc7333812d
The library developer should be able to handle specific types of Panics while passing through others (#3344)
Implements https://www.pivotaltracker.com/story/show/181569176

Also ensures that Dataflow Errors have proper stack traces (earlier they did not point at the right location).
2022-03-18 16:57:06 +00:00
Ilya Bogdanov
cdcc852e03
Node searcher zoom & edited node growth/shrink animation (#3327)
In this PR two things are implemented:
1. Node Searcher zoom factor (and therefore its size) is fixed no matter how you move the main camera. The node searcher is also positioned directly below currently edited node at all times.
2. Node growth/shrink animation when you start/finish node editing. After animation end the edited node zoom factor is also fixed and matches the zoom factor of the node searcher.

See attached video with different ways of editing/creating nodes:

https://user-images.githubusercontent.com/6566674/157348758-2880aa2b-494d-46e6-8eee-a22be84081ed.mp4


#### Technical details

1. Added several additional scene layers for separate rendering: `node_searcher`, `node_searcher_text`, `edited_node`, `edited_node_text`. Searcher is always rendered by `node_searcher` camera, edited node moves between its usual layers and `edited_node` layer. Because text rendering uses different API, all node components were modified to support change of the layer.
2. Also added `node_searcher` DOM layer, because documentation is implemented as a DOM object.
3. Added two FRP endpoints for `ensogl::Animation`: `on_end` and `set_value`. These endpoints are useful while implementing growth/shrink animation.
4. Added FRP endpoints for the `Camera2d`: `position` and `zoom` outputs. This allows to synchronize cameras easily using FRP networks.
5. Growth/shrink animation implemented in GraphEditor by blending two animations, similar to Node Snapping implementation. However, shrinking animation is a bit tricky to implement correctly, as we must always return node back to the `main` scene layer after editing is done.
2022-03-17 10:38:18 +00:00
Ilya Bogdanov
11dfd7bfc9
Return creating node with (+) button & fix a regression (#3338)
* Creating a new node with the (+) button (#3278)

[The Task](https://www.pivotaltracker.com/story/show/180887253)

A new (+) button on the left-bottom corner appeared. It may be clicked to open searcher in the middle of the scene, as an alternative to tab key.

https://user-images.githubusercontent.com/3919101/154514279-7972ed6a-0203-47cb-9a09-82dba948cf2f.mp4

* The window_control_buttons::common was extracted to separate crate `ensogl-component-button` almost without change.
* This includes a severe refactoring of adding nodes in general in the Graph Editor. The whole responsibility of adding new nodes (and starting their editing) was moved to Graph Editor - the Project View only reacts for GE events to show searcher properly.
* The status bar was moved from the bottom-left corner to the middle-top of the scene. It does not collide with (+) button, and plays "notification" role anyway.
* The `interface` debug scene was buggy. The problem was with one expression's span-tree. When I replaced it, the scene works.
* I've removed "new searcher" API, as it is completely outdated.
* I've changed code owners of integration tests to GUI team, as it is the team writing mostly the integration tests (int rust)

* Fix regression #181528359

* Add docs & remove unused function

* Fix & enable native Rust tests

* Fix formatting

Co-authored-by: Adam Obuchowicz <adam.obuchowicz@enso.org>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2022-03-16 21:02:47 +03:00
James Dunkerley
6c1c4554f5
Refactor table.group_by to table.aggregate (#3339)
Following UX work move to `table.aggregate` function.
2022-03-15 15:23:36 +01:00
Radosław Waśko
247b284316
Data analysts should be able to use Text.location_of to find indexes within string using various matchers (#3324)
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/181266029
2022-03-12 19:42:00 +00:00
Dmitry Bushev
3ef18ab5b8
Update Electron (#3316)
PR fixes the issue when the user is unable to sign in with Google.

In the end, my assumption about the `User-Agent` header was correct and Google sign-in works with the recent Electron out of the box.
2022-03-10 07:44:19 +00:00
Marcin Kostrzewa
4653bfeeab
Decorate values with arbitrary warnings (#3248) 2022-03-09 16:40:02 +01:00
Ilya Bogdanov
31be7c8b9a
Revert "Adding new node with (+) button" (#3328)
[ci no changelog needed]

This PR reverts commit [0836ce741d](0836ce741d) because of the spotted regression:

To reproduce:
1. Open a default project.
2. Without doing anything else, cmd + click on any node to edit it.
3. Abort editing by pressing escape.
4. Top-most node disappears (it is actually removed from scene)
If you start editing the bottom node - you will also see a visible regression in node searcher's position.

See thread https://discord.com/channels/401396655599124480/950730235719065620/950731247909478410 for details.
2022-03-09 11:44:39 +00:00
James Dunkerley
65465fb8ef
Restructuring the Faker type and creating tests for Group_By (#3318)
- Added Minimum, Maximum, Longest. Shortest, Mode, Percentile
- Added first and last to Map
- Restructured Faker type more inline with FakerJS
- Created 2,500 row data set
- Tests for group_by
- Performance tests for group_by
2022-03-09 10:31:02 +00:00
Hubert Plociniczak
f92108158c
Added compare_to to True/False (#3317) 2022-03-08 14:24:04 +01:00
Hubert Plociniczak
b26b38dc59
Fix CHANGELOG after mergeconflict (#3320) 2022-03-07 10:13:20 +01:00
Hubert Plociniczak
8bdca89917
New Text.insert function (#3311)
Implements https://www.pivotaltracker.com/n/projects/2539304
2022-03-04 16:40:34 +01:00
James Dunkerley
fb68f18739
Within Vector, use Array.Copy wherever possible (#3236)
Following the Slice and Array.Copy experiment, took just the Array.Copy parts out and built into the Vector class.

This gives big performance wins in common operations:

| Test | Ref | New |
| --- | --- | --- |
| New Vector | 41.5 | 41.4 |
| Append Single | 26.6 | 4.2 |
| Append Large | 26.6 | 4.2 |
| Sum | 230.1 | 99.1 |
| Drop First 20 and Sum | 343.5 | 96.9 |
| Drop Last 20 and Sum | 311.7 | 96.9 |
| Filter | 240.2 | 92.5 |
| Filter With Index | 364.9 | 237.2 |
| Partition | 772.6 | 280.4 |
| Partition With Index | 912.3 | 427.9 |
| Each | 110.2 | 113.3 |

*Benchmarks run on an AWS EC2 r5a.xlarge with 1,000,000 item count, 100  iteration size run 10 times.*

# Important Notes
Have generally tried to push the `@Tail_Call` down from the Vector class and move to calling functions on the range class.

- Expanded benchmarks on Vector
- Added `take` method to Vector
- Added `each_with_index` method to Vector
- Added `filter_with_index` method to Vector
2022-03-03 15:40:48 +00:00
James Dunkerley
ad1130587d
Updating Text.repeat and adding Text.* (#3310)
Updating the `Text.repeat` function:
- fix issue with negative count
- add * operator

Add tests of the function.
2022-03-02 19:00:47 +00:00
Radosław Waśko
40c851bf8b
Text.pad and Text.trim (#3309)
Implements https://www.pivotaltracker.com/story/show/181265516
2022-03-02 17:19:39 +00:00
James Dunkerley
738a691662
Table.group_by (#3305)
Functioning group_by based of Enso Map.

# Important Notes
This is an initial version which will be used to establish the API.
The grouping map will need to be moved to Java code for performance.
2022-03-01 16:18:11 +00:00
Radosław Waśko
0d96f59f44
Data analysts should be able to use Text.to_case to change the case of Text values (#3302)
* Move to_upper_case and to_lower_case into to_case

* Add an export, not sure about it

* Implement title case

TODO: some more tests would be good

* Add more tests

* explain title case

* fix todo

* changelog
2022-02-28 23:20:41 +00:00
Adam Obuchowicz
0836ce741d
Creating a new node with the (+) button (#3278)
[The Task](https://www.pivotaltracker.com/story/show/180887253)

A new (+) button on the left-bottom corner appeared. It may be clicked to open searcher in the middle of the scene, as an alternative to tab key.

https://user-images.githubusercontent.com/3919101/154514279-7972ed6a-0203-47cb-9a09-82dba948cf2f.mp4

# Important Notes
* The window_control_buttons::common was extracted to separate crate `ensogl-component-button` almost without change.
* This includes a severe refactoring of adding nodes in general in the Graph Editor. The whole responsibility of adding new nodes (and starting their editing) was moved to Graph Editor - the Project View only reacts for GE events to show searcher properly.
* The status bar was moved from the bottom-left corner to the middle-top of the scene. It does not collide with (+) button, and plays "notification" role anyway.
* The `interface` debug scene was buggy. The problem was with one expression's span-tree. When I replaced it, the scene works.
* I've removed "new searcher" API, as it is completely outdated.
* I've changed code owners of integration tests to GUI team, as it is the team writing mostly the integration tests (int rust)
2022-02-24 16:01:54 +00:00
Radosław Waśko
2ae636f63c
Data analysts should be able to use Text.starts_with and Text.ends_with (#3292)
Implements https://www.pivotaltracker.com/story/show/181265900
2022-02-23 16:48:33 +00:00
James Dunkerley
2e2c5562a8
Text.take and Text.drop (#3287)
Implementation of the Text take and drop APIs
- Added `Range.contains` function
- Added `Text_Sub_Range` type
- Added `Text_Utils.index_of` and `Text_Utils.last_index_of` based on ICU StringSearcher
2022-02-22 18:50:59 +00:00
Radosław Waśko
ae9d51555f
Data analysts should be able to use Text.contains to check for substring using various matcher techniques. (#3285)
* Add matching mode definitions

* Add stub for new method API and an initial test suite

* Fix tests, implement exact matching

* Implement Regex matching

* changelog

* Add benchmarks

* Wokraround for case insensitive regex locale support

* minor tweaks

* Unify Case_Insensitive

* Update edge cases

* Fix other affected places

* minor style change

* Add a problematic test

* Add a regex test for a similar situation

* Migrate to StringSearch:wq

* Add test cases for scharfes S edge case

* Add problematic Regex Unicode normalization test

* Document the regex accents peculiarity

* Do not apply the normalization in ASCII only mode

* cr
2022-02-22 15:41:56 +00:00
Ilya Bogdanov
67a1ae41a0
Restrict Graph Editor Zoom (#3295)
[Task link](#181181203).

This is a reincarnation of PR [3273](https://github.com/enso-org/enso/pull/3273).

The maximum zoom factor of Graph Editor is limited to 1.0x. It is not possible to zoom in from the default camera position.
Debug Mode (activated with `ctrl-shift-d` shortcut) allows to zoom up to 100.0x (the previous behavior of Graph Editor).

If you enable Debug Mode, then zoom in and disable Debug Mode - you won't see the immediate change of zoom factor back to 1.0x. But it will "jump" (with animation) back once you make a zoom in/out event with your controls.

Video:

https://user-images.githubusercontent.com/6566674/154037310-1d166737-353e-4ae6-aca1-f7840571ab16.mp4

# Important Notes
This is a reincarnation of PR [3273](https://github.com/enso-org/enso/pull/3273). There are two changes since that PR:
1. Fixed bug with GeoMap zooming described [here](https://github.com/enso-org/enso/pull/3290). This is done by restricting `ZoomEvent` API so that it will never contain `amount` which is equal to `0.0`.
2. A few refactoring changes from https://github.com/enso-org/enso/pull/3289 to simplify code a bit.
2022-02-22 11:23:59 +00:00
Ilya Bogdanov
9ffeaf3e15
Revert "Restrict maximum zoom in Graph Editor (#3273)" (#3290)
This reverts commit dbcc2548df.
2022-02-21 09:26:12 +01:00
Radosław Waśko
14f57271a2
Ensure that Text.compare_to compares strings according to grapheme clusters (#3282)
https://www.pivotaltracker.com/story/show/181175238
2022-02-17 17:09:41 +00:00
James Dunkerley
7afc8c48c5
Adding Integer.Parse (#3283)
* Integer parse via Longs

* Integer parse via Longs

* Benchmark for Number Parse

* CHANGELOG.md and Natural Order

* Expanded test set

* Number base tests

* Few more negative tests
2022-02-17 15:04:00 +00:00
Ilya Bogdanov
dbcc2548df
Restrict maximum zoom in Graph Editor (#3273) 2022-02-16 20:13:39 +00:00
James Dunkerley
68b85dea82
Improvement to the Natural Order Sort (#3276)
* Improved Natural Order
Data generator for benchmarking

* Missing Import
Benchmark script

* Update Natural_Order.enso

Restore missing ToDo

* Changelog

* PR Comments

* PR Comments

* Additional comments.

* Correction
2022-02-16 17:40:33 +00:00
Marcin Kostrzewa
67b4e59506
Properly expose stacktraces and related data to user code (#3271) 2022-02-16 10:36:19 +03:00
Radosław Waśko
fbf747d6cf
Implement Vector.flatten (#3259) 2022-02-15 16:16:08 +01:00
Ilya Bogdanov
03e105d42e
Debug Mode for Graph Editor (#3264) 2022-02-14 11:19:08 +01:00
James Dunkerley
585afd83ce
Adding Text.at and Text.is_digit functions (#3269)
* Add Text.at function

* Add tests for Text.at

* Add tests for Text.is_digit

* Change log

* Avoid memory allocation
2022-02-14 09:03:55 +00:00
Edward Kmett
0c25ee736c
Upgrade Truffle and Graal to Version 21.3.0 (#3258) 2022-02-11 19:05:13 +03:00
James Dunkerley
1814d3c4f1
Data analysts should be able to transform a Table using the rename_columns functions (#3249)
* Implement Natural_Order and sort_columns

* Starting on Rename

Align Column_Mapping

Add By_Position
Separating off the validation for By_Index so can reuse for rename

By_Position implemented

By_Index implemented
Adjusted behaviour following discussion with Ned, so that renames dominate untouched columns.

Moving to validation style checks for problems

Putting accumulator back

Rename work

* Add Range.find

* More work

* Regex support
Tidy of Unique Name Strategy

* Fix Regex support

* Warning messages
Tests for Unique Naming Strategy
Table rename working

* Database Table rename_columns
Fix for Table
**Must follow up on slice**

* Some tests

* More tests

* Complete test set
(and associated fixes)

* Functional use_first_row_as_names
Tests to go...

* Test for use_first_row_as_names

* Change log

* trailing space

Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2022-02-11 10:18:51 +00:00
Dmitry Bushev
eefd256862
misc: fix changelog formatting (#3263) 2022-02-10 09:09:13 +01:00
Marcin Kostrzewa
ee8df25fd5
Fix vector sorting with TCO comparators (#3256) 2022-02-09 22:17:43 +01:00
Radosław Waśko
8b24336604
Data analysts should be able to reorder columns into name order using sort_columns functions (#3250) 2022-02-08 17:28:46 +01:00
Michał Wawrzyniec Urbańczyk
4baad5f146
Nightly proccess preparations: Setting Enso version through the environment (#3241)
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Radosław Waśko <wasko.radek@gmail.com>
2022-02-07 15:14:32 +01:00