Commit Graph

2465 Commits

Author SHA1 Message Date
Marcin Kostrzewa
23e04f905f
Another attempt at M1 compilation (#3859) 2022-11-09 15:26:25 +00:00
Kaz Wesley
e8f3ad3979
Ensure parses of invalid inputs represent all tokens (#3860)
Ensure all tokens from the input are represented in trees resulting from invalid inputs--tests now cover every reachable code line that creates an `Invalid` node. (Also implemented stricter validation, mainly of `import`/`export` statements.)

See: https://www.pivotaltracker.com/story/show/183405907
2022-11-09 02:57:40 +00:00
Michał Wawrzyniec Urbańczyk
483028dbb0
Cloud dispatch & fixes (#3843)
This PR updates the build script:
* fixed issue where program version check was not properly triggering;
* improved `git-clean` command to correctly clear Scala artifacts;
* added `run.ps1` wrapper to the build script that works better with PowerShell than `run.cmd`;
* increased timeouts to work around failures on macOS nightly builds;
* replaced depracated GitHub Actions APIs (set-output) with their new equivalents;
* workaround for issue with electron builder (python2 lookup) on newer macOS runner images;
* GUI and backend dispatches to cloud were completed;
* release workflow allows creating RC releases.
2022-11-09 00:15:26 +01:00
Wojciech Daniło
cee7f27dc1
Text rendering quality improvements. (#3855) 2022-11-08 19:15:05 +01:00
James Dunkerley
45276b243d
Expanding Derived Columns and Expression Syntax (#3782)
- Added expression ANTLR4 grammar and sbt based build.
- Added expression support to `set` and `filter` on the Database and InMemory `Table`.
- Added expression support to `aggregate` on the Database and InMemory `Table`.
- Removed old aggregate functions (`sum`, `max`, `min` and `mean`) from `Column` types.
- Adjusted database `Column` `+` operator to do concatenation (`||`) when text types.
- Added power operator `^` to both `Column` types.
- Adjust `iif` to allow for columns to be passed for `when_true` and `when_false` parameters.
- Added `is_present` to database `Column` type.
- Added `coalesce`, `min` and `max` functions to both `Column` types performing row based operation.
- Added support for `Date`, `Time_Of_Day` and `Date_Time` constants in database.
- Added `read` method to InMemory `Column` returning `self` (or a slice).

# Important Notes
- Moved approximate type computation to `SQL_Type`.
- Fixed issue in `LongNumericOp` where it was always casting to a double.
- Removed `head` from InMemory Table (still has `first` method).
2022-11-08 15:57:59 +00:00
Jaroslav Tulach
c2633bc137
Metadata, in context and imports (#3856)
Another set of improvements extracted from #3611. This time it includes a fix to the Rust part of the parser.

# Important Notes
After digging into metadata parsing I realized the positions used to query the BTree data structure are wrong. This PR tries to address that by re-arranging the order of serialized fields and passing `startCode` and `endCode` locations in.

Originally I though I need changes on the Rust side to support `in` operator. Turned out I can do that just with changes on the Java side.

Qualified names in imports were missing UUIDs. Fixed now.
2022-11-07 19:05:19 +00:00
Dmitry Bushev
14012a751f
Add parent type field to suggestion (#3846)
PR adds `parentType` field to the `Type` suggestion. All Enso types have parent type except `Any`.
2022-11-07 13:21:04 +00:00
Paweł Grabarz
ed384bddb9
add scroll overshoot bounce animation (#3836)
Implements https://www.pivotaltracker.com/n/projects/2539304/stories/183390749

https://user-images.githubusercontent.com/919491/198984088-9c4f03db-922f-4503-83fa-f0301a11239e.mp4

# Important Notes
- The overshoot bounce animation is implemented as a animation-like FRP mixin. It is currently defined using `define_endpoints` macro, but should be migrated to `define_endpoints_2` as soon as it supports "frp mixins" - definitions that extend external networks.
2022-11-07 10:50:47 +00:00
Jaroslav Tulach
1273d8ac91
Updating the code to be more new parser friendly (#3853)
To minimize differences between #3611 and `develop` branch I propose to make following code changes that seem to work fine with the old as well as new parser. In addition to that there are new tests comparing the two parsers.

# Important Notes
Old parser is still used everywhere except `EnsoCompilerTest` where the test compares `IR`s constructed by both parsers.
2022-11-04 18:08:24 +00:00
James Dunkerley
b5881efdf0
Allow integers for take and drop. (#3854)
Allows passing an integer to take or drop as a shorthand.
2022-11-04 14:03:28 +00:00
Radosław Waśko
20cfea8422
Add docs explaining the purpose and mechanisms of launcher-shims project (#3834) 2022-11-04 07:04:57 +00:00
Jaroslav Tulach
56a4b8815b
Relaxed CodeLocationsTest and co. (#3849)
Another set of fixes extracted from #3611. `CodeLocationsTest` test has been made more flexible to allow _"off by one"_ differences in the offset locations between the old and new parser.
2022-11-04 06:04:56 +00:00
Paweł Grabarz
94eff1192a
reduce compile time by making output label generation non-generic (#3848)
Split `HasOutputTypeLabel::output_type_label` method implementation non-generic part into a separate function. This significantly reduces compile time without risking any performance regressions. That function is currently only used for debug network visualization using graphviz, but still contributed a significant compilation time.

I've made a single attempt to profile the compiler itself, and it turned out that the compiler spent a significant amount of time trying to resolve `Pattern` implementation for closures in that method. Each of those also had to separately go through codegen and optimization. That happened for each node type in the codebase, per crate. Moving that into separate non-inline function removed all those unnecessary duplicates from. I also took this opportunity to rewrite that small piece of parsing to make it a bit cleaner.

The method of measurement is explained here: https://blog.rust-lang.org/inside-rust/2020/02/25/intro-rustc-self-profile.html
In this specific case, I've used `self-profile` to generate a profile for all crates, then used `crox` and [perfetto](https://ui.perfetto.dev/) to analyze the output.

This also shows a pattern to be aware of - using closures in generic context forces the compiler to make a separate closure type per each instantiation, even if that closure doesn't close on anything and could be a static function. This in effect forces even more instantiations or unnecessary type resolutions for all code paths that touch that closure. Using static functions or separating the non-generic part away in those cases would likely continue to help with compile times and file size.

## Comparison

The measurement was done on same machine under same environment, cleaning the build artifacts inbetween runs.

| | before |  after |
|-|-|-|
|total build time|5m 5.5s|3m 59.0s|
| `ide-view-graph-editor` crate build time| 85.68s | 49.73s |
| `ide-view-component-list-panel-grid` crate build time | 49.88s | 32.07s |
| `ide-view` crate build time | 29.05s | 17.5s |
| `enso_gui.wasm` file size before wasm-opt | 83.6 MB | 80.9 MB |
| `enso_gui.wasm` file size after wasm-opt | 67.3 MB | 65.3 MB |

![image](https://user-images.githubusercontent.com/919491/199633193-64dada16-eb22-4020-8d31-3f24661497aa.png)
2022-11-03 14:43:56 +00:00
Jaroslav Tulach
db13084828
Changing the enso_parser library location to match Intel Mac (#3850) 2022-11-03 12:50:28 +01:00
Dmitry Bushev
73666df8fa
Cleanup old benchmark (#3845)
Old parser benchmark fails to compile, and it is not checked by CI. I assume it is safe to remove it.
2022-11-03 09:39:06 +00:00
Wojciech Daniło
48ce68cda1
Wip/wdanilo/text shape system single scene 183406745 (#3776) 2022-11-03 08:35:06 +01:00
Jaroslav Tulach
85f71cbfc8
Including enso_parser library in the engine distribution (#3842)
Make sure `libenso_parser.so`, `.dll` or `.dylib` are packaged and included when `sbt buildEngineDistribution`.

# Important Notes
There was [a discussion](https://discord.com/channels/401396655599124480/1036562819644141598) about proper location of the library. It was concluded that _"there's no functional difference between a dylib and a jar."_ and as such the library is placed in `component` folder.

Currently the old parser is still used for parsing. This PR just integrates the build system changes and makes us ready for smooth flipping of the parser in the future as part of #3611.
2022-11-02 17:13:53 +00:00
Paweł Grabarz
418120f16c
box Application to reduce component struct sizes (#3839)
Wrap big and commonly copied `Application` struct into `Rc` to avoid large structure sizes for all widgets.

The `Application`  struct by itself takes 704 bytes, measured by `size_of::<Application>()`. This structure is very often directly copied into components and widgets. Most notably, the generic `Widget` struct contains it. There are also cases, where structs (especially various `Model`s) contain multiple other child widgets directly, and end up indirectly copying the application multiple times.

All of `Application` clones are logical references (via `CloneRef`), so wrapping it into extra `Rc` doesn't change semantics in any way, but makes all structs that clone it way smaller. This reduces the amount of `memcpy`s and overall volume of allocated memory.

Measurement of a few example structs:

Before change:
```
size_of Application: 704 B
size_of Scrollbar: 712 B
size_of Scrollbar Model: 1576 B
```

After change:
```
size_of Application: 4 B
size_of ApplicationData: 704 B
size_of Scrollbar: 12 B
size_of Scrollbar Model: 176 B
```

Ideally we would not need to clone application reference into each component, but that's out of scope of this PR since it requires a lot more effort.
2022-11-02 16:01:50 +00:00
Marcin Kostrzewa
1c9e57099c
Introduce the Self keyword (#3844)
Introduces the `Self` keyword, aliasing the current type.
2022-11-02 13:39:33 +00:00
Dmitry Bushev
a6ce49e8a5
Split Atom suggestion entry to Type and Constructor (#3835)
Changelog:
- update: split `Atom` suggestion to `Type` and `Constructor`
- update: gui API
- update: JSONRPC doc
2022-11-02 09:53:40 +00:00
Radosław Waśko
438a284346
Implement missing Table.take/drop While (#3840)
Implements https://www.pivotaltracker.com/story/show/183677982
2022-11-01 12:11:50 +00:00
Kaz Wesley
330612119a
Parse the standard library (#3830)
Fix bugs in `TreeToIr` (rewrite) and parser. Implement more undocumented features in parser. Emulate some old parser bugs and quirks for compatibility.

Changes in libs:
- Fix some bugs.
- Clean up some odd syntaxes that the old parser translates idiosyncratically.
- Constructors are now required to precede methods.

# Important Notes
Out of 221 files:
- 215 match the old parser
- 6 contain complex types the old parser is known not to handle correctly

So, compared to the old parser, the new parser parses 103% of files correctly.
2022-10-31 16:19:12 +00:00
Radosław Waśko
f60e9e9d8e
Add a Visualization to the Table.Row type (#3837)
Follow up to https://www.pivotaltracker.com/story/show/182307026

When working in the IDE I noticed that the default vis is bad, so this should make it better.
2022-10-31 14:20:13 +00:00
Pavel Marek
f8a4e2a9d2
Add Period type (#3818)
This PR adds `Period` type, which is a date-only complement to `Duration` builtin type.

# Important Notes
- `Period` replaces `Date_Period`, and `Time_Period`.
- Added shorthand constructors for `Duration` and `Period`. For example: `Period.days 10` instead of `Period.new days=10`.
- `Period` can be compared to other `Period` in some cases, other cases throw an error.
2022-10-28 17:27:20 +00:00
Adam Obuchowicz
c4a7e28fb5
Recognize unqualified import syntax in double representation library. (#3816)
Fixes [#183511669](https://www.pivotaltracker.com/story/show/183511669)

This PR improves the `ImportInfo` structure from Double Representation crate so it provides information what names are brought into the scope with this import. The from_ast method also recognized [various ways of defining imports](https://github.com/enso-org/enso/blob/develop/docs/syntax/imports.md)

# Important Notes
The parser seems to not recognize properly imports with aliases. I have not fixed that, as we expect a new parser to be merged very soon.
2022-10-27 13:22:06 +00:00
Dmitry Bushev
3a8aa90f1b
Document no changelog label (#3833) 2022-10-27 15:28:34 +03:00
Pavel Marek
28243a0fd1
Define Enso epoch start as 15th October 1582. (#3804)
Define start of Enso epoch as 15th of October 1582 - start of the Gregorian calendar.

# Important Notes
- Some (Gregorian) calendar related functionalities within `Date` and `Date_Time` now produces a warning  if the receiving Date/Date_Time is before the epoch start, e.g., `week_of_year`, `is_leap_year`, etc.
2022-10-27 10:16:43 +00:00
Paweł Grabarz
d7954bf6da
Scrollbar LMB click and hold scrolling (#3824) 2022-10-26 20:20:44 +02:00
Marcin Kostrzewa
901760816c
State rework & IO Contexts (#3828)
1. Changes how we do monadic state – rather than a haskelly solution, we now have an implicit env with mutable data inside. It's better for the JVM. It also opens the possibility to have state ratained on exceptions (previously not possible) – both can now be implemented.
2. Introduces permission check system for IO actions.
2022-10-26 16:22:08 +00:00
Radosław Waśko
bb29833da5
Create a Table Row Type and expose as a Vector on In-Memory Table with .rows property (#3827)
Implements https://www.pivotaltracker.com/story/show/182307026
2022-10-26 11:21:33 +00:00
Dmitry Bushev
46441ca7a8
Add isStatic method field (#3829)
PR adds `isStatic` field to suggestion. The field is required for Component Browser.
2022-10-26 09:12:45 +00:00
Hubert Plociniczak
55b9bea352
Initialize Builtins at Native Image build time (#3821)
Moved loading of Builtin Types and Methods to a static initializer. That way the information is available at Native Image build time and one does not have to update a corresponding entry in `reflect-config` which would be a real pain when we start using it in anger.
Fixes https://www.pivotaltracker.com/story/show/183374932
2022-10-25 18:28:10 +00:00
Adam Obuchowicz
639f612adf
Update shortcuts to reflect Component Browser design docs. (#3823)
When no node is edited, `Enter` creates a new one (opening Component Browser). The shortcut for entering node was changed to `cmd + Enter`
2022-10-25 17:21:12 +00:00
Ilya Bogdanov
c2b82b9934
Editing via esc does not correctly update node view (#3799)
Fixes https://www.pivotaltracker.com/story/show/182926584
[Task link](https://www.pivotaltracker.com/story/show/183426449)

This PR fixes an IDE freeze introduced by https://github.com/enso-org/enso/pull/3732 and reimplements reverting edited nodes to their previous state.

The cause of the IDE freeze is quite interesting. A detailed investigation is available [here](https://gist.github.com/vitvakatu/785e34881368b8cfda61715d7543cbd0).

The graph editor needs to update the Presenter state only if the user is editing the node. Before this PR, the graph editor notified the Presenter with a visual representation of the node content instead of code expression. It caused inconsistency between the states of the controller and Presenter and caused severe performance issues.

https://user-images.githubusercontent.com/6566674/195831224-6d6e8258-e347-48b4-890a-d89c7300bc39.mp4

# Important Notes
- ~~There is a more complex alternative solution – it requires refactoring of the `component::node::input::area` module. The Presenter can be notified with `expression.code` changes, not `expression.viz_code`. I found a simpler solution (`.gate(&edit_mode)`), which has the same effect but does not require additional refactoring.~~ Said solution is implemented in a separate commit
2022-10-25 15:37:49 +00:00
Hubert Plociniczak
f6379fb1f7
Accept Array-like objects seamlessly in builtins (#3817)
Most of the time, rather than defining the type of the parameter of the builtin, we want to accept every Array-like object i.e. Vector, Array, polyglot Array etc.
Rather than writing all possible combinations, and likely causing bugs on the way anyway as we already saw, one should use `CoerceArrayNode` to convert to Java's `Object[]`.
Added various test cases to illustrate the problem.
2022-10-25 12:44:48 +00:00
Radosław Waśko
bc09c7b4c2
Remove obsolete Rust parser experiment, superseded by the much more mature new Rust parser integration (#3815)
We've had an old attempt at integrating a Rust parser with our Scala/Java projects. It seems to have been abandoned and is not used anywhere - it is also superseded by the new integration of the Rust parser. I think it was used as an experiment to see how to approach such an integration.

Since it is not used anymore - it make sense to remove it, because it only adds some (slight, but non-zero) maintenance effort. We can always bring it back from git history if necessary.
2022-10-24 14:56:07 +00:00
Radosław Waśko
2bc0611869
Add support for using Columns within Is_In (#3822)
Implements https://www.pivotaltracker.com/story/show/183560222
2022-10-24 12:51:15 +00:00
James Dunkerley
f0f6deef2a
Load the File_Format types via a ServiceLoader (#3813)
Moves the File.read method into the `File` type.
Uses the ServiceLoader to find all types for the File_Format.
2022-10-24 09:55:18 +00:00
Jaroslav Tulach
d8882f606d
Few more properly parsed files (#3826)
Another part of #3611 with few more `TreeToIr` improvements.

# Important Notes
Unofficial `LoadParser.sh` check from #3611 of all library files now reports just 54 failures out of 222 files - e.g. 75% success rate.
2022-10-24 08:53:37 +00:00
Radosław Waśko
c1c9be2998
Update CODEOWNERS to avoid project files being owned by just one person (#3820)
Only exception: CI files, as no-one apart from @mwu-tow is currently modifying them anyway
2022-10-22 05:45:55 +00:00
Jaroslav Tulach
42d0e40db7
Order of types in method signatures and more robust comment handling (#3825)
Another part of #3611 to integrate into `develop` branch.
2022-10-22 04:19:13 +00:00
Jaroslav Tulach
7bcd76ac5c
Test to make sure number of files accepted by the new parser only grows (#3814)
A new test to verify that some `Standard.Base` library files can be parsed by the new parser is added. Use it as:
```bash
$ sbt bootstrap
$ sbt buildEngineDistribution
$ sbt "runtime/testOnly *ParseStdLibTest"
```
By having this test instantly available we can block integration of PRs that would decrease the number of parseable files or introduce new files that aren't paseable.

# Important Notes
The test contains a _black list_ of classes that are currently known to fail. Over the time the _black list_ shall get smaller and smaller.
2022-10-21 15:09:32 +00:00
Hubert Plociniczak
6c440beecc
Move logic calculating the index in Vector.at to a builtin method to make the performance of Vector to be on par with Array (#3811)
The main culprit of a Vector slowdown (when compared to Array) was the normalization of the index when accessing the elements. Turns out that the Graal was very persistent on **not** inlining that particular fragment and that was degrading the results in benchmarks.

Being unable to force it to do it (looks like a combination of thunk execution and another layer of indirection) we resorted to just moving the normalization to the builtin method. That makes Array and Vector perform roughly the same.

Moved all handling of invalid index into the builtin as well, simplifying the Enso implementation. This also meant that `Vector.unsafe_at` is now obsolete.
Additionally, added support for negative indices in Array, to behave in the same way as for Vector.

# Important Notes
Note that this workaround only addresses this particular perf issue. I'm pretty sure we will have more of such scenarios.
Before the change `averageOverVector` benchmark averaged around `0.033 ms/op` now it does consistently `0.016 ms/op`, similarly to `averageOverArray`.
2022-10-20 12:50:44 +00:00
Radosław Waśko
cc76e7d36a
Add support for Blank_Columns to Table and Database (#3812)
Implements https://www.pivotaltracker.com/story/show/183390281 and https://www.pivotaltracker.com/story/show/183390394
2022-10-20 09:11:08 +00:00
Mateusz Czapliński
81e5e77ae8
A caption when hovering the marketplace button on the left bar of Component Browser (#3783)
When hovering the mouse pointer over the Marketplace button on the left bar of the Component Browser, show a caption informing that the Marketplace will be available soon.

https://www.pivotaltracker.com/story/show/182613789

#### Visuals

The video below demonstrates the caption shown when hovering the Marketplace button on the left bar of the Component Browser. It shows the caption disappearing after a hardcoded time, or when the mouse pointer is moved away from the button.







https://user-images.githubusercontent.com/273837/196195809-45a712e1-ad86-47d8-99ff-1475a0b74c6e.mov

# Important Notes
- The "Label" visual component was fixed. Previously, the width calculation of the background was not synchronized correctly with the text width. As a result, a zero-width background was displayed when a Label was shown for the first time.
2022-10-19 16:07:40 +00:00
Adam Obuchowicz
c4e89a8afd
Fix potential dobule-borrow panic. (#3807)
There is one place in code, where we can potentially panic on double borrow, because the drop routine is done while having data borrowed.

# Important Notes
The issue was easily reproducible on @Frizi machine.
2022-10-19 12:11:38 +00:00
Jaroslav Tulach
21fe0b8865
Process documentation in type definitions (#3806)
Another part of #3611 ready for integration into `develop` branch.

# Important Notes
Test `org.enso.compiler.EnsoCompilerTest.testTestGroup` is ignored as it has problems with source offsets - identifiers don't have the appropriate names due to `Tree.codeRepr()` being _off_.
2022-10-19 09:19:42 +00:00
Kaz Wesley
feb8eb4f83
fix span bug in doc comments (#3808) 2022-10-18 20:37:36 +00:00
Radosław Waśko
17f73988e8
Update drop_missing_rows to filter_blank_rows API. (#3805)
Implements https://www.pivotaltracker.com/story/show/183390042 and https://www.pivotaltracker.com/story/show/183390370
2022-10-18 15:58:50 +00:00
Pavel Marek
a53fbc79be
Improve Unsupported_Argument_Types message. (#3803)
Improve `Unsupported_Argument_Types` error so that it includes the message from the original exception. `arguments` field is retained, but not included in `to_display_text` method.
2022-10-18 12:03:25 +00:00