Commit Graph

199 Commits

Author SHA1 Message Date
Michał Wawrzyniec Urbańczyk
c6d0843a2c
Submodule CI issue workaround. (#3615) 2022-07-27 04:37:47 +02:00
Michał W. Urbańczyk
7e2998bd27 Cleaning the repository before build to avoid infinite sbt compilation issue. 2022-07-21 12:08:48 +02:00
Michał Wawrzyniec Urbańczyk
b5c8296aaf
Build script bump & benchmark workflow (#3563) 2022-07-05 22:03:29 +02:00
Michał Wawrzyniec Urbańczyk
43a893cae6
Bump the build script (#3535) 2022-07-01 03:58:14 +02:00
Jaroslav Tulach
479f74cc84
Enso Integration with Ideal Graph Visualizer (#3533)
This PR adds sources for Enso language support in IGV (and NetBeans). The support is based on TextMate grammar shown in the editor and registration of the Enso language so IGV can find it. Then this PR adds new GitHub Actions workflow file to build the project using Maven.
2022-06-23 04:43:49 +00:00
James Dunkerley
a0c6fa9c96
Removing old functions and tidy up of Table types (#3519)
- Removed `select` method.
- Removed `group` method.
- Removed `Aggregate_Table` type.
- Removed `Order_Rule` type.
- Removed `sort` method from Table.
- Expanded comments on `order_by`.
- Update comment on `aggregate` on Database.
- Update Visualisation to use new APIs.
- Updated Data Science examples to use new APIs.
- Moved Examples test out of Tests to own test.

# Important Notes
Need to get Examples_Tests added to CI.
2022-06-14 13:37:20 +00:00
Kaz Wesley
a1bf0974ce
Better release build time; new maximum-performance production profile. (#3498)
### Pull Request Description

Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`):
- `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`.
- `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental.  
(See key in first chart for the settings defining these profiles.)

We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes:
- `release`: A profile that supports fast developer iteration, while offering realistic performance.
- `production`: A maximally-optimized profile, for nightly builds and actual releases.

Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags.

### Performance details

![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png)

As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster.

<details>
  <summary>Methodology (click to show)</summary>

I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data:

Build the application with profiling:
`./run.sh ide build --profiling-level=debug`

Run the `open_project` workflow repeatedly:
`for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done`

For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV:
`echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do`
(Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...)

The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data.
</details>

#### Build times
![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png)

In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds.

(To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 00:09:54 +02:00
Hubert Plociniczak
9e92694dfc
Create non-existent directories in curl (#3506)
Got annoyed by all those FP notifications I get for legacy CI build.
Either this, or we turn it off please.
2022-06-02 16:11:25 +02:00
Michał Wawrzyniec Urbańczyk
6fc4947764
Bumped the build script (#3489)
* The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). 
* Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`.
* Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. 
* Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.)
* If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue.
* Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. 
* Whitelisting dependabot from changelog checks, so PRs created by it are mergeable.
* Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds.
* Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax.
* Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there.

Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 13:44:40 +02:00
Michał Wawrzyniec Urbańczyk
42d82bd8b7
Added a missing checkout step for changelog assertion. (#3481) 2022-05-25 11:38:40 +02:00
Michał Wawrzyniec Urbańczyk
f5f6d264dd
Fix changelog assertion (#3479) 2022-05-25 03:19:45 +02:00
Michał Wawrzyniec Urbańczyk
14a01c4635
New IDE build script (#3466) 2022-05-23 04:16:04 +02:00
Radosław Waśko
a32644dd85
Make it possible to run Enso with assertions enabled (#3450)
Before, when running Enso with `-ea`, some assertions were broken and the interpreter would not start.

This PR fixes two very minor bugs that were the cause of this - now we can successfully run Enso with `-ea`, to test that any assertions in Truffle or in our own libraries are indeed satisfied.

Additionally, this PR adds a setting to SBT that ensures that IntelliJ uses the right language level (Java 17) for our projects.
2022-05-13 15:38:52 +00:00
Hubert Plociniczak
63ff67b621
Add formatting checks for Scala and Java (#3394) 2022-04-20 16:11:24 +02:00
Radosław Waśko
20be5516a5
Aggregates in the Database library - MVP (#3353)
Implements infrastructure for new aggregations in the Database. It comes with only some basic aggregations and limited error-handling. More aggregations and problem handling will be added in subsequent PRs.

# Important Notes
This introduces basic aggregations using our existing codegen and sets-up our testing infrastructure to be able to use the same aggregate tests as in-memory backend for the database backends.

Many aggregations are not yet implemented - they will be added in subsequent tasks.

There are some TODOs left - they will be addressed in the next tasks.
2022-03-28 15:51:37 +00:00
Michał Wawrzyniec Urbańczyk
5bc9811f6a
Remove Windows code signing environment. (#3343) 2022-03-15 16:07:52 +01:00
Michał W. Urbańczyk
5c6c882c38 [CI] Fix for the nightly workflow. 2022-02-21 22:34:34 +01:00
Michał Wawrzyniec Urbańczyk
f9069517db
The new nightly workflow. (#3294) 2022-02-21 16:09:48 +01:00
Michał Wawrzyniec Urbańczyk
9f051ad69c
Automatic cancelling of previous Engine workflow runs (#3281) 2022-02-16 17:04:18 +01:00
Michał Wawrzyniec Urbańczyk
ef9cbe04e6
Update for the build script (#3277) 2022-02-15 14:20:47 +01:00
Edward Kmett
0c25ee736c
Upgrade Truffle and Graal to Version 21.3.0 (#3258) 2022-02-11 19:05:13 +03:00
Michał Wawrzyniec Urbańczyk
4baad5f146
Nightly proccess preparations: Setting Enso version through the environment (#3241)
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Radosław Waśko <wasko.radek@gmail.com>
2022-02-07 15:14:32 +01:00
Michał Wawrzyniec Urbańczyk
ed0e918bff
Fix the new engine CI workflow (#180855729) (#3219)
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
2022-01-17 19:21:34 +01:00
Ilya Bogdanov
a4355876fa
Developers should be able to build and test desktop IDE on M1 Mac (#3194) 2022-01-10 12:18:34 +03:00
Michał Wawrzyniec Urbańczyk
66c256a1f7
Self-hosted Engine CI for Windows and Linux (#3182) 2021-12-27 17:56:35 +01:00
Adam Obuchowicz
c4d22102cf
Switch to 2021 edition (#3173) 2021-12-01 16:06:57 +01:00
Radosław Waśko
46c31bb9a5
Preinstalling With Dependencies (#1981) 2021-11-23 11:51:17 +03:00
Adam Obuchowicz
bb3b145af5
Refactor gui/src/rust/ide to two app/gui and app/ide-desktop (#3157) 2021-11-16 10:04:56 +01:00
Adam Obuchowicz
942464cbaf
Refactoring: merge utils into prelude; merge workspaces. (#3151) 2021-11-10 14:36:08 +01:00
Michał Wawrzyniec Urbańczyk
a7023cf585
Update docs.yml
Remove the Engine changelog check altogether.
2021-11-09 00:45:15 +01:00
Michał Wawrzyniec Urbańczyk
8fc51bfe44
Apply unified prettier style to engine codebase (#3145) 2021-11-08 16:45:29 +01:00
Michael Mauderer
c822256e6c
Update code style to use rust fmt (#3131) 2021-11-02 14:05:43 +01:00
Michael Mauderer
ab0f50a7a3
Bump rustc to nightly-2021-10-21 (#3129)
Bump rustc to nightly-2021-10-21, upgrade nightly features and fix new lints.
2021-11-01 23:41:52 +01:00
Michał Wawrzyniec Urbańczyk
76553dd217
Post-merge fixes (#3130)
Add GUI codeowners. Require jobs from GUI CI as part of PR. Keep prettier under control.
2021-11-01 20:44:15 +01:00
Michał W. Urbańczyk
99053decd8 various default branch switch - related updates 2021-11-01 01:37:30 +01:00
Michał W. Urbańczyk
a57da75ef0 update most of the CI workflows to use develop instead of main 2021-10-30 17:35:45 +02:00
Michał W. Urbańczyk
0d2653f835 one more path to fix 2021-10-30 17:24:54 +02:00
Michał W. Urbańczyk
d8ed4cd26f CI fixes for GUI (IDE). 2021-10-30 17:06:21 +02:00
Michał W. Urbańczyk
7648deb492 single wprkspace, update ci 2021-10-30 15:04:07 +02:00
Ara Adkins
d6465e9e97
Implement a --compile command for the engine runner (#1998) 2021-09-24 12:24:44 +01:00
Ara Adkins
1cd2706ba8
Load IR Caches from Disk (#1996) 2021-09-18 13:48:13 +01:00
Marcin Kostrzewa
a81257b402
Google Spreadsheet Reading (#1976) 2021-09-03 21:41:12 +02:00
Radosław Waśko
338743a832
Fix Nightly Builds on Windows (#1953) 2021-08-16 14:54:05 +02:00
Radosław Waśko
be6e60509a
Tool To Bump Stdlib Version (#1947) 2021-08-13 18:14:20 +02:00
Radosław Waśko
fac0405fd0
Updating Editions (#1944) 2021-08-12 16:55:23 +02:00
Radosław Waśko
270479c9a8
Fix Artifact Upload Issue on CI (#1936)
Fixes the nightly builds.
2021-08-10 21:05:39 +02:00
Radosław Waśko
31167fd6ae
Revert "Revert "Library Publishing MVP (#1898)"" (#1933) 2021-08-09 16:00:04 +02:00
Dmitry Bushev
10b0e11218
Add Project Template Files (#1915)
Add new templates
2021-08-02 19:01:15 +03:00
Ara Adkins
a7478bc573
Revert "Library Publishing MVP (#1898)"
This reverts commit 1bd4e5824e.
2021-07-26 10:32:19 +01:00
Radosław Waśko
1bd4e5824e
Library Publishing MVP (#1898) 2021-07-23 18:16:34 +02:00
Radosław Waśko
f3bc88b91e
Library Downloads MVP (#1885) 2021-07-22 08:24:06 +02:00
Radosław Waśko
042a0f81f1
Fix Nightly Workflows and Update Release Policy (#1845) 2021-07-10 01:10:31 +02:00
Radosław Waśko
b58e895b18
Editions Integration (#1832) 2021-07-08 15:38:20 +02:00
Marcin Kostrzewa
189272c15f
Fix nightly builds [release: nightly] (#1825)
* fix?

* ... [no-changelog]
2021-06-29 17:04:01 +02:00
Radosław Waśko
241a1e7d74
Define Editions (#1797) 2021-06-18 16:39:45 +02:00
Ara Adkins
af4558a93c
Refactor the LS to support multiple content roots (#1800) 2021-06-16 11:00:42 +01:00
Radosław Waśko
db537bf0a8
Add some tweaks to the native builds (#1779) 2021-06-15 10:13:22 +01:00
Ara Adkins
1bdf87ce76
Update the memory limit for Native Image (#1746) 2021-05-20 14:03:14 +01:00
Maciej Mikołajek
f74d386acb
Rewrite Docs Generator from Python to Scala (#1729) 2021-05-17 18:44:21 +02:00
Ara Adkins
48bcebc723
Update to GraalVM 21.1.0 (#1738) 2021-05-14 13:08:39 +01:00
Dmitry Bushev
46725e07c3
Remove reflective access when loading OpenCV (#1727) 2021-05-05 17:26:01 +01:00
Radosław Waśko
c3e4901eb1
Add a workflow for building nightly releases (#1689) 2021-05-05 11:58:14 +01:00
Dmitry Bushev
65e9cca5a4
Add Python and R to the GraalVM Bundle (#1644)
CI release pipeline is updated to install
python and R GraalVM components
2021-04-07 17:19:23 +03:00
Michał Wawrzyniec Urbańczyk
5b57960da3
Histogram and Scatterplot visualizations support for Table (#1608) 2021-03-25 17:47:22 +01:00
Radosław Waśko
21f667323e
PostgreSQL Support in Database Library (#1565)
Co-authored-by: Marcin Kostrzewa <marckostrzewa@gmail.com>
2021-03-16 17:53:04 +01:00
Ara Adkins
96697ddc97
Fix a crash due to shadowed project names (#1571) 2021-03-16 12:45:19 +00:00
Ara Adkins
8290dedcda
Bump sbt to fix an issue with the scala REPL (#1562) 2021-03-10 12:00:35 +00:00
Marcin Kostrzewa
f298fbd3cf
R Interop (#1559) 2021-03-09 16:19:05 +01:00
Marcin Kostrzewa
03fa549daf
Python interop (#1541) 2021-03-05 12:18:37 +01:00
Dmitry Bushev
b5e9895fa7
Initialize language server on client connection (#1535) 2021-03-04 16:59:05 +00:00
Radosław Waśko
6544c2478d
Implement the first part of the database library (#1475) 2021-02-25 13:48:18 +00:00
Ara Adkins
f7c8c1ea4a
Enable the docker pipeline for releases (#1500) 2021-02-23 14:12:24 +00:00
Ara Adkins
8ec2a92b77
Upgrade GraalVM to 21.0 (#1487) 2021-02-22 13:32:55 +00:00
Ara Adkins
521acc7e72
Add the Geo.point function (#1474) 2021-02-12 15:27:54 +00:00
Ara Adkins
4baaa71a58
Back out the 0.2.2 Release (#1464) 2021-02-10 16:38:09 +00:00
Ara Adkins
9ce5999629
Update the docs CI token name 2021-02-05 10:12:06 +00:00
Łukasz Olczak
f726cfe119
Docker refinement (#1420) 2021-01-25 21:26:46 +01:00
Radosław Waśko
10bccf6b56
Better component handling in the project manager (#1399) 2021-01-15 15:26:51 +00:00
Łukasz Olczak
82ae12ea09
Release hot fix (#1373) 2020-12-28 10:36:34 +01:00
Łukasz Olczak
291cbfa6ee
Runtime Containerisation (#1345) 2020-12-23 18:56:30 +01:00
Radosław Waśko
25561e4c8c
Project Manager Native Image (#1318) 2020-12-09 14:58:11 +01:00
Ara Adkins
ab7b49536f
Fix some CI timeouts (#1321) 2020-12-03 13:01:16 +00:00
Radosław Waśko
9e1b49d245
Forking Language Server in the Project Manager (#1305) 2020-12-02 16:56:47 +01:00
Marcin Kostrzewa
150771c0e2
Simple CSV parser (#1268) 2020-11-05 16:53:50 +01:00
Dmitry Bushev
36694e2496
Restructure the Standard Library (#1245)
Split std-lib into two libraries: Base and Test.
2020-10-29 08:48:48 +03:00
Dmitry Bushev
11e4241921
HTTP Library (#1220)
Add `Base.Net.Http` library
2020-10-27 14:45:10 +03:00
Ara Adkins
746521f8b2
Bump SBT and Scalafmt (#1203)
Co-authored-by: Radosław Waśko <radoslaw.wasko@enso.org>
Co-authored-by: Dmitry Bushev <bushevdv@gmail.com>
2020-10-22 16:12:28 +02:00
Radosław Waśko
0740905306
Use Environment Files in Actions (#1218)
Updates and/or pins versions of some of the GitHub Actions that we use.
2020-10-21 16:48:05 +02:00
Radosław Waśko
989aa4b7d7
Initial Legal Review (#1210)
The reports will now be sent to the legal team for review.
2020-10-19 10:50:12 +02:00
Radosław Waśko
0a9e2a42ce
Automate License Information Gathering (#1198) 2020-10-09 16:19:58 +02:00
Marcin Kostrzewa
42dcc1a003
Run Stdlib tests on CI (#1196) 2020-10-06 17:45:02 +02:00
Ara Adkins
22f1b76428
Bump the Node LTS patch version (#1182) 2020-10-02 11:25:48 +01:00
Radosław Waśko
3d88e7f227
Change sbt links to direct GitHub releases links (#1176) 2020-10-01 10:37:46 +01:00
Marcin Kostrzewa
a1748c3978
Enso's Text Type (#1166) 2020-09-30 13:33:57 +02:00
Ara Adkins
bd9689a2b4
Disable TCP offloading on macOS (#1159) 2020-09-22 09:46:29 +01:00
Radosław Waśko
2155fa8e88
Fallback for Launcher Upgrades (#1139) 2020-09-14 11:05:37 +02:00
Radosław Waśko
044a0fa664
Implement Launcher Self-Update (#1125) 2020-09-09 15:37:26 +02:00
Ara Adkins
259302c229
Improve the CI configuration (#1122) 2020-09-07 12:24:34 +01:00
Radosław Waśko
125af6b7fe
Bump GraalVM Version to 20.2.0 (#1094) 2020-08-28 13:03:09 +02:00
Dmitry Bushev
170df5ebd8
Fix SBT Exit During Boot With Code 1 (#1108)
Fix the issue when sbt exits during the boot with error code 1
2020-08-27 13:00:01 +03:00