enso/.github/workflows/gui.yml

662 lines
31 KiB
YAML
Raw Normal View History

2022-05-23 05:16:04 +03:00
---
2021-10-30 18:06:21 +03:00
name: GUI CI
"on":
2021-10-30 18:06:21 +03:00
push:
branches:
- develop
pull_request: {}
workflow_dispatch: {}
jobs:
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-assert-changelog-linux:
2022-05-25 04:19:45 +03:00
name: Assert if CHANGELOG.md was updated (on pull request)
runs-on:
- self-hosted
- Linux
- engine
steps:
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
with:
clean: true
2022-05-25 04:19:45 +03:00
- id: changed_files
run: "git fetch\nlist=`git diff --name-only origin/develop HEAD | tr '\\n' ' '`\necho $list\necho \"::set-output name=list::'$list'\""
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
- run: "if [[ ${{ contains(steps.changed_files.outputs.list,'CHANGELOG.md') || contains(github.event.head_commit.message,'[ci no changelog needed]') || contains(github.event.pull_request.body,'[ci no changelog needed]') || github.event.pull_request.user.login == 'dependabot' }} == false ]]; then exit 1; fi"
2022-05-25 04:19:45 +03:00
if: "github.base_ref == 'develop' || github.base_ref == 'unstable' || github.base_ref == 'stable'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-build-backend-linux:
name: Build Backend (linux)
2022-05-23 05:16:04 +03:00
runs-on:
- self-hosted
- Linux
- engine
2021-10-30 18:06:21 +03:00
steps:
2022-05-23 05:16:04 +03:00
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
2021-10-30 18:06:21 +03:00
with:
2022-05-23 05:16:04 +03:00
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2021-10-30 18:06:21 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run backend get"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-build-backend-macos:
name: Build Backend (macos)
2022-05-23 05:16:04 +03:00
runs-on:
2022-05-25 04:19:45 +03:00
- macos-latest
2021-10-30 18:06:21 +03:00
steps:
2022-05-23 05:16:04 +03:00
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2021-10-30 18:06:21 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run backend get"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-build-backend-windows:
name: Build Backend (windows)
2022-05-23 05:16:04 +03:00
runs-on:
- self-hosted
2022-05-25 04:19:45 +03:00
- Windows
2022-05-23 05:16:04 +03:00
- engine
2021-10-30 18:06:21 +03:00
steps:
2022-05-23 05:16:04 +03:00
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2021-10-30 18:06:21 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run backend get"
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-build-wasm-linux:
2022-05-23 05:16:04 +03:00
name: Build GUI (WASM) (linux)
runs-on:
- self-hosted
- Linux
- engine
2021-10-30 18:06:21 +03:00
steps:
2022-05-23 05:16:04 +03:00
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2021-10-30 18:06:21 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
2022-07-01 04:58:14 +03:00
- run: "./run --upload-artifacts ${{ runner.os == 'Linux' }} wasm build"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-build-wasm-macos:
2022-05-25 04:19:45 +03:00
name: Build GUI (WASM) (macos)
2022-05-23 05:16:04 +03:00
runs-on:
2022-05-25 04:19:45 +03:00
- macos-latest
2021-10-30 18:06:21 +03:00
steps:
2022-05-23 05:16:04 +03:00
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2021-10-30 18:06:21 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
2022-07-01 04:58:14 +03:00
- run: "./run --upload-artifacts ${{ runner.os == 'Linux' }} wasm build"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-build-wasm-windows:
2022-05-25 04:19:45 +03:00
name: Build GUI (WASM) (windows)
2022-05-23 05:16:04 +03:00
runs-on:
- self-hosted
2022-05-25 04:19:45 +03:00
- Windows
2022-05-23 05:16:04 +03:00
- engine
steps:
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2022-05-23 05:16:04 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
2022-07-01 04:58:14 +03:00
- run: "./run --upload-artifacts ${{ runner.os == 'Linux' }} wasm build"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-cancel-workflow-linux:
2022-05-25 04:19:45 +03:00
name: Cancel Previous Runs
2022-05-23 05:16:04 +03:00
runs-on:
2022-05-25 04:19:45 +03:00
- X64
steps:
- name: Cancel Previous Runs
uses: styfle/cancel-workflow-action@0.9.1
with:
access_token: "${{ github.token }}"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-integration-test-linux:
2022-05-25 04:19:45 +03:00
name: IDE integration tests (linux)
needs:
2022-07-01 04:58:14 +03:00
- enso-build-cli-ci-gen-job-build-backend-linux
2022-05-25 04:19:45 +03:00
runs-on:
- self-hosted
- Linux
- engine
2022-05-23 05:16:04 +03:00
steps:
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2022-05-23 05:16:04 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run ide integration-test --backend-source current-ci-run"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-lint-linux:
2022-05-25 04:19:45 +03:00
name: Lint (linux)
2022-05-23 05:16:04 +03:00
runs-on:
- self-hosted
2022-05-25 04:19:45 +03:00
- Linux
2022-05-23 05:16:04 +03:00
- engine
2021-10-30 18:06:21 +03:00
steps:
2022-05-23 05:16:04 +03:00
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2022-05-23 05:16:04 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run lint"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-native-test-linux:
2022-05-25 04:19:45 +03:00
name: Native GUI tests (linux)
2022-05-23 05:16:04 +03:00
runs-on:
2022-05-25 04:19:45 +03:00
- self-hosted
- Linux
- engine
2022-05-23 05:16:04 +03:00
steps:
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2021-10-30 18:06:21 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run wasm test --no-wasm"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-package-ide-linux:
2022-05-25 04:19:45 +03:00
name: Package IDE (linux)
needs:
2022-07-01 04:58:14 +03:00
- enso-build-cli-ci-gen-job-build-backend-linux
- enso-build-cli-ci-gen-job-build-wasm-linux
2022-05-23 05:16:04 +03:00
runs-on:
- self-hosted
2022-05-25 04:19:45 +03:00
- Linux
2022-05-23 05:16:04 +03:00
- engine
steps:
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2022-05-23 05:16:04 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run ide build --wasm-source current-ci-run --backend-source current-ci-run"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-package-ide-macos:
2022-05-23 05:16:04 +03:00
name: Package IDE (macos)
2021-10-30 18:06:21 +03:00
needs:
2022-07-01 04:58:14 +03:00
- enso-build-cli-ci-gen-job-build-backend-macos
- enso-build-cli-ci-gen-job-build-wasm-linux
2022-05-23 05:16:04 +03:00
runs-on:
- macos-latest
2021-10-30 18:06:21 +03:00
steps:
2022-05-23 05:16:04 +03:00
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2021-10-30 18:06:21 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run ide build --wasm-source current-ci-run --backend-source current-ci-run"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-package-ide-windows:
2022-05-25 04:19:45 +03:00
name: Package IDE (windows)
needs:
2022-07-01 04:58:14 +03:00
- enso-build-cli-ci-gen-job-build-backend-windows
- enso-build-cli-ci-gen-job-build-wasm-linux
2022-05-23 05:16:04 +03:00
runs-on:
- self-hosted
2022-05-25 04:19:45 +03:00
- Windows
2022-05-23 05:16:04 +03:00
- engine
steps:
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2022-05-23 05:16:04 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
- run: "./run ide build --wasm-source current-ci-run --backend-source current-ci-run"
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
WIN_CSC_KEY_PASSWORD: "${{ secrets.MICROSOFT_CODE_SIGNING_CERT_PASSWORD }}"
WIN_CSC_LINK: "${{ secrets.MICROSOFT_CODE_SIGNING_CERT }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
2022-07-01 04:58:14 +03:00
enso-build-cli-ci-gen-job-wasm-test-linux:
2022-05-25 04:19:45 +03:00
name: WASM GUI tests (linux)
2022-05-23 05:16:04 +03:00
runs-on:
- self-hosted
- Linux
- engine
steps:
- name: Setup conda (GH runners only)
uses: s-weigand/setup-conda@v1.0.5
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
update-conda: false
conda-channels: "anaconda, conda-forge"
- name: Installing wasm-pack
uses: jetli/wasm-pack-action@v0.3.0
if: "startsWith(runner.name, 'GitHub Actions') || startsWith(runner.name, 'Hosted Agent')"
with:
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
version: v0.10.2
2022-05-23 05:16:04 +03:00
- name: Setup the Artifact API environment
uses: actions/github-script@v6
with:
script: "core.exportVariable(\"ACTIONS_RUNTIME_TOKEN\", process.env[\"ACTIONS_RUNTIME_TOKEN\"])\ncore.exportVariable(\"ACTIONS_RUNTIME_URL\", process.env[\"ACTIONS_RUNTIME_URL\"])\ncore.exportVariable(\"GITHUB_RETENTION_DAYS\", process.env[\"GITHUB_RETENTION_DAYS\"])"
- name: "Workaround for https://github.com/actions/checkout/issues/590 (Windows)"
run: "\"c:\\Program Files\\Git\\bin\\bash.exe\" -c \"git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :\""
2022-07-27 05:37:47 +03:00
if: "runner.os == 'Windows'"
shell: cmd
- name: "Workaround for https://github.com/actions/checkout/issues/590 (non-Windows)"
run: "git checkout -f $(git -c user.name=x -c user.email=x@x commit-tree $(git hash-object -t tree /dev/null) < /dev/null) || :"
2022-07-27 05:37:47 +03:00
if: "runner.os != 'Windows'"
shell: bash
2022-05-23 05:16:04 +03:00
- name: Checking out the repository
2022-07-01 04:58:14 +03:00
uses: actions/checkout@v2
2022-05-23 05:16:04 +03:00
with:
clean: true
2022-07-01 04:58:14 +03:00
- name: Build Script Setup
run: "./run --help"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- run: "./run wasm test --no-native"
Bumped the build script (#3489) * The bash entry point was renamed `run.sh` -> `run`. Thanks to that `./run` works both on Linux and Windows with PowerShell (sadly not on CMD). * Everyone's favorite checks for WASM size and program versions are back. These can be disabled through `--wasm-size-limit=0` and `--skip-version-check` respectively. WASM size limit is stored in `build-config.yaml`. * Improved diagnostics for case when downloaded CI run artifact archive cannot be extracted. * Added GH API authentication to the build script calls on CI. This should fix the macOS build failures that were occurring from time to time. (Actually they were due to runner being GitHub-hosted, not really an OS-specific issue by itself.) * If the GH API Personal Access Token is provided, it will be validated. Later on it is difficult to say, whether fail was caused by wrong PAT or other issue. * Renamed `clean` to `git-clean` as per suggestion to reduce risk of user accidently deleting unstaged work. * Whitelisting dependabot from changelog checks, so PRs created by it are mergeable. * Fixing issue where wasm-pack-action (third party) randomly failed to recognize the latest version of wasm-pack (macOS runners), leading to failed builds. * Build logs can be filtered using `ENSO_BUILD_LOG` environment variable. See https://docs.rs/tracing-subscriber/0.3.11/tracing_subscriber/struct.EnvFilter.html#directives for the supported syntax. * Improve help for ci-run source, to make clear that PAT token is required and what scope is expected there. Also, JS parts were updated with some cleanups and fixes following the changes made when introducing the build script.
2022-06-01 14:44:40 +03:00
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
- name: List files if failed (Windows)
run: Get-ChildItem -Force -Recurse
if: "failure() && runner.os == 'Windows'"
- name: List files if failed (non-Windows)
run: ls -lAR
if: "failure() && runner.os != 'Windows'"
Better `release` build time; new maximum-performance `production` profile. (#3498) ### Pull Request Description Using the new tooling (#3491), I investigated the **performance / compile-time tradeoff** of different codegen options for release mode builds. By scripting the testing procedure, I was able to explore many possible combinations of options, which is important because their interactions (on both application performance and build time) are complex. I found **two candidate profiles** that offer specific advantages over the current `release` settings (`baseline`): - `thin16`: Supports incremental compiles in 1/3 the time of `baseline` in common cases. Application runs about 2% slower than `baseline`. - `fat1-O4`: Application performs 13% better than `baseline`. Compile time is almost 3x `baseline`, and non-incremental. (See key in first chart for the settings defining these profiles.) We can build faster or run faster, though not in the same build. Because the effect sizes are large enough to be impactful to developer and user experience, respectively, I think we should consider having it both ways. We could **split the `release` profile** into two profiles to serve different purposes: - `release`: A profile that supports fast developer iteration, while offering realistic performance. - `production`: A maximally-optimized profile, for nightly builds and actual releases. Since `wasm-pack` doesn't currently support custom profiles (rustwasm/wasm-pack#1111), we can't use a Cargo profile for `production`; however, we can implement our own profile by overriding rustc flags. ### Performance details ![perf](https://user-images.githubusercontent.com/1047859/170788530-ab6d7910-5253-4a2b-b432-8bfa0b4735ba.png) As you can see, `thin16` is slightly slower than `baseline`; `fat1-O4` is dramatically faster. <details> <summary>Methodology (click to show)</summary> I developed a procedure for benchmarking "whole application" performance, using the new "open project" workflow (which opens the IDE and loads a complex project), and some statistical analysis to account for variance. To gather this data: Build the application with profiling: `./run.sh ide build --profiling-level=debug` Run the `open_project` workflow repeatedly: `for i in $(seq 0 9); do dist/ide/linux-unpacked/enso --entry-point profile --workflow open_project --save-profile open_project_thin16_${i}.json; done` For each profile recorded, take the new `total_self_time` output of the `intervals` tool; gather into CSV: `echo $(for i in $(seq 0 9); do target/rust/debug/intervals < open_project_thin16_${i}.json | tail -n1 | awk '{print $2}'; do` (Note that the output of intervals should not be considered stable; this command may need modification in the future. Eventually it would be nice to support formatted outputs...) The data is ready to graph. I used the `boxplot` method of the [seaborn](https://seaborn.pydata.org/index.html) package, in order to show the distribution of data. </details> #### Build times ![thin16](https://user-images.githubusercontent.com/1047859/170788539-1578e41b-bc30-4f30-9b71-0b0181322fa5.png) In the case of changing a file in `enso-prelude`, with the current `baseline` settings rebuilding takes over 3 minutes. With the `thin16` settings, the same rebuild completes in 40 seconds. (To gather this data on different hardware or in the future, just run the new `bench-build.sh` script for each case to be measured.)
2022-06-11 01:09:54 +03:00
env:
ENSO_BUILD_SKIP_VERSION_CHECK: "true"