Flake lock file updates: • Updated input 'flake-parts': 'github:hercules-ci/flake-parts/7f53fdb7bdc5bb237da7fefef12d099e4fd611ca' (2023-09-01) → 'github:hercules-ci/flake-parts/c9afaba3dfa4085dbd2ccb38dfade5141e33d9d4' (2023-10-03) • Updated input 'flake-parts/nixpkgs-lib': 'github:NixOS/nixpkgs/3e52e76b70d5508f3cec70b882a29199f4d1ee85?dir=lib' (2023-08-31) → 'github:NixOS/nixpkgs/f5892ddac112a1e9b3612c39af1b72987ee5783a?dir=lib' (2023-09-29) • Updated input 'nixpkgs': 'github:NixOS/nixpkgs/efd23a1c9ae8c574e2ca923c2b2dc336797f4cc4' (2023-09-06) → 'github:NixOS/nixpkgs/51d906d2341c9e866e48c2efcaac0f2d70bfd43e' (2023-10-21) • Updated input 'treefmt-nix': 'github:numtide/treefmt-nix/b8d3a059f5487d6767d07c3716386753e3132d9f' (2023-09-04) → 'github:numtide/treefmt-nix/aae39f64f5ecbe89792d05eacea5cb241891292a' (2023-10-15) |
||
---|---|---|
.github/workflows | ||
bin | ||
nix_fast_build | ||
tests | ||
.envrc | ||
.gitignore | ||
.mergify.yml | ||
default.nix | ||
flake.lock | ||
flake.nix | ||
LICENSE.md | ||
pyproject.toml | ||
README.md | ||
treefmt.nix |
nix-fast-build 🚀 (previously known as nix-ci-build)
Combine the power of nix-eval-jobs
with nix-output-monitor
to speed-up your
evaluation and building process. nix-fast-build
an also integrates with remote
machines by uploading the current flake, performing the evaluation/build
remotely, and then transferring the resultant store paths back to you.
Why nix-fast-build
?
Problem: Evaluating and building big flakes i.e. with numerous NixOS
machines can be painfully slow. For instance, rebuilding the already-compiled
disko integration test suite demands
1:50 minutes on an AMD Ryzen 9 7950X3D. But, it only takes a
10 seconds with
nix-fast-build
.
Solution: nix-fast-build
makes builds faster by evaluating and building
your nix packages concurrently, reducing the overall time.
How Does It Work?
Under the hood:
- It leverages the output from
nix-eval-jobs
to evaluate flake attributes in parallel. - As soon as attributes complete evaluation,
nix-fast-build
initiates their build, even if the overall evaluation is ongoing. - Lastly,
nix-output-monitor
to show the build progress nicely. - (Optional) Once a build finishes,
nix-fast-build
can initiate its upload to a designated remote binary cache.
Usage
To get started, run:
$ nix-fast-build
or:
$ nix run github:Mic92/nix-fast-build
This command will concurrently evaluate all systems in .#checks
and build the
attributes in .#checks.$currentSystem
.
Enjoy faster and more efficient NixOS builds with nix-fast-build
!
Remote building
When leveraging the remote-builder protocol, uploading pre-built paths or
sources from the local machine can often turn into a bottleneck.
nix-fast-build
does not use the remote-builder protocol. Instead it uploads
only the flake and executes all evaluation/build operations on the remote end.
At the end nix-fast-build
will download the finished builds to the local
machine while not having to download all build dependencies in between.
Here is how to use it:
nix run github:Mic92/nix-fast-build -- --remote youruser@yoursshhostname
Replace youruser@yoursshhostname
with your SSH login credentials for the
target machine. Please note that as of now, you must be recognized as a trusted
user on the remote endpoint to access this feature.
CI-Friendly Output
By default, Nix-output-monitor
(abbreviated as nom
) updates its output every
0.5 seconds. In standard terminal environments, this frequent update is
unnoticeable, as nom
erases the previous output before displaying the new one.
However, in Continuous Integration (CI) systems, each update appears as a
separate line of output.
To make output more concise for CI environments, use the --no-nom
flag. This
replaces nom
with a streamlined status reporter, which updates only when
there's a change in the number of pending builds, uploads, or downloads.
Avoiding Redundant Package Downloads
By default, nix build
will download pre-built packages, leading to needless
downloads even when there are no changes to any package. This can be especially
burdensome for CI environments without a persistent Nix store, such as GitHub
Actions.
To optimize this, use the --skip-cached
flag with nix-fast-build
. This
ensures that only those packages missing from the binary caches will be built.
Specifying Build Systems
By default, nix-fast-build
evaluates all architectures but only initiates
builds for the current system. You can modify this behavior with the --systems
flag. For instance, using --systems "aarch64-linux x86_64-linux"
will prompt
builds for both aarch64-linux
and x86_64-linux
architectures. Ensure that
your system is capable of building for the specified architectures, either
locally or through the remote builder protocol.
Building different flake attributes
nix-fast-build
by default builds .#checks.$currentSystem
, which refers to
all checks for the current flake. You can modify this default behavior by using
the --flake
flag to specify a different attribute path.
Example:
$ nix run github:Mic92/nix-fast-build -- --flake github:NixOS/nixpkgs#legacyPackages.x86_64-linux.hello
Note: Always provide the complete flake path. Unlike nix build
,
nix-fast-build
does not iterate over different attributes; the full path must
be explicitly stated.
Only evaluate the current system
By default nix-fast-build will evaluate all systems in .#checks
, you can limit
it to the current system by using this command:
$ nix run github:Mic92/nix-fast-build -- --skip-cached --no-nom --flake ".#checks.$(nix eval --raw --impure --expr builtins.currentSystem)"
Reference
usage: nix-fast-build [-h] [-f FLAKE] [-j MAX_JOBS] [--option name value] [--no-nom] [--systems SYSTEMS]
[--retries RETRIES] [--remote REMOTE] [--always-upload-source] [--no-download]
[--skip-cached] [--copy-to COPY_TO] [--verbose]
[--eval-max-memory-size EVAL_MAX_MEMORY_SIZE] [--eval-workers EVAL_WORKERS]
options:
-h, --help show this help message and exit
-f FLAKE, --flake FLAKE
Flake url to evaluate/build (default: .#checks
-j MAX_JOBS, --max-jobs MAX_JOBS
Maximum number of build jobs to run in parallel (0 for unlimited)
--option name value Nix option to set
--no-nom Use nix-output-monitor to print build output (default: false)
--systems SYSTEMS Comma-separated list of systems to build for (default: current system)
--retries RETRIES Number of times to retry failed builds
--remote REMOTE Remote machine to build on
--always-upload-source
Always upload sources to remote machine. This is needed if the remote machine cannot
access all sources (default: false)
--no-download Do not download build results from remote machine
--skip-cached Skip builds that are already present in the binary cache (default: false)
--copy-to COPY_TO Copy build results to the given path (passed to nix copy, i.e.
file:///tmp/cache?compression=none)
--verbose Print verbose output
--eval-max-memory-size EVAL_MAX_MEMORY_SIZE
Maximum memory size for nix-eval-jobs (in MiB) per worker. After the limit is
reached, the worker is restarted.
--eval-workers EVAL_WORKERS
Number of evaluation threads spawned