ipetkov/crane
1
1
Fork 0
mirror of https://github.com/ipetkov/crane.git synced 2024-10-04 01:37:57 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add Crane book (#199)

Browse Source
This commit is contained in:
Ivan Petkov 2022-12-26 14:27:13 -08:00 committed by GitHub
parent 67b1799c33
commit a4d70a26e7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
25 changed files with 1231 additions and 384 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

41
.github/workflows/pages.yml vendored Normal file
View File

@ -0,0 +1,41 @@
name: "pages"
on:
push:
branches:
- "master"
permissions:
contents: read
jobs:
deploy:
runs-on: ubuntu-latest
env:
SSH_AUTH_SOCK: /tmp/ssh_agent.sock
steps:
- uses: actions/checkout@v3
- uses: cachix/install-nix-action@v18
- uses: cachix/cachix-action@v12
with:
name: crane
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- name: mdbook build
run: |
mkdir output
nix build .#book --out-link result --print-build-logs
rsync -r -L ./result/ ./output
- name: git commit
working-directory: output
run: |
git init
git config user.name "GitHub Actions"
git config user.email "github-actions-bot@users.noreply.github.com"
git branch -M gh-pages
git add .
git commit -m "Deploying site"
- name: git push
working-directory: output
run: |
ssh-agent -a $SSH_AUTH_SOCK > /dev/null
ssh-add - <<< "${{ secrets.DEPLOY_KEY }}"
git push --force "git@github.com:${GITHUB_REPOSITORY}.git" gh-pages

391
README.md
View File

@ -1,6 +1,6 @@
# Crane
A [Nix](https://nixos.org/) library for building [cargo](https://doc.rust-lang.org/cargo/) projects.
A [Nix] library for building [cargo] projects.
* **Source fetching**: automatically done using a Cargo.lock file
* **Incremental**: build your workspace dependencies just once, then quickly lint,
@ -10,8 +10,6 @@ A [Nix](https://nixos.org/) library for building [cargo](https://doc.rust-lang.o
## Features
Examples can be found [here](./examples). Detailed [API docs] are available, but
at a glance, the following are supported:
* Automatic vendoring of dependencies in a way that works with Nix
- Alternative cargo registries are supported (with a minor configuration
change)
@ -20,9 +18,11 @@ at a glance, the following are supported:
- Cargo retains the flexibility to only use these dependencies when they are
actually needed, without forcing an override for the entire workspace.
* Reusing dependency artifacts after only building them once
* [clippy](https://github.com/rust-lang/rust-clippy) checks
* [rustfmt](https://github.com/rust-lang/rustfmt) checks
* [cargo-tarpaulin](https://github.com/xd009642/tarpaulin) for code coverage
* [clippy] checks
* [rustfmt] checks
* [cargo-audit] for vulnerability scanning
* [cargo-nextest] a next-generation test runner
* [cargo-tarpaulin] for code coverage
## Getting Started
@ -78,231 +78,6 @@ following contents at the root of your cargo workspace:
}
```
## Philosophy
Crane is designed around the idea of composing cargo invocations such that they
can take advantage of the artifacts generated in previous invocations. This
allows for both flexible configurations and great caching (à la Cachix) in CI
and local development builds.
Here's how it works at a high level: when a cargo workspace is built its source
is first transformed such that only the dependencies listed by the `Cargo.toml`
and `Cargo.lock` files are built, and none of the crate's real source is
included. This allows cargo to build all dependency crates and prevents Nix from
invalidating the derivation whenever the source files are updated. Then, a
second derivation is built, this time using the real source files, which also
imports the cargo artifacts generated in the first step.
This pattern can be used with any arbitrary sequence of commands, regardless of
whether those commands are running additional lints, performing code coverage
analysis, or even generating types from a model schema. Let's take a look at two
examples at how very similar configurations can give us very different behavior!
### Example One
Suppose we are developing a crate and want to run our CI assurance checks
via `nix flake check`. Perhaps we want the CI gate to be very strict and block
any changes which raise warnings when run with `cargo clippy`. Oh, and we want
to enforce some code coverage too!
Except we do not want to push our strict guidelines on any downstream consumers
who may want to build our crate. Suppose they need to build the crate with a
different compiler version (for one reason or another) which comes with a new lint
whose warnings we have not yet addressed. We don't want to make their life
harder, so we want to make sure we do not run `cargo clippy` as part of the
crate's actual derivation, but at the same time, we don't want to have to
rebuild dependencies from scratch.
Here's how we can set up our flake to achieve our goals:
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
crane.url = "github:ipetkov/crane";
crane.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, crane, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
};
craneLib = crane.lib.${system};
# Common derivation arguments used for all builds
commonArgs = {
src = craneLib.cleanCargoSource ./.;
buildInputs = with pkgs; [
# Add extra build inputs here, etc.
# openssl
];
nativeBuildInputs = with pkgs; [
# Add extra native build inputs here, etc.
# pkg-config
];
};
# Build *just* the cargo dependencies, so we can reuse
# all of that work (e.g. via cachix) when running in CI
cargoArtifacts = craneLib.buildDepsOnly (commonArgs // {
# Additional arguments specific to this derivation can be added here.
# Be warned that using `//` will not do a deep copy of nested
# structures
pname = "mycrate-deps";
});
# Run clippy (and deny all warnings) on the crate source,
# resuing the dependency artifacts (e.g. from build scripts or
# proc-macros) from above.
#
# Note that this is done as a separate derivation so it
# does not impact building just the crate by itself.
myCrateClippy = craneLib.cargoClippy (commonArgs // {
# Again we apply some extra arguments only to this derivation
# and not every where else. In this case we add some clippy flags
inherit cargoArtifacts;
cargoClippyExtraArgs = "--all-targets -- --deny warnings";
});
# Build the actual crate itself, reusing the dependency
# artifacts from above.
myCrate = craneLib.buildPackage (commonArgs // {
inherit cargoArtifacts;
});
# Also run the crate tests under cargo-tarpaulin so that we can keep
# track of code coverage
myCrateCoverage = craneLib.cargoTarpaulin (commonArgs // {
inherit cargoArtifacts;
});
in
{
packages.default = myCrate;
checks = {
inherit
# Build the crate as part of `nix flake check` for convenience
myCrate
myCrateClippy
myCrateCoverage;
};
});
}
```
When we run `nix flake check` the following will happen:
1. The sources for any dependency crates will be fetched
1. They will be built without our crate's code and the artifacts propagated
1. Our crate, the clippy checks, and code coverage collection will be built,
each reusing the same set of artifacts from the initial source-free build. If
enough cores are available to Nix it may build all three derivations
completely in parallel, or schedule them in some arbitrary order.
Splitting up our builds like this also gives us the benefit of granular control
over what is rebuilt. Suppose we change our mind and decide to adjust the clippy
flags (e.g. to allow certain lints or forbid others). Doing so will _only_
rebuild the clippy derivation, without having to rebuild and rerun any of our
other tests!
### Example Two
Let's take an alternative approach to the example above. Suppose instead that we
care more about not wasting any resources building certain tests (even if they
would succeed!) if another particular test fails. Perhaps binary substitutes are
readily available so that we do not mind if anyone building from source is bound
by our rules, and we can be sure that all tests have passed as part of the
build.
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
crane.url = "github:ipetkov/crane";
crane.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, crane, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
};
craneLib = crane.lib.${system};
# Common derivation arguments used for all builds
commonArgs = {
src = craneLib.cleanCargoSource ./.;
buildInputs = with pkgs; [
# Add extra build inputs here, etc.
# openssl
];
nativeBuildInputs = with pkgs; [
# Add extra native build inputs here, etc.
# pkg-config
];
};
# Build *just* the cargo dependencies, so we can reuse
# all of that work (e.g. via cachix) when running in CI
cargoArtifacts = craneLib.buildDepsOnly (commonArgs // {
# Additional arguments specific to this derivation can be added here.
# Be warned that using `//` will not do a deep copy of nested
# structures
pname = "mycrate-deps";
});
# First, run clippy (and deny all warnings) on the crate source.
myCrateClippy = craneLib.cargoClippy (commonArgs // {
# Again we apply some extra arguments only to this derivation
# and not every where else. In this case we add some clippy flags
inherit cargoArtifacts;
cargoClippyExtraArgs = "--all-targets -- --deny warnings";
});
# Next, we want to run the tests and collect code-coverage, _but only if
# the clippy checks pass_ so we do not waste any extra cycles.
myCrateCoverage = craneLib.cargoTarpaulin (commonArgs // {
cargoArtifacts = myCrateClippy;
});
# Build the actual crate itself, _but only if the previous tests pass_.
myCrate = craneLib.buildPackage (commonArgs // {
cargoArtifacts = myCrateCoverage;
});
in
{
packages.default = myCrate;
checks = {
inherit
# Build the crate as part of `nix flake check` for convenience
myCrate
myCrateCoverage;
};
});
}
```
When we run `nix flake check` the following will happen:
1. The sources for any dependency crates will be fetched
1. They will be built without our crate's code and the artifacts propagated
1. Next the clippy checks will run, reusing the dependency artifacts above.
1. Next the code coverage tests will run, reusing the artifacts from the clippy
run
1. Finally the actual crate itself is built
In this case we lose the ability to build derivations independently, but we gain
the ability to enforce a strict build order. However, we can easily change our
mind, which would be much more difficult if we had written everything as one
giant derivation.
## Compatibility Policy
Breaking changes can land on the `master` branch at any time, so it is
@ -316,152 +91,6 @@ The test suite is run against the latest stable nixpkgs release, as well as
`nixpkgs-unstable`. Any breakage on those channels is considered a bug and
should be reported as such.
## FAQs
### I want to use a custom version of nixpkgs or another specific system
The crane library can be instantiated with a specific version of nixpkgs as
follows. For more information, see the [API docs] for `mkLib`.
```nix
crane.mkLib (import nixpkgs { system = "armv7l-linux"; })
```
### I want to override a particular package used by the crane library
Specific inputs can be overridden for the entire library via the
`overrideScope'` API as follows. For more information, see the [API docs] for
`mkLib`/`overrideToolchain`, or checkout the [custom-toolchain] example.
```nix
crane.lib.${system}.overrideScope' (final: prev: {
cargo-tarpaulin = myCustomCargoTarpaulinVersion;
})
```
```nix
crane.lib.${system}.overrideToolchain myCustomToolchain
```
### Nix is complaining about IFD (import from derivation)
If a derivation's `pname` and `version` attributes are not explicitly set,
crane will inspect the project's `Cargo.toml` file to set them as a convenience
to avoid duplicating that information by hand. This works well when the source
is a local path, but can cause issues if the source is being fetched remotely,
or flakes are not being used (since flakes have IFD enabled on by default).
One easy workaround for this issue (besides enabling the
`allow-import-from-derivation` option in Nix) is to explicitly set
`{ pname = "..."; version = "..."; }` in the derivation.
You'll know you've run into this issue if you see error messages along the lines
of:
* `cannot build '/nix/store/...-source.drv' during evaluation because the option 'allow-import-from-derivation' is disabled`
* `a 'aarch64-darwin' with features {} is required to build '/nix/store/...', but I am a 'x86_64-linux' with features {}`
### I'm getting rebuilds all of the time, especially when I change `flake.nix`
Nix will rebuild a derivation if any of its inputs change, which includes any
file contained by the source that is passed in. For example, if the build
expression specifies `src = ./.;` then the crate will be rebuilt when _any_ file
changes (including "unrelated" changes to `flake.nix`)!
There are two main ways to avoid unnecessary builds:
1. Use a [source cleaning] function which can omit any files know to not be
needed while building the crate (for example, all `*.nix` sources,
`flake.lock`, and so on). For example `cleanCargoSource` (see [API docs] for
details) implements some good defaults for ignoring irrelevant files which
are not needed by cargo.
1. Another option is to put the crate's source files into its own subdirectory
(e.g. `./mycrate`) and then set the build expression's source to that
subdirectory (e.g. `src = ./mycrate;`). Then, changes to files _outside_ of
that directory will be ignored and will not cause a rebuild
### I'm trying to build another cargo project from source which has no lock file
First consider if there is a release of this project available _with_ a lock
file as it may be simpler and more consistent to use the exact dependencies
published by the project itself. Projects published on crates.io always come
with a lock file and `nixpkgs` has a `fetchCrate` fetcher which pulls straight
from crates.io.
If that is not an option, the next best thing is to generate your own
`Cargo.lock` file and pass it in as an override by setting `cargoLock =
./path/to/Cargo.lock`. If you are calling `buildDepsOnly` or `vendorCargoDeps`
directly the value must be passed there; otherwise you can pass it into
`buildPackage` or `cargoBuild` and it will automatically passed through.
Note that the `Cargo.lock` file must be accessible _at evaluation time_ for the
dependency vendoring to work, meaning the file cannot be generated within the
same derivation that builds the project. It _may_ come from another derivation,
but it may require enabling IFD if flakes are not used.
### I need to patch `Cargo.lock` but when I do the build fails
Dependency crates are vendored by reading `Cargo.lock` _at evaluation time_ and
not at build time. Thus using `patches = [ ./patch-which-updates-lockfile.patch ];`
may result in a situation where any new crates introduced by the patch cannot be
found by cargo.
It is possible to work around this limitation by patching `Cargo.lock` in a
stand-alone derivation and passing that result to `vendorCargoDeps` before
building the rest of the workspace.
```nix
let
patchedCargoLock = src = pkgs.stdenv.mkDerivation {
src = ./path/to/Cargo.lock;
patches = [
./update-cargo-lock.patch
];
installPhase = ''
runHook preInstall
mkdir -p $out
cp Cargo.lock $out
runHook postInstall
'';
};
in
craneLib.buildPackage {
cargoVendorDir = craneLib.vendorCargoDeps {
src = patchedCargoLock;
};
src = craneLib.cleanCargoSource ./.;
patches = [
./update-cargo-lock.patch
./some-other.patch
];
}
```
### How can I build only a subset of a given cargo workspace?
By default, cargo will build the crate at the current directory when invoked; if
the current directory holds a workspace, cargo will then build all crates within
that workspace.
Sometimes it can be useful to only build a subset of a given workspace (e.g.
only specific binaries are needed, or some crates cannot be built for certain
platforms, etc.), and cargo [can be instructed to do
so](https://doc.rust-lang.org/cargo/commands/cargo-build.html).
Notably, it is possible to set:
* `cargoExtraArgs = "-p foo -p bar";` to only build the `foo` and `bar` crates
only, but nothing else in the workspace
* `cargoExtraArgs = "--bin baz";` to only build the `baz` binary (from whatever
crate defines it)
* `cargoExtraArgs = "--workspace --exclude qux";` to build the entire cargo
workspace _except for the `qux` crate_.
Consider setting `pname = "NAME_OF_THE_EXECUTABLE";` when building a single
executable from the workspace. Having the name of the package match the
executable name will allow the result to easily run via `nix run` without
further configuration.
## License
This project is licensed under the [MIT license].
@ -473,9 +102,15 @@ for inclusion by you, shall be licensed as MIT, without any additional terms or
conditions.
[API docs]: ./docs/API.md
[cargo-audit]: https://rustsec.org/
[cargo]: https://doc.rust-lang.org/cargo/
[cargo-nextest]: https://nexte.st/
[cargo-tarpaulin]: https://github.com/xd009642/tarpaulin
[CHANGELOG]: ./CHANGELOG.md
[clippy]: https://github.com/rust-lang/rust-clippy
[custom-toolchain]: ./examples/custom-toolchain/flake.nix
[MIT license]: ./LICENSE
[niv]: https://github.com/nmattia/niv
[Nix]: https://nixos.org/
[rustfmt]: https://github.com/rust-lang/rustfmt
[Semantic Versioning]: http://semver.org/spec/v2.0.0.html
[source cleaning]: https://nixos.org/manual/nixpkgs/unstable/#sec-functions-library-sources

4
checks/default.nix
View File

@ -1,4 +1,4 @@
{ pkgs, myLib }:
{ pkgs, myLib, myPkgs }:
let
inherit (pkgs) lib;
@ -155,6 +155,8 @@ in
features = callPackage ./features { };
flakePackages = pkgs.linkFarmFromDrvs "flake-packages" (builtins.attrValues myPkgs);
gitOverlappingRepo = myLib.buildPackage {
src = ./git-overlapping;
};

1
docs/.gitignore vendored Normal file
View File

@ -0,0 +1 @@
book

26
docs/SUMMARY.md Normal file
View File

@ -0,0 +1,26 @@
# Summary
[Home](../README.md)
[Changelog](../CHANGELOG.md)
---
* [Introduction](./introduction.md)
* [Artifact reuse](./introduction/artifact-reuse.md)
* [Sequential builds](./introduction/sequential-builds.md)
* [Getting started](./getting-started.md)
* [Quick start](./examples/quick-start.md)
* [Quick start (simple)](./examples/quick-start-simple.md)
* [Custom toolchain](./examples/custom-toolchain.md)
* [Alternative registry](./examples/alt-registry.md)
* [Cross compiling](./examples/cross-rust-overlay.md)
* [Cross compiling with musl](./examples/cross-musl.md)
---
- [API Reference](./API.md)
---
* [Troubleshooting/FAQ](./faq/faq.md)
* [Customizing nixpkgs and other inputs](./faq/custom-nixpkgs.md)
* [IFD (import from derivation) errors](./faq/ifd-error.md)
* [Constantly rebuilding from scratch](./faq/constant-rebuilds.md)
* [Building upstream cargo crate with no `Cargo.lock`](./faq/no-cargo-lock.md)
* [Patching `Cargo.lock` during build](./faq/patching-cargo-lock.md)
* [Building a subset of a workspace](./faq/build-workspace-subset.md)

16
docs/book.toml Normal file
View File

@ -0,0 +1,16 @@
[book]
authors = ["Ivan Petkov"]
language = "en"
multilingual = false
src = "."
title = "crane"
description = "A Nix library for building cargo projects."
[build]
create-missing = false
[output.html]
default-theme = "ayu"
preferred-dark-theme = "ayu"
git-repository-url = "https://github.com/ipetkov/crane"
edit-url-template = "https://github.com/ipetkov/crane/edit/master/docs/{path}"

11
docs/examples/alt-registry.md Normal file
View File

@ -0,0 +1,11 @@
Build a cargo project which uses another crate registry:
```sh
nix flake init -t github:ipetkov/crane#alt-registry
```
Alternatively, copy and paste the following `flake.nix`:
```nix
{{#include ../../examples/alt-registry/flake.nix}}
```

11
docs/examples/cross-musl.md Normal file
View File

@ -0,0 +1,11 @@
Build a cargo project with musl to crate statically linked binaries:
```sh
nix flake init -t github:ipetkov/crane#cross-musl
```
Alternatively, copy and paste the following `flake.nix`:
```nix
{{#include ../../examples/cross-musl/flake.nix}}
```

11
docs/examples/cross-rust-overlay.md Normal file
View File

@ -0,0 +1,11 @@
Cross compile a rust project using `oxalica/rust-overlay`:
```sh
nix flake init -t github:ipetkov/crane#cross-rust-overlay
```
Alternatively, copy and paste the following `flake.nix`:
```nix
{{#include ../../examples/cross-rust-overlay/flake.nix}}
```

11
docs/examples/custom-toolchain.md Normal file
View File

@ -0,0 +1,11 @@
Build a cargo project with a custom toolchain (e.g. WASM builds):
```sh
nix flake init -t github:ipetkov/crane#custom-toolchain
```
Alternatively, copy and paste the following `flake.nix`:
```nix
{{#include ../../examples/custom-toolchain/flake.nix}}
```

11
docs/examples/quick-start-simple.md Normal file
View File

@ -0,0 +1,11 @@
Build a cargo project without extra tests:
```sh
nix flake init -t github:ipetkov/crane#quick-start-simple
```
Alternatively, copy and paste the following `flake.nix`:
```nix
{{#include ../../examples/quick-start-simple/flake.nix}}
```

11
docs/examples/quick-start.md Normal file
View File

@ -0,0 +1,11 @@
Build a cargo project with a comprehensive test suite:
```sh
nix flake init -t github:ipetkov/crane#quick-start
```
Alternatively, copy and paste the following `flake.nix`:
```nix
{{#include ../../examples/quick-start/flake.nix}}
```

22
docs/faq/build-workspace-subset.md Normal file
View File

@ -0,0 +1,22 @@
## How can I build only a subset of a given cargo workspace?
By default, cargo will build the crate at the current directory when invoked; if
the current directory holds a workspace, cargo will then build all crates within
that workspace.
Sometimes it can be useful to only build a subset of a given workspace (e.g.
only specific binaries are needed, or some crates cannot be built for certain
platforms, etc.), and cargo [can be instructed to do so](https://doc.rust-lang.org/cargo/commands/cargo-build.html).
Notably, it is possible to set:
* `cargoExtraArgs = "-p foo -p bar";` to only build the `foo` and `bar` crates
only, but nothing else in the workspace
* `cargoExtraArgs = "--bin baz";` to only build the `baz` binary (from whatever
crate defines it)
* `cargoExtraArgs = "--workspace --exclude qux";` to build the entire cargo
workspace _except for the `qux` crate_.
Consider setting `pname = "NAME_OF_THE_EXECUTABLE";` when building a single
executable from the workspace. Having the name of the package match the
executable name will allow the result to easily run via `nix run` without
further configuration.

20
docs/faq/constant-rebuilds.md Normal file
View File

@ -0,0 +1,20 @@
## I'm getting rebuilds all of the time, especially when I change `flake.nix`
Nix will rebuild a derivation if any of its inputs change, which includes any
file contained by the source that is passed in. For example, if the build
expression specifies `src = ./.;` then the crate will be rebuilt when _any_ file
changes (including "unrelated" changes to `flake.nix`)!
There are two main ways to avoid unnecessary builds:
1. Use a [source cleaning] function which can omit any files know to not be
needed while building the crate (for example, all `*.nix` sources,
`flake.lock`, and so on). For example `cleanCargoSource` (see [API docs] for
details) implements some good defaults for ignoring irrelevant files which
are not needed by cargo.
1. Another option is to put the crate's source files into its own subdirectory
(e.g. `./mycrate`) and then set the build expression's source to that
subdirectory (e.g. `src = ./mycrate;`). Then, changes to files _outside_ of
that directory will be ignored and will not cause a rebuild
[source cleaning]: https://nixos.org/manual/nixpkgs/unstable/#sec-functions-library-sources

48
docs/faq/custom-nixpkgs.md Normal file
View File

@ -0,0 +1,48 @@
The crane library can be instantiated with a specific version of nixpkgs as
follows. For more information, see the [API docs] for `mkLib`.
```nix
# Instantiating for a specific `system`
crane.mkLib (import nixpkgs {
system = "armv7l-linux";
})
```
```nix
# Instantiating for cross compiling
crane.mkLib (import nixpkgs {
localSystem = "x86_64-linux";
crossSystem = "aarch64-linux";
})
```
The crane library can also be instantiated with a particular rust toolchain:
```nix
# For example, using rust-overlay
let
system = "x86_64-linux";
pkgs = import nixpkgs {
inherit system;
overlays = [ (import rust-overlay) ];
};
rustToolchain = pkgs.rust-bin.stable.latest.default.override {
targets = [ "wasm32-wasi" ];
};
in
(crane.mkLib pkgs).overrideToolchain rustToolchain
```
Finally, specific inputs can be overridden for the entire library via the
`overrideScope'` API as follows. For more information, see the [API
docs](../API.md) for `mkLib`/`overrideToolchain`, or checkout the
[custom-toolchain](../../examples/custom-toolchain) example.
```nix
crane.lib.${system}.overrideScope' (final: prev: {
cargo-tarpaulin = myCustomCargoTarpaulinVersion;
})
```
[API docs]: ../API.md

5
docs/faq/faq.md Normal file
View File

@ -0,0 +1,5 @@
## Troubleshooting and Frequently Asked Questions
This chapter captures a list of common questions or issues and how to resolve
them. If you happen to run into an issue that is not documented here, please
consider submitting a pull request!

16
docs/faq/ifd-error.md Normal file
View File

@ -0,0 +1,16 @@
## Nix is complaining about IFD (import from derivation)
If a derivation's `pname` and `version` attributes are not explicitly set,
crane will inspect the project's `Cargo.toml` file to set them as a convenience
to avoid duplicating that information by hand. This works well when the source
is a local path, but can cause issues if the source is being fetched remotely,
or flakes are not being used (since flakes have IFD enabled on by default).
One easy workaround for this issue (besides enabling the
`allow-import-from-derivation` option in Nix) is to explicitly set
`{ pname = "..."; version = "..."; }` in the derivation.
You'll know you've run into this issue if you see error messages along the lines
of:
* `cannot build '/nix/store/...-source.drv' during evaluation because the option 'allow-import-from-derivation' is disabled`
* `a 'aarch64-darwin' with features {} is required to build '/nix/store/...', but I am a 'x86_64-linux' with features {}`

18
docs/faq/no-cargo-lock.md Normal file
View File

@ -0,0 +1,18 @@
## I'm trying to build another cargo project from source which has no lock file
First consider if there is a release of this project available _with_ a lock
file as it may be simpler and more consistent to use the exact dependencies
published by the project itself. Projects published on crates.io always come
with a lock file and `nixpkgs` has a `fetchCrate` fetcher which pulls straight
from crates.io.
If that is not an option, the next best thing is to generate your own
`Cargo.lock` file and pass it in as an override by setting `cargoLock =
./path/to/Cargo.lock`. If you are calling `buildDepsOnly` or `vendorCargoDeps`
directly the value must be passed there; otherwise you can pass it into
`buildPackage` or `cargoBuild` and it will automatically passed through.
Note that the `Cargo.lock` file must be accessible _at evaluation time_ for the
dependency vendoring to work, meaning the file cannot be generated within the
same derivation that builds the project. It _may_ come from another derivation,
but it may require enabling IFD if flakes are not used.

39
docs/faq/patching-cargo-lock.md Normal file
View File

@ -0,0 +1,39 @@
## I need to patch `Cargo.lock` but when I do the build fails
Dependency crates are vendored by reading `Cargo.lock` _at evaluation time_ and
not at build time. Thus using `patches = [ ./patch-which-updates-lockfile.patch ];`
may result in a situation where any new crates introduced by the patch cannot be
found by cargo.
It is possible to work around this limitation by patching `Cargo.lock` in a
stand-alone derivation and passing that result to `vendorCargoDeps` before
building the rest of the workspace.
```nix
let
patchedCargoLock = src = pkgs.stdenv.mkDerivation {
src = ./path/to/Cargo.lock;
patches = [
./update-cargo-lock.patch
];
installPhase = ''
runHook preInstall
mkdir -p $out
cp Cargo.lock $out
runHook postInstall
'';
};
in
craneLib.buildPackage {
cargoVendorDir = craneLib.vendorCargoDeps {
src = patchedCargoLock;
};
src = craneLib.cleanCargoSource ./.;
patches = [
./update-cargo-lock.patch
./some-other.patch
];
}
```

53
docs/getting-started.md Normal file
View File

@ -0,0 +1,53 @@
## Getting Started
The easiest way to get started is to initialize a flake from a template:
```sh
# Start with a comprehensive suite of tests
nix flake init -t github:ipetkov/crane#quick-start
# Or if you want something simpler
nix flake init -t github:ipetkov/crane#quick-start-simple
# If you need a custom rust toolchain (e.g. to build WASM targets):
nix flake init -t github:ipetkov/crane#custom-toolchain
# If you need to use another crate registry besides crates.io
nix flake init -t github:ipetkov/crane#alt-registry
# If you need cross-compilation, you can also try out
nix flake init -t github:ipetkov/crane#cross-rust-overlay
# For statically linked binaries using musl
nix flake init -t github:ipetkov/crane#cross-musl
```
For an even more lean, no frills set up, create a `flake.nix` file with the
following contents at the root of your cargo workspace:
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
crane.url = "github:ipetkov/crane";
crane.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, crane, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
craneLib = crane.lib.${system};
in
{
packages.default = craneLib.buildPackage {
src = craneLib.cleanCargoSource ./.;
# Add extra inputs here or any other derivation settings
# doCheck = true;
# buildInputs = [];
# nativeBuildInputs = [];
};
});
}
```

592
docs/highlight.js Normal file
View File

File diff suppressed because one or more lines are too long

19
docs/introduction.md Normal file
View File

@ -0,0 +1,19 @@
## Philosophy
Crane is designed around the idea of composing cargo invocations such that they
can take advantage of the artifacts generated in previous invocations. This
allows for both flexible configurations and great caching (à la Cachix) in CI
and local development builds.
Here's how it works at a high level: when a cargo workspace is built its source
is first transformed such that only the dependencies listed by the `Cargo.toml`
and `Cargo.lock` files are built, and none of the crate's real source is
included. This allows cargo to build all dependency crates and prevents Nix from
invalidating the derivation whenever the source files are updated. Then, a
second derivation is built, this time using the real source files, which also
imports the cargo artifacts generated in the first step.
This pattern can be used with any arbitrary sequence of commands, regardless of
whether those commands are running additional lints, performing code coverage
analysis, or even generating types from a model schema. Let's take a look at two
examples at how very similar configurations can give us very different behavior!

110
docs/introduction/artifact-reuse.md Normal file
View File

@ -0,0 +1,110 @@
### Example One: Artifact Reuse
Suppose we are developing a crate and want to run our CI assurance checks
via `nix flake check`. Perhaps we want the CI gate to be very strict and block
any changes which raise warnings when run with `cargo clippy`. Oh, and we want
to enforce some code coverage too!
Except we do not want to push our strict guidelines on any downstream consumers
who may want to build our crate. Suppose they need to build the crate with a
different compiler version (for one reason or another) which comes with a new lint
whose warnings we have not yet addressed. We don't want to make their life
harder, so we want to make sure we do not run `cargo clippy` as part of the
crate's actual derivation, but at the same time, we don't want to have to
rebuild dependencies from scratch.
Here's how we can set up our flake to achieve our goals:
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
crane.url = "github:ipetkov/crane";
crane.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, crane, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
};
craneLib = crane.lib.${system};
# Common derivation arguments used for all builds
commonArgs = {
src = craneLib.cleanCargoSource ./.;
buildInputs = with pkgs; [
# Add extra build inputs here, etc.
# openssl
];
nativeBuildInputs = with pkgs; [
# Add extra native build inputs here, etc.
# pkg-config
];
};
# Build *just* the cargo dependencies, so we can reuse
# all of that work (e.g. via cachix) when running in CI
cargoArtifacts = craneLib.buildDepsOnly (commonArgs // {
# Additional arguments specific to this derivation can be added here.
# Be warned that using `//` will not do a deep copy of nested
# structures
pname = "mycrate-deps";
});
# Run clippy (and deny all warnings) on the crate source,
# reusing the dependency artifacts (e.g. from build scripts or
# proc-macros) from above.
#
# Note that this is done as a separate derivation so it
# does not impact building just the crate by itself.
myCrateClippy = craneLib.cargoClippy (commonArgs // {
# Again we apply some extra arguments only to this derivation
# and not every where else. In this case we add some clippy flags
inherit cargoArtifacts;
cargoClippyExtraArgs = "--all-targets -- --deny warnings";
});
# Build the actual crate itself, reusing the dependency
# artifacts from above.
myCrate = craneLib.buildPackage (commonArgs // {
inherit cargoArtifacts;
});
# Also run the crate tests under cargo-tarpaulin so that we can keep
# track of code coverage
myCrateCoverage = craneLib.cargoTarpaulin (commonArgs // {
inherit cargoArtifacts;
});
in
{
packages.default = myCrate;
checks = {
inherit
# Build the crate as part of `nix flake check` for convenience
myCrate
myCrateClippy
myCrateCoverage;
};
});
}
```
When we run `nix flake check` the following will happen:
1. The sources for any dependency crates will be fetched
1. They will be built without our crate's code and the artifacts propagated
1. Our crate, the clippy checks, and code coverage collection will be built,
each reusing the same set of artifacts from the initial source-free build. If
enough cores are available to Nix it may build all three derivations
completely in parallel, or schedule them in some arbitrary order.
Splitting up our builds like this also gives us the benefit of granular control
over what is rebuilt. Suppose we change our mind and decide to adjust the clippy
flags (e.g. to allow certain lints or forbid others). Doing so will _only_
rebuild the clippy derivation, without having to rebuild and rerun any of our
other tests!

93
docs/introduction/sequential-builds.md Normal file
View File

@ -0,0 +1,93 @@
### Example Two: Sequential Builds
Let's take an alternative approach to the previous example. Suppose instead that we
care more about not wasting any resources building certain tests (even if they
would succeed!) if another particular check fails. Perhaps binary substitutes are
readily available so that we do not mind if anyone building from source is bound
by our rules, and we can be sure that all tests have passed as part of the
build.
```nix
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
crane.url = "github:ipetkov/crane";
crane.inputs.nixpkgs.follows = "nixpkgs";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, crane, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
};
craneLib = crane.lib.${system};
# Common derivation arguments used for all builds
commonArgs = {
src = craneLib.cleanCargoSource ./.;
buildInputs = with pkgs; [
# Add extra build inputs here, etc.
# openssl
];
nativeBuildInputs = with pkgs; [
# Add extra native build inputs here, etc.
# pkg-config
];
};
# Build *just* the cargo dependencies, so we can reuse
# all of that work (e.g. via cachix) when running in CI
cargoArtifacts = craneLib.buildDepsOnly (commonArgs // {
# Additional arguments specific to this derivation can be added here.
# Be warned that using `//` will not do a deep copy of nested
# structures
pname = "mycrate-deps";
});
# First, run clippy (and deny all warnings) on the crate source.
myCrateClippy = craneLib.cargoClippy (commonArgs // {
# Again we apply some extra arguments only to this derivation
# and not every where else. In this case we add some clippy flags
inherit cargoArtifacts;
cargoClippyExtraArgs = "--all-targets -- --deny warnings";
});
# Next, we want to run the tests and collect code-coverage, _but only if
# the clippy checks pass_ so we do not waste any extra cycles.
myCrateCoverage = craneLib.cargoTarpaulin (commonArgs // {
cargoArtifacts = myCrateClippy;
});
# Build the actual crate itself, _but only if the previous tests pass_.
myCrate = craneLib.buildPackage (commonArgs // {
cargoArtifacts = myCrateCoverage;
});
in
{
packages.default = myCrate;
checks = {
inherit
# Build the crate as part of `nix flake check` for convenience
myCrate
myCrateCoverage;
};
});
}
```
When we run `nix flake check` the following will happen:
1. The sources for any dependency crates will be fetched
1. They will be built without our crate's code and the artifacts propagated
1. Next the clippy checks will run, reusing the dependency artifacts above.
1. Next the code coverage tests will run, reusing the artifacts from the clippy
run
1. Finally the actual crate itself is built
In this case we lose the ability to build derivations independently, but we gain
the ability to enforce a strict build order. However, we can easily change our
mind, which would be much more difficult if we had written everything as one
giant derivation.

35
flake.nix
View File

@ -63,6 +63,33 @@
};
} // flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
};
packages.book =
let
inherit (pkgs) lib;
root = ./.;
rootPrefix = toString root;
cleanedSrc = lib.cleanSourceWith {
src = root;
filter = path: type:
let
relativePath = lib.removePrefix rootPrefix path;
in
lib.any (prefix: lib.hasPrefix prefix relativePath) [
"/docs" # Build the docs directory
"/examples" # But also include examples as we cross-reference them
"/README.md"
"/CHANGELOG.md"
];
};
in
pkgs.runCommand "crane-book" { } ''
${pkgs.mdbook}/bin/mdbook build --dest-dir $out ${cleanedSrc}/docs
'';
checks =
let
pkgsChecks = import nixpkgs {
@ -73,23 +100,21 @@
pkgsChecks.callPackages ./checks {
pkgs = pkgsChecks;
myLib = mkLib pkgsChecks;
myPkgs = packages;
};
pkgs = import nixpkgs {
inherit system;
};
# To override do: lib.overrideScope' (self: super: { ... });
lib = mkLib pkgs;
in
{
inherit checks lib;
inherit checks lib packages;
formatter = pkgs.nixpkgs-fmt;
devShells.default = pkgs.mkShell {
nativeBuildInputs = with pkgs; [
jq
mdbook
nixpkgs-fmt
];
};