2022-01-18 02:01:21 +03:00
|
|
|
# Crane
|
|
|
|
|
|
|
|
A [Nix](https://nixos.org/) library for building [cargo](https://doc.rust-lang.org/cargo/) projects.
|
|
|
|
|
2022-01-22 22:47:18 +03:00
|
|
|
* **Source fetching**: automatically done using a Cargo.lock file
|
2022-01-24 01:52:16 +03:00
|
|
|
* **Incremental**: build your workspace dependencies just once, then quickly lint,
|
|
|
|
build, and test changes to your project without slowing down
|
2022-01-18 02:01:21 +03:00
|
|
|
* **Composable**: split builds and tests into granular steps. Gate CI without
|
|
|
|
burdening downstream consumers building from source.
|
|
|
|
|
2022-01-19 07:51:11 +03:00
|
|
|
## Features
|
|
|
|
|
2022-01-22 21:17:05 +03:00
|
|
|
Examples can be found [here](./examples). Detailed [API docs] are available, but
|
|
|
|
at a glance, the following are supported:
|
2022-01-19 07:51:11 +03:00
|
|
|
* Automatic vendoring of dependencies in a way that works with Nix
|
2022-01-31 02:06:46 +03:00
|
|
|
- Alternative cargo registries are supported (with a minor configuration
|
|
|
|
change)
|
2022-02-12 07:33:11 +03:00
|
|
|
- Git dependencies are automatically supported without additional
|
|
|
|
configuration.
|
|
|
|
- Cargo retains the flexibility to only use these dependencies when they are
|
|
|
|
actually needed, without forcing an override for the entire workspace.
|
2022-01-19 07:51:11 +03:00
|
|
|
* Reusing dependency artifacts after only building them once
|
|
|
|
* [clippy](https://github.com/rust-lang/rust-clippy) checks
|
|
|
|
* [rustfmt](https://github.com/rust-lang/rustfmt) checks
|
|
|
|
* [cargo-tarpaulin](https://github.com/xd009642/tarpaulin) for code coverage
|
2022-01-18 02:01:21 +03:00
|
|
|
|
|
|
|
## Getting Started
|
|
|
|
|
|
|
|
The easiest way to get started is to initialize a flake from a template:
|
|
|
|
|
|
|
|
```sh
|
|
|
|
# Start with a comprehensive suite of tests
|
|
|
|
nix flake init -t github:ipetkov/crane#quick-start
|
|
|
|
|
|
|
|
# Or if you want something simpler
|
|
|
|
nix flake init -t github:ipetkov/crane#quick-start-simple
|
|
|
|
|
|
|
|
# If you need a custom rust toolchain (e.g. to build WASM targets):
|
|
|
|
nix flake init -t github:ipetkov/crane#custom-toolchain
|
2022-01-31 02:06:46 +03:00
|
|
|
|
|
|
|
# If you need to use another crate registry besides crates.io
|
|
|
|
nix flake init -t github:ipetkov/crane#alt-registry
|
2022-02-16 05:42:59 +03:00
|
|
|
|
|
|
|
# If you need cross-compilation, you can also try out
|
|
|
|
nix flake init -t github:ipetkov/crane#cross-rust-overlay
|
2022-02-17 07:50:59 +03:00
|
|
|
|
|
|
|
# For statically linked binaries using musl
|
|
|
|
nix flake init -t github:ipetkov/crane#cross-musl
|
2022-01-18 02:01:21 +03:00
|
|
|
```
|
|
|
|
|
|
|
|
For an even more lean, no frills set up, create a `flake.nix` file with the
|
|
|
|
following contents at the root of your cargo workspace:
|
|
|
|
|
|
|
|
```nix
|
|
|
|
{
|
|
|
|
inputs = {
|
|
|
|
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
|
|
|
|
crane.url = "github:ipetkov/crane";
|
|
|
|
crane.inputs.nixpkgs.follows = "nixpkgs";
|
|
|
|
flake-utils.url = "github:numtide/flake-utils";
|
|
|
|
};
|
|
|
|
|
|
|
|
outputs = { self, nixpkgs, crane, flake-utils, ... }:
|
|
|
|
flake-utils.lib.eachDefaultSystem (system: {
|
2022-06-02 03:09:29 +03:00
|
|
|
packages.default = crane.lib.${system}.buildPackage {
|
2022-01-18 02:01:21 +03:00
|
|
|
src = ./.;
|
2022-02-09 03:18:12 +03:00
|
|
|
|
|
|
|
# Add extra inputs here or any other derivation settings
|
|
|
|
# doCheck = true;
|
|
|
|
# buildInputs = [];
|
|
|
|
# nativeBuildInputs = [];
|
2022-01-18 02:01:21 +03:00
|
|
|
};
|
|
|
|
});
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
2022-01-19 07:51:11 +03:00
|
|
|
## Philosophy
|
|
|
|
|
|
|
|
Crane is designed around the idea of composing cargo invocations such that they
|
|
|
|
can take advantage of the artifacts generated in previous invocations. This
|
|
|
|
allows for both flexible configurations and great caching (à la Cachix) in CI
|
|
|
|
and local development builds.
|
|
|
|
|
|
|
|
Here's how it works at a high level: when a cargo workspace is built its source
|
|
|
|
is first transformed such that only the dependencies listed by the `Cargo.toml`
|
|
|
|
and `Cargo.lock` files are built, and none of the crate's real source is
|
|
|
|
included. This allows cargo to build all dependency crates and prevents Nix from
|
|
|
|
invalidating the derivation whenever the source files are updated. Then, a
|
|
|
|
second derivation is built, this time using the real source files, which also
|
|
|
|
imports the cargo artifacts generated in the first step.
|
|
|
|
|
2022-01-23 00:36:41 +03:00
|
|
|
This pattern can be used with any arbitrary sequence of commands, regardless of
|
|
|
|
whether those commands are running additional lints, performing code coverage
|
|
|
|
analysis, or even generating types from a model schema. Let's take a look at two
|
|
|
|
examples at how very similar configurations can give us very different behavior!
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
### Example One
|
|
|
|
|
|
|
|
Suppose we are developing a crate and want to run our CI assurance checks
|
|
|
|
via `nix flake check`. Perhaps we want the CI gate to be very strict and block
|
|
|
|
any changes which raise warnings when run with `cargo clippy`. Oh, and we want
|
|
|
|
to enforce some code coverage too!
|
|
|
|
|
|
|
|
Except we do not want to push our strict guidelines on any downstream consumers
|
|
|
|
who may want to build our crate. Suppose they need to build the crate with a
|
|
|
|
different compiler version (for one reason or another) which comes with a new lint
|
|
|
|
whose warnings we have not yet addressed. We don't want to make their life
|
|
|
|
harder, so we want to make sure we do not run `cargo clippy` as part of the
|
|
|
|
crate's actual derivation, but at the same time, we don't want to have to
|
|
|
|
rebuild dependencies from scratch.
|
|
|
|
|
|
|
|
Here's how we can set up our flake to achieve our goals:
|
|
|
|
|
|
|
|
```nix
|
|
|
|
{
|
|
|
|
inputs = {
|
|
|
|
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
|
|
|
|
crane.url = "github:ipetkov/crane";
|
|
|
|
crane.inputs.nixpkgs.follows = "nixpkgs";
|
|
|
|
flake-utils.url = "github:numtide/flake-utils";
|
|
|
|
};
|
|
|
|
|
|
|
|
outputs = { self, nixpkgs, crane, flake-utils, ... }:
|
|
|
|
flake-utils.lib.eachDefaultSystem (system:
|
|
|
|
let
|
|
|
|
pkgs = import nixpkgs {
|
|
|
|
inherit system;
|
|
|
|
};
|
|
|
|
|
|
|
|
craneLib = crane.lib.${system};
|
2022-05-13 02:06:10 +03:00
|
|
|
|
|
|
|
# Common derivation arguments used for all builds
|
|
|
|
commonArgs = {
|
|
|
|
src = ./.;
|
|
|
|
|
|
|
|
buildInputs = with pkgs; [
|
|
|
|
# Add extra build inputs here, etc.
|
|
|
|
# openssl
|
|
|
|
];
|
|
|
|
|
|
|
|
nativeBuildInputs = with pkgs; [
|
|
|
|
# Add extra native build inputs here, etc.
|
|
|
|
# pkg-config
|
|
|
|
];
|
|
|
|
};
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# Build *just* the cargo dependencies, so we can reuse
|
|
|
|
# all of that work (e.g. via cachix) when running in CI
|
2022-05-13 02:06:10 +03:00
|
|
|
cargoArtifacts = craneLib.buildDepsOnly (commonArgs // {
|
|
|
|
# Additional arguments specific to this derivation can be added here.
|
|
|
|
# Be warned that using `//` will not do a deep copy of nested
|
|
|
|
# structures
|
|
|
|
pname = "mycrate-deps";
|
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# Run clippy (and deny all warnings) on the crate source,
|
|
|
|
# resuing the dependency artifacts (e.g. from build scripts or
|
|
|
|
# proc-macros) from above.
|
|
|
|
#
|
|
|
|
# Note that this is done as a separate derivation so it
|
|
|
|
# does not impact building just the crate by itself.
|
2022-05-13 02:06:10 +03:00
|
|
|
myCrateClippy = craneLib.cargoClippy (commonArgs // {
|
|
|
|
# Again we apply some extra arguments only to this derivation
|
|
|
|
# and not every where else. In this case we add some clippy flags
|
|
|
|
inherit cargoArtifacts;
|
2022-08-12 02:58:32 +03:00
|
|
|
cargoClippyExtraArgs = "--all-targets -- --deny warnings";
|
2022-05-13 02:06:10 +03:00
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# Build the actual crate itself, reusing the dependency
|
|
|
|
# artifacts from above.
|
2022-05-13 02:06:10 +03:00
|
|
|
myCrate = craneLib.buildPackage (commonArgs // {
|
|
|
|
inherit cargoArtifacts;
|
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# Also run the crate tests under cargo-tarpaulin so that we can keep
|
|
|
|
# track of code coverage
|
2022-05-13 02:06:10 +03:00
|
|
|
myCrateCoverage = craneLib.cargoTarpaulin (commonArgs // {
|
2022-05-13 02:16:21 +03:00
|
|
|
inherit cargoArtifacts;
|
2022-05-13 02:06:10 +03:00
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
in
|
|
|
|
{
|
2022-06-02 03:09:29 +03:00
|
|
|
packages.default = myCrate;
|
2022-01-19 07:51:11 +03:00
|
|
|
checks = {
|
|
|
|
inherit
|
|
|
|
# Build the crate as part of `nix flake check` for convenience
|
2022-05-13 02:06:10 +03:00
|
|
|
myCrate
|
|
|
|
myCrateClippy
|
|
|
|
myCrateCoverage;
|
2022-01-19 07:51:11 +03:00
|
|
|
};
|
|
|
|
});
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
When we run `nix flake check` the following will happen:
|
|
|
|
1. The sources for any dependency crates will be fetched
|
|
|
|
1. They will be built without our crate's code and the artifacts propagated
|
|
|
|
1. Our crate, the clippy checks, and code coverage collection will be built,
|
|
|
|
each reusing the same set of artifacts from the initial source-free build. If
|
|
|
|
enough cores are available to Nix it may build all three derivations
|
|
|
|
completely in parallel, or schedule them in some arbitrary order.
|
|
|
|
|
|
|
|
Splitting up our builds like this also gives us the benefit of granular control
|
|
|
|
over what is rebuilt. Suppose we change our mind and decide to adjust the clippy
|
|
|
|
flags (e.g. to allow certain lints or forbid others). Doing so will _only_
|
|
|
|
rebuild the clippy derivation, without having to rebuild and rerun any of our
|
|
|
|
other tests!
|
|
|
|
|
|
|
|
### Example Two
|
|
|
|
|
|
|
|
Let's take an alternative approach to the example above. Suppose instead that we
|
|
|
|
care more about not wasting any resources building certain tests (even if they
|
|
|
|
would succeed!) if another particular test fails. Perhaps binary substitutes are
|
|
|
|
readily available so that we do not mind if anyone building from source is bound
|
|
|
|
by our rules, and we can be sure that all tests have passed as part of the
|
|
|
|
build.
|
|
|
|
|
|
|
|
```nix
|
|
|
|
{
|
|
|
|
inputs = {
|
|
|
|
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
|
|
|
|
crane.url = "github:ipetkov/crane";
|
|
|
|
crane.inputs.nixpkgs.follows = "nixpkgs";
|
|
|
|
flake-utils.url = "github:numtide/flake-utils";
|
|
|
|
};
|
|
|
|
|
|
|
|
outputs = { self, nixpkgs, crane, flake-utils, ... }:
|
|
|
|
flake-utils.lib.eachDefaultSystem (system:
|
|
|
|
let
|
|
|
|
pkgs = import nixpkgs {
|
|
|
|
inherit system;
|
|
|
|
};
|
|
|
|
|
|
|
|
craneLib = crane.lib.${system};
|
2022-05-13 02:16:21 +03:00
|
|
|
# Common derivation arguments used for all builds
|
|
|
|
commonArgs = {
|
|
|
|
src = ./.;
|
|
|
|
|
|
|
|
buildInputs = with pkgs; [
|
|
|
|
# Add extra build inputs here, etc.
|
|
|
|
# openssl
|
|
|
|
];
|
|
|
|
|
|
|
|
nativeBuildInputs = with pkgs; [
|
|
|
|
# Add extra native build inputs here, etc.
|
|
|
|
# pkg-config
|
|
|
|
];
|
|
|
|
};
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# Build *just* the cargo dependencies, so we can reuse
|
|
|
|
# all of that work (e.g. via cachix) when running in CI
|
2022-05-13 02:16:21 +03:00
|
|
|
cargoArtifacts = craneLib.buildDepsOnly (commonArgs // {
|
|
|
|
# Additional arguments specific to this derivation can be added here.
|
|
|
|
# Be warned that using `//` will not do a deep copy of nested
|
|
|
|
# structures
|
|
|
|
pname = "mycrate-deps";
|
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# First, run clippy (and deny all warnings) on the crate source.
|
2022-05-13 02:16:21 +03:00
|
|
|
myCrateClippy = craneLib.cargoClippy (commonArgs // {
|
|
|
|
# Again we apply some extra arguments only to this derivation
|
|
|
|
# and not every where else. In this case we add some clippy flags
|
|
|
|
inherit cargoArtifacts;
|
2022-08-12 02:58:32 +03:00
|
|
|
cargoClippyExtraArgs = "--all-targets -- --deny warnings";
|
2022-05-13 02:16:21 +03:00
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# Next, we want to run the tests and collect code-coverage, _but only if
|
|
|
|
# the clippy checks pass_ so we do not waste any extra cycles.
|
2022-05-13 02:16:21 +03:00
|
|
|
myCrateCoverage = craneLib.cargoTarpaulin (commonArgs // {
|
|
|
|
cargoArtifacts = myCrateClippy;
|
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
|
|
|
|
# Build the actual crate itself, _but only if the previous tests pass_.
|
2022-05-13 02:16:21 +03:00
|
|
|
myCrate = craneLib.buildPackage (commonArgs // {
|
|
|
|
cargoArtifacts = myCrateCoverage;
|
|
|
|
});
|
2022-01-19 07:51:11 +03:00
|
|
|
in
|
|
|
|
{
|
2022-06-02 03:09:29 +03:00
|
|
|
packages.default = myCrate;
|
2022-01-19 07:51:11 +03:00
|
|
|
checks = {
|
|
|
|
inherit
|
|
|
|
# Build the crate as part of `nix flake check` for convenience
|
2022-05-13 02:16:21 +03:00
|
|
|
myCrate
|
|
|
|
myCrateCoverage;
|
2022-01-19 07:51:11 +03:00
|
|
|
};
|
|
|
|
});
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
When we run `nix flake check` the following will happen:
|
|
|
|
1. The sources for any dependency crates will be fetched
|
|
|
|
1. They will be built without our crate's code and the artifacts propagated
|
|
|
|
1. Next the clippy checks will run, reusing the dependency artifacts above.
|
|
|
|
1. Next the code coverage tests will run, reusing the artifacts from the clippy
|
|
|
|
run
|
|
|
|
1. Finally the actual crate itself is built
|
|
|
|
|
|
|
|
In this case we lose the ability to build derivations independently, but we gain
|
|
|
|
the ability to enforce a strict build order. However, we can easily change our
|
|
|
|
mind, which would be much more difficult if we had written everything as one
|
|
|
|
giant derivation.
|
|
|
|
|
2022-01-31 03:26:55 +03:00
|
|
|
## Compatibility Policy
|
|
|
|
|
|
|
|
Breaking changes can land on the `master` branch at any time, so it is
|
|
|
|
recommended you use a versioning strategy when consuming this library (for
|
|
|
|
example, using something like flakes or [niv]).
|
|
|
|
|
|
|
|
Tagged releases will be cut periodically and changes will be documented in the
|
|
|
|
[CHANGELOG]. Release versions will follow [Semantic Versioning].
|
|
|
|
|
|
|
|
The test suite is run against the latest stable nixpkgs release, as well as
|
|
|
|
`nixpkgs-unstable`. Any breakage on those channels is considered a bug and
|
|
|
|
should be reported as such.
|
|
|
|
|
2022-01-18 02:01:21 +03:00
|
|
|
## FAQs
|
|
|
|
|
|
|
|
### I want to use a custom version of nixpkgs or another specific system
|
|
|
|
|
|
|
|
The crane library can be instantiated with a specific version of nixpkgs as
|
|
|
|
follows. For more information, see the [API docs] for `mkLib`.
|
|
|
|
|
|
|
|
```nix
|
|
|
|
crane.mkLib (import nixpkgs { system = "armv7l-linux"; })
|
|
|
|
```
|
|
|
|
|
|
|
|
### I want to override a particular package used by the crane library
|
|
|
|
|
|
|
|
Specific inputs can be overridden for the entire library via the
|
|
|
|
`overrideScope'` API as follows. For more information, see the [API docs] for
|
2022-06-26 21:25:40 +03:00
|
|
|
`mkLib`/`overrideToolchain`, or checkout the [custom-toolchain] example.
|
2022-01-18 02:01:21 +03:00
|
|
|
|
|
|
|
```nix
|
|
|
|
crane.lib.${system}.overrideScope' (final: prev: {
|
2022-06-26 21:25:40 +03:00
|
|
|
cargo-tarpaulin = myCustomCargoTarpaulinVersion;
|
2022-01-18 02:01:21 +03:00
|
|
|
})
|
|
|
|
```
|
|
|
|
|
2022-06-26 21:25:40 +03:00
|
|
|
```nix
|
|
|
|
crane.lib.${system}.overrideToolchain myCustomToolchain
|
|
|
|
```
|
|
|
|
|
2022-01-18 02:01:21 +03:00
|
|
|
### Nix is complaining about IFD (import from derivation)
|
|
|
|
|
|
|
|
If a derivation's `pname` and `version` attributes are not explicitly set,
|
|
|
|
crane will inspect the project's `Cargo.toml` file to set them as a convenience
|
|
|
|
to avoid duplicating that information by hand. This works well when the source
|
|
|
|
is a local path, but can cause issues if the source is being fetched remotely,
|
|
|
|
or flakes are not being used (since flakes have IFD enabled on by default).
|
|
|
|
|
|
|
|
One easy workaround for this issue (besides enabling the
|
|
|
|
`allow-import-from-derivation` option in Nix) is to explicitly set
|
|
|
|
`{ pname = "..."; version = "..."; }` in the derivation.
|
|
|
|
|
|
|
|
You'll know you've run into this issue if you see error messages along the lines
|
|
|
|
of:
|
|
|
|
* `cannot build '/nix/store/...-source.drv' during evaluation because the option 'allow-import-from-derivation' is disabled`
|
|
|
|
* `a 'aarch64-darwin' with features {} is required to build '/nix/store/...', but I am a 'x86_64-linux' with features {}`
|
|
|
|
|
2022-08-14 06:22:43 +03:00
|
|
|
### I'm getting rebuilds all of the time, especially when I change `flake.nix`
|
|
|
|
|
|
|
|
Nix will rebuild a derivation if any of its inputs change, which includes any
|
|
|
|
file contained by the source that is passed in. For example, if the build
|
|
|
|
expression specifies `src = ./.;` then the crate will be rebuilt when _any_ file
|
|
|
|
changes (including "unrelated" changes to `flake.nix`)!
|
|
|
|
|
|
|
|
There are two main ways to avoid unnecessary builds:
|
|
|
|
|
|
|
|
1. One option is to put the crate's source files into its own subdirectory (e.g.
|
|
|
|
`./mycrate`) and then set the build expression's source to that subdirectory
|
|
|
|
(e.g. `src = ./mycrate;`). Then, changes to files _outside_ of that directory
|
|
|
|
will be ignored and will not cause a rebuild
|
|
|
|
1. The other option is to use a [source cleaning] function from nixpkgs which
|
|
|
|
can omit any files know to not be needed while building the crate (for
|
|
|
|
example, all `*.nix` sources, `flake.lock`, and so on)
|
|
|
|
|
2022-01-18 02:05:45 +03:00
|
|
|
## License
|
|
|
|
|
|
|
|
This project is licensed under the [MIT license].
|
|
|
|
|
|
|
|
### Contribution
|
|
|
|
|
|
|
|
Unless you explicitly state otherwise, any contribution intentionally submitted
|
|
|
|
for inclusion by you, shall be licensed as MIT, without any additional terms or
|
|
|
|
conditions.
|
|
|
|
|
2022-01-18 02:01:21 +03:00
|
|
|
[API docs]: ./docs/API.md
|
2022-01-31 03:26:55 +03:00
|
|
|
[CHANGELOG]: ./CHANGELOG.md
|
2022-01-18 02:01:21 +03:00
|
|
|
[custom-toolchain]: ./examples/custom-toolchain/flake.nix
|
2022-01-18 02:05:45 +03:00
|
|
|
[MIT license]: ./LICENSE
|
2022-01-31 03:26:55 +03:00
|
|
|
[niv]: https://github.com/nmattia/niv
|
|
|
|
[Semantic Versioning]: http://semver.org/spec/v2.0.0.html
|
2022-08-14 06:22:43 +03:00
|
|
|
[source cleaning]: https://nixos.org/manual/nixpkgs/unstable/#sec-functions-library-sources
|