oxalica/rust-overlay
1
1
Fork 0
mirror of https://github.com/oxalica/rust-overlay.git synced 2024-10-04 05:07:13 +03:00
Code Issues Packages Projects Releases Wiki Activity

Decouple self-dependencies and impl lib.mkRustBin interface (#177)

Browse Source
This commit is contained in:
oxalica 2024-06-03 17:20:56 +08:00 committed by GitHub
commit 711ff40be1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
16 changed files with 418 additions and 212 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
.github/workflows/ci.yaml vendored
View File

@ -138,12 +138,16 @@ jobs:
[[ "$(< out)" == *"warning: this loop never actually loops"* ]]
test-cross-examples:
name: Test cross compilation examples
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
example: [cross-aarch64, cross-wasi, cross-mingw]
kind: [shell]
include:
- example: cross-aarch64
kind: flake
name: Test cross compilation examples (${{ matrix.kind }})
steps:
- name: Checkout
@ -158,15 +162,23 @@ jobs:
- name: Don't let ~/.cargo/bin mess things up
run: rm -rf ~/.cargo/bin
- name: Prepare nix-shell
- name: Prepare dev env
run: |
cd examples/${{ matrix.example }}
nix-shell --pure --command true
if [[ ${{ matrix.kind }} == 'shell' ]]; then
nix-shell --pure --command true
else
nix develop --ignore-environment --override-input rust-overlay ../..
fi
- name: Build and run example
run: |
cd examples/${{ matrix.example }}
set -o pipefail
# No --pure since it requires CA cert access for crates downloading.
nix-shell --command "make run" | tee out
if [[ ${{ matrix.kind }} == 'shell' ]]; then
# No --pure since it requires CA cert access for crates downloading.
nix-shell --pure --command "make run" | tee out
else
nix develop --ignore-environment --override-input rust-overlay ../.. --command make run | tee out
fi
[[ "$(< out)" == *"Hello, world!"* ]]

2
.gitignore vendored
View File

@ -2,3 +2,5 @@ target
result
result-*
*.tmp
examples/cross-aarch64/flake.lock

35
README.md
View File

@ -3,20 +3,32 @@
[![CI status](https://github.com/oxalica/rust-overlay/workflows/CI/badge.svg)](https://github.com/oxalica/rust-overlay/actions/workflows/ci.yaml)
[![sync-channels status](https://github.com/oxalica/rust-overlay/workflows/sync-channels/badge.svg)](https://github.com/oxalica/rust-overlay/actions/workflows/sync-channels.yaml)
*Pure and reproducible* overlay for binary distributed rust toolchains.
A compatible but better replacement for rust overlay of [nixpkgs-mozilla].
*Pure and reproducible* packaging of binary distributed rust toolchains.
A compatible but better replacement for rust overlay of [nixpkgs-mozilla],
with also non-overlay and [Nix Flake][flake] interfaces (despite the project name).
Hashes of toolchain components are pre-fetched in tree, so the evaluation is *pure* and
no need to have network access. It also works well with [Nix Flakes](https://nixos.wiki/wiki/Flakes).
[flake]: https://wiki.nixos.org/wiki/Flakes
For migration from [nixpkgs-mozilla], see [this section](#migration-from-nixpkgs-mozilla).
Features:
- Hashes of toolchain components are pre-fetched in tree, so the evaluation is
*pure* and no need to have network access.
- These hashes are auto-updated daily using GitHub Actions.
- The toolchain hashes are auto-updated daily using GitHub Actions.
- Current oldest supported version is stable 1.29.0 and beta/nightly 2018-09-13
(which are randomly picked and may change over time).
rust-overlay targets nixos-unstable and supported releases of NixOS, on x86_64-linux.
They are tested on CI. It may also work on other channels but is not guarenteed.
- We targets nixos-unstable and supported releases of NixOS, on x86\_64-linux.
They are tested on CI. Other platforms and nixpkgs channels may also work but
is not guaranteed.
For migration from [nixpkgs-mozilla], see [this section](#migration-from-nixpkgs-mozilla).
Documentations:
- [API References](docs/reference.md)
- [Cross compilation](docs/cross_compilation.md)
## Installation
@ -41,8 +53,6 @@ And then feel free to use it anywhere like
### Nix Flakes
This repository already has flake support.
**Warning: Only the output `overlay`/`overlays` are currently stable. Use other outputs at your own risk!**
For a quick play, just use `nix shell` to bring the latest stable rust toolchain into scope.
@ -218,11 +228,6 @@ Running `nix develop` will create a shell with the default beta Rust toolchain i
- There also an cross-compilation example in [`examples/cross-aarch64`].
## More documentations
- [Reference of all public attributes](docs/reference.md)
- [Cross compilation](docs/cross_compilation.md)
## License
MIT licensed.

72
default.nix
View File

@ -1,2 +1,72 @@
# Overlay interface for non-flake Nix.
final: prev:
(import ./rust-overlay.nix final) (prev // (import ./manifest.nix final prev))
let
inherit (builtins) mapAttrs readFile;
inherit (final) lib rust-bin;
inherit (rust-bin._internal) toolchainFromManifest selectManifest;
# Same as `toolchainFromManifest` but read from a manifest file.
toolchainFromManifestFile = path: toolchainFromManifest (fromTOML (readFile path));
# Override all pkgs of a toolchain set.
overrideToolchain = attrs: mapAttrs (name: pkg: pkg.override attrs);
# This is eagerly evaluated and disallow overriding. Get this from `final`
# will easily encounter infinite recursion without manually expand all attrs
# from `rust-bin.nix` output like `mkIf` does.
# This is considered internal anyway.
manifests = import ./lib/manifests.nix {
inherit lib;
inherit (rust-bin) distRoot;
};
in {
rust-bin = (prev.rust-bin or { }) // {
# The overridable dist url for fetching.
distRoot = import ./lib/dist-root.nix;
} // import ./lib/rust-bin.nix {
inherit lib manifests;
inherit (final.rust) toRustTarget;
inherit (rust-bin) nightly;
pkgs = final;
};
# All attributes below are for compatibility with mozilla overlay.
lib = (prev.lib or { }) // {
rustLib = (prev.lib.rustLib or { }) // {
manifest_v2_url = throw ''
`manifest_v2_url` is not supported.
Select a toolchain from `rust-bin` or using `rustChannelOf` instead.
See also README at https://github.com/oxalica/rust-overlay
'';
fromManifest = throw ''
`fromManifest` is not supported due to network access during evaluation.
Select a toolchain from `rust-bin` or using `rustChannelOf` instead.
See also README at https://github.com/oxalica/rust-overlay
'';
fromManifestFile = manifestFilePath: { stdenv, fetchurl, patchelf }@deps: builtins.trace ''
`fromManifestFile` is deprecated.
Select a toolchain from `rust-bin` or using `rustChannelOf` instead.
See also README at https://github.com/oxalica/rust-overlay
'' (overrideToolchain deps (toolchainFromManifestFile manifestFilePath));
};
};
rustChannelOf = manifestArgs: toolchainFromManifest (selectManifest manifestArgs);
latest = (prev.latest or {}) // {
rustChannels = {
stable = rust-bin.stable.latest;
beta = rust-bin.beta.latest;
nightly = rust-bin.nightly.latest;
};
};
rustChannelOfTargets = channel: date: targets:
(final.rustChannelOf { inherit channel date; })
.rust.override { inherit targets; };
rustChannels = final.latest.rustChannels;
}

36
docs/cross_compilation.md
View File

@ -1,5 +1,7 @@
# Cross compilation with `rust-overlay`
## `shell.nix` and `nix shell` development environment
There are examples for cross compilation in [`example` directory](../examples).
To try examples,
1. `cd` into `example/cross-aarch64` (or other directory).
@ -42,7 +44,7 @@ mkShell {
# See: https://github.com/NixOS/nixpkgs/pull/146583
depsBuildBuild = [ qemu ];
# Run-time dependencies. build = your-matchine, host = target = aarch64
# Run-time dependencies. build = your-machine, host = target = aarch64
# Usually are libraries to be linked.
buildInputs = [ openssl ];
@ -57,4 +59,34 @@ mkShell {
```
For more details about these different kinds of dependencies,
see also [Nix Wiki - Cross Compiling](https://nixos.wiki/wiki/Cross_Compiling#How_to_specify_dependencies).
see also [Nix Wiki - Cross Compiling][wiki-cross]
## Flakes and `nix develop` development environment
Unfortunately flake output layout does not natively support cross-compilation
(see [NixOS/nix#5157][flake-cross-issue]). We provide `mkRustBin` to allow
construction of `rust-bin` on an existing nixpkgs to leverage flake benefits of
not-importing nixpkgs again, with the cost of re-instantiating `rust-bin` for
each host-target tuples.
Pass the spliced packages for your cross-compilation target to `mkRustBin` to
get corresponding compiler toolchains for them.
```nix
let
pkgs = nixpkgs.legacyPackages.x86_64-linux.pkgsCross.aarch64-multiplatform;
rust-bin = rust-overlay.lib.mkRustBin { } pkgs.buildPackages;
in pkgs.mkShell {
nativeBuildInputs = [ rust-bin.stable.latest.minimal ];
}
```
The full example can be seen in
[`examples/cross-aarch64/flake.nix`](../examples/cross-aarch64/flake.nix).
To try it,
1. `cd` into `example/cross-aarch64`.
2. `nix develop` to enter the development environment.
3. `make run` to build and run the program in an emulator.
[wiki-cross]: https://wiki.nixos.org/wiki/Cross_Compiling#How_to_specify_dependencies
[flake-cross-issue]: https://github.com/NixOS/nix/issues/5157

209
docs/reference.md
View File

@ -1,86 +1,155 @@
# `rust-overlay` reference
# Reference
All public attributes provided by the overlay are below. Fields not defined here are for internal usage.
We provides two entry interfaces:
1. Overlay interface, via `default.nix` (non-flake) or
`rust-overlay.overlays.default` (flake).
This provides a [nixpkgs overlay][nixpkgs-overlay] to be used when
importing nixpkgs. The overlay will (try its best) not modify any existing
attributes of nixpkgs, but adds these new attributes:
- `rust-bin`,
where all main functionalities resides.
Its structure is documented in the next section.
- `latest`, `rustChannelOf`, `rustChannelOfTargets`, `rustChannels`,
for compatibility with [nixpkgs-mozilla].
They are built on top of `rust-bin` with only structure/argument
differences, and will provide the same corresponding derivations as
`rust-bin`.
[nixpkgs-overlay]: https://wiki.nixos.org/wiki/Overlays
[nixpkgs-mozilla]: https://github.com/mozilla/nixpkgs-mozilla
2. Builder interface, via `rust-overlay.lib.mkRustBin` (flake only).
It is a function with type
`{ distRoot ? ... } -> (pkgs: attrset) -> attrset`.
It returns an attrset with the same structure as `rust-bin` attribute
mentioned above, with the argument of an already imported (instantiated)
nixpkgs.
Notably, only the attrset `rust-bin` is returned, so this builder will not
give you any nixpkgs-mozilla compatible attributes.
## `rust-bin` structure
```nix
{
rust-bin = {
# The default dist url for fetching.
# Override it if you want to use a mirror server.
distRoot = "https://static.rust-lang.org/dist";
# The default dist url for fetching.
#
# For the overlay interface, this attribute is overridable. Changing it will
# also change all dependent URLs for source derivations (FOD). Useful for
# setting mirrors.
distRoot = "https://static.rust-lang.org/dist";
# Select a toolchain and aggregate components by rustup's `rust-toolchain` file format.
# See: https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file
fromRustupToolchain = { channel, components ? [], targets ? [] }: «derivation»;
# Same as `fromRustupToolchain` but read from a `rust-toolchain` file (legacy one-line string or in TOML).
fromRustupToolchainFile = rust-toolchain-file-path: «derivation»;
# Select a toolchain and aggregate components by rustup's `rust-toolchain` file format.
# See: https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file
fromRustupToolchain = { channel, components ? [], targets ? [] }: «derivation»;
# Same as `fromRustupToolchain` but read from a `rust-toolchain` file (legacy one-line string or in TOML).
fromRustupToolchainFile = rust-toolchain-file-path: «derivation»;
# Select the latest nightly toolchain which have specific components or profile available.
# This helps nightly users in case of latest nightly may not contains all components they want.
#
# `selectLatestNightlyWith (toolchain: toolchain.default)` selects the latest nightly toolchain
# with all `default` components (rustc, cargo, rustfmt, ...) available.
selectLatestNightlyWith = selector: «derivation»;
# Select the latest nightly toolchain which have specific components or profile available.
# This helps nightly users in case of latest nightly may not contains all components they want.
#
# `selectLatestNightlyWith (toolchain: toolchain.default)` selects the latest nightly toolchain
# with all `default` components (rustc, cargo, rustfmt, ...) available.
selectLatestNightlyWith = selector: «derivation»;
# Custom toolchain from a specific rustc git revision.
# This does almost the same thing as `rustup-toolchain-install-master`. (https://crates.io/crates/rustup-toolchain-install-master)
# Parameter `components` should be an attrset with component name as key and its SRI hash as value.
fromRustcRev = { pname ? …, rev, components, target ? … }: «derivation»;
# Custom toolchain from a specific rustc git revision.
# This does almost the same thing as `rustup-toolchain-install-master`. (https://crates.io/crates/rustup-toolchain-install-master)
# Parameter `components` should be an attrset with component name as key and its SRI hash as value.
fromRustcRev = { pname ? …, rev, components, target ? … }: «derivation»;
stable = {
# The latest stable toolchain.
latest = {
# Profiles, predefined component sets.
# See: https://rust-lang.github.io/rustup/concepts/profiles.html
minimal = «derivation»; # Only `cargo`, `rustc` and `rust-std`.
default = «derivation»; # The default profile of `rustup`. Good for general use.
complete = «derivation»; # Do not use it. It almost always fails.
stable = {
# The latest stable toolchain.
latest = {
# Profiles, predefined component sets.
# See: https://rust-lang.github.io/rustup/concepts/profiles.html
minimal = «derivation»; # Only `cargo`, `rustc` and `rust-std`.
default = «derivation»; # The default profile of `rustup`. Good for general use.
complete = «derivation»; # Do not use it. It almost always fails.
# Pre-aggregated package provided by upstream, the most commonly used package in `mozilla-overlay`.
# It consists of an uncertain number of components, usually more than the `default` profile of `rustup`
# but less than `complete` profile.
rust = «derivation»;
# Pre-aggregated package provided by upstream, the most commonly used package in `mozilla-overlay`.
# It consists of an uncertain number of components, usually more than the `default` profile of `rustup`
# but less than `complete` profile.
rust = «derivation»;
# Individial components.
rustc = «derivation»;
cargo = «derivation»;
rust-std = «derivation»;
# ... other components
};
"1.49.0" = { /* toolchain */ };
"1.48.0" = { /* toolchain */ };
# ... other versions.
# Individual components.
rustc = «derivation»;
cargo = «derivation»;
rust-std = «derivation»;
# ... other components
};
beta = {
# The latest beta toolchain.
latest = { /* toolchain */ };
"2021-01-01" = { /* toolchain */ };
"2020-12-30" = { /* toolchain */ };
# ... other versions.
};
nightly = {
# The latest nightly toolchain.
# It is preferred to use `selectLatestNightlyWith` instead of this since
# nightly toolchain may have components (like `rustfmt` or `rls`) missing,
# making `default` profile unusable.
latest = { /* toolchain */ };
"2020-12-31" = { /* toolchain */ };
"2020-12-30" = { /* toolchain */ };
# ... other versions.
};
# ... Some internal attributes omitted.
"1.49.0" = { /* toolchain */ };
"1.48.0" = { /* toolchain */ };
# ... other versions.
};
# These are for compatibility with nixpkgs-mozilla and
# provide same toolchains as `rust-bin.*`.
latest.rustChannels = /* ... */;
rustChannelOf = /* ... */;
rustChannelOfTargets = /* ... */;
rustChannels = /* ... */;
beta = {
# The latest beta toolchain.
latest = { /* toolchain */ };
"2021-01-01" = { /* toolchain */ };
"2020-12-30" = { /* toolchain */ };
# ... other versions.
};
nightly = {
# The latest nightly toolchain.
# It is preferred to use `selectLatestNightlyWith` instead of this since
# nightly toolchain may have components (like `rustfmt` or `rls`) missing,
# making `default` profile unusable.
latest = { /* toolchain */ };
"2020-12-31" = { /* toolchain */ };
"2020-12-30" = { /* toolchain */ };
# ... other versions.
};
# ... Some internal attributes omitted.
}
```
For more details, see also the source code of [`rust-overlay.nix`](../rust-overlay.nix).
## Flake output structure
```nix
{
lib.mkRustBin = { distRoot ? /*...*/ }: pkgs: «attrset rust-bin»;
overlays = {
# The overlay.
default = final: prev: «attrset»;
# Alias to `default`.
rust-overlay = final: prev: «attrset»;
};
# WARNING: The structure of `packages` here is to-be-determined and may be
# renamed or modified in the future.
packages.${system} = {
# Stable toolchain of release major.minor.patch
# The derivation is the "default" profile, and the "minimal" profile can be
# accessed via `minimal` attribute.
# This type is abbreviated as `«toolchain»` below.
"rust_${major}_${minor}_${patch}" = «derivation» // { minimal = «derivation»; };
# Alias to `rust_${latest-major-minor-patch}`.
rust = «toolchain»;
# Alias to `rust`.
default = «toolchain»;
# Nightly toolchain of a specific date.
"rust-nightly_${yyyy}-${mm}-${dd}" = «toolchain»;
# Alias to `rust-nightly_${latest-nightly-yyyy-mm-dd}`.
rust-nightly = «toolchain»;
# Beta toolchain of a specific date.
"rust-beta_${yyyy}-${mm}-${dd}" = «toolchain»;
# Alias to `rust-beta_${latest-beta-yyyy-mm-dd}`.
rust-beta = «toolchain»;
};
# ... Some internal attributes omitted.
}
```
For more details, see also the source code of [`lib/rust-bin.nix`](../lib/rust-bin.nix).

29
examples/cross-aarch64/flake.nix Normal file
View File

@ -0,0 +1,29 @@
# Example flake for `nix develop`.
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
rust-overlay.url = "github:oxalica/rust-overlay";
rust-overlay.inputs.nixpkgs.follows = "nixpkgs";
};
outputs = { self, rust-overlay, nixpkgs }: {
devShells.x86_64-linux.default = let
pkgs = nixpkgs.legacyPackages.x86_64-linux.pkgsCross.aarch64-multiplatform;
rust-bin = rust-overlay.lib.mkRustBin { } pkgs.buildPackages;
in
pkgs.mkShell {
nativeBuildInputs = [
rust-bin.stable.latest.minimal
pkgs.buildPackages.pkg-config
];
depsBuildBuild = [ pkgs.pkgsBuildBuild.qemu ];
buildInputs = [ pkgs.openssl ];
env = {
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER = "${pkgs.stdenv.cc.targetPrefix}cc";
CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUNNER = "qemu-aarch64";
};
};
};
}

1
examples/cross-aarch64/shell.nix
View File

@ -1,3 +1,4 @@
# Example flake for `nix shell`.
# See docs/cross_compilation.md for details.
(import <nixpkgs> {
crossSystem = "aarch64-linux";

55
flake.nix
View File

@ -10,11 +10,10 @@
};
outputs = { self, nixpkgs, flake-utils }: let
inherit (nixpkgs.lib)
inherit (nixpkgs) lib;
inherit (lib)
elem filterAttrs head mapAttrs' optionalAttrs replaceStrings;
overlay = import ./.;
allSystems = [
"aarch64-darwin"
"aarch64-linux"
@ -30,13 +29,43 @@
"x86_64-linux"
];
overlay = import ./.;
defaultDistRoot = import ./lib/dist-root.nix;
mkManifests = distRoot: import ./lib/manifests.nix { inherit lib distRoot; };
# Builder to construct `rust-bin` interface on an existing `pkgs`.
# This would be immutable, non-intrusive and (hopefully) can benefit from
# flake eval-cache.
#
# Note that this does not contain compatible attrs for mozilla-overlay.
mkRustBin =
{ distRoot ? defaultDistRoot }:
pkgs:
lib.fix (rust-bin: import ./lib/rust-bin.nix {
inherit lib pkgs;
inherit (pkgs.rust) toRustTarget;
inherit (rust-bin) nightly;
manifests = mkManifests distRoot;
});
in {
lib = {
# Internal use only!
_internal = {
defaultManifests = mkManifests defaultDistRoot;
};
inherit mkRustBin;
};
overlays = {
default = overlay;
rust-overlay = overlay;
};
} // flake-utils.lib.eachSystem allSystems (system: let
pkgs = import nixpkgs { inherit system; overlays = [ overlay ]; };
pkgs = nixpkgs.legacyPackages.${system};
rust-bin = mkRustBin {} pkgs;
in {
# TODO: Flake outputs except `overlay[s]` are not stabilized yet.
@ -54,25 +83,27 @@
then "rust"
else "rust_${replaceStrings ["."] ["_"] version}";
value = select version comps;
}) pkgs.rust-bin.stable //
}) rust-bin.stable //
mapAttrs' (version: comps: {
name = if version == "latest"
then "rust-nightly"
else "rust-nightly_${version}";
value = select version comps;
}) pkgs.rust-bin.nightly //
}) rust-bin.nightly //
mapAttrs' (version: comps: {
name = if version == "latest"
then "rust-beta"
else "rust-beta_${version}";
value = select version comps;
}) pkgs.rust-bin.beta;
}) rust-bin.beta;
result' = filterAttrs (name: drv: drv != null) result;
in result' // { default = result'.rust; };
checks = let
inherit (pkgs) rust-bin rustChannelOf;
inherit (pkgs.rust-bin) fromRustupToolchain fromRustupToolchainFile stable beta nightly;
inherit (rust-bin) fromRustupToolchain fromRustupToolchainFile stable beta nightly;
pkgs-compat = import nixpkgs { inherit system; overlays = [ overlay ]; };
inherit (pkgs-compat) latest rustChannelOf;
rustHostPlatform = pkgs.rust.toRustTarget pkgs.hostPlatform;
@ -100,9 +131,9 @@
rename-unavailable = assertEq (stable."1.30.0" ? rustfmt) false;
rename-available = assertEq stable."1.48.0".rustfmt stable."1.48.0".rustfmt-preview;
latest-stable-legacy = assertEq pkgs.latest.rustChannels.stable.rustc stable.latest.rustc;
latest-beta-legacy = assertEq pkgs.latest.rustChannels.beta.rustc beta.latest.rustc;
latest-nightly-legacy = assertEq pkgs.latest.rustChannels.nightly.rustc nightly.latest.rustc;
latest-stable-legacy = assertEq latest.rustChannels.stable.rustc stable.latest.rustc;
latest-beta-legacy = assertEq latest.rustChannels.beta.rustc beta.latest.rustc;
latest-nightly-legacy = assertEq latest.rustChannels.nightly.rustc nightly.latest.rustc;
rust-channel-of-stable = assertEq (rustChannelOf { channel = "stable"; }).rustc stable.latest.rustc;
rust-channel-of-beta = assertEq (rustChannelOf { channel = "beta"; }).rustc beta.latest.rustc;

0
cargo-miri-wrapper.sh → lib/cargo-miri-wrapper.sh
View File

1
lib/dist-root.nix Normal file
View File

@ -0,0 +1 @@
"https://static.rust-lang.org/dist"

27
manifest.nix → lib/manifests.nix
View File

@ -1,15 +1,15 @@
final: prev:
# Manifests which describe the content of each version.
{ lib, distRoot }:
let
inherit (builtins) match isString toString;
inherit (final.lib)
inherit (lib)
attrNames concatMap elemAt filter hasAttr mapAttrs mapAttrs' removeSuffix;
targets = import ./manifests/targets.nix // { _ = "*"; };
renamesList = import ./manifests/renames.nix;
profilesList = import ./manifests/profiles.nix;
targets = import ../manifests/targets.nix // { _ = "*"; };
renamesList = import ../manifests/renames.nix;
profilesList = import ../manifests/profiles.nix;
inherit (final.rust-bin) distRoot;
# Extensions for mixed `rust` pkg.
components = [
@ -110,16 +110,7 @@ let
in ret // { latest = ret.${set.latest}; };
in {
rust-bin = (prev.rust-bin or {}) // {
# The dist url for fetching.
# Override it if you want to use a mirror server.
distRoot = "https://static.rust-lang.org/dist";
# For internal usage.
manifests = {
stable = uncompressManifestSet "stable" (import ./manifests/stable);
beta = uncompressManifestSet "beta" (import ./manifests/beta);
nightly = uncompressManifestSet "nightly" (import ./manifests/nightly);
};
};
stable = uncompressManifestSet "stable" (import ../manifests/stable);
beta = uncompressManifestSet "beta" (import ../manifests/beta);
nightly = uncompressManifestSet "nightly" (import ../manifests/nightly);
}

0
mk-aggregated.nix → lib/mk-aggregated.nix
View File

0
mk-component-set.nix → lib/mk-component-set.nix
View File

137
rust-overlay.nix → lib/rust-bin.nix
View File

@ -1,14 +1,22 @@
# Define component resolution and utility functions.
self: super:
# Component resolution, aggregation and other utility functions.
# Provide the content of `rust-bin`.
{
lib,
pkgs,
toRustTarget,
manifests,
nightly,
}:
let
inherit (builtins) compareVersions fromTOML match readFile tryEval;
inherit (self.lib)
inherit (lib)
any attrNames attrValues concatStringsSep elem elemAt filter flatten foldl'
hasPrefix head isString length listToAttrs makeOverridable mapAttrs
mapAttrsToList optional optionalAttrs replaceStrings substring trace unique;
inherit (pkgs) stdenv callPackage fetchurl;
# Remove keys from attrsets whose value is null.
removeNulls = set:
removeAttrs set
@ -16,27 +24,26 @@ let
(attrNames set));
# FIXME: https://github.com/NixOS/nixpkgs/pull/146274
toRustTarget = platform:
toRustTarget' = platform:
if platform.isWasi then
"${platform.parsed.cpu.name}-wasi"
else
platform.rust.rustcTarget or (super.rust.toRustTarget platform);
platform.rust.rustcTarget or (toRustTarget platform);
# The platform where `rustc` is running.
rustHostPlatform = toRustTarget self.stdenv.hostPlatform;
rustHostPlatform = toRustTarget' stdenv.hostPlatform;
# The platform of binary which `rustc` produces.
rustTargetPlatform = toRustTarget self.stdenv.targetPlatform;
rustTargetPlatform = toRustTarget' stdenv.targetPlatform;
mkComponentSet = self.callPackage ./mk-component-set.nix {
inherit toRustTarget removeNulls;
mkComponentSet = callPackage ./mk-component-set.nix {
inherit removeNulls;
toRustTarget = toRustTarget';
};
mkAggregated = self.callPackage ./mk-aggregated.nix {};
mkAggregated = callPackage ./mk-aggregated.nix {};
# Manifest selector.
selectManifest = { channel, date ? null }: let
inherit (self.rust-bin) manifests;
assertWith = cond: msg: body: if cond then body else throw msg;
# https://rust-lang.github.io/rustup/concepts/toolchains.html#toolchain-specification
@ -127,7 +134,7 @@ let
matchParenPart = match ".*/([^ /]*) [(][^)]*[)](.*)" url;
name = if matchParenPart == null then "" else (elemAt matchParenPart 0) + (elemAt matchParenPart 1);
in
self.fetchurl { inherit name sha256; url = url'; };
fetchurl { inherit name sha256; url = url'; };
# Resolve final components to install from mozilla-overlay style `extensions`, `targets` and `targetExtensions`.
#
@ -221,7 +228,7 @@ let
${concatStringsSep "\n" (errors ++ notes)}
'';
# Genereate the toolchain set from a parsed manifest.
# Generate the toolchain set from a parsed manifest.
#
# Manifest files are organized as follow:
# { date = "2017-03-03";
@ -252,8 +259,6 @@ let
# *Attention* If you want to install an extension like rust-src, that has no fixed architecture (arch *),
# you will need to specify this extension in the extensions options or it will not be installed!
toolchainFromManifest = manifest: let
maybeRename = name: manifest.renames.${name}.to or name;
# platform -> true
# For fail-fast test.
allPlatformSet =
@ -348,12 +353,6 @@ let
_manifest = manifest;
};
# Same as `toolchainFromManifest` but read from a manifest file.
toolchainFromManifestFile = path: toolchainFromManifest (fromTOML (readFile path));
# Override all pkgs of a toolchain set.
overrideToolchain = attrs: mapAttrs (name: pkg: pkg.override attrs);
# From a git revision of rustc.
# This does the same thing as crate `rustup-toolchain-install-master`.
# But you need to manually provide component hashes.
@ -370,7 +369,7 @@ let
target ? rustTargetPlatform
}: let
hashToSrc = compName: hash:
self.fetchurl {
fetchurl {
url = if compName == "rust-src"
then "https://ci-artifacts.rust-lang.org/rustc-builds/${rev}/${compName}-nightly.tar.xz"
else "https://ci-artifacts.rust-lang.org/rustc-builds/${rev}/${compName}-nightly-${target}.tar.xz";
@ -394,10 +393,10 @@ let
# `selectLatestNightlyWith (toolchain: toolchain.default.override { extensions = ["llvm-tools-preview"]; })`
selectLatestNightlyWith = selector:
let
nightlyDates = attrNames (removeAttrs self.rust-bin.nightly [ "latest" ]);
nightlyDates = attrNames (removeAttrs nightly [ "latest" ]);
dateLength = length nightlyDates;
go = idx:
let ret = selector (self.rust-bin.nightly.${elemAt nightlyDates idx}); in
let ret = selector (nightly.${elemAt nightlyDates idx}); in
if idx == 0 then
ret
else if dateLength - idx >= 256 then
@ -409,67 +408,31 @@ let
in
go (length nightlyDates - 1);
in {
# For each channel:
# rust-bin.stable.latest.{minimal,default,complete} # Profiles.
# rust-bin.stable.latest.rust # Pre-aggregate from upstream.
# rust-bin.stable.latest.cargo # Components...
# rust-bin.stable.latest.rustc
# rust-bin.stable.latest.rust-docs
# ...
#
# For a specific version of stable:
# rust-bin.stable."1.47.0".default
#
# For a specific date of beta:
# rust-bin.beta."2021-01-01".default
#
# For a specific date of nightly:
# rust-bin.nightly."2020-01-01".default
rust-bin =
(super.rust-bin or {}) //
mapAttrs (channel: mapAttrs (version: toolchainFromManifest)) super.rust-bin.manifests //
{
inherit fromRustupToolchain fromRustupToolchainFile;
inherit selectLatestNightlyWith;
inherit fromRustcRev;
};
in
# For each channel:
# rust-bin.stable.latest.{minimal,default,complete} # Profiles.
# rust-bin.stable.latest.rust # Pre-aggregate from upstream.
# rust-bin.stable.latest.cargo # Components...
# rust-bin.stable.latest.rustc
# rust-bin.stable.latest.rust-docs
# ...
#
# For a specific version of stable:
# rust-bin.stable."1.47.0".default
#
# For a specific date of beta:
# rust-bin.beta."2021-01-01".default
#
# For a specific date of nightly:
# rust-bin.nightly."2020-01-01".default
mapAttrs (channel: mapAttrs (version: toolchainFromManifest)) manifests //
{
inherit fromRustupToolchain fromRustupToolchainFile;
inherit selectLatestNightlyWith;
inherit fromRustcRev;
# All attributes below are for compatiblity with mozilla overlay.
lib = (super.lib or {}) // {
rustLib = (super.lib.rustLib or {}) // {
manifest_v2_url = throw ''
`manifest_v2_url` is not supported.
Select a toolchain from `rust-bin` or using `rustChannelOf` instead.
See also README at https://github.com/oxalica/rust-overlay
'';
fromManifest = throw ''
`fromManifest` is not supported due to network access during evaluation.
Select a toolchain from `rust-bin` or using `rustChannelOf` instead.
See also README at https://github.com/oxalica/rust-overlay
'';
fromManifestFile = manifestFilePath: { stdenv, fetchurl, patchelf }@deps: trace ''
`fromManifestFile` is deprecated.
Select a toolchain from `rust-bin` or using `rustChannelOf` instead.
See also README at https://github.com/oxalica/rust-overlay
'' (overrideToolchain deps (toolchainFromManifestFile manifestFilePath));
};
_internal = {
inherit toolchainFromManifest;
inherit selectManifest;
};
rustChannelOf = manifestArgs: toolchainFromManifest (selectManifest manifestArgs);
latest = (super.latest or {}) // {
rustChannels = {
stable = self.rust-bin.stable.latest;
beta = self.rust-bin.beta.latest;
nightly = self.rust-bin.nightly.latest;
};
};
rustChannelOfTargets = channel: date: targets:
(self.rustChannelOf { inherit channel date; })
.rust.override { inherit targets; };
rustChannels = self.latest.rustChannels;
}

4
scripts/fetch.py
View File

@ -290,7 +290,7 @@ def sync_stable_channel(*, stop_if_exists, max_update=None):
update_stable_index()
def sync_beta_channel(*, stop_if_exists, max_update=None):
# Fetch the global nightly manifest to retrive the latest nightly version.
# Fetch the global nightly manifest to retrieve the latest nightly version.
print('Fetching latest beta version')
manifest = fetch_url(f'{DIST_ROOT}/channel-rust-beta.toml').text
date = datetime.date.fromisoformat(toml.loads(manifest)['date'])
@ -313,7 +313,7 @@ def sync_beta_channel(*, stop_if_exists, max_update=None):
update_beta_index()
def sync_nightly_channel(*, stop_if_exists, max_update=None):
# Fetch the global nightly manifest to retrive the latest nightly version.
# Fetch the global nightly manifest to retrieve the latest nightly version.
print('Fetching latest nightly version')
manifest = fetch_url(f'{DIST_ROOT}/channel-rust-nightly.toml').text
date = datetime.date.fromisoformat(toml.loads(manifest)['date'])