Add support for materializing generated nix files (#356)

This change adds a `materialized` argument to the functions that
generate nix files with `plan-to-nix` and `stack-to-nix`.

* Provides `builtins.trace` messages with pointers on how to do so.

* Includes a `checkMaterialized` option that can be used
  to verify the materialized files.

* Documents how to script updates with `plan-nix` or `stack-nix`.

* Outputs warnings if materialized files are used without hash
  (or without an `index-state` for cabal projects).

* Provides materialized nix files for `alex`, `happy` and `hscolour`
This commit is contained in:
Hamish Mackenzie 2019-12-10 16:42:25 +13:00 committed by GitHub
parent 7e996d5f96
commit f5d1f82952
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 1073 additions and 19 deletions

View File

@ -0,0 +1,270 @@
# Materialization
## What is materialization?
Capturing and storing the nix files for a project so that they do
not need to be built (or checked). This allows us to cache the input
of an IFD (import from derviation).
## Why use materialization?
Using functions like `cabalProject`, `stackProject` and `hackage-package`
results in a lot of dependencies (all the dependencies of nix-tools
for instance).
* They can be slow to calculate (even if no work needs to be done it
is not unusual for it to take 5 seconds per project).
* They can be slow to build (or download) on machines that do not
yet have them in the nix store.
* Hydra does not show progress because it does not provide feedback
until it has a list of jobs and the list of jobs cannot depends
on the nix being present (although this is often blamed on IFD
it would be the same if it wrote out JSON files and read them in)
## When is it ok to materialize?
* The nix is unlikely to change frequently (and when it does you
are happy to manually update it).
* You are happy to script something to update the materialized
nix files automatically.
* You are certain that the IFD you materialize is not `system`-dependent. If it was you'd
obtain different nix expressions depending on which `system` the IFD was evaluated.
## How can we materialize the nix files?
Lets say we want to build `hlint`. We might start with an `hlint`
file that looks like this:
```nix
((import ./nixpkgs (import ./.)).haskell-nix.hackage-package {
name = "hlint";
version = "2.2.4";
}).components.exes.hlint
```
Building this may result in a lot of output, but if youb build
it again it should give just:
```
$ nix-build hlint.nix
trace: Using latest index state for hlint!
trace: Using index-state: 2019-12-09T00:00:00Z for hlint
/nix/store/7wwqq5v42gm6iiz2d3ngbnkyw7i4py11-hlint-2.2.4-exe-hlint
```
To materialize the nix files we need to take care to pin down the
inputs. For `cabalProject` and `hackage-package` this means
we must specify the `index-state` of hackage we want to use:
```nix
((import ./nixpkgs (import ./.)).haskell-nix.hackage-package {
name = "hlint";
version = "2.2.4";
index-state = "2019-12-03T00:00:00Z";
}).components.exes.hlint
```
Now if we build again we get a hint telling use how to
calculate a suitable sha256 hash to turn the derivation
containing the nix files into a fixed output derivation:
```
$ nix-build hlint.nix
trace: Using index-state: 2019-12-03T00:00:00Z for hlint
trace: Get `plan-sha256` with `nix-hash --base32 --type sha256 /nix/store/qk1fvza1alkvs51vzmpjp2xsg8xklyxk-hlint-plan-to-nix-pkgs/`
/nix/store/7wwqq5v42gm6iiz2d3ngbnkyw7i4py11-hlint-2.2.4-exe-hlint
$ nix-hash --base32 --type sha256 /nix/store/qk1fvza1alkvs51vzmpjp2xsg8xklyxk-hlint-plan-to-nix-pkgs/
1a4rhv3h2daz6dzwzfl3w7l1v556n7aqfiagw6m0rvqq230iabss
```
We can add the hash as `plan-sha256` or (`stack-sha256` for
`stackProject`)
```nix
((import ./nixpkgs (import ./.)).haskell-nix.hackage-package {
name = "hlint";
version = "2.2.4";
index-state = "2019-12-03T00:00:00Z";
plan-sha256 = "1a4rhv3h2daz6dzwzfl3w7l1v556n7aqfiagw6m0rvqq230iabss";
}).components.exes.hlint
```
Just adding the hash might help reuse of the cached nix, but nix will
still calculate all the dependencies (which can add seconds to
`nix-build` and `nix-shell` commands when no other work is needed)
and users who do not yet have the dependencies in their store will have
to wait while they are built or downloaded.
Running nix build again gives us a hint on what we can do next:
```
$ nix-build hlint.nix
trace: Using index-state: 2019-12-03T00:00:00Z for hlint
trace: To materialize, point `materialized` to a copy of /nix/store/0xalcphb7ifvy5fc9dpwj40fij6nn5av-hlint-plan-to-nix-pkgs
/nix/store/7wwqq5v42gm6iiz2d3ngbnkyw7i4py11-hlint-2.2.4-exe-hlint
```
To capture the nix we can do something like:
```nix
((import ./nixpkgs (import ./.)).haskell-nix.hackage-package {
name = "hlint";
version = "2.2.4";
index-state = "2019-12-03T00:00:00Z";
plan-sha256 = "1a4rhv3h2daz6dzwzfl3w7l1v556n7aqfiagw6m0rvqq230iabss";
materialized = ./hlint.materialized;
}).components.exes.hlint
```
Now we can copy the nix files needed and build with:
```
$ cp -r /nix/store/0xalcphb7ifvy5fc9dpwj40fij6nn5av-hlint-plan-to-nix-pkgs hlint.materialized
$ nix-build hlint.nix
/nix/store/7wwqq5v42gm6iiz2d3ngbnkyw7i4py11-hlint-2.2.4-exe-hlint
```
We may want to run `cmod -R +w hlint.materialized` as the files copied from the
store will be read only.
## How can we check `sha256` and `materialized` are up to date?
Let's pretend we had to go back to `hlint` version `2.2.3`.
We can change `version` and temporarily add
`checkMaterialization = true;`:
```nix
((import ./nixpkgs (import ./.)).haskell-nix.hackage-package {
name = "hlint";
version = "2.2.3";
index-state = "2019-12-03T00:00:00Z";
plan-sha256 = "1a4rhv3h2daz6dzwzfl3w7l1v556n7aqfiagw6m0rvqq230iabss";
materialized = ./hlint.materialized;
checkMaterialization = true;
}).components.exes.hlint
```
This will fail and report the details of what is wrong:
```
$ nix-build hlint.nix
trace: Using index-state: 2019-12-03T00:00:00Z for hlint
building '/nix/store/zmif4gk52ynh57pf4dikzgsk30haqi2b-hlint-plan-to-nix-pkgs.drv'...
Changes to hlint-plan-to-nix-pkgs not reflected in plan-sha256
diff -ru /nix/store/0xalcphb7ifvy5fc9dpwj40fij6nn5av-hlint-plan-to-nix-pkgs/.plan.nix/hlint.nix /nix/store/h5j8k3y5lansyfss25gd7knbninzr6z4-hlint-plan-to-nix-pkgs/.plan.nix/hlint.nix
--- /nix/store/0xalcphb7ifvy5fc9dpwj40fij6nn5av-hlint-plan-to-nix-pkgs/.plan.nix/hlint.nix 1970-01-01 00:00:01.000000000 +0000
+++ /nix/store/h5j8k3y5lansyfss25gd7knbninzr6z4-hlint-plan-to-nix-pkgs/.plan.nix/hlint.nix 1970-01-01 00:00:01.000000000 +0000
@@ -42,7 +42,7 @@
flags = { threaded = true; gpl = true; ghc-lib = false; };
package = {
specVersion = "1.18";
- identifier = { name = "hlint"; version = "2.2.4"; };
+ identifier = { name = "hlint"; version = "2.2.3"; };
license = "BSD-3-Clause";
copyright = "Neil Mitchell 2006-2019";
maintainer = "Neil Mitchell <ndmitchell@gmail.com>";
@@ -95,7 +95,6 @@
(hsPkgs."extra" or (buildDepError "extra"))
(hsPkgs."refact" or (buildDepError "refact"))
(hsPkgs."aeson" or (buildDepError "aeson"))
- (hsPkgs."filepattern" or (buildDepError "filepattern"))
(hsPkgs."syb" or (buildDepError "syb"))
(hsPkgs."mtl" or (buildDepError "mtl"))
] ++ (if !flags.ghc-lib && (compiler.isGhc && (compiler.version).ge "8.8.0") && (compiler.isGhc && (compiler.version).lt "8.9.0")
diff -ru /nix/store/0xalcphb7ifvy5fc9dpwj40fij6nn5av-hlint-plan-to-nix-pkgs/default.nix /nix/store/h5j8k3y5lansyfss25gd7knbninzr6z4-hlint-plan-to-nix-pkgs/default.nix
--- /nix/store/0xalcphb7ifvy5fc9dpwj40fij6nn5av-hlint-plan-to-nix-pkgs/default.nix 1970-01-01 00:00:01.000000000 +0000
+++ /nix/store/h5j8k3y5lansyfss25gd7knbninzr6z4-hlint-plan-to-nix-pkgs/default.nix 1970-01-01 00:00:01.000000000 +0000
@@ -76,7 +76,7 @@
"tagged".revision = (((hackage."tagged")."0.8.6").revisions).default;
"tagged".flags.transformers = true;
"tagged".flags.deepseq = true;
- "haskell-src-exts".revision = (((hackage."haskell-src-exts")."1.22.0").revisions).default;
+ "haskell-src-exts".revision = (((hackage."haskell-src-exts")."1.21.1").revisions).default;
"unliftio-core".revision = (((hackage."unliftio-core")."0.1.2.0").revisions).default;
"ghc-lib-parser".revision = (((hackage."ghc-lib-parser")."8.8.1").revisions).default;
"containers".revision = (((hackage."containers")."0.6.0.1").revisions).default;
@@ -116,7 +116,6 @@
"hpc".revision = (((hackage."hpc")."0.6.0.3").revisions).default;
"filepath".revision = (((hackage."filepath")."1.4.2.1").revisions).default;
"process".revision = (((hackage."process")."1.6.5.0").revisions).default;
- "filepattern".revision = (((hackage."filepattern")."0.1.1").revisions).default;
"libyaml".revision = (((hackage."libyaml")."0.1.1.1").revisions).default;
"libyaml".flags.system-libyaml = false;
"libyaml".flags.no-unicode = false;
Calculated hash is 1qjmhlb4rw6mggs7y57f6zr5zjmkhkx7sn9q8pb18308n5nxgxcs expected hash was 1a4rhv3h2daz6dzwzfl3w7l1v556n7aqfiagw6m0rvqq230iabss for hlint-plan-to-nix-pkgs
builder for '/nix/store/zmif4gk52ynh57pf4dikzgsk30haqi2b-hlint-plan-to-nix-pkgs.drv' failed with exit code 1
error: build of '/nix/store/zmif4gk52ynh57pf4dikzgsk30haqi2b-hlint-plan-to-nix-pkgs.drv' failed
```
Checking the materialization requires nix to do all the work that materialization
avoids. So while it might be tempting to leave `checkMaterialization = true` all
the time, we would be better off just removing `materialized` and `plan-sha256`.
## How can we update the nix files with a script?
There are versions of the functions (`cabalProject'`, `stackProject'`
and `hackage-project`) that also return the nix as `plan-nix` or
`stack-nix`. By calling one of these functions without the
hash and materialized nix we can find out what nix files should be.
For instance:
```nix
(import ./nixpkgs (import ./.)).haskell-nix.hackage-project {
name = "hlint";
version = "2.2.4";
index-state = "2019-12-03T00:00:00Z";
}
```
```
$ nix-build hlint.nix -A plan-nix
trace: Using index-state: 2019-12-03T00:00:00Z for hlint
trace: Get `plan-sha256` with `nix-hash --base32 --type sha256 /nix/store/qk1fvza1alkvs51vzmpjp2xsg8xklyxk-hlint-plan-to-nix-pkgs/`
/nix/store/qk1fvza1alkvs51vzmpjp2xsg8xklyxk-hlint-plan-to-nix-pkgs
```
We can have the script copy `$(nix-build hlint.nix -A plan-nix --no-out-link)`
and use `nix-hash` to calculate the new value for `plan-sha256`.
## Can we skip making a copy and use `materialized = /nix/store/...`?
Yes and it gives us the same speed improvement, however:
* It does not help at all in `restricted-eval` mode (Hydra).
* Users will still wind up building or downloading the dependencies
needed to build the nix fileds (if they do not have them).
For instance:
```nix
((import ./nixpkgs (import ./.)).haskell-nix.hackage-package {
name = "hlint";
version = "2.2.4";
index-state = "2019-12-03T00:00:00Z";
plan-sha256 = "1a4rhv3h2daz6dzwzfl3w7l1v556n7aqfiagw6m0rvqq230iabss";
materialized = /nix/store/qk1fvza1alkvs51vzmpjp2xsg8xklyxk-hlint-plan-to-nix-pkgs;
}).components.exes.hlint
```
Running when no building is needed is still slow in restricted evaluation mode.
```
$ time nix-build --option restrict-eval true -I . --option allowed-uris "https://github.com/NixOS https://github.com/input-output-hk" hlint.nix
trace: Using index-state: 2019-12-03T00:00:00Z for hlint
/nix/store/7wwqq5v42gm6iiz2d3ngbnkyw7i4py11-hlint-2.2.4-exe-hlint
real 0m10.066s
user 0m8.563s
sys 0m0.630s
$ time nix-build hlint.nix
/nix/store/7wwqq5v42gm6iiz2d3ngbnkyw7i4py11-hlint-2.2.4-exe-hlint
real 0m4.628s
user 0m3.889s
sys 0m0.389s
```

View File

@ -1,9 +1,11 @@
{ dotCabal, pkgs, runCommand, nix-tools, cabal-install, ghc, hpack, symlinkJoin, cacert, index-state-hashes, haskellLib }@defaults:
{ dotCabal, pkgs, runCommand, nix-tools, cabal-install, ghc, hpack, symlinkJoin, cacert, index-state-hashes, haskellLib, materialize }@defaults:
{ name ? null # optional name for better error messages
, src
, index-state ? null # Hackage index-state, eg. "2019-10-10T00:00:00Z"
, index-sha256 ? null # The hash of the truncated hackage index-state
, plan-sha256 ? null # The hash of the plan-to-nix output (makes the plan-to-nix step a fixed output derivation)
, materialized ? null # Location of a materialized copy of the nix files
, checkMaterialization ? null # If true the nix files will be generated used to check plan-sha256 and material
, cabalProject ? null # Cabal project file (when null uses "${src}/cabal.project")
, ghc ? defaults.ghc
, nix-tools ? defaults.nix-tools
@ -156,20 +158,24 @@ let
}
else replaceSoureRepos rawCabalProject;
plan-nix = runCommand (if name == null then "plan-to-nix-pkgs" else name + "-plan-to-nix-pkgs") ({
plan-nix = materialize ({
inherit materialized;
sha256 = plan-sha256;
sha256Arg = "plan-sha256";
# Before pinning stuff down we need an index state to use
reasonNotSafe =
if index-state == null
then "index-state is not set"
else null;
} // pkgs.lib.optionalAttrs (checkMaterialization != null) {
inherit checkMaterialization;
}) (runCommand (if name == null then "plan-to-nix-pkgs" else name + "-plan-to-nix-pkgs") {
nativeBuildInputs = [ nix-tools ghc hpack cabal-install pkgs.rsync pkgs.git ];
# Needed or stack-to-nix will die on unicode inputs
LANG = "en_US.UTF-8";
meta.platforms = pkgs.lib.platforms.all;
preferLocalBuild = false;
} // (if plan-sha256 == null
then {}
else {
outputHashMode = "recursive";
outputHashAlgo = "sha256";
outputHash = plan-sha256;
})
) (''
} ''
tmp=$(mktemp -d)
cd $tmp
# if maybeCleanedSource is empty, this means it's a new
@ -222,6 +228,11 @@ let
# proper relative paths.
(cd $out && plan-to-nix --full --plan-json $tmp/dist-newstyle/cache/plan.json -o .)
# Remove the non nix files ".project" ".cabal" "package.yaml" files
# as they should not be in the output hash (they may change slightly
# without affecting the nix).
find $out -type f ! -name '*.nix' -exec rm "{}" \;
# move pkgs.nix to default.nix ensure we can just nix `import` the result.
mv $out/pkgs.nix $out/default.nix
'');

View File

@ -5,8 +5,12 @@
*
* see also `call-cabal-project-to-nix`!
*/
{ runCommand, nix-tools, pkgs, mkCacheFile }:
{ src, stackYaml ? null, ignorePackageYaml ? false, cache ? null, ... }:
{ runCommand, nix-tools, pkgs, mkCacheFile, materialize }:
{ src, stackYaml ? null, ignorePackageYaml ? false, cache ? null
, stack-sha256 ? null
, materialized ? null # Location of a materialized copy of the nix files
, checkMaterialization ? null # If true the nix files will be generated used to check plan-sha256 and material
, ... }:
let
stackToNixArgs = builtins.concatStringsSep " " [
"--full"
@ -14,7 +18,14 @@ let
(if ignorePackageYaml then "--ignore-package-yaml" else "")
"-o ."
];
stack = runCommand "stack-to-nix-pkgs" {
stack = materialize ({
inherit materialized;
sha256 = stack-sha256;
sha256Arg = "stack-sha256";
reasonNotSafe = null;
} // pkgs.lib.optionalAttrs (checkMaterialization != null) {
inherit checkMaterialization;
}) (runCommand "stack-to-nix-pkgs" {
nativeBuildInputs = [ nix-tools pkgs.nix-prefetch-git pkgs.cacert ];
# Needed or stack-to-nix will die on unicode inputs
LOCALE_ARCHIVE = pkgs.lib.optionalString (pkgs.stdenv.hostPlatform.libc == "glibc") "${pkgs.glibcLocales}/lib/locale/locale-archive";
@ -36,5 +47,5 @@ let
# move pkgs.nix to default.nix ensure we can just nix `import` the result.
mv $out/pkgs.nix $out/default.nix
'');
''));
in { projectNix = stack; inherit src; sourceRepos = []; }

89
lib/materialize.nix Normal file
View File

@ -0,0 +1,89 @@
{ pkgs, nix, runCommand, checkMaterialization }@defaults:
{ sha256
, sha256Arg
, materialized
, reasonNotSafe
, checkMaterialization ? defaults.checkMaterialization
}: derivation:
let
inherit (derivation) name;
traceIgnoringSha256 = reason: x:
if sha256 != null
then builtins.trace ("Warning: ignoring sha256 for " + name + " " + reason) x
else x;
traceIgnoringMaterialized = reason: x:
if materialized != null
then builtins.trace ("Warning: ignoring materialized for " + name + " " + reason) x
else x;
unchecked =
if reasonNotSafe != null
then
# Warn the user if they tried to pin stuff down when it is not safe
traceIgnoringSha256 reasonNotSafe
(traceIgnoringMaterialized reasonNotSafe calculateNoHash)
else if sha256 == null
then
# Let the user know how to calculate a sha256 to use to make this
# a fixed output derivation.
builtins.trace ("Get `${sha256Arg}` with `nix-hash --base32 --type sha256 "
+ toString calculateNoHash + "/`") (traceIgnoringMaterialized
"${sha256Arg} is not set" calculateNoHash)
else if materialized == null
then
# To avoid any IFD dependencies add a `materialized` copy somewhere
# and pass it in.
builtins.trace ("To materialize, point `materialized` to a copy of " + toString calculateUseHash)
calculateUseHash
else
# Everything is in place we can safely use the sha256 and materialized
calculateUseAll;
# Build fully and check the hash and materialized versions
checked = runCommand name {
buildInputs = [ nix ];
} (
(pkgs.lib.optionalString (sha256 != null) ''
NEW_HASH=$(nix-hash --base32 --type sha256 ${calculateNoHash})
if [ "${sha256}" != "$NEW_HASH" ]; then
echo Changes to ${name} not reflected in ${sha256Arg}
diff -ru ${calculateUseHash} ${calculateNoHash} || true
echo Calculated hash is $NEW_HASH expected hash was ${sha256} for ${name}
false
else
echo ${sha256Arg} used for ${name} is correct
fi
'')
+ (pkgs.lib.optionalString (materialized != null) ''
if diff -qr ${materialized} ${calculateNoHash} &>/dev/null; then
echo materialized nix used for ${name} is correct
else
echo Changes to plan not reflected in materialized nix for ${name}
diff -ru ${materialized} ${calculateNoHash}
fi
'')
+ ''
cp -r ${unchecked} $out
''
);
hashArgs = {
outputHashMode = "recursive";
outputHashAlgo = "sha256";
outputHash = sha256;
};
calculateNoHash = derivation;
calculateUseHash = derivation.overrideAttrs (_: hashArgs);
calculateUseAll =
# Skip right to expectedPath if it already exists
if materialized != null && builtins.pathExists materialized
then runCommand name hashArgs "cp -r ${materialized} $out"
else calculateUseHash;
in
if checkMaterialization
then checked
else unchecked

View File

@ -0,0 +1,191 @@
let
buildDepError = pkg:
builtins.throw ''
The Haskell package set does not contain the package: ${pkg} (build dependency).
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
sysDepError = pkg:
builtins.throw ''
The Nixpkgs package set does not contain the package: ${pkg} (system dependency).
You may need to augment the system package mapping in haskell.nix so that it can be found.
'';
pkgConfDepError = pkg:
builtins.throw ''
The pkg-conf packages does not contain the package: ${pkg} (pkg-conf dependency).
You may need to augment the pkg-conf package mapping in haskell.nix so that it can be found.
'';
exeDepError = pkg:
builtins.throw ''
The local executable components do not include the component: ${pkg} (executable dependency).
'';
legacyExeDepError = pkg:
builtins.throw ''
The Haskell package set does not contain the package: ${pkg} (executable dependency).
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
buildToolDepError = pkg:
builtins.throw ''
Neither the Haskell package set or the Nixpkgs package set contain the package: ${pkg} (build tool dependency).
If this is a system dependency:
You may need to augment the system package mapping in haskell.nix so that it can be found.
If this is a Haskell dependency:
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
in { system, compiler, flags, pkgs, hsPkgs, pkgconfPkgs, ... }:
{
flags = { small_base = true; };
package = {
specVersion = "1.8";
identifier = { name = "alex"; version = "3.2.4"; };
license = "BSD-3-Clause";
copyright = "(c) Chis Dornan, Simon Marlow";
maintainer = "Simon Marlow <marlowsd@gmail.com>";
author = "Chris Dornan and Simon Marlow";
homepage = "http://www.haskell.org/alex/";
url = "";
synopsis = "Alex is a tool for generating lexical analysers in Haskell";
description = "Alex is a tool for generating lexical analysers in Haskell.\nIt takes a description of tokens based on regular\nexpressions and generates a Haskell module containing code\nfor scanning text efficiently. It is similar to the tool\nlex or flex for C/C++.";
buildType = "Simple";
isLocal = true;
detailLevel = "FullDetails";
licenseFiles = [ "LICENSE" ];
dataDir = "data/";
dataFiles = [
"AlexTemplate"
"AlexTemplate-ghc"
"AlexTemplate-ghc-nopred"
"AlexTemplate-ghc-debug"
"AlexTemplate-debug"
"AlexWrapper-basic"
"AlexWrapper-basic-bytestring"
"AlexWrapper-strict-bytestring"
"AlexWrapper-posn"
"AlexWrapper-posn-bytestring"
"AlexWrapper-monad"
"AlexWrapper-monad-bytestring"
"AlexWrapper-monadUserState"
"AlexWrapper-monadUserState-bytestring"
"AlexWrapper-gscan"
];
extraSrcFiles = [
"CHANGELOG.md"
"README.md"
"TODO"
"alex.spec"
"doc/Makefile"
"doc/aclocal.m4"
"doc/alex.1.in"
"doc/alex.xml"
"doc/config.mk.in"
"doc/configure.ac"
"doc/docbook-xml.mk"
"doc/fptools.css"
"examples/Makefile"
"examples/Tokens.x"
"examples/Tokens_gscan.x"
"examples/Tokens_posn.x"
"examples/examples.x"
"examples/haskell.x"
"examples/lit.x"
"examples/pp.x"
"examples/state.x"
"examples/tiny.y"
"examples/words.x"
"examples/words_monad.x"
"examples/words_posn.x"
"src/Parser.y.boot"
"src/Scan.x.boot"
"src/ghc_hooks.c"
"templates/GenericTemplate.hs"
"templates/wrappers.hs"
"tests/Makefile"
"tests/simple.x"
"tests/null.x"
"tests/tokens.x"
"tests/tokens_gscan.x"
"tests/tokens_posn.x"
"tests/tokens_bytestring.x"
"tests/tokens_posn_bytestring.x"
"tests/tokens_scan_user.x"
"tests/tokens_strict_bytestring.x"
"tests/tokens_monad_bytestring.x"
"tests/tokens_monadUserState_bytestring.x"
"tests/tokens_bytestring_unicode.x"
"tests/basic_typeclass.x"
"tests/basic_typeclass_bytestring.x"
"tests/default_typeclass.x"
"tests/gscan_typeclass.x"
"tests/posn_typeclass.x"
"tests/monad_typeclass.x"
"tests/monad_typeclass_bytestring.x"
"tests/monadUserState_typeclass.x"
"tests/monadUserState_typeclass_bytestring.x"
"tests/posn_typeclass_bytestring.x"
"tests/strict_typeclass.x"
"tests/unicode.x"
];
extraTmpFiles = [];
extraDocFiles = [];
};
components = {
exes = {
"alex" = {
depends = [
(hsPkgs."base" or (buildDepError "base"))
] ++ (if flags.small_base
then [
(hsPkgs."base" or (buildDepError "base"))
(hsPkgs."array" or (buildDepError "array"))
(hsPkgs."containers" or (buildDepError "containers"))
(hsPkgs."directory" or (buildDepError "directory"))
]
else [ (hsPkgs."base" or (buildDepError "base")) ]);
buildable = true;
modules = [
"AbsSyn"
"CharSet"
"DFA"
"DFAMin"
"DFS"
"Info"
"Map"
"NFA"
"Output"
"Paths_alex"
"Parser"
"ParseMonad"
"Scan"
"Set"
"Sort"
"Util"
"UTF8"
"Data/Ranged"
"Data/Ranged/Boundaries"
"Data/Ranged/RangedSet"
"Data/Ranged/Ranges"
];
hsSourceDirs = [ "src" ];
mainPath = [ "Main.hs" ] ++ [ "" ];
};
};
tests = {
"tests" = {
depends = [
(hsPkgs."base" or (buildDepError "base"))
(hsPkgs."process" or (buildDepError "process"))
];
build-tools = [
(hsPkgs.buildPackages.alex or (pkgs.buildPackages.alex or (buildToolDepError "alex")))
];
buildable = true;
mainPath = [ "test.hs" ];
};
};
};
} // rec { src = (pkgs.lib).mkDefault ../.; }

View File

@ -0,0 +1,49 @@
{
pkgs = hackage:
{
packages = {
"ghc-prim".revision = (((hackage."ghc-prim")."0.5.2.0").revisions).default;
"unix".revision = (((hackage."unix")."2.7.2.2").revisions).default;
"rts".revision = (((hackage."rts")."1.0").revisions).default;
"deepseq".revision = (((hackage."deepseq")."1.4.3.0").revisions).default;
"directory".revision = (((hackage."directory")."1.3.1.5").revisions).default;
"containers".revision = (((hackage."containers")."0.5.11.0").revisions).default;
"bytestring".revision = (((hackage."bytestring")."0.10.8.2").revisions).default;
"base".revision = (((hackage."base")."4.11.1.0").revisions).default;
"time".revision = (((hackage."time")."1.8.0.2").revisions).default;
"filepath".revision = (((hackage."filepath")."1.4.2").revisions).default;
"process".revision = (((hackage."process")."1.6.3.0").revisions).default;
"array".revision = (((hackage."array")."0.5.2.0").revisions).default;
"integer-gmp".revision = (((hackage."integer-gmp")."1.0.2.0").revisions).default;
};
compiler = {
version = "8.4.4";
nix-name = "ghc844";
packages = {
"ghc-prim" = "0.5.2.0";
"unix" = "2.7.2.2";
"rts" = "1.0";
"deepseq" = "1.4.3.0";
"directory" = "1.3.1.5";
"containers" = "0.5.11.0";
"bytestring" = "0.10.8.2";
"base" = "4.11.1.0";
"time" = "1.8.0.2";
"filepath" = "1.4.2";
"process" = "1.6.3.0";
"array" = "0.5.2.0";
"integer-gmp" = "1.0.2.0";
};
};
};
extras = hackage:
{ packages = { alex = ./.plan.nix/alex.nix; }; };
modules = [
({ lib, ... }:
{
packages = {
"alex" = { flags = { "small_base" = lib.mkOverride 900 true; }; };
};
})
];
}

View File

@ -0,0 +1,223 @@
let
buildDepError = pkg:
builtins.throw ''
The Haskell package set does not contain the package: ${pkg} (build dependency).
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
sysDepError = pkg:
builtins.throw ''
The Nixpkgs package set does not contain the package: ${pkg} (system dependency).
You may need to augment the system package mapping in haskell.nix so that it can be found.
'';
pkgConfDepError = pkg:
builtins.throw ''
The pkg-conf packages does not contain the package: ${pkg} (pkg-conf dependency).
You may need to augment the pkg-conf package mapping in haskell.nix so that it can be found.
'';
exeDepError = pkg:
builtins.throw ''
The local executable components do not include the component: ${pkg} (executable dependency).
'';
legacyExeDepError = pkg:
builtins.throw ''
The Haskell package set does not contain the package: ${pkg} (executable dependency).
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
buildToolDepError = pkg:
builtins.throw ''
Neither the Haskell package set or the Nixpkgs package set contain the package: ${pkg} (build tool dependency).
If this is a system dependency:
You may need to augment the system package mapping in haskell.nix so that it can be found.
If this is a Haskell dependency:
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
in { system, compiler, flags, pkgs, hsPkgs, pkgconfPkgs, ... }:
{
flags = { small_base = true; };
package = {
specVersion = "1.8";
identifier = { name = "happy"; version = "1.19.11"; };
license = "BSD-2-Clause";
copyright = "(c) Andy Gill, Simon Marlow";
maintainer = "Simon Marlow <marlowsd@gmail.com>";
author = "Andy Gill and Simon Marlow";
homepage = "https://www.haskell.org/happy/";
url = "";
synopsis = "Happy is a parser generator for Haskell";
description = "Happy is a parser generator for Haskell. Given a grammar\nspecification in BNF, Happy generates Haskell code to parse the\ngrammar. Happy works in a similar way to the @yacc@ tool for C.";
buildType = "Custom";
isLocal = true;
setup-depends = [
(hsPkgs.buildPackages.Cabal or (pkgs.buildPackages.Cabal or (buildToolDepError "Cabal")))
(hsPkgs.buildPackages.base or (pkgs.buildPackages.base or (buildToolDepError "base")))
(hsPkgs.buildPackages.directory or (pkgs.buildPackages.directory or (buildToolDepError "directory")))
(hsPkgs.buildPackages.filepath or (pkgs.buildPackages.filepath or (buildToolDepError "filepath")))
];
detailLevel = "FullDetails";
licenseFiles = [ "LICENSE" ];
dataDir = "";
dataFiles = [];
extraSrcFiles = [
"ANNOUNCE"
"CHANGES"
"Makefile"
"README.md"
"TODO"
"doc/Makefile"
"doc/aclocal.m4"
"doc/config.mk.in"
"doc/configure.ac"
"doc/docbook-xml.mk"
"doc/fptools.css"
"doc/happy.1.in"
"doc/happy.xml"
"examples/glr/nlp/Main.lhs"
"examples/glr/nlp/Makefile"
"examples/glr/nlp/README"
"examples/glr/nlp/English.y"
"examples/glr/nlp/Hugs.lhs"
"examples/glr/Makefile"
"examples/glr/Makefile.defs"
"examples/glr/expr-eval/Main.lhs"
"examples/glr/expr-eval/Makefile"
"examples/glr/expr-eval/Expr.y"
"examples/glr/expr-eval/README"
"examples/glr/expr-eval/Hugs.lhs"
"examples/glr/expr-tree/Main.lhs"
"examples/glr/expr-tree/Makefile"
"examples/glr/expr-tree/Expr.y"
"examples/glr/expr-tree/README"
"examples/glr/expr-tree/Tree.lhs"
"examples/glr/expr-tree/Hugs.lhs"
"examples/glr/highly-ambiguous/Main.lhs"
"examples/glr/highly-ambiguous/Makefile"
"examples/glr/highly-ambiguous/Expr.y"
"examples/glr/highly-ambiguous/README"
"examples/glr/highly-ambiguous/Hugs.lhs"
"examples/glr/hidden-leftrec/Main.lhs"
"examples/glr/hidden-leftrec/Makefile"
"examples/glr/hidden-leftrec/Expr.y"
"examples/glr/hidden-leftrec/README"
"examples/glr/hidden-leftrec/Hugs.lhs"
"examples/glr/expr-monad/Main.lhs"
"examples/glr/expr-monad/Makefile"
"examples/glr/expr-monad/Expr.y"
"examples/glr/expr-monad/README"
"examples/glr/expr-monad/Hugs.lhs"
"examples/glr/bio-eg/Main.lhs"
"examples/glr/bio-eg/Makefile"
"examples/glr/bio-eg/Bio.y"
"examples/glr/bio-eg/README"
"examples/glr/bio-eg/1-1200.dna"
"examples/glr/bio-eg/1-600.dna"
"examples/glr/common/DV_lhs"
"examples/glr/common/DaVinciTypes.hs"
"examples/glr/packing/Main.lhs"
"examples/glr/packing/Makefile"
"examples/glr/packing/Expr.y"
"examples/glr/packing/README"
"examples/glr/packing/Hugs.lhs"
"examples/PgnParser.ly"
"examples/MonadTest.ly"
"examples/igloo/ParserM.hs"
"examples/igloo/Makefile"
"examples/igloo/Parser.y"
"examples/igloo/Foo.hs"
"examples/igloo/README"
"examples/igloo/Lexer.x"
"examples/README"
"examples/Calc.ly"
"examples/DavesExample.ly"
"examples/ErrorTest.ly"
"examples/ErlParser.ly"
"examples/SimonsExample.ly"
"examples/LexerTest.ly"
"happy.spec"
"src/ARRAY-NOTES"
"templates/GLR_Base.hs"
"templates/GenericTemplate.hs"
"templates/GLR_Lib.hs"
"tests/AttrGrammar001.y"
"tests/AttrGrammar002.y"
"tests/Makefile"
"tests/Partial.ly"
"tests/Test.ly"
"tests/TestMulti.ly"
"tests/TestPrecedence.ly"
"tests/bogus-token.y"
"tests/bug001.ly"
"tests/bug002.y"
"tests/error001.stderr"
"tests/error001.stdout"
"tests/error001.y"
"tests/monad001.y"
"tests/monad002.ly"
"tests/monaderror.y"
"tests/precedence001.ly"
"tests/precedence002.y"
"tests/test_rules.y"
"tests/issue91.y"
"tests/issue93.y"
"tests/issue94.y"
"tests/issue95.y"
"tests/monaderror-explist.y"
"tests/typeclass_monad001.y"
"tests/typeclass_monad002.ly"
"tests/typeclass_monad_lexer.y"
"tests/rank2.y"
];
extraTmpFiles = [];
extraDocFiles = [];
};
components = {
exes = {
"happy" = {
depends = [
(hsPkgs."base" or (buildDepError "base"))
(hsPkgs."array" or (buildDepError "array"))
(hsPkgs."containers" or (buildDepError "containers"))
(hsPkgs."mtl" or (buildDepError "mtl"))
];
buildable = true;
modules = [
"Paths_happy"
"AbsSyn"
"First"
"GenUtils"
"Grammar"
"Info"
"LALR"
"Lexer"
"ParseMonad"
"Parser"
"ProduceCode"
"ProduceGLRCode"
"NameSet"
"Target"
"AttrGrammar"
"AttrGrammarParser"
"ParamRules"
"PrettyGrammar"
];
hsSourceDirs = [ "src" ];
mainPath = [ "Main.lhs" ];
};
};
tests = {
"tests" = {
depends = [
(hsPkgs."base" or (buildDepError "base"))
(hsPkgs."process" or (buildDepError "process"))
];
buildable = true;
mainPath = [ "test.hs" ];
};
};
};
} // rec { src = (pkgs.lib).mkDefault ../.; }

View File

@ -0,0 +1,63 @@
{
pkgs = hackage:
{
packages = {
"binary".revision = (((hackage."binary")."0.8.5.1").revisions).default;
"ghc-prim".revision = (((hackage."ghc-prim")."0.5.2.0").revisions).default;
"unix".revision = (((hackage."unix")."2.7.2.2").revisions).default;
"mtl".revision = (((hackage."mtl")."2.2.2").revisions).default;
"rts".revision = (((hackage."rts")."1.0").revisions).default;
"deepseq".revision = (((hackage."deepseq")."1.4.3.0").revisions).default;
"parsec".revision = (((hackage."parsec")."3.1.13.0").revisions).default;
"directory".revision = (((hackage."directory")."1.3.1.5").revisions).default;
"containers".revision = (((hackage."containers")."0.5.11.0").revisions).default;
"bytestring".revision = (((hackage."bytestring")."0.10.8.2").revisions).default;
"text".revision = (((hackage."text")."1.2.3.1").revisions).default;
"Cabal".revision = (((hackage."Cabal")."2.2.0.1").revisions).default;
"base".revision = (((hackage."base")."4.11.1.0").revisions).default;
"time".revision = (((hackage."time")."1.8.0.2").revisions).default;
"transformers".revision = (((hackage."transformers")."0.5.5.0").revisions).default;
"filepath".revision = (((hackage."filepath")."1.4.2").revisions).default;
"process".revision = (((hackage."process")."1.6.3.0").revisions).default;
"pretty".revision = (((hackage."pretty")."1.1.3.6").revisions).default;
"array".revision = (((hackage."array")."0.5.2.0").revisions).default;
"integer-gmp".revision = (((hackage."integer-gmp")."1.0.2.0").revisions).default;
};
compiler = {
version = "8.4.4";
nix-name = "ghc844";
packages = {
"binary" = "0.8.5.1";
"ghc-prim" = "0.5.2.0";
"unix" = "2.7.2.2";
"mtl" = "2.2.2";
"rts" = "1.0";
"deepseq" = "1.4.3.0";
"parsec" = "3.1.13.0";
"directory" = "1.3.1.5";
"containers" = "0.5.11.0";
"bytestring" = "0.10.8.2";
"text" = "1.2.3.1";
"Cabal" = "2.2.0.1";
"base" = "4.11.1.0";
"time" = "1.8.0.2";
"transformers" = "0.5.5.0";
"filepath" = "1.4.2";
"process" = "1.6.3.0";
"pretty" = "1.1.3.6";
"array" = "0.5.2.0";
"integer-gmp" = "1.0.2.0";
};
};
};
extras = hackage:
{ packages = { happy = ./.plan.nix/happy.nix; }; };
modules = [
({ lib, ... }:
{
packages = {
"happy" = { flags = { "small_base" = lib.mkOverride 900 true; }; };
};
})
];
}

View File

@ -0,0 +1,101 @@
let
buildDepError = pkg:
builtins.throw ''
The Haskell package set does not contain the package: ${pkg} (build dependency).
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
sysDepError = pkg:
builtins.throw ''
The Nixpkgs package set does not contain the package: ${pkg} (system dependency).
You may need to augment the system package mapping in haskell.nix so that it can be found.
'';
pkgConfDepError = pkg:
builtins.throw ''
The pkg-conf packages does not contain the package: ${pkg} (pkg-conf dependency).
You may need to augment the pkg-conf package mapping in haskell.nix so that it can be found.
'';
exeDepError = pkg:
builtins.throw ''
The local executable components do not include the component: ${pkg} (executable dependency).
'';
legacyExeDepError = pkg:
builtins.throw ''
The Haskell package set does not contain the package: ${pkg} (executable dependency).
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
buildToolDepError = pkg:
builtins.throw ''
Neither the Haskell package set or the Nixpkgs package set contain the package: ${pkg} (build tool dependency).
If this is a system dependency:
You may need to augment the system package mapping in haskell.nix so that it can be found.
If this is a Haskell dependency:
If you are using Stackage, make sure that you are using a snapshot that contains the package. Otherwise you may need to update the Hackage snapshot you are using, usually by updating haskell.nix.
'';
in { system, compiler, flags, pkgs, hsPkgs, pkgconfPkgs, ... }:
{
flags = {};
package = {
specVersion = "1.6";
identifier = { name = "hscolour"; version = "1.24.4"; };
license = "LicenseRef-LGPL";
copyright = "2003-2017 Malcolm Wallace; 2006 Bjorn Bringert";
maintainer = "Malcolm Wallace";
author = "Malcolm Wallace";
homepage = "http://code.haskell.org/~malcolm/hscolour/";
url = "";
synopsis = "Colourise Haskell code.";
description = "hscolour is a small Haskell script to colourise Haskell code. It currently\nhas six output formats:\nANSI terminal codes (optionally XTerm-256colour codes),\nHTML 3.2 with <font> tags,\nHTML 4.01 with CSS,\nHTML 4.01 with CSS and mouseover annotations,\nXHTML 1.0 with inline CSS styling,\nLaTeX,\nand mIRC chat codes.";
buildType = "Simple";
isLocal = true;
detailLevel = "FullDetails";
licenseFiles = [ "LICENCE-LGPL" ];
dataDir = "";
dataFiles = [ "hscolour.css" "data/rgb24-example-.hscolour" ];
extraSrcFiles = [];
extraTmpFiles = [];
extraDocFiles = [];
};
components = {
"library" = {
depends = [
(hsPkgs."base" or (buildDepError "base"))
(hsPkgs."containers" or (buildDepError "containers"))
];
buildable = true;
modules = [
"Language/Haskell/HsColour"
"Language/Haskell/HsColour/ANSI"
"Language/Haskell/HsColour/Anchors"
"Language/Haskell/HsColour/ACSS"
"Language/Haskell/HsColour/CSS"
"Language/Haskell/HsColour/Classify"
"Language/Haskell/HsColour/ColourHighlight"
"Language/Haskell/HsColour/Colourise"
"Language/Haskell/HsColour/General"
"Language/Haskell/HsColour/HTML"
"Language/Haskell/HsColour/InlineCSS"
"Language/Haskell/HsColour/LaTeX"
"Language/Haskell/HsColour/MIRC"
"Language/Haskell/HsColour/Options"
"Language/Haskell/HsColour/Output"
"Language/Haskell/HsColour/TTY"
];
};
exes = {
"HsColour" = {
depends = [
(hsPkgs."base" or (buildDepError "base"))
(hsPkgs."containers" or (buildDepError "containers"))
];
buildable = true;
mainPath = [ "HsColour.hs" ];
};
};
};
} // rec { src = (pkgs.lib).mkDefault ../.; }

View File

@ -0,0 +1,33 @@
{
pkgs = hackage:
{
packages = {
"ghc-prim".revision = (((hackage."ghc-prim")."0.5.2.0").revisions).default;
"rts".revision = (((hackage."rts")."1.0").revisions).default;
"deepseq".revision = (((hackage."deepseq")."1.4.3.0").revisions).default;
"containers".revision = (((hackage."containers")."0.5.11.0").revisions).default;
"base".revision = (((hackage."base")."4.11.1.0").revisions).default;
"array".revision = (((hackage."array")."0.5.2.0").revisions).default;
"integer-gmp".revision = (((hackage."integer-gmp")."1.0.2.0").revisions).default;
};
compiler = {
version = "8.4.4";
nix-name = "ghc844";
packages = {
"ghc-prim" = "0.5.2.0";
"rts" = "1.0";
"deepseq" = "1.4.3.0";
"containers" = "0.5.11.0";
"base" = "4.11.1.0";
"array" = "0.5.2.0";
"integer-gmp" = "1.0.2.0";
};
};
};
extras = hackage:
{ packages = { hscolour = ./.plan.nix/hscolour.nix; }; };
modules = [
({ lib, ... }:
{ packages = { "hscolour" = { flags = {}; }; }; })
];
}

View File

@ -265,7 +265,8 @@ self: super: {
inherit (bootstrap.packages) cabal-install nix-tools hpack;
name = "alex"; version = "3.2.4";
index-state = "2019-10-20T00:00:00Z";
plan-sha256 = "086kd6aa5bir3y4aqb1wl5zkj6agz5q4wp4snvdnf6cidz5wra06";
plan-sha256 = "1adn8s46msqm2rl6yf01z2r81maa2001qh441j491gpmc3ki36n0";
materialized = ../materialized/alex;
};
alex = bootstrap.packages.alex-project.hsPkgs.alex.components.exes.alex;
happy-project = hackage-project {
@ -274,7 +275,8 @@ self: super: {
inherit (bootstrap.packages) cabal-install nix-tools hpack;
name = "happy"; version = "1.19.11";
index-state = "2019-10-20T00:00:00Z";
plan-sha256 = "011bxlxdv239psgi80j00km2wcgb68j16sz3fng67d03sqf5i37w";
plan-sha256 = "0swpwhlym4p3209qv90mfgq6zsaw99ipznm4pvd32mxzwq9s5q8i";
materialized = ../materialized/happy;
};
happy = bootstrap.packages.happy-project.hsPkgs.happy.components.exes.happy;
hscolour-project = hackage-project {
@ -283,7 +285,8 @@ self: super: {
inherit (bootstrap.packages) cabal-install nix-tools hpack;
name = "hscolour"; version = "1.24.4";
index-state = "2019-10-20T00:00:00Z";
plan-sha256 = "021rwcmkshibc3mfr833ay5hfr19kq6k32lxyrrb6vp9vmihgw4b";
plan-sha256 = "0cnkczsh1xy7cc60q3blwa51qrjhf5mc89s34y9ab3x702a26b75";
materialized = ../materialized/hscolour;
};
hscolour = bootstrap.packages.hscolour-project.hsPkgs.hscolour.components.exes.HsColour;
};

View File

@ -193,6 +193,16 @@ self: super: {
HOME=$out cabal new-update cached
'';
checkMaterialization = false; # This is the default. Use an overlay to set it to true and test all the materialized files
# Helps materialize the output of derivations
materialize = import ../lib/materialize.nix {
inherit (self) nix;
inherit (self.haskell-nix) checkMaterialization;
pkgs = self.buildPackages.pkgs;
inherit (self.buildPackages.pkgs) runCommand;
};
update-index-state-hashes = import ../scripts/update-index-state-hashes.nix {
inherit (self.haskell-nix) indexStateHashesPath nix-tools;
inherit (self) coreutils nix writeShellScriptBin stdenv;
@ -202,7 +212,7 @@ self: super: {
callStackToNix = import ../lib/call-stack-to-nix.nix {
pkgs = self.buildPackages.pkgs;
inherit (self.buildPackages.pkgs) runCommand;
inherit (self.buildPackages.haskell-nix) nix-tools mkCacheFile;
inherit (self.buildPackages.haskell-nix) nix-tools mkCacheFile materialize;
};
# given a source location call `cabal-to-nix` (from nix-tools) on it
@ -330,7 +340,7 @@ self: super: {
# Resulting nix files are added to nix-plan subdirectory.
callCabalProjectToNix = import ../lib/call-cabal-project-to-nix.nix {
index-state-hashes = import indexStateHashesPath;
inherit (self.buildPackages.haskell-nix) dotCabal nix-tools haskellLib;
inherit (self.buildPackages.haskell-nix) dotCabal nix-tools haskellLib materialize;
pkgs = self.buildPackages.pkgs;
inherit (self.buildPackages.haskell-nix.haskellPackages.hpack.components.exes) hpack;
inherit (self.buildPackages.haskell-nix) cabal-install ghc;