haskell.nix/package-set.nix

107 lines
4.2 KiB
Nix
Raw Normal View History

let f = { hackage, pkgs, pkg-def, pkg-def-extras ? [], modules ? [] }: let
buildModules = f { inherit hackage pkg-def pkg-def-extras modules; pkgs = pkgs.buildPackages; };
2019-01-25 12:58:53 +03:00
in pkgs.lib.evalModules {
modules = modules ++ [
Overlays (#261) The Overlays branch This is a major reorganization in how haskell.nix is used, and marks our 1.0 release. The branch doesn't build due to numerous issues that we believe to be with the CI and not this branch. We expect only very minor adjustments prior to calling this the official 1.0 release. * Move iohk-nix patches into haskell.nix This moves the customizations we did in iohk-nix into haskell.nix via overlays and config. Add bootPkgs logic this moves the nuking of bootPkgs where it belongs. This should eventually still be removed and replaced by a proper solution, that doesn't require the nuking of bootPkgs. Allow us to bootstrap binary ghcs and a cabal-install With this we can do the following: ``` $ nix repl compiler/old-ghc-nix nix-repl> :b (let pkgs = import <nixpkgs> {}; in with import ./. {}; import ./compiler/bootstrap/cabal-install.nix { inherit (pkgs) fetchurl stdenv zlib; inherit hackage ; ghc = ghc844; src = pkgs.fetchurl { url = "https://github.com/haskell/cabal/archive/Cabal-v3.0.0.0-rc3.tar.gz"; sha256 = "1zl2mgg8307ykq3v8nmafc6zdhhj1cw7w8ffpap16dsm6 5lbnx33"; }; }) ``` which wile it may look daunting, will allow us to bootstrap a cabal-install with a ghc. From that point onwards, we should be able to build any hackage package via haskell.nix. Pass through cabal-install version Better threading of arguments. Add bootstrap overlay Allow alex + happy to be built This still has a wart: we need nix-tools, and for that we use the ghc865 from nixpkgs. Ideally we'd build nix-tools against a specific ghc, but then we'd need a build expression for that. Make ghcjs work Building something like this: ``` nix build '(with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; })).components.exes.hello' ``` will now work. Assuming `nixpkgs` has been appropriately patched to support the `js-unknown-ghcjs` triple. Also: this will need an additional `Cabal` patch, to make `Cabal` understand what it needs to do with: `dist/build/hello/hello: copyFile: does not exist (No such file or directory)` It needs to learn that `dist/build/hello/hello.jsexe` is what it wants to copy and that that is a directory. Luckily we do have some code in Cabal already that does this for `.exe` on windows. Build `js-unknown-ghcjs` packages with haskell.nix Using the following expression: ``` with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; let Cabal = buildPackages.haskell-nix.hackage-package { name = "Cabal"; version = "2.4.1.0"; modules = [ { packages.Cabal.patches = [ ./Cabal-install-folder.diff ]; } ]; }; in (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; modules = [ ({config, ... }:{ packages.hello.package.setup-depends = [ Cabal ]; }) ];}).components.exes.hello ``` in a `test.nix` file. And running ``` nix build -f ./test.nix ``` on it, will produce ``` ./result ├── bin │ └── hello.jsexe │ ├── all.js │ ├── all.js.externs │ ├── index.html │ ├── lib.js │ ├── manifest.webapp │ ├── out.frefs.js │ ├── out.frefs.json │ ├── out.js │ ├── out.stats │ ├── rts.js │ └── runmain.js └── share └── doc └── x86_64-linux-ghc-8.6.5 └── hello-1.0.0.2 └── LICENSE 6 directories, 12 files ```
2019-10-21 15:07:58 +03:00
({ config, lib, ... }: {
2019-01-25 12:58:53 +03:00
# Provide all modules with haskellLib, pkgs, and pkgconfPkgs arguments
_module.args = {
# this is *not* the hasekllLib from nixpkgs; it is rather our own
# library from haskell.nix
Update ghc 8.4.4 based tools to ghc 8.6.5 (#618) Although the default ghc used by haskell.nix is ghc 8.6.5 many of the tools used in haskell.nix are still built with the boot compiler ghc 8.4.4. These include * haskell-nix.cabal-install * haskell-nix.alex * haskell-nix.happy This change updates those to ghc 8.6.5 and includes materializations for the new versions. When cabal-install is built it is careful to disable materialization checks on the version of itself used during the build to avoid infinite recursion. There was a version of nix-tools built with the boot ghc which was only used when `checkMaterialization = true`. It was used for the boot versions of alex, happy and hscolour. These have been update to use the default (ghc 8.6.5) version of nix-tools and checkMaterialization is forced off when they are being used to build ghc. This means the materialization will only be checked for these when they are built independently (they are included in the test set via haskellNixRoots). Three new arguments are added to `default.nix`: * `defaultCompilerNixName` if not specified "ghc865" is used * `checkMaterialization` makes it easier to switch on materialization checks * `system` defaults to `builtins.currentSystem` This change also moves the work needed for hydra eval to the eval system using a new `evalPackages` feature. This includes: * Fetching from git with `fetchgit` * Building scripts with `runCommand` and `writeTextFile` * `git ls-files` in `cleanGit` * running `cabal v2-configure` * copying materialized files (we are not sure why this is necessary but if we do not `cp -r` the files nix will not load them on hydra) Reduce size of `make-config-files.nix` strings by around 80%. These are unlikely to be the cause of hydra eval time memory issues in the GB range, but were still quite large (around 10MB for the `cabal-simple` test case). There was issue causing excessive builds of the `git` package when cross compiling. Gory details are a comment in `lib/defaults.nix` but in short if you use `git` you need an extra `.buildPackages` one is not enough because it depends on `gdb` and that will be different in `buildPackages` compared to `buildPackages.buildPackages`. Adds missing materialization files for ghc 8.4.4 (only needed when `checkMaterialization` is on because of other materialiazations, but good to have).
2020-05-21 02:31:26 +03:00
haskellLib = let hl = import ./lib { inherit pkgs lib; inherit (pkgs) stdenv recurseIntoAttrs srcOnly; haskellLib = hl; }; in hl;
2019-01-25 12:58:53 +03:00
# The package descriptions depend on pkgs, which are used to resolve system package dependencies
# as well as pkgconfPkgs, which are used to resolve pkgconfig name to nixpkgs names. We simply
# augment the existing pkgs set with the specific mappings:
pkgs = import ./lib/system-pkgs.nix pkgs;
pkgconfPkgs = import ./lib/pkgconf-nixpkgs-map.nix pkgs;
2019-01-25 12:58:53 +03:00
inherit buildModules;
};
2019-01-25 12:58:53 +03:00
# Set the hackage DB for modules/hackage.nix
hackage.db = hackage;
2019-01-25 12:58:53 +03:00
# Set the plan for modules/plan.nix
plan.pkg-def = hackage: with builtins;
Overlays (#261) The Overlays branch This is a major reorganization in how haskell.nix is used, and marks our 1.0 release. The branch doesn't build due to numerous issues that we believe to be with the CI and not this branch. We expect only very minor adjustments prior to calling this the official 1.0 release. * Move iohk-nix patches into haskell.nix This moves the customizations we did in iohk-nix into haskell.nix via overlays and config. Add bootPkgs logic this moves the nuking of bootPkgs where it belongs. This should eventually still be removed and replaced by a proper solution, that doesn't require the nuking of bootPkgs. Allow us to bootstrap binary ghcs and a cabal-install With this we can do the following: ``` $ nix repl compiler/old-ghc-nix nix-repl> :b (let pkgs = import <nixpkgs> {}; in with import ./. {}; import ./compiler/bootstrap/cabal-install.nix { inherit (pkgs) fetchurl stdenv zlib; inherit hackage ; ghc = ghc844; src = pkgs.fetchurl { url = "https://github.com/haskell/cabal/archive/Cabal-v3.0.0.0-rc3.tar.gz"; sha256 = "1zl2mgg8307ykq3v8nmafc6zdhhj1cw7w8ffpap16dsm6 5lbnx33"; }; }) ``` which wile it may look daunting, will allow us to bootstrap a cabal-install with a ghc. From that point onwards, we should be able to build any hackage package via haskell.nix. Pass through cabal-install version Better threading of arguments. Add bootstrap overlay Allow alex + happy to be built This still has a wart: we need nix-tools, and for that we use the ghc865 from nixpkgs. Ideally we'd build nix-tools against a specific ghc, but then we'd need a build expression for that. Make ghcjs work Building something like this: ``` nix build '(with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; })).components.exes.hello' ``` will now work. Assuming `nixpkgs` has been appropriately patched to support the `js-unknown-ghcjs` triple. Also: this will need an additional `Cabal` patch, to make `Cabal` understand what it needs to do with: `dist/build/hello/hello: copyFile: does not exist (No such file or directory)` It needs to learn that `dist/build/hello/hello.jsexe` is what it wants to copy and that that is a directory. Luckily we do have some code in Cabal already that does this for `.exe` on windows. Build `js-unknown-ghcjs` packages with haskell.nix Using the following expression: ``` with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; let Cabal = buildPackages.haskell-nix.hackage-package { name = "Cabal"; version = "2.4.1.0"; modules = [ { packages.Cabal.patches = [ ./Cabal-install-folder.diff ]; } ]; }; in (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; modules = [ ({config, ... }:{ packages.hello.package.setup-depends = [ Cabal ]; }) ];}).components.exes.hello ``` in a `test.nix` file. And running ``` nix build -f ./test.nix ``` on it, will produce ``` ./result ├── bin │ └── hello.jsexe │ ├── all.js │ ├── all.js.externs │ ├── index.html │ ├── lib.js │ ├── manifest.webapp │ ├── out.frefs.js │ ├── out.frefs.json │ ├── out.js │ ├── out.stats │ ├── rts.js │ └── runmain.js └── share └── doc └── x86_64-linux-ghc-8.6.5 └── hello-1.0.0.2 └── LICENSE 6 directories, 12 files ```
2019-10-21 15:07:58 +03:00
# pkg-def's may reference boot packages, but those
# are not guaranteed to be available on hackage, as
# it is a manual process. They might eventually show
# up much later on hackage; but are not installable
# anyway. Therefore we just strip them out of the
# pkg-def's packages.
#
# Note: these will need to be provided by alternative
# means outside of hackage.
let strip-pkg-def = pkg-def: hackage:
lib.mapAttrs (k: v: if k == "packages"
then lib.filterAttrs (k: _: !(builtins.elem k config.bootPkgs)) v
else v)
(pkg-def hackage);
in let pkg-def' = strip-pkg-def pkg-def;
2019-01-25 12:58:53 +03:00
# The desugar reason.
#
# it is quite cumbersome to write
2019-01-25 12:58:53 +03:00
# (hackage: { packages.x.revision = hackage...;
# packages.y.revision = import ./foo.nix; })
# where we'd rather write:
# (hackage: { x = hackage...; })
# or
# { y = ./foo.nix; }
# As such the desugarer desugars this short hand syntax.
Overlays (#261) The Overlays branch This is a major reorganization in how haskell.nix is used, and marks our 1.0 release. The branch doesn't build due to numerous issues that we believe to be with the CI and not this branch. We expect only very minor adjustments prior to calling this the official 1.0 release. * Move iohk-nix patches into haskell.nix This moves the customizations we did in iohk-nix into haskell.nix via overlays and config. Add bootPkgs logic this moves the nuking of bootPkgs where it belongs. This should eventually still be removed and replaced by a proper solution, that doesn't require the nuking of bootPkgs. Allow us to bootstrap binary ghcs and a cabal-install With this we can do the following: ``` $ nix repl compiler/old-ghc-nix nix-repl> :b (let pkgs = import <nixpkgs> {}; in with import ./. {}; import ./compiler/bootstrap/cabal-install.nix { inherit (pkgs) fetchurl stdenv zlib; inherit hackage ; ghc = ghc844; src = pkgs.fetchurl { url = "https://github.com/haskell/cabal/archive/Cabal-v3.0.0.0-rc3.tar.gz"; sha256 = "1zl2mgg8307ykq3v8nmafc6zdhhj1cw7w8ffpap16dsm6 5lbnx33"; }; }) ``` which wile it may look daunting, will allow us to bootstrap a cabal-install with a ghc. From that point onwards, we should be able to build any hackage package via haskell.nix. Pass through cabal-install version Better threading of arguments. Add bootstrap overlay Allow alex + happy to be built This still has a wart: we need nix-tools, and for that we use the ghc865 from nixpkgs. Ideally we'd build nix-tools against a specific ghc, but then we'd need a build expression for that. Make ghcjs work Building something like this: ``` nix build '(with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; })).components.exes.hello' ``` will now work. Assuming `nixpkgs` has been appropriately patched to support the `js-unknown-ghcjs` triple. Also: this will need an additional `Cabal` patch, to make `Cabal` understand what it needs to do with: `dist/build/hello/hello: copyFile: does not exist (No such file or directory)` It needs to learn that `dist/build/hello/hello.jsexe` is what it wants to copy and that that is a directory. Luckily we do have some code in Cabal already that does this for `.exe` on windows. Build `js-unknown-ghcjs` packages with haskell.nix Using the following expression: ``` with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; let Cabal = buildPackages.haskell-nix.hackage-package { name = "Cabal"; version = "2.4.1.0"; modules = [ { packages.Cabal.patches = [ ./Cabal-install-folder.diff ]; } ]; }; in (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; modules = [ ({config, ... }:{ packages.hello.package.setup-depends = [ Cabal ]; }) ];}).components.exes.hello ``` in a `test.nix` file. And running ``` nix build -f ./test.nix ``` on it, will produce ``` ./result ├── bin │ └── hello.jsexe │ ├── all.js │ ├── all.js.externs │ ├── index.html │ ├── lib.js │ ├── manifest.webapp │ ├── out.frefs.js │ ├── out.frefs.json │ ├── out.js │ ├── out.stats │ ├── rts.js │ └── runmain.js └── share └── doc └── x86_64-linux-ghc-8.6.5 └── hello-1.0.0.2 └── LICENSE 6 directories, 12 files ```
2019-10-21 15:07:58 +03:00
in let desugar = extras:
2019-01-25 12:58:53 +03:00
let
isPath = x: builtins.typeOf x == "path";
# rewrite
# { ... }
# into
# { package = { ... }; }
inject-packages = o: if o ? "packages" then o else { packages = o; };
# rewrite
# x = pkg;
# into
# x.revision = pkg;
inject-revision = pkg: if pkg ? "revision" then pkg else { revision = pkg; };
# rewrite
# x.revision = ./some/path;
# into
# x.revision = import ./some/path;
expand-paths = pkg: if !(isPath pkg.revision) then pkg else { revision = import pkg.revision; };
# apply injection and expansion to the "packages" in extras.
2019-01-25 12:58:53 +03:00
in lib.mapAttrs (k: v: if k != "packages"
then v
else lib.mapAttrs (_: pkg: (expand-paths (inject-revision pkg))) v)
(inject-packages extras);
# fold any potential `pkg-def-extras`
2019-01-25 12:58:53 +03:00
# onto the `pkg-def`.
#
# This means you can have a base definition (e.g. stackage)
# and augment it with custom packages to your liking.
in foldl' lib.recursiveUpdate
Overlays (#261) The Overlays branch This is a major reorganization in how haskell.nix is used, and marks our 1.0 release. The branch doesn't build due to numerous issues that we believe to be with the CI and not this branch. We expect only very minor adjustments prior to calling this the official 1.0 release. * Move iohk-nix patches into haskell.nix This moves the customizations we did in iohk-nix into haskell.nix via overlays and config. Add bootPkgs logic this moves the nuking of bootPkgs where it belongs. This should eventually still be removed and replaced by a proper solution, that doesn't require the nuking of bootPkgs. Allow us to bootstrap binary ghcs and a cabal-install With this we can do the following: ``` $ nix repl compiler/old-ghc-nix nix-repl> :b (let pkgs = import <nixpkgs> {}; in with import ./. {}; import ./compiler/bootstrap/cabal-install.nix { inherit (pkgs) fetchurl stdenv zlib; inherit hackage ; ghc = ghc844; src = pkgs.fetchurl { url = "https://github.com/haskell/cabal/archive/Cabal-v3.0.0.0-rc3.tar.gz"; sha256 = "1zl2mgg8307ykq3v8nmafc6zdhhj1cw7w8ffpap16dsm6 5lbnx33"; }; }) ``` which wile it may look daunting, will allow us to bootstrap a cabal-install with a ghc. From that point onwards, we should be able to build any hackage package via haskell.nix. Pass through cabal-install version Better threading of arguments. Add bootstrap overlay Allow alex + happy to be built This still has a wart: we need nix-tools, and for that we use the ghc865 from nixpkgs. Ideally we'd build nix-tools against a specific ghc, but then we'd need a build expression for that. Make ghcjs work Building something like this: ``` nix build '(with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; })).components.exes.hello' ``` will now work. Assuming `nixpkgs` has been appropriately patched to support the `js-unknown-ghcjs` triple. Also: this will need an additional `Cabal` patch, to make `Cabal` understand what it needs to do with: `dist/build/hello/hello: copyFile: does not exist (No such file or directory)` It needs to learn that `dist/build/hello/hello.jsexe` is what it wants to copy and that that is a directory. Luckily we do have some code in Cabal already that does this for `.exe` on windows. Build `js-unknown-ghcjs` packages with haskell.nix Using the following expression: ``` with import ./. { nixpkgs = ../nixpkgs; nixpkgsArgs = { crossSystem = { config = "js-unknown-ghcjs"; }; }; }; let Cabal = buildPackages.haskell-nix.hackage-package { name = "Cabal"; version = "2.4.1.0"; modules = [ { packages.Cabal.patches = [ ./Cabal-install-folder.diff ]; } ]; }; in (haskell-nix.hackage-package { name = "hello"; version = "1.0.0.2"; modules = [ ({config, ... }:{ packages.hello.package.setup-depends = [ Cabal ]; }) ];}).components.exes.hello ``` in a `test.nix` file. And running ``` nix build -f ./test.nix ``` on it, will produce ``` ./result ├── bin │ └── hello.jsexe │ ├── all.js │ ├── all.js.externs │ ├── index.html │ ├── lib.js │ ├── manifest.webapp │ ├── out.frefs.js │ ├── out.frefs.json │ ├── out.js │ ├── out.stats │ ├── rts.js │ └── runmain.js └── share └── doc └── x86_64-linux-ghc-8.6.5 └── hello-1.0.0.2 └── LICENSE 6 directories, 12 files ```
2019-10-21 15:07:58 +03:00
(pkg-def' hackage)
(map (p: desugar (if builtins.isFunction p then p hackage else p)) pkg-def-extras)
2019-01-25 12:58:53 +03:00
;
2019-01-25 12:58:53 +03:00
})
# Error handlers
./modules/error-handler.nix
2019-01-25 12:58:53 +03:00
# Supplies metadata
./modules/cabal.nix
2019-01-25 12:58:53 +03:00
# Converts config.packages into config.hsPkgs
# Replace this with compat-driver.nix to use nixpkgs haskell build infra
./modules/component-driver.nix
2019-01-25 12:58:53 +03:00
# Converts config.hackage.db to config.hackage.configs
./modules/hackage.nix
2019-01-25 12:58:53 +03:00
# Converts config.hackage.configs and pkg-def to config.packages
./modules/plan.nix
2019-01-25 12:58:53 +03:00
# Configuration that applies to all plans
./modules/configuration-nix.nix
];
};
in f