git.iko.soy/typeable/cabal2nix
on v20160308 help sign in
typeable/cabal2nix/doc/nixcon-2015-slides.md
357 lines 10 kB history raw
1% The Nixpkgs Haskell Infrastructure
2% Peter Simons \<simons@cryp.to\>
3% NixCon 2015
5-------------------------------------------------------------------------------
7# Players Involved in Haskell Packaging
9![](pipeline-overview.pdf)\
12-------------------------------------------------------------------------------
14# The Packaging Process
16![](pipeline-details.pdf)\
19-------------------------------------------------------------------------------
21# Hackage
23- 62,423 Cabal files (+41.6 per day)
25- 9,028 distinct packages (+4.9 per day)
27- $\mu=6.9, \sigma=\pm 10.3$ releases per package:
28 ![](hackage-version-boxplot.pdf)
30- Top 10 most active packages: egison (149), git-annex (145), hakyll (145),
31 purescript (141), yesod-core (140), warp (136), lens (129), wai-extra (119),
32 yesod (116), and hlint (114).
34-------------------------------------------------------------------------------
36# Hackage: Structure of the Git Repository
38 3d-graphics-examples
39 3dmodels
40 4Blocks
41 [...]
42 mtl
43 +-- 1.0
44 | +-- mtl.cabal
45 | +-- mtl.json
46 +-- [...]
47 +-- 2.2.1
48 +-- mtl.cabal
49 +-- mtl.json
50 [...]
51 ztail
52 Zwaluw
54-------------------------------------------------------------------------------
56# Hackage: `mtl/2.2.1/mtl.cabal`
58 name: mtl
59 version: 2.2.1
60 license: BSD3
61 synopsis: Monad classes, using functional dependencies
62 homepage: http://github.com/ekmett/mtl
63 build-type: Simple
65 Library
66 exposed-modules:
67 Control.Monad.Cont
68 [...]
69 build-depends: base < 6, transformers == 0.4.*
70 extensions:
71 MultiParamTypeClasses
72 FunctionalDependencies
73 FlexibleInstances
75-------------------------------------------------------------------------------
77# Hackage: `mtl/2.2.1/mtl.json`
79 { "package-hashes" : { "MD5" : "96a2f12b...",
80 "Skein512_512" : "73b5858d...",
81 "SHA1" : "c244f8ec...",
82 "SHA256" : "cae59d79...",
83 "SHA512" : "5c31626b..."
84 },
85 "package-locations" : [
86 "https://hackage.haskell.org/package/mtl-2.2.1/" \
87 "mtl-2.2.1.tar.gz",
88 "https://s3.amazonaws.com/hackage.fpcomplete.com/" \
89 "package/mtl-2.2.1.tar.gz"
90 ],
91 "package-size" : 15391
92 }
94-------------------------------------------------------------------------------
96# Hackage: Nix Build for mtl 2.2.1
98 { mkDerivation, base, stdenv, transformers }:
100 mkDerivation {
101 pname = "mtl";
102 version = "2.2.1";
103 sha256 = "cae59d79f3a16f8e9f3c9adc1010c7c6...";
104 libraryHaskellDepends = [ base transformers ];
105 homepage = "http://github.com/ekmett/mtl";
106 description = "Monad classes, using functional ...";
107 license = stdenv.lib.licenses.bsd3;
108 }
110-------------------------------------------------------------------------------
112# Hackage: Destructive Editing
114`MonadRandom/0.4/MonadRandom.cabal` contains a line:
116 x-revision: 2
118This translates to Nix:
120 revision = "2";
121 editedCabalFile = "2e218afd5b29c868...";
123The build log will say:
125> Replace Cabal file with edited version from
126> http://hackage.haskell.org/package/MonadRandom-0.4/revision/2.cabal.
128-------------------------------------------------------------------------------
130# Hackage: Conditional Values
132Values in Cabal files may depend on OS, architecture, compiler, and flags.
133For example, hint 0.4.2.3 specifies:
135 Library
136 if impl(ghc >= 6.8)
137 build-depends: random, directory
138 if impl(ghc >= 6.10)
139 build-depends: base >= 4, base < 5,
140 ghc-mtl == 1.2.1.*
141 else
142 build-depends: base >= 3, base < 4
143 else
144 build-depends: utf8-string < 0.3
145 if !os(windows)
146 build-depends: unix >= 2.2.0.0
148-------------------------------------------------------------------------------
150# Hackage: Cabal Flags
152Flags can be specified by the user or auto-configured by the build to fit the
153environment. For example, pandoc 1.15.1.1 defines:
155 Flag https
156 Description: Enable support for downloading over https.
158 Library
159 Build-Depends: base >= 4.2 && <5,
160 syb >= 0.1 && < 0.7,
161 [...]
162 ghc-prim >= 0.2
163 if flag(https)
164 Build-Depends: http-client >= 0.3.2 && < 0.5,
165 http-client-tls >= 0.2 && < 0.3,
166 http-types >= 0.8 && < 0.10
168-------------------------------------------------------------------------------
170# Stackage: Stable Hackage
172- 1,605 packages (18% of Hackage)
174- Build instructions published via Git specify parameters such as "don't run
175 test suite", "don't run Haddock", etc.
177- Testing takes place on 64-bit Linux.
179- Nightly Snapshots contain the latest version of every package such that the
180 package set is consistent.
182- LTS minor releases are created weekly and contain only point releases that
183 don't change the API.
185- LTS major releases contain API-breaking changes and usually conincide with a
186 compiler update.
188-------------------------------------------------------------------------------
190# cabal2nix & hackage2nix
192- Code lives at <https://github.com/NixOS/cabal2nix>:
193 - `distribution-nixpkgs` library
194 - `cabal2nix` executable
195 - `hackage2nix` executable
197- [`update-nixpkgs.sh`](https://github.com/NixOS/cabal2nix/blob/master/hackage2nix/update-nixpkgs.sh)
198 script runs automatically once per hour.
200- Changes go to `haskell-updates` branch in [peti's Nixpkgs fork on Github](https://github.com/peti/nixpkgs).
202- [`hydra.cryp.to`](http://hydra.cryp.to/jobset/nixpkgs/haskell-updates)
203 continuously builds Stackage Nightly and the latest LTS Haskell package set.
205- If Hydra builds work sufficiently well, updates are manually merged into
206 `master` branch of main repository.
208- [`generate-nixpkgs-haskell-package-list`](https://github.com/peti/package-list)
209 utility manually publishes new state of Nixpkgs on Hackage.
211-------------------------------------------------------------------------------
213# Nixpkgs: `hackage-packages.nix`
215The package set is represented as a primitive recursive function:
217 ps = self: {
218 "3d-graphics-examples" = callPackage ...;
219 ...
220 "mtl_2_1_3_1" = callPackage ...;
221 "mtl" = callPackage ...;
222 ...
223 "ztail" = callPackage ...;
224 };
226The Nixpkgs attribute set `haskellPackages` refers to the fixpoint of that
227function which is computed by the expression:
229 let self = ps self;
230 in # = ps (ps self) = ps (ps (ps (self))) = ...
231 self
233-------------------------------------------------------------------------------
235# Nixpkgs: hackage-packages.nix
237We can modify the package set using OO-style inheritance:
239 fix = f: let self = f self; in self;
240 extend = rattrs: f: self: let super = rattrs self;
241 in super // f self super;
243 ps = self: {
244 foo = "foo"; bar = "bar";
245 foobar = self.foo + self.bar;
246 };
248 f = self: super: { foo = reverse super.foo; };
250- `(fix ps).foobar` $\equiv$ `"foobar"`
251- `(fix (extend ps f)).foobar` $\equiv$ `"oofbar"`
252- `(fix (extend (extend ps f) f)).foobar` $\equiv$ `"foobar"`
254-------------------------------------------------------------------------------
256# Nixpkgs: pkgs/development/haskell-modules/default.nix
258 { compilerConfig ? (self: super: {})
259 , packageSetConfig ? (self: super: {})
260 , overrides ? (self: super: {})
261 }:
263 fix
264 (extend
265 (extend
266 (extend
267 (extend haskellPackages commonConfiguration)
268 compilerConfig)
269 packageSetConfig)
270 overrides)
272-------------------------------------------------------------------------------
274# Nixpkgs: Multiple Views Of The Package Set
276- `hackage-packages.nix` represents "Stackage Nightly".
278- `configuration-common.nix` "extends" that to work around deficiencies in
279 `hackage2nix`.
281- `configuration-ghc-x.y.nix` "extends" that with fixes relevant only for
282 particular compiler versions: `haskell.packages.ghcXY`.
284- `configuration-lts-v.w.nix` "extends" those sets to choose default versions
285 according to "LTS Haskell x.y": `haskell.packages.lts-x_y`.
287-------------------------------------------------------------------------------
289# Nixpkgs: `overrideCabal`
291Haskell derivations take the `mkDerivation` function as an argument:
293 { mkDerivation, ... }: mkDerivation {
294 pname = "mtl";
295 version = "2.2.1";
296 ...
297 }
299The `override` method changes the arguments passed to a derivation --- so we
300can change the definition of `mkDerivation`:
302 overrideCabal = drv: f: drv.override (args: args // {
303 mkDerivation = drv: args.mkDerivation (drv // f drv);
304 });
306-------------------------------------------------------------------------------
308# Nixpkgs: `haskell.lib` Helper Functions
310 doHaddock = ...;
311 dontHaddock = drv: overrideCabal drv (drv: {
312 doHaddock = false;
313 });
315 enableLibraryProfiling = ...;
316 disableLibraryProfiling = drv: overrideCabal drv (drv: {
317 enableLibraryProfiling = false;
318 });
320 addBuildTool = drv: x: addBuildTools drv [x];
321 addBuildTools = drv: xs: overrideCabal drv (drv: {
322 buildTools = (drv.buildTools or []) ++ xs;
323 });
325-------------------------------------------------------------------------------
327# Nixpkgs: Deep Overriding
329 fix = f: let x = f x // { __unfix__ = f; }; in x;
331 haskellPackages = self:
332 let
333 callPackage = callPackageWithScope self;
335 callPackageWithScope = scope: drv: args:
336 (stdenv.lib.callPackageWith scope drv args) // {
337 overrideScope = f:
338 callPackageWithScope
339 (fix (extend scope.__unfix__ f))
340 drv args;
341 };
342 in
343 { "git-annex" = callPackage ...; };
345-------------------------------------------------------------------------------
347# Other Resources
349- <http://hackage.haskell.org/>
351- <http://www.stackage.org/>
353- <http://nixos.org/nixpkgs/manual/#users-guide-to-the-haskell-infrastructure>
355- <http://lists.science.uu.nl/mailman/listinfo/nix-dev>
357- `#nixos` at `irc.freenode.org`