1% Managing the Haskell Dependency Hell with Nix
7The Nix package manager has proven very well suited for managing sophisticated
8Haskell development environments. It has out-of-the-box support for installing
9Haskell Platforms 2009.2.0.2, 2010.1.0.0, 2010.2.0.0, 2011.2.0.0, 2011.2.0.1,
102011.4.0.0, 2012.2.0.0, and 2012.4.0.0 on any machine running Linux or Darwin.
11Furthermore, Nix provides easy access to well over 700 popular Haskell packages
12from Hackage. Developers can use Nix to automate building and testing of their
13Haskell packages, either by running those builds locally on-demand, or by
14relying on the Continuous Integration service Hydra. Last but not least,
15Haskell programmers will probably enjoy the fact that Nix packages are written
16in a language that is, essentially, lazily evaluated lambda calculus.
18This presentation introduces the principles of the Nix package manager, and
19provides a hands-on tour through its essential features, teaching Haskell
22 - manage their working environment with Nix,
23 - extend the Nix packages database with their own packages, and
24 - utilize the advanced continuous integration features.
28Purely functional programming languages like Haskell encourage software
29engineers to write re-usable code. As of today, the Haskell community's central
30repository "Hackage" registers 5015 packages, 3944 of which provide function
31libraries. This means that approximately 78% of all Haskell packages are
32intended to be re-used by other developers. For Haskell programmers, this is a
33good situation to be in! There is a lot of re-usable code out there, and the
34pool to choose from is growing steadily.
36However, the fact that Haskell packages re-use each other a lot comes at a
37price: a phenomenon lovingly referred to as "dependency hell". The task of
38installing a Haskell package includes the installation of all its dependencies
39(and their respective dependencies). Updating a Haskell package typically
40requires updates of that package's dependencies as well. And if a package is
41de-installed, then chances are that some of its dependencies become unused in
42the process and ought to be de-installed, too.
44Many Haskell programmers turn to Cabal for help --- only to find that Cabal
45doesn't really address those tasks. Cabal can install a package, alright, but
46it cannot perform reliable updates or clean up the system after a
47de-installation. Arguably, Cabal doesn't even address the installation part
48satisfactorily, because it is unaware of all software outside of the domain of
49Haskell. The package "hsdns", for example, needs the GNU ADNS library installed
50to compile, but Cabal doesn't know anything about GNU ADNS. It cannot install
51that library as part of the hsdns build. All it can do is abort with an error
52message, saying "the adns library is missing", but it's the users problem to
53remedy that situation somehow.
55Traditional package managers like `apt-get`, `rpm`, `pacman`, or `BSD ports`
56are designed to solve exactly those issues. These programs are really good at
57installing system packages. They know how to keep the installation up-to-date
58without breaking anything, and they know how clean-up after themselves when a
59package has been removed. Yet, those package managers are somewhat unprepared
60for the complexity of Haskell's package system. For example, most package
61managers assume that for any given package `A`, only one version of `A` can be
62installed at the same time. This is certainly not true for Haskell, where
63oftentimes a whole zoo of mutually incompatible library versions need to be
64installed at the same time to build all installation targets. The same problem
65applies to the installation of the compiler. Developers who care about portable
66code need many different versions of GHC installed in parallel, so that their
67code can be tested with different compiler variants, but the ability to manage
68an installation of this complexity is far out of the reach of traditional
71# The Nix Package Manager
73Nix [1] is a package manager very much like `apt-get` or `rpm`. At the same
74time, Nix differs from traditional package managers, because it breaks with the
75established Unix file system layout. Instead of installing executable programs
76into `/usr/bin` and libraries into `/usr/lib`, Nix installs every package into
77a directory hierarchy of its own. For example, all files that belong to the
78package GHC 7.6.2 reside underneath the prefix:
80 $ ls /nix/store/f2c7ia72ial5y8h5k1w7z5n87xnr5gm6-ghc-7.6.2/
83This installation scheme allows any number of GHC versions to co-exist, because
84they live in completely different places:
86 $ ls /nix/store/*-ghc-*/bin/ghc
87 /nix/store/cczdxjhrx27wmf4rpk4rk70zabw2fggd-ghc-6.10.4/bin/ghc
88 /nix/store/n6ygpkh29dfp9di27l70681k7llnhama-ghc-6.12.3/bin/ghc
89 /nix/store/frziw92yvb0rgm7sx7zzsmgy7cyl16d9-ghc-7.0.4/bin/ghc
90 /nix/store/82lkfs5c4l41p244apv7q5dmd91h712d-ghc-7.4.2/bin/ghc
91 /nix/store/ch0cq5l0l8p7g80skifivrg4d40s8sdm-ghc-7.6.2/bin/ghc
93Furthermore, packages compiled by Nix generally don't refer to any system path
94outside of `/nix/store`. GHC depends on system libraries like glibc, libgmp,
95and curses, but all those dependencies are satisfied entirely within Nix:
98 /nix/store/ml128hlikbl14q4cwviafpaskskx1a3f-ncurses-5.9/lib/libncursesw.so.5
99 /nix/store/zpr8jdx10napidkyz62f3nzl5fm87jyr-glibc-2.17/lib/librt.so.1
100 /nix/store/zpr8jdx10napidkyz62f3nzl5fm87jyr-glibc-2.17/lib/libutil.so.1
101 /nix/store/zpr8jdx10napidkyz62f3nzl5fm87jyr-glibc-2.17/lib/libdl.so.2
102 /nix/store/zpr8jdx10napidkyz62f3nzl5fm87jyr-glibc-2.17/lib/libm.so.6
103 /nix/store/zpr8jdx10napidkyz62f3nzl5fm87jyr-glibc-2.17/lib/libpthread.so.0
104 /nix/store/zpr8jdx10napidkyz62f3nzl5fm87jyr-glibc-2.17/lib/libc.so.6
105 /nix/store/989njbbf770sxyqqpyrw2m0cbfdynzbb-gmp-5.0.5/lib/libgmp.so.10
107The `/nix/store` hierarchy is completely self-contained. A Haskell development
108environment that's been created on one machine can be copied to any other
109machine simply by running "`rsync -a /nix other-machine:/`".
111A quick glance at the store paths shown above reveals yet another powerful
112feature. The exact path used to store an installed package contains a
113cryptographic hash that uniquely identifies
115 1. the shell command sequence that built this particular package and
116 2. all other store paths that this package depends on.
118Suppose that GHC 7.6.2 would be compiled two times. The first build uses GCC
119version 4.6.3 to build the Haskell compiler, but the second build uses GCC 4.7.
120Then the results of those two builds would be stored in paths like
122 - `/nix/store/ch0cq5l0l8p7g80skifivrg4d40s8sdm-ghc-7.6.2` and
123 - `/nix/store/f2c7ia72ial5y8h5k1w7z5n87xnr5gm6-ghc-7.6.2`.
125The cryptographic hashes assigned to those two paths differs because both
126variants of GHC 7.6.2 have different dependencies (GCC 4.6.3 and 4.7
127respectively). Because of this approach, Nix is able to satisfy the
128requirements of the Haskell package system, which needs to keep multiple
129versions of the same package around concurrently. Nix can easily do that,
130simply by having a separate store path for each variant of the package.
132These capabilities make Nix rather well suited for maintaining sophisticated
133Haskell development environments, and the remainder of this article is intended
134to provide a hands-on tutorial of how to use Nix for exactly that purpose.
136# Installing the Haskell Platform
138The installation procedure of Nix itself is fairly straightforward.
139Ready-to-use binary packages are available for many Linux platforms
140(particularly on x86, x86_64, and PowerPC), for Mac OS X, and for FreeBSD.
141Furthermore, Nix can be compiled from source code on any POSIX-like operating
142system. Detailed installation instructions can be found at [2].
144Once Nix itself is installed, it can be used to obtain an up-to-date copy of
145the Nix package database by running:
147 $ nix-channel --add http://nixos.org/channels/nixpkgs-unstable
148 $ nix-channel --update
150Now, it is possible to see the entire set of packages known to Nix by running:
153 pkgs.pong3d 3dpong-0.5.0
154 pkgs.rdf4store 4store-v1.1.5
155 pkgs.foursuite 4suite-1.0.2
158The output consists of two columns. The right hand-side shows the
159human-readable name and version of the package. The left hand-side shows a
160unique identifier for that particular package in the Nix package database. Both
161identifiers can be used to install packages:
163 $ nix-env -i 3dpong-0.5.0 # install by name
164 $ nix-env -iA pkgs.pong3d # install by attribute
166Now, to install the Haskell Platform, we first need to make up our mind which
167version we want, because there are plenty to choose from. The command
169 $ nix-env -qaP '*' | grep haskell-platform
171shows that all of the versions 2009.2.0.2, 2010.1.0.0, 2010.2.0.0, 2011.2.0.0,
1722011.2.0.1, 2011.4.0.0, 2012.2.0.0, and 2012.4.0.0 are available, and each of
173those versions of Haskell Platform can be compiled with GHC versions 6.10.4,
1746.12.1, 6.12.2, 6.12.3, 7.0.1, 7.0.2, 7.0.3, 7.0.4, 7.2.1, 7.2.2, 7.4.1, 7.4.2,
1757.6.1, and 7.6.2 respectively!
177For the sake of the example, let's just choose the latest official version:
179 $ nix-env -iA haskellPackages.haskellPlatform
180 installing `haskell-haskell-platform-ghc7.4.2-2012.4.0.0'
182 building path(s) `/nix/store/j1gq9xciqj3390himzi3p9xcln9hnxl4-user-environment'
183 created 1159 symlinks in user environment
185When that command is run for the first time, it will download and install a
186*lot* of packages, because the Haskell Platform depends on many other things
187such as the Linux kernel headers, glibc, GCC, GMP, Perl, and so on. Of course,
188these packages will be re-used when installing a different version of the
189Haskell Platform later, so one doesn't have to worry that every single install
190is going to be that expensive.
192In its last step, `nix-env` creates a "user environment" at
193`$HOME/.nix-profile`, which simplies use of the installed packages a lot. That
194environment uses symlinks to create a view of the Nix store that combines those
195packages that have been installed into a single hierarchy. For example, the
196path `~/.nix-profile/bin/ghc` will be a symlink to
197`/nix/store/vlx3ikjjxlfvqjjlx74cg07p79y43mdq-ghc-7.4.2/bin/ghc`. The path
198`~/.nix-profile/bin/alex`, on the other hand, points to
199`/nix/store/7rfilvshj9wzm3rx86m4mkpf76l24fhs-alex-3.0.2/bin/alex`. The binaries
200`alex` and `ghc` live in different store paths, but the user environment joins
201all those paths into a single hierarchy. This means that it is sufficient to
202add `$HOME/.nix-profile/bin` to `$PATH` to use the Haskell Platform from Nix.
206[1] Nix, the purely functional package manager: <http://nixos.org/nix/>
208[2] Nix Installation Instructions: <http://hydra.nixos.org/build/4480483/download/1/manual/#chap-installation>
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list