Merge remote-tracking branch 'upstream/master' into gcc-6

doc/cross-compilation.xml

@@ -30,7 +30,7 @@
   <section>
     <title>Platform parameters</title>
     <para>
-      The three GNU Autoconf platforms, <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>cross</wordasword>, are historically the result of much confusion.
+      The three GNU Autoconf platforms, <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>target</wordasword>, are historically the result of much confusion.
       <link xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html" /> clears this up somewhat but there is more to be said.
       An important advice to get out the way is, unless you are packaging a compiler or other build tool, just worry about the build and host platforms.
       Dealing with just two platforms usually better matches people's preconceptions, and in this case is completely correct.
@@ -62,8 +62,8 @@
             The "target platform" is black sheep.
             The other two intrinsically apply to all compiled software—or any build process with a notion of "build-time" followed by "run-time".
             The target platform only applies to programming tools, and even then only is a good for for some of them.
-            Briefly, GCC, Binutils, GHC, and certain other tools are written in such a way such that a single build can only compiler code for a single platform.
-            Thus, when building them, one must think ahead about what platforms they wish to use the tool to produce machine code for, and build binaries for each.
+            Briefly, GCC, Binutils, GHC, and certain other tools are written in such a way such that a single build can only compile code for a single platform.
+            Thus, when building them, one must think ahead about which platforms they wish to use the tool to produce machine code for, and build binaries for each.
           </para>
           <para>
             There is no fundamental need to think about the target ahead of time like this.
@@ -85,7 +85,8 @@
       This field is obsolete and will soon disappear—please do not use it.
     </para></note>
     <para>
-      The exact scheme these fields is a bit ill-defined due to a long and convoluted evolution, but this is slowly being cleaned up.
+      The exact schema these fields follow is a bit ill-defined due to a long and convoluted evolution, but this is slowly being cleaned up.
+      You can see examples of ones used in practice in <literal>lib.systems.examples</literal>; note how they are not all very consistent.
       For now, here are few fields can count on them containing:
     </para>
     <variablelist>
@@ -118,8 +119,27 @@
             This is a nix representation of a parsed LLVM target triple with white-listed components.
             This can be specified directly, or actually parsed from the <varname>config</varname>.
             [Technically, only one need be specified and the others can be inferred, though the precision of inference may not be very good.]
-            See <literal>lib.systems.parse</literal> for the exact representation, along with some <literal>is*</literal>predicates.
-            These predicates are superior to the ones in <varname>stdenv</varname> as they aren't tied to the build platform (host, as previously discussed, would be a saner default).
+            See <literal>lib.systems.parse</literal> for the exact representation.
+          </para>
+        </listitem>
+      </varlistentry>
+      <varlistentry>
+        <term><varname>libc</varname></term>
+        <listitem>
+          <para>
+            This is a string identifying the standard C library used.
+            Valid identifiers include "glibc" for GNU libc, "libSystem" for Darwin's Libsystem, and "uclibc" for µClibc.
+            It should probably be refactored to use the module system, like <varname>parse</varname>.
+          </para>
+        </listitem>
+      </varlistentry>
+      <varlistentry>
+        <term><varname>is*</varname></term>
+        <listitem>
+          <para>
+            These predicates are defined in <literal>lib.systems.inspect</literal>, and slapped on every platform.
+            They are superior to the ones in <varname>stdenv</varname> as they force the user to be explicit about which platform they are inspecting.
+            Please use these instead of those.
           </para>
         </listitem>
       </varlistentry>
@@ -128,7 +148,7 @@
         <listitem>
           <para>
             This is, quite frankly, a dumping ground of ad-hoc settings (it's an attribute set).
-            See <literal>lib.systems.platforms</literal> for examples—there's hopefully one in there that will work verbatim for each platform one is working.
+            See <literal>lib.systems.platforms</literal> for examples—there's hopefully one in there that will work verbatim for each platform that is working.
             Please help us triage these flags and give them better homes!
           </para>
         </listitem>
@@ -158,10 +178,10 @@
       The depending package's target platform is unconstrained by the sliding window principle, which makes sense in that one can in principle build cross compilers targeting arbitrary platforms.
     </para></note>
     <para>
-      How does this work in practice? Nixpkgs is now structured so that build-time dependencies are taken from from <varname>buildPackages</varname>, whereas run-time dependencies are taken from the top level attribute set.
+      How does this work in practice? Nixpkgs is now structured so that build-time dependencies are taken from <varname>buildPackages</varname>, whereas run-time dependencies are taken from the top level attribute set.
       For example, <varname>buildPackages.gcc</varname> should be used at build time, while <varname>gcc</varname> should be used at run time.
       Now, for most of Nixpkgs's history, there was no <varname>buildPackages</varname>, and most packages have not been refactored to use it explicitly.
-      Instead, one can use the four attributes used for specifying dependencies as documented in <link linkend="ssec-stdenv-attributes" />.
+      Instead, one can use the four attributes used for specifying dependencies as documented in <xref linkend="ssec-stdenv-attributes"/>.
       We "splice" together the run-time and build-time package sets with <varname>callPackage</varname>, and then <varname>mkDerivation</varname> for each of four attributes pulls the right derivation out.
       This splicing can be skipped when not cross compiling as the package sets are the same, but is a bit slow for cross compiling.
       Because of this, a best-of-both-worlds solution is in the works with no splicing or explicit access of <varname>buildPackages</varname> needed.
@@ -184,11 +204,27 @@
     More information needs to moved from the old wiki, especially <link xlink:href="https://nixos.org/wiki/CrossCompiling" />, for this section.
   </para></note>
   <para>
-    Many sources (manual, wiki, etc) probably mention passing <varname>system</varname>, <varname>platform</varname>, and, optionally, <varname>crossSystem</varname> to nixpkgs:
-    <literal>import &lt;nixpkgs&gt; { system = ..; platform = ..; crossSystem = ..; }</literal>.
-    <varname>system</varname> and <varname>platform</varname> together determine the system on which packages are built, and <varname>crossSystem</varname> specifies the platform on which packages are ultimately intended to run, if it is different.
-    This still works, but with more recent changes, one can alternatively pass <varname>localSystem</varname>, containing <varname>system</varname> and <varname>platform</varname>, for symmetry.
+    Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in which case there is no cross compiling and everything is built by and for that system,
+    or also with <varname>crossSystem</varname>, in which case packages run on the latter, but all building happens on the former.
+    Both parameters take the same schema as the 3 (build, host, and target) platforms defined in the previous section.
+    As mentioned above, <literal>lib.systems.examples</literal> has some platforms which are used as arguments for these parameters in practice.
+    You can use them programmatically, or on the command line like <command>nix-build &lt;nixpkgs&gt; --arg crossSystem '(import &lt;nixpkgs/lib&gt;).systems.examples.fooBarBaz'</command>.
   </para>
+  <para>
+    While one is free to pass both parameters in full, there's a lot of logic to fill in missing fields.
+    As discussed in the previous section, only one of <varname>system</varname>, <varname>config</varname>, and <varname>parsed</varname> is needed to infer the other two.
+    Additionally, <varname>libc</varname> will be inferred from <varname>parse</varname>.
+    Finally, <literal>localSystem.system</literal> is also <emphasis>impurely</emphasis> inferred based on the platform evaluation occurs.
+    This means it is often not necessary to pass <varname>localSystem</varname> at all, as in the command-line example in the previous paragraph.
+  </para>
+  <note>
+    <para>
+      Many sources (manual, wiki, etc) probably mention passing <varname>system</varname>, <varname>platform</varname>, along with the optional <varname>crossSystem</varname> to nixpkgs:
+      <literal>import &lt;nixpkgs&gt; { system = ..; platform = ..; crossSystem = ..; }</literal>.
+      Passing those two instead of <varname>localSystem</varname> is still supported for compatibility, but is discouraged.
+      Indeed, much of the inference we do for these parameters is motivated by compatibility as much as convenience.
+    </para>
+  </note>
   <para>
     One would think that <varname>localSystem</varname> and <varname>crossSystem</varname> overlap horribly with the three <varname>*Platforms</varname> (<varname>buildPlatform</varname>, <varname>hostPlatform,</varname> and <varname>targetPlatform</varname>; see <varname>stage.nix</varname> or the manual).
     Actually, those identifiers are purposefully not used here to draw a subtle but important distinction:

doc/languages-frameworks/python.md

@@ -710,7 +710,7 @@ nix-env -if build.nix
 ```
 Now you can use the Python interpreter, as well as the extra packages that you added to the environment.
 
-#### Environment defined in `~/.nixpkgs/config.nix`
+#### Environment defined in `~/.config/nixpkgs/config.nix`
 
 If you prefer to, you could also add the environment as a package override to the Nixpkgs set.
 ```nix

lib/default.nix

@@ -5,8 +5,9 @@
  */
 let
 
-  # trivial, often used functions
+  # often used, or depending on very little
   trivial = import ./trivial.nix;
+  fixedPoints = import ./fixed-points.nix;
 
   # datatypes
   attrsets = import ./attrsets.nix;
@@ -42,7 +43,7 @@ let
   filesystem = import ./filesystem.nix;
 
 in
-  { inherit trivial
+  { inherit trivial fixedPoints
             attrsets lists strings stringsWithDeps
             customisation maintainers meta sources
             modules options types
@@ -55,6 +56,7 @@ in
   }
   # !!! don't include everything at top-level; perhaps only the most
   # commonly used functions.
-  // trivial // lists // strings // stringsWithDeps // attrsets // sources
+  // trivial // fixedPoints
+  // lists // strings // stringsWithDeps // attrsets // sources
   // options // types // meta // debug // misc // modules
   // customisation

lib/fixed-points.nix

@@ -0,0 +1,78 @@
+rec {
+  # Compute the fixed point of the given function `f`, which is usually an
+  # attribute set that expects its final, non-recursive representation as an
+  # argument:
+  #
+  #     f = self: { foo = "foo"; bar = "bar"; foobar = self.foo + self.bar; }
+  #
+  # Nix evaluates this recursion until all references to `self` have been
+  # resolved. At that point, the final result is returned and `f x = x` holds:
+  #
+  #     nix-repl> fix f
+  #     { bar = "bar"; foo = "foo"; foobar = "foobar"; }
+  #
+  #  Type: fix :: (a -> a) -> a
+  #
+  # See https://en.wikipedia.org/wiki/Fixed-point_combinator for further
+  # details.
+  fix = f: let x = f x; in x;
+
+  # A variant of `fix` that records the original recursive attribute set in the
+  # result. This is useful in combination with the `extends` function to
+  # implement deep overriding. See pkgs/development/haskell-modules/default.nix
+  # for a concrete example.
+  fix' = f: let x = f x // { __unfix__ = f; }; in x;
+
+  # Modify the contents of an explicitly recursive attribute set in a way that
+  # honors `self`-references. This is accomplished with a function
+  #
+  #     g = self: super: { foo = super.foo + " + "; }
+  #
+  # that has access to the unmodified input (`super`) as well as the final
+  # non-recursive representation of the attribute set (`self`). `extends`
+  # differs from the native `//` operator insofar as that it's applied *before*
+  # references to `self` are resolved:
+  #
+  #     nix-repl> fix (extends g f)
+  #     { bar = "bar"; foo = "foo + "; foobar = "foo + bar"; }
+  #
+  # The name of the function is inspired by object-oriented inheritance, i.e.
+  # think of it as an infix operator `g extends f` that mimics the syntax from
+  # Java. It may seem counter-intuitive to have the "base class" as the second
+  # argument, but it's nice this way if several uses of `extends` are cascaded.
+  extends = f: rattrs: self: let super = rattrs self; in super // f self super;
+
+  # Compose two extending functions of the type expected by 'extends'
+  # into one where changes made in the first are available in the
+  # 'super' of the second
+  composeExtensions =
+    f: g: self: super:
+      let fApplied = f self super;
+          super' = super // fApplied;
+      in fApplied // g self super';
+
+  # Create an overridable, recursive attribute set. For example:
+  #
+  #     nix-repl> obj = makeExtensible (self: { })
+  #
+  #     nix-repl> obj
+  #     { __unfix__ = «lambda»; extend = «lambda»; }
+  #
+  #     nix-repl> obj = obj.extend (self: super: { foo = "foo"; })
+  #
+  #     nix-repl> obj
+  #     { __unfix__ = «lambda»; extend = «lambda»; foo = "foo"; }
+  #
+  #     nix-repl> obj = obj.extend (self: super: { foo = super.foo + " + "; bar = "bar"; foobar = self.foo + self.bar; })
+  #
+  #     nix-repl> obj
+  #     { __unfix__ = «lambda»; bar = "bar"; extend = «lambda»; foo = "foo + "; foobar = "foo + bar"; }
+  makeExtensible = makeExtensibleWithCustomName "extend";
+
+  # Same as `makeExtensible` but the name of the extending attribute is
+  # customized.
+  makeExtensibleWithCustomName = extenderName: rattrs:
+    fix' rattrs // {
+      ${extenderName} = f: makeExtensibleWithCustomName extenderName (extends f rattrs);
+   };
+}

lib/maintainers.nix

@@ -132,6 +132,7 @@
   deepfire = "Kosyrev Serge <_deepfire@feelingofgreen.ru>";
   demin-dmitriy = "Dmitriy Demin <demindf@gmail.com>";
   DerGuteMoritz = "Moritz Heidkamp <moritz@twoticketsplease.de>";
+  dermetfan = "Robin Stumm <serverkorken@gmail.com>";
   DerTim1 = "Tim Digel <tim.digel@active-group.de>";
   desiderius = "Didier J. Devroye <didier@devroye.name>";
   devhell = "devhell <\"^\"@regexmail.net>";
@@ -161,6 +162,7 @@
   ehegnes = "Eric Hegnes <eric.hegnes@gmail.com>";
   ehmry = "Emery Hemingway <emery@vfemail.net>";
   eikek = "Eike Kettner <eike.kettner@posteo.de>";
+  ekleog = "Leo Gaspard <leo@gaspard.io>";
   elasticdog = "Aaron Bull Schaefer <aaron@elasticdog.com>";
   eleanor = "Dejan Lukan <dejan@proteansec.com>";
   elitak = "Eric Litak <elitak@gmail.com>";

lib/strings.nix

@@ -438,8 +438,13 @@ rec {
        => true
        isStorePath pkgs.python
        => true
+       isStorePath [] || isStorePath 42 || isStorePath {} || …
+       => false
   */
-  isStorePath = x: builtins.substring 0 1 (toString x) == "/" && dirOf (builtins.toPath x) == builtins.storeDir;
+  isStorePath = x:
+       builtins.isString x
+    && builtins.substring 0 1 (toString x) == "/"
+    && dirOf (builtins.toPath x) == builtins.storeDir;
 
   /* Convert string to int
      Obviously, it is a bit hacky to use fromJSON that way.

lib/systems/default.nix

@@ -5,6 +5,7 @@ rec {
   parse = import ./parse.nix;
   inspect = import ./inspect.nix;
   platforms = import ./platforms.nix;
+  examples = import ./examples.nix;
 
   # Elaborate a `localSystem` or `crossSystem` so that it contains everything
   # necessary.

lib/systems/examples.nix

@@ -0,0 +1,130 @@
+# These can be passed to nixpkgs as either the `localSystem` or
+# `crossSystem`. They are put here for user convenience, but also used by cross
+# tests and linux cross stdenv building, so handle with care!
+
+let platforms = import ./platforms.nix; in
+
+rec {
+  #
+  # Linux
+  #
+
+  sheevaplug = rec {
+    config = "armv5tel-unknown-linux-gnueabi";
+    bigEndian = false;
+    arch = "armv5tel";
+    float = "soft";
+    withTLS = true;
+    libc = "glibc";
+    platform = platforms.sheevaplug;
+    openssl.system = "linux-generic32";
+    inherit (platform) gcc;
+  };
+
+  raspberryPi = rec {
+    config = "armv6l-unknown-linux-gnueabihf";
+    bigEndian = false;
+    arch = "armv6l";
+    float = "hard";
+    fpu = "vfp";
+    withTLS = true;
+    libc = "glibc";
+    platform = platforms.raspberrypi;
+    openssl.system = "linux-generic32";
+    inherit (platform) gcc;
+  };
+
+  armv7l-hf-multiplatform = rec {
+    config = "arm-unknown-linux-gnueabihf";
+    bigEndian = false;
+    arch = "armv7-a";
+    float = "hard";
+    fpu = "vfpv3-d16";
+    withTLS = true;
+    libc = "glibc";
+    platform = platforms.armv7l-hf-multiplatform;
+    openssl.system = "linux-generic32";
+    inherit (platform) gcc;
+  };
+
+  aarch64-multiplatform = rec {
+    config = "aarch64-unknown-linux-gnu";
+    bigEndian = false;
+    arch = "aarch64";
+    withTLS = true;
+    libc = "glibc";
+    platform = platforms.aarch64-multiplatform;
+    inherit (platform) gcc;
+  };
+
+  scaleway-c1 = armv7l-hf-multiplatform // rec {
+    platform = platforms.scaleway-c1;
+    inherit (platform) gcc;
+    inherit (gcc) fpu;
+  };
+
+  pogoplug4 = rec {
+    arch = "armv5tel";
+    config = "armv5tel-softfloat-linux-gnueabi";
+    float = "soft";
+
+    platform = platforms.pogoplug4;
+
+    inherit (platform) gcc;
+    libc = "glibc";
+
+    withTLS = true;
+    openssl.system = "linux-generic32";
+  };
+
+  fuloongminipc = rec {
+    config = "mips64el-unknown-linux-gnu";
+    bigEndian = false;
+    arch = "mips";
+    float = "hard";
+    withTLS = true;
+    libc = "glibc";
+    platform = platforms.fuloong2f_n32;
+    openssl.system = "linux-generic32";
+    inherit (platform) gcc;
+  };
+
+  #
+  # Darwin
+  #
+
+  iphone64 = {
+    config = "aarch64-apple-darwin14";
+    arch = "arm64";
+    libc = "libSystem";
+    platform = {};
+  };
+
+  iphone32 = {
+    config = "arm-apple-darwin10";
+    arch = "armv7-a";
+    libc = "libSystem";
+    platform = {};
+  };
+
+  #
+  # Windows
+  #
+
+  # 32 bit mingw-w64
+  mingw32 = {
+    config = "i686-pc-mingw32";
+    arch = "x86"; # Irrelevant
+    libc = "msvcrt"; # This distinguishes the mingw (non posix) toolchain
+    platform = {};
+  };
+
+  # 64 bit mingw-w64
+  mingwW64 = {
+    # That's the triplet they use in the mingw-w64 docs.
+    config = "x86_64-pc-mingw32";
+    arch = "x86_64"; # Irrelevant
+    libc = "msvcrt"; # This distinguishes the mingw (non posix) toolchain
+    platform = {};
+  };
+}

lib/systems/platforms.nix

@@ -255,6 +255,10 @@ rec {
       arch = "armv6";
       fpu = "vfp";
       float = "hard";
+      # TODO(@Ericson2314) what is this and is it a good idea? It was
+      # used in some cross compilation examples but not others.
+      #
+      # abi = "aapcs-linux";
     };
   };
 
@@ -460,7 +464,10 @@ rec {
     '';
     kernelTarget = "vmlinux";
     uboot = null;
-    gcc.arch = "loongson2f";
+    gcc = {
+      arch = "loongson2f";
+      abi = "n32";
+    };
   };
 
   beaglebone = armv7l-hf-multiplatform // {

lib/tests/misc.nix

@@ -1,7 +1,6 @@
 # to run these tests:
 # nix-instantiate --eval --strict nixpkgs/lib/tests/misc.nix
 # if the resulting list is empty, all tests passed
-let inherit (builtins) add; in
 with import ../default.nix;
 
 runTests {
@@ -88,6 +87,37 @@ runTests {
     expected = [ "2001" "db8" "0" "0042" "" "8a2e" "370" "" ];
   };
 
+  testIsStorePath =  {
+    expr =
+      let goodPath =
+            "/nix/store/d945ibfx9x185xf04b890y4f9g3cbb63-python-2.7.11";
+      in {
+        storePath = isStorePath goodPath;
+        storePathAppendix = isStorePath
+          "${goodPath}/bin/python";
+        nonAbsolute = isStorePath (concatStrings (tail (stringToCharacters goodPath)));
+        asPath = isStorePath (builtins.toPath goodPath);
+        otherPath = isStorePath "/something/else";
+        otherVals = {
+          attrset = isStorePath {};
+          list = isStorePath [];
+          int = isStorePath 42;
+        };
+      };
+    expected = {
+      storePath = true;
+      storePathAppendix = false;
+      nonAbsolute = false;
+      asPath = true;
+      otherPath = false;
+      otherVals = {
+        attrset = false;
+        list = false;
+        int = false;
+      };
+    };
+  };
+
 # LISTS
 
   testFilter = {
@@ -266,14 +296,14 @@ runTests {
           res4 = let x = defaultOverridableDelayableArgs id { a = 7; };
                 in (x.merge) ( x: { b = 10; });
           res5 = let x = defaultOverridableDelayableArgs id { a = 7; };
-                in (x.merge) ( x: { a = add x.a 3; });
-          res6 = let x = defaultOverridableDelayableArgs id { a = 7; mergeAttrBy = { a = add; }; };
+                in (x.merge) ( x: { a = builtins.add x.a 3; });
+          res6 = let x = defaultOverridableDelayableArgs id { a = 7; mergeAttrBy = { a = builtins.add; }; };
                      y = x.merge {};
                 in (y.merge) { a = 10; };
 
           resRem7 = res6.replace (a: removeAttrs a ["a"]);
 
-          resReplace6 = let x = defaultOverridableDelayableArgs id { a = 7; mergeAttrBy = { a = add; }; };
+          resReplace6 = let x = defaultOverridableDelayableArgs id { a = 7; mergeAttrBy = { a = builtins.add; }; };
                             x2 = x.merge { a = 20; }; # now we have 27
                         in (x2.replace) { a = 10; }; # and override the value by 10
 

lib/trivial.nix

@@ -43,84 +43,6 @@ rec {
   */
   mergeAttrs = x: y: x // y;
 
-
-  # Compute the fixed point of the given function `f`, which is usually an
-  # attribute set that expects its final, non-recursive representation as an
-  # argument:
-  #
-  #     f = self: { foo = "foo"; bar = "bar"; foobar = self.foo + self.bar; }
-  #
-  # Nix evaluates this recursion until all references to `self` have been
-  # resolved. At that point, the final result is returned and `f x = x` holds:
-  #
-  #     nix-repl> fix f
-  #     { bar = "bar"; foo = "foo"; foobar = "foobar"; }
-  #
-  #  Type: fix :: (a -> a) -> a
-  #
-  # See https://en.wikipedia.org/wiki/Fixed-point_combinator for further
-  # details.
-  fix = f: let x = f x; in x;
-
-  # A variant of `fix` that records the original recursive attribute set in the
-  # result. This is useful in combination with the `extends` function to
-  # implement deep overriding. See pkgs/development/haskell-modules/default.nix
-  # for a concrete example.
-  fix' = f: let x = f x // { __unfix__ = f; }; in x;
-
-  # Modify the contents of an explicitly recursive attribute set in a way that
-  # honors `self`-references. This is accomplished with a function
-  #
-  #     g = self: super: { foo = super.foo + " + "; }
-  #
-  # that has access to the unmodified input (`super`) as well as the final
-  # non-recursive representation of the attribute set (`self`). `extends`
-  # differs from the native `//` operator insofar as that it's applied *before*
-  # references to `self` are resolved:
-  #
-  #     nix-repl> fix (extends g f)
-  #     { bar = "bar"; foo = "foo + "; foobar = "foo + bar"; }
-  #
-  # The name of the function is inspired by object-oriented inheritance, i.e.
-  # think of it as an infix operator `g extends f` that mimics the syntax from
-  # Java. It may seem counter-intuitive to have the "base class" as the second
-  # argument, but it's nice this way if several uses of `extends` are cascaded.
-  extends = f: rattrs: self: let super = rattrs self; in super // f self super;
-
-  # Compose two extending functions of the type expected by 'extends'
-  # into one where changes made in the first are available in the
-  # 'super' of the second
-  composeExtensions =
-    f: g: self: super:
-      let fApplied = f self super;
-          super' = super // fApplied;
-      in fApplied // g self super';
-
-  # Create an overridable, recursive attribute set. For example:
-  #
-  #     nix-repl> obj = makeExtensible (self: { })
-  #
-  #     nix-repl> obj
-  #     { __unfix__ = «lambda»; extend = «lambda»; }
-  #
-  #     nix-repl> obj = obj.extend (self: super: { foo = "foo"; })
-  #
-  #     nix-repl> obj
-  #     { __unfix__ = «lambda»; extend = «lambda»; foo = "foo"; }
-  #
-  #     nix-repl> obj = obj.extend (self: super: { foo = super.foo + " + "; bar = "bar"; foobar = self.foo + self.bar; })
-  #
-  #     nix-repl> obj
-  #     { __unfix__ = «lambda»; bar = "bar"; extend = «lambda»; foo = "foo + "; foobar = "foo + bar"; }
-  makeExtensible = makeExtensibleWithCustomName "extend";
-
-  # Same as `makeExtensible` but the name of the extending attribute is
-  # customized.
-  makeExtensibleWithCustomName = extenderName: rattrs:
-    fix' rattrs // {
-      ${extenderName} = f: makeExtensibleWithCustomName extenderName (extends f rattrs);
-   };
-
   # Flip the order of the arguments of a binary function.
   flip = f: a: b: f b a;
 

lib/types.nix

@@ -92,6 +92,8 @@ rec {
     };
 
 
+  # When adding new types don't forget to document them in
+  # nixos/doc/manual/development/option-types.xml!
   types = rec {
 
     unspecified = mkOptionType {
@@ -257,6 +259,7 @@ rec {
       functor = (defaultFunctor name) // { wrapped = elemType; };
     };
 
+    # Value of given type but with no merging (i.e. `uniq list`s are not concatenated).
     uniq = elemType: mkOptionType rec {
       name = "uniq";
       inherit (elemType) description check;
@@ -267,6 +270,7 @@ rec {
       functor = (defaultFunctor name) // { wrapped = elemType; };
     };
 
+    # Null or value of ...
     nullOr = elemType: mkOptionType rec {
       name = "nullOr";
       description = "null or ${elemType.description}";
@@ -283,6 +287,7 @@ rec {
       functor = (defaultFunctor name) // { wrapped = elemType; };
     };
 
+    # A submodule (like typed attribute set). See NixOS manual.
     submodule = opts:
       let
         opts' = toList opts;
@@ -314,6 +319,7 @@ rec {
         };
       };
 
+    # A value from a set of allowed ones.
     enum = values:
       let
         show = v:
@@ -329,6 +335,7 @@ rec {
         functor = (defaultFunctor name) // { payload = values; binOp = a: b: unique (a ++ b); };
       };
 
+    # Either value of type `t1` or `t2`.
     either = t1: t2: mkOptionType rec {
       name = "either";
       description = "${t1.description} or ${t2.description}";
@@ -352,6 +359,8 @@ rec {
       functor = (defaultFunctor name) // { wrapped = [ t1 t2 ]; };
     };
 
+    # Either value of type `finalType` or `coercedType`, the latter is
+    # converted to `finalType` using `coerceFunc`.
     coercedTo = coercedType: coerceFunc: finalType:
       assert coercedType.getSubModules == null;
       mkOptionType rec {

maintainers/scripts/rebuild-amount.sh

@@ -1,260 +1,115 @@
-#!/bin/sh
+#!/usr/bin/env bash
+set -e
 
-usage () {
-  echo 1>&2 "
-usage:
-  $0
-    [--git commit..commit | --git commit]
-    [--svn rev:rev | --svn rev]
-    [--path path[:path]*]
-    [--help]
-
-This program is used to investigate how any changes inside your nixpkgs
-repository may hurt.  With these kind of information you may choose wisely
-where you should commit your changes.
-
-This program adapts it-self to your versionning system to avoid too much
-effort on your Internet bandwidth.  If you need to check more than one
-commits / revisions, you may use the following commands:
-
-  --git remotes/trunk..master
-  --svn 17670:17677
-
-    Check the differences between each commit separating the first and the
-    last commit.
-
-  --path /etc/nixos/nixpkgs:/tmp/nixpkgs_1:/tmp/nixpkgs_2
-
-    Check the differences between multiple directories containing different
-    versions of nixpkgs.
-
-All these options exist with one commit / revision argument.  Such options
-are used to compare your \$NIXPKGS path with the specified version.
-
-If you omit to mention any other commit / revision, then your \$NIXPKGS path
-is compared with its last update.  This command is useful to test code from
-a dirty repository.
-
-"
-
-  exit 1;
-}
-
-#####################
-# Process Arguments #
-#####################
-
-: ${NIXPKGS=/etc/nixos/nixpkgs/}
-
-vcs=""
-gitCommits=""
-svnRevisions=""
-pathLocations=""
-verbose=false
-
-argfun=""
-for arg; do
-  if test -z "$argfun"; then
-    case $arg in
-      --git) vcs="git"; argfun="set_gitCommits";;
-      --svn) vcs="svn"; argfun="set_svnRevisions";;
-      --path) vcs="path"; argfun="set_pathLocations";;
-      --verbose) verbose=true;;
-      --help) usage;;
-      *) usage;;
-    esac
-  else
-    case $argfun in
-      set_*)
-        var=$(echo $argfun | sed 's,^set_,,')
-        eval $var=$arg
-        ;;
-    esac
-    argfun=""
-  fi
-done
-
-if $verbose; then
-  set -x
-else
-  set +x
+if [ "$#" != 1 ] && [ "$#" != 2 ]; then
+	cat <<-EOF
+	Usage: $0 commit-spec [commit-spec]
+	    You need to be in a git-controlled nixpkgs tree.
+	    The current state of the tree will be used if the second commit is missing.
+	EOF
+	exit 1
 fi
 
-############################
-# Find the repository type #
-############################
+# A slightly hacky way to get the config.
+parallel="$(echo 'config.rebuild-amount.parallel or false' | nix-repl . 2>/dev/null \
+			| grep -v '^\(nix-repl.*\)\?$' | tail -n 1 || true)"
 
-if test -z "$vcs"; then
-  if test -x "$NIXPKGS/.git"; then
-    if git --git-dir="$NIXPKGS/.git" branch > /dev/null 2>&1; then
-      vcs="git"
-      gitCommits=$(git --git-dir="$NIXPKGS/.git" log -n 1 --pretty=format:%H 2> /dev/null)
-    fi
-  elif test -x "$NIXPKGS/.svn"; then
-    cd "$NIXPKGS"
-    if svn info > /dev/null 2>&1; then
-      vcs="svn";
-      svnRevisions=$(svn info | sed -n 's,Revision: ,,p')
-    fi
-    cd -
-  else
-    usage
-  fi
-fi
+echo "Estimating rebuild amount by counting changed Hydra jobs."
 
-###############################
-# Define a storage directory. #
-###############################
+toRemove=()
 
-pkgListDir=""
-exitCode=1
-cleanup(){
-  test -e "$pkgListDir" && rm -rf "$pkgListDir"
-  exit $exitCode;
+cleanup() {
+	rm -rf "${toRemove[@]}"
 }
-
 trap cleanup EXIT SIGINT SIGQUIT ERR
 
-pkgListDir=$(mktemp --tmpdir -d rebuild-amount-XXXXXXXX)
-vcsDir="$pkgListDir/.vcs"
+MKTEMP='mktemp --tmpdir nix-rebuild-amount-XXXXXXXX'
 
-###########################
-# Versionning for Dummies #
-###########################
+nixexpr() {
+	cat <<-EONIX
+		let
+		  lib = import $1/lib;
+		  hydraJobs = import $1/pkgs/top-level/release.nix
+		    # Compromise: accuracy vs. resources needed for evaluation.
+		    { supportedSystems = cfg.systems or [ "x86_64-linux" "x86_64-darwin" ]; };
+		  cfg = (import $1 {}).config.rebuild-amount or {};
 
-path_init() {
-  if test "${pathLocations#*:}" = "$pathLocations"; then
-    pathLocations="$NIXPKGS:$pathLocations"
+		  recurseIntoAttrs = attrs: attrs // { recurseForDerivations = true; };
+
+		  # hydraJobs leaves recurseForDerivations as empty attrmaps;
+		  # that would break nix-env and we also need to recurse everywhere.
+		  tweak = lib.mapAttrs
+		    (name: val:
+		      if name == "recurseForDerivations" then true
+		      else if lib.isAttrs val && val.type or null != "derivation"
+		              then recurseIntoAttrs (tweak val)
+		      else val
+		    );
+
+		  # Some of these contain explicit references to platform(s) we want to avoid;
+		  # some even (transitively) depend on ~/.nixpkgs/config.nix (!)
+		  blacklist = [
+		    "tarball" "metrics" "manual"
+		    "darwin-tested" "unstable" "stdenvBootstrapTools"
+		    "moduleSystem" "lib-tests" # these just confuse the output
+		  ];
+		
+		in
+		  tweak (builtins.removeAttrs hydraJobs blacklist)
+	EONIX
+}
+
+# Output packages in tree $2 that weren't in $1.
+# Changing the output hash or name is taken as a change.
+# Extra nix-env parameters can be in $3
+newPkgs() {
+	# We use files instead of pipes, as running multiple nix-env processes
+	# could eat too much memory for a standard 4GiB machine.
+	local -a list
+	for i in 1 2; do
+		local l="$($MKTEMP)"
+		list[$i]="$l"
+		toRemove+=("$l")
+
+		local expr="$($MKTEMP)"
+		toRemove+=("$expr")
+		nixexpr "${!i}" > "$expr"
+
+		nix-env -f "$expr" -qaP --no-name --out-path --show-trace $3 \
+			| sort > "${list[$i]}" &
+
+		if [ "$parallel" != "true" ]; then
+			wait
 		fi
-  pathLocations="${pathLocations}:"
+	done
+
+	wait
+	comm -13 "${list[@]}"
 }
 
-path_getNext() {
-  pathLoc="${pathLocations%%:*}"
-  pathLocations="${pathLocations#*:}"
-}
+# Prepare nixpkgs trees.
+declare -a tree
+for i in 1 2; do
+	if [ -n "${!i}" ]; then # use the given commit
+		dir="$($MKTEMP -d)"
+		tree[$i]="$dir"
+		toRemove+=("$dir")
 
-path_setPath() {
-  path="$pathLoc"
-}
-
-path_setName() {
-  name=$(echo "$pathLoc" | tr '/' '_')
-}
-
-################
-# Git Commands #
-################
-
-git_init() {
-  git clone "$NIXPKGS/.git" "$vcsDir" > /dev/null 2>&1
-  if echo "gitCommits" | grep -c "\.\." > /dev/null 2>&1; then
-    gitCommits=$(git --git-dir="$vcsDir/.git" log --reverse --pretty=format:%H $gitCommits 2> /dev/null)
-  else
-    pathLocations="$vcsDir:$NIXPKGS"
-    vcs="path"
-    path_init
+		git clone --shared --no-checkout --quiet . "${tree[$i]}"
+		(cd "${tree[$i]}" && git checkout --quiet "${!i}")
+	else #use the current tree
+		tree[$i]="$(pwd)"
 	fi
-}
-
-git_getNext() {
-  git --git-dir="$vcsDir/.git" checkout $(echo "$gitCommits" | head -n 1) > /dev/null 2>&1
-  gitCommits=$(echo "$gitCommits" | sed '1 d')
-}
-
-git_setPath() {
-  path="$vcsDir"
-}
-
-git_setName() {
-  name=$(git --git-dir="$vcsDir/.git" log -n 1 --pretty=format:%H  2> /dev/null)
-}
-
-#######################
-# Subversion Commands #
-#######################
-
-svn_init() {
-  cp -r "$NIXPKGS" "$vcsDir" > /dev/null 2>&1
-  if echo "svnRevisions" | grep -c ":" > /dev/null 2>&1; then
-    svnRevisions=$(seq ${svnRevisions%:*} ${svnRevisions#*:})
-  else
-    pathLocations="$vcsDir:$NIXPKGS"
-    vcs="path"
-    path_init
-  fi
-}
-
-svn_getNext() {
-  cd "$vcsDir"
-  svn checkout $(echo "$svnRevisions" | head -n 1) > /dev/null 2>&1
-  cd -
-  svnRevisions=$(echo "$svnRevisions" | sed '1 d')
-}
-
-svn_setPath() {
-  path="$vcsDir"
-}
-
-svn_setName() {
-  name=$(svn info  2> /dev/null | sed -n 's,Revision: ,,p')
-}
-
-####################
-# Logical Commands #
-####################
-
-init    () { ${vcs}_init; }
-getNext () { ${vcs}_getNext; }
-setPath () { ${vcs}_setPath; }
-setName () { ${vcs}_setName; }
-
-
-#####################
-# Check for Rebuild #
-#####################
-
-# Generate the list of all derivations that could be build from a nixpkgs
-# respository.  This list of derivation hashes is compared with previous
-# lists and a brief summary is produced on the output.
-
-compareNames () {
-    nb=$(diff -y --suppress-common-lines --speed-large-files "$pkgListDir/$1.drvs" "$pkgListDir/$2.drvs" 2> /dev/null | wc -l)
-    echo "$1 -> $2: $nb"
-}
-
-echo "Please wait, this may take some minutes ..."
-
-init
-first=""
-oldPrev=""
-
-prev=""
-curr=""
-
-while true; do
-  getNext
-  setPath # set path=...
-  setName # set name=...
-  curr="$name"
-
-  test -z "$curr" && break || true
-
-  nix-instantiate "$path" > "$pkgListDir/$curr.drvs" > /dev/null 2>&1 || true
-
-  if test -n "$prev"; then
-    compareNames "$prev" "$curr"
-  else
-    echo "Number of package to rebuild:"
-    first="$curr"
-  fi
-  oldPrev="$prev"
-  prev="$curr"
 done
 
-if test "$first" != "$oldPrev"; then
-  echo "Number of package to rebuild (first -> last):"
-  compareNames "$first" "$curr"
-fi
+newlist="$($MKTEMP)"
+toRemove+=("$newlist")
+# Notes:
+#	- the evaluation is done on x86_64-linux, like on Hydra.
+#	- using $newlist file so that newPkgs() isn't in a sub-shell (because of toRemove)
+newPkgs "${tree[1]}" "${tree[2]}" '--argstr system "x86_64-linux"' > "$newlist"
+
+# Hacky: keep only the last word of each attribute path and sort.
+sed -n 's/\([^. ]*\.\)*\([^. ]*\) .*$/\2/p' < "$newlist" \
+	| sort | uniq -c
 
-exitCode=0

maintainers/scripts/update-python-libraries

@@ -0,0 +1,250 @@
+#! /usr/bin/env nix-shell
+#! nix-shell -i python3 -p 'python3.withPackages(ps: with ps; [ requests toolz ])'
+
+"""
+Update a Python package expression by passing in the `.nix` file, or the directory containing it.
+You can pass in multiple files or paths.
+
+You'll likely want to use
+``
+  $ ./update-python-libraries ../../pkgs/development/python-modules/*
+``
+to update all libraries in that folder.
+"""
+
+import argparse
+import logging
+import os
+import re
+import requests
+import toolz
+
+INDEX = "https://pypi.io/pypi"
+"""url of PyPI"""
+
+EXTENSIONS = ['tar.gz', 'tar.bz2', 'tar', 'zip', '.whl']
+"""Permitted file extensions. These are evaluated from left to right and the first occurance is returned."""
+
+def _get_value(attribute, text):
+    """Match attribute in text and return it."""
+    regex = '{}\s+=\s+"(.*)";'.format(attribute)
+    regex = re.compile(regex)
+    value = regex.findall(text)
+    n = len(value)
+    if n > 1:
+        raise ValueError("Found too many values for {}".format(attribute))
+    elif n == 1:
+        return value[0]
+    else:
+        raise ValueError("No value found for {}".format(attribute))
+
+def _get_line_and_value(attribute, text):
+    """Match attribute in text. Return the line and the value of the attribute."""
+    regex = '({}\s+=\s+"(.*)";)'.format(attribute)
+    regex = re.compile(regex)
+    value = regex.findall(text)
+    n = len(value)
+    if n > 1:
+        raise ValueError("Found too many values for {}".format(attribute))
+    elif n == 1:
+        return value[0]
+    else:
+        raise ValueError("No value found for {}".format(attribute))
+
+
+def _replace_value(attribute, value, text):
+    """Search and replace value of attribute in text."""
+    old_line, old_value = _get_line_and_value(attribute, text)
+    new_line = old_line.replace(old_value, value)
+    new_text = text.replace(old_line, new_line)
+    return new_text
+
+def _fetch_page(url):
+    r = requests.get(url)
+    if r.status_code == requests.codes.ok:
+        return r.json()
+    else:
+        raise ValueError("Request for {} failed".format(url))
+
+def _get_latest_version(package, extension):
+
+
+    url = "{}/{}/json".format(INDEX, package)
+    json = _fetch_page(url)
+
+    data = extract_relevant_nix_data(json, extension)[1]
+
+    version = data['latest_version']
+    if version in data['versions']:
+        sha256 = data['versions'][version]['sha256']
+    else:
+        sha256 = None   # Its possible that no file was uploaded to PyPI
+
+    return version, sha256
+
+
+def extract_relevant_nix_data(json, extension):
+    """Extract relevant Nix data from the JSON of a package obtained from PyPI.
+
+    :param json: JSON obtained from PyPI
+    """
+    def _extract_license(json):
+        """Extract license from JSON."""
+        return json['info']['license']
+
+    def _available_versions(json):
+        return json['releases'].keys()
+
+    def _extract_latest_version(json):
+        return json['info']['version']
+
+    def _get_src_and_hash(json, version, extensions):
+        """Obtain url and hash for a given version and list of allowable extensions."""
+        if not json['releases']:
+            msg = "Package {}: No releases available.".format(json['info']['name'])
+            raise ValueError(msg)
+        else:
+            # We use ['releases'] and not ['urls'] because we want to have the possibility for different version.
+            for possible_file in json['releases'][version]:
+                for extension in extensions:
+                    if possible_file['filename'].endswith(extension):
+                        src = {'url': str(possible_file['url']),
+                               'sha256': str(possible_file['digests']['sha256']),
+                                }
+                        return src
+            else:
+                msg = "Package {}: No release with valid file extension available.".format(json['info']['name'])
+                logging.info(msg)
+                return None
+                #raise ValueError(msg)
+
+    def _get_sources(json, extensions):
+        versions = _available_versions(json)
+        releases = {version: _get_src_and_hash(json, version, extensions) for version in versions}
+        releases = toolz.itemfilter(lambda x: x[1] is not None, releases)
+        return releases
+
+    # Collect data)
+    name = str(json['info']['name'])
+    latest_version = str(_extract_latest_version(json))
+    #src = _get_src_and_hash(json, latest_version, EXTENSIONS)
+    sources = _get_sources(json, [extension])
+
+    # Collect meta data
+    license = str(_extract_license(json))
+    license = license if license != "UNKNOWN" else None
+    summary = str(json['info'].get('summary')).strip('.')
+    summary = summary if summary != "UNKNOWN" else None
+    #description = str(json['info'].get('description'))
+    #description = description if description != "UNKNOWN" else None
+    homepage = json['info'].get('home_page')
+
+    data = {
+        'latest_version'    : latest_version,
+        'versions'  : sources,
+        #'src'           : src,
+        'meta'          : {
+            'description'            : summary if summary else None,
+            #'longDescription'        : description,
+            'license'                : license,
+            'homepage'               : homepage,
+            },
+        }
+    return name, data
+
+
+def _update_package(path):
+
+    # We need to read and modify a Nix expression.
+    if os.path.isdir(path):
+        path = os.path.join(path, 'default.nix')
+
+    if not os.path.isfile(path):
+        logging.warning("Path does not exist: {}".format(path))
+        return False
+
+    if not path.endswith(".nix"):
+        logging.warning("Path does not end with `.nix`, skipping: {}".format(path))
+        return False
+
+    with open(path, 'r') as f:
+        text = f.read()
+
+    try:
+        pname = _get_value('pname', text)
+    except ValueError as e:
+        logging.warning("Path {}: {}".format(path, str(e)))
+        return False
+
+    try:
+        version = _get_value('version', text)
+    except ValueError as e:
+        logging.warning("Path {}: {}".format(path, str(e)))
+        return False
+
+    # If we use a wheel, then we need to request a wheel as well
+    try:
+        format = _get_value('format', text)
+    except ValueError as e:
+        # No format mentioned, then we assume we have setuptools
+        # and use a .tar.gz
+        logging.info("Path {}: {}".format(path, str(e)))
+        extension = ".tar.gz"
+    else:
+        if format == 'wheel':
+            extension = ".whl"
+        else:
+            try:
+                url = _get_value('url', text)
+                extension = os.path.splitext(url)[1]
+                if 'pypi' not in url:
+                    logging.warning("Path {}: uses non-PyPI url, not updating.".format(path))
+                    return False
+            except ValueError as e:
+                logging.info("Path {}: {}".format(path, str(e)))
+                extension = ".tar.gz"
+
+    try:
+        new_version, new_sha256 = _get_latest_version(pname, extension)
+    except ValueError as e:
+        logging.warning("Path {}: {}".format(path, str(e)))
+    else:
+        if not new_sha256:
+            logging.warning("Path has no valid file available: {}".format(path))
+            return False
+        if new_version != version:
+            try:
+                text = _replace_value('version', new_version, text)
+            except ValueError as e:
+                logging.warning("Path {}: {}".format(path, str(e)))
+            try:
+                text = _replace_value('sha256', new_sha256, text)
+            except ValueError as e:
+                logging.warning("Path {}: {}".format(path, str(e)))
+
+            with open(path, 'w') as f:
+                f.write(text)
+
+            logging.info("Updated {} from {} to {}".format(pname, version, new_version))
+
+        else:
+            logging.info("No update available for {} at {}".format(pname, version))
+
+    return True
+
+
+def main():
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument('package', type=str, nargs='+')
+
+    args = parser.parse_args()
+
+    packages = args.package
+
+    count = list(map(_update_package, packages))
+
+    #logging.info("{} package(s) updated".format(sum(count)))
+
+if __name__ == '__main__':
+    main()

nixos/doc/manual/development/option-types.xml

@@ -68,8 +68,7 @@
 
  <section><title>Value Types</title>
 
-   <para>Value types are type that take a value parameter. The only value type 
-     in the library is <literal>enum</literal>.</para>
+   <para>Value types are type that take a value parameter.</para>
 
 <variablelist>
   <varlistentry>
@@ -141,6 +140,17 @@
           str</literal>. Multiple definitions cannot be 
         merged.</para></listitem>
   </varlistentry>
+  <varlistentry>
+    <term><varname>types.coercedTo</varname> <replaceable>from</replaceable>
+        <replaceable>f</replaceable> <replaceable>to</replaceable></term>
+    <listitem><para>Type <replaceable>to</replaceable> or type
+        <replaceable>from</replaceable> which will be coerced to
+	type <replaceable>to</replaceable> using function
+	<replaceable>f</replaceable> which takes an argument of type
+        <replaceable>from</replaceable> and return a value of type
+	<replaceable>to</replaceable>. Can be used to preserve backwards
+        compatibility of an option if its type was changed.</para></listitem>
+  </varlistentry>
 </variablelist>
 
 </section>

nixos/doc/manual/release-notes/rl-1709.xml

@@ -68,6 +68,27 @@ following incompatible changes:</para>
       <literal>db-config.sqlite</literal> which will be automatically recreated.
     </para>
   </listitem>
+  <listitem>
+    <para>
+      The ipfs package now doesn't ignore the <literal>dataDir</literal> option anymore. If you've ever set this option to anything other than the default you'll have to either unset it (so the default gets used) or migrate the old data manually with
+<programlisting>
+dataDir=&lt;valueOfDataDir&gt;
+mv /var/lib/ipfs/.ipfs/* $dataDir
+rmdir /var/lib/ipfs/.ipfs
+</programlisting>
+    </para>
+  </listitem>
+  <listitem>
+    <para>
+      The <literal>postgres</literal> default version was changed from 9.5 to 9.6.
+    </para>
+    <para>
+      The <literal>postgres</literal> superuser name has changed from <literal>root</literal> to <literal>postgres</literal> to more closely follow what other Linux distributions are doing.
+    </para>
+    <para>
+      The <literal>postgres</literal> default <literal>dataDir</literal> has changed from <literal>/var/db/postgres</literal> to <literal>/var/lib/postgresql/$psqlSchema</literal> where $psqlSchema is 9.6 for example.
+    </para>
+  </listitem>
 </itemizedlist>
 
 

nixos/modules/config/networking.nix

@@ -222,13 +222,11 @@ in
             '' + cfg.extraResolvconfConf + ''
             '';
 
-      } // (optionalAttrs config.services.resolved.enable (
-        if dnsmasqResolve then {
-          "dnsmasq-resolv.conf".source = "/run/systemd/resolve/resolv.conf";
-        } else {
+      } // optionalAttrs config.services.resolved.enable {
         "resolv.conf".source = "/run/systemd/resolve/resolv.conf";
-        }
-      ));
+      } // optionalAttrs (config.services.resolved.enable && dnsmasqResolve) {
+        "dnsmasq-resolv.conf".source = "/run/systemd/resolve/resolv.conf";
+      };
 
       networking.proxy.envVars =
         optionalAttrs (cfg.proxy.default != null) {

nixos/modules/config/nsswitch.nix

@@ -10,10 +10,12 @@ let
   inherit (config.services.samba) nsswins;
   ldap = (config.users.ldap.enable && config.users.ldap.nsswitch);
   sssd = config.services.sssd.enable;
+  resolved = config.services.resolved.enable;
 
   hostArray = [ "files" "mymachines" ]
     ++ optionals nssmdns [ "mdns_minimal [!UNAVAIL=return]" ]
     ++ optionals nsswins [ "wins" ]
+    ++ optionals resolved ["resolv [!UNAVAIL=return]"]
     ++ [ "dns" ]
     ++ optionals nssmdns [ "mdns" ]
     ++ ["myhostname" ];

nixos/modules/config/pulseaudio.nix

@@ -240,11 +240,14 @@ in {
       };
       systemd.user = {
         services.pulseaudio = {
+          restartIfChanged = true;
           serviceConfig = {
             RestartSec = "500ms";
+            PassEnvironment = "DISPLAY";
           };
-          environment = { DISPLAY = ":${toString config.services.xserver.display}"; };
-          restartIfChanged = true;
+        };
+        sockets.pulseaudio = {
+          wantedBy = [ "sockets.target" ];
         };
       };
     })

nixos/modules/hardware/nitrokey.nix

@@ -0,0 +1,41 @@
+{ config, lib, pkgs, ... }:
+
+with lib;
+
+let
+
+  cfg = config.hardware.nitrokey;
+
+in
+
+{
+  options.hardware.nitrokey = {
+    enable = mkOption {
+      type = types.bool;
+      default = false;
+      description = ''
+        Enables udev rules for Nitrokey devices. By default grants access
+        to users in the "nitrokey" group. You may want to install the
+        nitrokey-app package, depending on your device and needs.
+      '';
+    };
+
+    group = mkOption {
+      type = types.str;
+      default = "nitrokey";
+      example = "wheel";
+      description = ''
+        Grant access to Nitrokey devices to users in this group.
+      '';
+    };
+  };
+
+  config = mkIf cfg.enable {
+    services.udev.packages = [
+      (pkgs.nitrokey-udev-rules.override (attrs:
+        { inherit (cfg) group; }
+      ))
+    ];
+    users.extraGroups."${cfg.group}" = {};
+  };
+}

nixos/modules/hardware/video/nvidia.nix

@@ -28,7 +28,7 @@ let
   nvidia_libs32 = (nvidiaForKernel pkgs_i686.linuxPackages).override { libsOnly = true; kernel = null; };
 
   nvidiaPackage = nvidia: pkgs:
-    if !nvidia.useGLVND then nvidia
+    if !nvidia.useGLVND then nvidia.out
     else pkgs.buildEnv {
       name = "nvidia-libs";
       paths = [ pkgs.libglvnd nvidia.out ];
@@ -56,7 +56,8 @@ in
     hardware.opengl.package = nvidiaPackage nvidia_x11 pkgs;
     hardware.opengl.package32 = nvidiaPackage nvidia_libs32 pkgs_i686;
 
-    environment.systemPackages = [ nvidia_x11.bin nvidia_x11.settings nvidia_x11.persistenced ];
+    environment.systemPackages = [ nvidia_x11.bin nvidia_x11.settings ]
+      ++ lib.filter (p: p != null) [ nvidia_x11.persistenced ];
 
     boot.extraModulePackages = [ nvidia_x11.bin ];
 

nixos/modules/module-list.nix

@@ -39,6 +39,7 @@
   ./hardware/network/intel-3945abg.nix
   ./hardware/network/ralink.nix
   ./hardware/network/rtl8192c.nix
+  ./hardware/nitrokey.nix
   ./hardware/opengl.nix
   ./hardware/pcmcia.nix
   ./hardware/usb-wwan.nix
@@ -79,6 +80,7 @@
   ./programs/environment.nix
   ./programs/fish.nix
   ./programs/freetds.nix
+  ./programs/gnupg.nix
   ./programs/gphoto2.nix
   ./programs/info.nix
   ./programs/java.nix
@@ -129,6 +131,8 @@
   ./security/rtkit.nix
   ./security/wrappers/default.nix
   ./security/sudo.nix
+  ./service-managers/docker.nix
+  ./service-managers/trivial.nix
   ./services/admin/salt/master.nix
   ./services/admin/salt/minion.nix
   ./services/amqp/activemq/default.nix
@@ -239,8 +243,9 @@
   ./services/logging/logrotate.nix
   ./services/logging/logstash.nix
   ./services/logging/rsyslogd.nix
-  ./services/logging/syslogd.nix
+  ./services/logging/SystemdJournal2Gelf.nix
   ./services/logging/syslog-ng.nix
+  ./services/logging/syslogd.nix
   ./services/mail/dovecot.nix
   ./services/mail/dspam.nix
   ./services/mail/exim.nix

nixos/modules/programs/environment.nix

@@ -20,7 +20,7 @@ in
       { NIXPKGS_CONFIG = "/etc/nix/nixpkgs-config.nix";
         PAGER = mkDefault "less -R";
         EDITOR = mkDefault "nano";
-        XCURSOR_PATH = "$HOME/.icons";
+        XCURSOR_PATH = [ "$HOME/.icons" ];
       };
 
     environment.profiles =

nixos/modules/programs/gnupg.nix

@@ -0,0 +1,75 @@
+{ config, lib, pkgs, ... }:
+
+with lib;
+
+let
+
+  cfg = config.programs.gnupg;
+
+in
+
+{
+
+  options.programs.gnupg = {
+    agent.enable = mkOption {
+      type = types.bool;
+      default = false;
+      description = ''
+        Enables GnuPG agent with socket-activation for every user session.
+      '';
+    };
+
+    agent.enableSSHSupport = mkOption {
+      type = types.bool;
+      default = true;
+      description = ''
+        Enable SSH agent support in GnuPG agent. Also sets SSH_AUTH_SOCK
+        environment variable correctly. This will disable socket-activation
+        and thus always start a GnuPG agent per user session.
+      '';
+    };
+  };
+
+  config = mkIf cfg.agent.enable {
+    systemd.user.services.gpg-agent = {
+      serviceConfig = {
+        ExecStart = [
+          ""
+          ("${pkgs.gnupg}/bin/gpg-agent --supervised "
+            + optionalString cfg.agent.enableSSHSupport "--enable-ssh-support")
+        ];
+      };
+    };
+
+    systemd.user.sockets.gpg-agent = {
+      wantedBy = [ "sockets.target" ];
+    };
+
+    systemd.user.sockets.gpg-agent-ssh = mkIf cfg.agent.enableSSHSupport {
+      wantedBy = [ "sockets.target" ];
+    };
+
+    systemd.packages = [ pkgs.gnupg ];
+
+    environment.extraInit = ''
+      # Bind gpg-agent to this TTY if gpg commands are used.
+      export GPG_TTY=$(tty)
+
+    '' + (optionalString cfg.agent.enableSSHSupport ''
+      # SSH agent protocol doesn't support changing TTYs, so bind the agent
+      # to every new TTY.
+      ${pkgs.gnupg}/bin/gpg-connect-agent --quiet updatestartuptty /bye > /dev/null
+
+      if [ -z "$SSH_AUTH_SOCK" ]; then
+        export SSH_AUTH_SOCK=$(${pkgs.gnupg}/bin/gpgconf --list-dirs agent-ssh-socket)
+      fi
+    '');
+
+    assertions = [
+      { assertion = cfg.agent.enableSSHSupport && !config.programs.ssh.startAgent;
+        message = "You can't use ssh-agent and GnuPG agent with SSH support enabled at the same time!";
+      }
+    ];
+  };
+
+}

nixos/modules/programs/ssh.nix

@@ -199,9 +199,8 @@ in
     environment.etc."ssh/ssh_known_hosts".text = knownHostsText;
 
     # FIXME: this should really be socket-activated for über-awesomeness.
-    systemd.user.services.ssh-agent =
-      { enable = cfg.startAgent;
-        description = "SSH Agent";
+    systemd.user.services.ssh-agent = mkIf cfg.startAgent
+      { description = "SSH Agent";
         wantedBy = [ "default.target" ];
         serviceConfig =
           { ExecStartPre = "${pkgs.coreutils}/bin/rm -f %t/ssh-agent";

nixos/modules/programs/zsh/oh-my-zsh.nix

@@ -3,11 +3,11 @@
 with lib;
 
 let
-  cfg = config.programs.zsh.oh-my-zsh;
+  cfg = config.programs.zsh.ohMyZsh;
 in
   {
     options = {
-      programs.zsh.oh-my-zsh = {
+      programs.zsh.ohMyZsh = {
         enable = mkOption {
           default = false;
           description = ''

nixos/modules/rename.nix

@@ -210,5 +210,9 @@ with lib;
     (mkRenamedOptionModule [ "programs" "zsh" "syntax-highlighting" "enable" ] [ "programs" "zsh" "syntaxHighlighting" "enable" ])
     (mkRenamedOptionModule [ "programs" "zsh" "syntax-highlighting" "highlighters" ] [ "programs" "zsh" "syntaxHighlighting" "highlighters" ])
     (mkRenamedOptionModule [ "programs" "zsh" "syntax-highlighting" "patterns" ] [ "programs" "zsh" "syntaxHighlighting" "patterns" ])
+    (mkRenamedOptionModule [ "programs" "zsh" "oh-my-zsh" "enable" ] [ "programs" "zsh" "ohMyZsh" "enable" ])
+    (mkRenamedOptionModule [ "programs" "zsh" "oh-my-zsh" "theme" ] [ "programs" "zsh" "ohMyZsh" "theme" ])
+    (mkRenamedOptionModule [ "programs" "zsh" "oh-my-zsh" "custom" ] [ "programs" "zsh" "ohMyZsh" "custom" ])
+    (mkRenamedOptionModule [ "programs" "zsh" "oh-my-zsh" "plugins" ] [ "programs" "zsh" "ohMyZsh" "plugins" ])
   ];
 }

nixos/modules/service-managers/docker.nix

@@ -0,0 +1,29 @@
+{ config, lib, pkgs, ... }:
+
+with lib;
+
+let
+  cfg = config.docker-containers;
+
+  containerModule = {
+    script = mkOption {
+      type = types.lines;
+      description = "Shell commands executed as the service's main process.";
+    };
+  };
+
+  toContainer = name: value: pkgs.dockerTools.buildImage {
+    inherit name;
+    config = {
+      Cmd = [ value.script ];
+    };
+  };
+in {
+  options.docker-containers = mkOption {
+    default = {};
+    type = with types; attrsOf (types.submodule containerModule);
+    description = "Definition of docker containers";
+  };
+
+  config.system.build.toplevel-docker = lib.mapAttrs toContainer cfg;
+}

nixos/modules/service-managers/trivial.nix

@@ -0,0 +1,35 @@
+{ config, lib, pkgs, ... }:
+
+with lib;
+
+let
+  cfg = config.trivial-services;
+
+  serviceModule.options = {
+    script = mkOption {
+      type = types.lines;
+      description = "Shell commands executed as the service's main process.";
+    };
+
+    environment = mkOption {
+      default = {};
+      type = types.attrs; # FIXME
+      example = { PATH = "/foo/bar/bin"; LANG = "nl_NL.UTF-8"; };
+      description = "Environment variables passed to the service's processes.";
+    };
+  };
+
+  launcher = name: value: pkgs.writeScript name ''
+    #!${pkgs.stdenv.shell} -eu
+
+    ${pkgs.writeScript "${name}-entry" value.script}
+  '';
+in {
+  options.trivial-services = mkOption {
+    default = {};
+    type = with types; attrsOf (types.submodule serviceModule);
+    description = "Definition of trivial services";
+  };
+
+  config.system.build.toplevel-trivial = lib.mapAttrs launcher cfg;
+}

nixos/modules/services/databases/mysql.nix

@@ -72,7 +72,7 @@ in
 
       dataDir = mkOption {
         type = types.path;
-        default = "/var/mysql"; # !!! should be /var/db/mysql
+        example = "/var/lib/mysql";
         description = "Location where MySQL stores its table files";
       };
 
@@ -166,6 +166,10 @@ in
 
   config = mkIf config.services.mysql.enable {
 
+    services.mysql.dataDir =
+      mkDefault (if versionAtLeast config.system.stateVersion "17.09" then "/var/lib/mysql"
+                 else "/var/mysql");
+
     users.extraUsers.mysql = {
       description = "MySQL server user";
       group = "mysql";

nixos/modules/services/databases/postgresql.nix

@@ -38,6 +38,10 @@ let
 
   pre84 = versionOlder (builtins.parseDrvName postgresql.name).version "8.4";
 
+  # NixOS traditionally used `root` as superuser, most other distros use `postgres`. From 17.09
+  # we also try to follow this standard
+  superuser = (if versionAtLeast config.system.stateVersion "17.09" then "postgres" else "root");
+
 in
 
 {
@@ -74,7 +78,7 @@ in
 
       dataDir = mkOption {
         type = types.path;
-        default = "/var/db/postgresql";
+        example = "/var/lib/postgresql/9.6";
         description = ''
           Data directory for PostgreSQL.
         '';
@@ -160,7 +164,13 @@ in
       # Note: when changing the default, make it conditional on
       # ‘system.stateVersion’ to maintain compatibility with existing
       # systems!
-      mkDefault (if versionAtLeast config.system.stateVersion "16.03" then pkgs.postgresql95 else pkgs.postgresql94);
+      mkDefault (if versionAtLeast config.system.stateVersion "17.09" then pkgs.postgresql96
+            else if versionAtLeast config.system.stateVersion "16.03" then pkgs.postgresql95
+            else pkgs.postgresql94);
+
+    services.postgresql.dataDir =
+      mkDefault (if versionAtLeast config.system.stateVersion "17.09" then "/var/lib/postgresql/${config.services.postgresql.package.psqlSchema}"
+                 else "/var/db/postgresql");
 
     services.postgresql.authentication = mkAfter
       ''
@@ -205,7 +215,7 @@ in
           ''
             # Initialise the database.
             if ! test -e ${cfg.dataDir}/PG_VERSION; then
-              initdb -U root
+              initdb -U ${superuser}
               # See postStart!
               touch "${cfg.dataDir}/.first_startup"
             fi
@@ -237,14 +247,14 @@ in
         # Wait for PostgreSQL to be ready to accept connections.
         postStart =
           ''
-            while ! psql --port=${toString cfg.port} postgres -c "" 2> /dev/null; do
+            while ! ${pkgs.sudo}/bin/sudo -u ${superuser} psql --port=${toString cfg.port} -d postgres -c "" 2> /dev/null; do
                 if ! kill -0 "$MAINPID"; then exit 1; fi
                 sleep 0.1
             done
 
             if test -e "${cfg.dataDir}/.first_startup"; then
               ${optionalString (cfg.initialScript != null) ''
-                psql -f "${cfg.initialScript}" --port=${toString cfg.port} postgres
+                ${pkgs.sudo}/bin/sudo -u ${superuser} psql -f "${cfg.initialScript}" --port=${toString cfg.port} -d postgres
               ''}
               rm -f "${cfg.dataDir}/.first_startup"
             fi

nixos/modules/services/logging/SystemdJournal2Gelf.nix

@@ -0,0 +1,59 @@
+{ config, lib, pkgs, ... }:
+
+with lib;
+
+let cfg = config.services.SystemdJournal2Gelf;
+in
+
+{ options = {
+    services.SystemdJournal2Gelf = {
+      enable = mkOption {
+        type = types.bool;
+        default = false;
+        description = ''
+          Whether to enable SystemdJournal2Gelf.
+        '';
+      };
+
+      graylogServer = mkOption {
+        type = types.string;
+        example = "graylog2.example.com:11201";
+        description = ''
+          Host and port of your graylog2 input. This should be a GELF
+          UDP input.
+        '';
+      };
+
+      extraOptions = mkOption {
+        type = types.string;
+        default = "";
+        description = ''
+          Any extra flags to pass to SystemdJournal2Gelf. Note that
+          these are basically <literal>journalctl</literal> flags.
+        '';
+      };
+
+      package = mkOption {
+        type = types.package;
+        default = pkgs.systemd-journal2gelf;
+        description = ''
+          SystemdJournal2Gelf package to use.
+        '';
+      };
+
+    };
+  };
+
+  config = mkIf cfg.enable {
+    systemd.services.SystemdJournal2Gelf = {
+      description = "SystemdJournal2Gelf";
+      after = [ "network.target" ];
+      wantedBy = [ "multi-user.target" ];
+      serviceConfig = {
+        ExecStart = "${cfg.package}/bin/SystemdJournal2Gelf ${cfg.graylogServer} --follow ${cfg.extraOptions}";
+        Restart = "on-failure";
+        RestartSec = "30";
+      };
+    };
+  };
+}

nixos/modules/services/mail/mlmmj.nix

@@ -4,6 +4,8 @@ with lib;
 
 let
 
+  concatMapLines = f: l: lib.concatStringsSep "\n" (map f l);
+
   cfg = config.services.mlmmj;
   stateDir = "/var/lib/mlmmj";
   spoolDir = "/var/spool/mlmmj";
@@ -16,12 +18,23 @@ let
   listAddress = domain: list: "${list}@${domain}";
   customHeaders = domain: list: [ "List-Id: ${list}" "Reply-To: ${list}@${domain}" ];
   footer = domain: list: "To unsubscribe send a mail to ${list}+unsubscribe@${domain}";
-  createList = d: l: ''
-    ${pkgs.coreutils}/bin/mkdir -p ${listCtl d l}
-    echo ${listAddress d l} > ${listCtl d l}/listaddress
-    echo "${lib.concatStringsSep "\n" (customHeaders d l)}" > ${listCtl d l}/customheaders
-    echo ${footer d l} > ${listCtl d l}/footer
-    echo ${subjectPrefix l} > ${listCtl d l}/prefix
+  createList = d: l:
+    let ctlDir = listCtl d l; in
+    ''
+      for DIR in incoming queue queue/discarded archive text subconf unsubconf \
+                 bounce control moderation subscribers.d digesters.d requeue \
+                 nomailsubs.d
+      do
+             mkdir -p '${listDir d l}'/"$DIR"
+      done
+      ${pkgs.coreutils}/bin/mkdir -p ${ctlDir}
+      echo ${listAddress d l} > '${ctlDir}/listaddress'
+      [ ! -e ${ctlDir}/customheaders ] && \
+          echo "${lib.concatStringsSep "\n" (customHeaders d l)}" > '${ctlDir}/customheaders'
+      [ ! -e ${ctlDir}/footer ] && \
+          echo ${footer d l} > '${ctlDir}/footer'
+      [ ! -e ${ctlDir}/prefix ] && \
+          echo ${subjectPrefix l} > '${ctlDir}/prefix'
     '';
 in
 
@@ -63,6 +76,16 @@ in
         description = "The collection of hosted maillists";
       };
 
+      maintInterval = mkOption {
+        type = types.str;
+        default = "20min";
+        description = ''
+          Time interval between mlmmj-maintd runs, see
+          <citerefentry><refentrytitle>systemd.time</refentrytitle>
+          <manvolnum>7</manvolnum></citerefentry> for format information.
+        '';
+      };
+
     };
 
   };
@@ -93,7 +116,7 @@ in
         mlmmj unix - n n - - pipe flags=ORhu user=mlmmj argv=${pkgs.mlmmj}/bin/mlmmj-receive -F -L ${spoolDir}/$nexthop
       '';
 
-      extraAliases = concatMapStrings (alias cfg.listDomain) cfg.mailLists;
+      extraAliases = concatMapLines (alias cfg.listDomain) cfg.mailLists;
 
       extraConfig = ''
         transport_maps = hash:${stateDir}/transports
@@ -107,17 +130,15 @@ in
     system.activationScripts.mlmmj = ''
           ${pkgs.coreutils}/bin/mkdir -p ${stateDir} ${spoolDir}/${cfg.listDomain}
           ${pkgs.coreutils}/bin/chown -R ${cfg.user}:${cfg.group} ${spoolDir}
-          ${lib.concatMapStrings (createList cfg.listDomain) cfg.mailLists}
-          echo ${lib.concatMapStrings (virtual cfg.listDomain) cfg.mailLists} > ${stateDir}/virtuals
-          echo ${lib.concatMapStrings (transport cfg.listDomain) cfg.mailLists} > ${stateDir}/transports
+          ${concatMapLines (createList cfg.listDomain) cfg.mailLists}
+          echo "${concatMapLines (virtual cfg.listDomain) cfg.mailLists}" > ${stateDir}/virtuals
+          echo "${concatMapLines (transport cfg.listDomain) cfg.mailLists}" > ${stateDir}/transports
           ${pkgs.postfix}/bin/postmap ${stateDir}/virtuals
           ${pkgs.postfix}/bin/postmap ${stateDir}/transports
       '';
 
     systemd.services."mlmmj-maintd" = {
       description = "mlmmj maintenance daemon";
-      wantedBy = [ "multi-user.target" ];
-
       serviceConfig = {
         User = cfg.user;
         Group = cfg.group;
@@ -125,6 +146,11 @@ in
       };
     };
 
+    systemd.timers."mlmmj-maintd" = {
+      description = "mlmmj maintenance timer";
+      timerConfig.OnUnitActiveSec = cfg.maintInterval;
+      wantedBy = [ "timers.target" ];
+    };
   };
 
 }

nixos/modules/services/monitoring/arbtt.nix

@@ -48,7 +48,8 @@ in {
   config = mkIf cfg.enable {
     systemd.user.services.arbtt = {
       description = "arbtt statistics capture service";
-      wantedBy = [ "default.target" ];
+      wantedBy = [ "graphical-session.target" ];
+      partOf = [ "graphical-session.target" ];
 
       serviceConfig = {
         Type = "simple";

nixos/modules/services/monitoring/dd-agent/dd-agent-defaults.nix

@@ -1,8 +0,0 @@
-# Generated using update-dd-agent-default, please re-run after updating dd-agent. DO NOT EDIT MANUALLY.
-[
-  "auto_conf"
-  "agent_metrics.yaml.default"
-  "disk.yaml.default"
-  "network.yaml.default"
-  "ntp.yaml.default"
-]

nixos/modules/services/monitoring/dd-agent/dd-agent.nix

@@ -16,100 +16,24 @@ let
     forwarder_log_file: /var/log/datadog/forwarder.log
     dogstatsd_log_file: /var/log/datadog/dogstatsd.log
     pup_log_file:       /var/log/datadog/pup.log
-
-    # proxy_host: my-proxy.com
-    # proxy_port: 3128
-    # proxy_user: user
-    # proxy_password: password
-
-    # tags: mytag0, mytag1
     ${optionalString (cfg.tags != null ) "tags: ${concatStringsSep "," cfg.tags }"}
-
-    # collect_ec2_tags: no
-    # recent_point_threshold: 30
-    # use_mount: no
-    # listen_port: 17123
-    # graphite_listen_port: 17124
-    # non_local_traffic: no
-    # use_curl_http_client: False
-    # bind_host: localhost
-
-    # use_pup: no
-    # pup_port: 17125
-    # pup_interface: localhost
-    # pup_url: http://localhost:17125
-
-    # dogstatsd_port : 8125
-    # dogstatsd_interval : 10
-    # dogstatsd_normalize : yes
-    # statsd_forward_host: address_of_own_statsd_server
-    # statsd_forward_port: 8125
-
-    # device_blacklist_re: .*\/dev\/mapper\/lxc-box.*
-
-    # ganglia_host: localhost
-    # ganglia_port: 8651
+    ${cfg.extraDdConfig}
   '';
 
-  diskConfig = pkgs.writeText "disk.yaml" ''
-    init_config:
-
-    instances:
-      - use_mount: no
-  '';
-  
-  networkConfig = pkgs.writeText "network.yaml" ''
-    init_config:
-
-    instances:
-      # Network check only supports one configured instance
-      - collect_connection_state: false
-        excluded_interfaces:
-          - lo
-          - lo0
-  '';
-  
-  postgresqlConfig = pkgs.writeText "postgres.yaml" cfg.postgresqlConfig;
-  nginxConfig = pkgs.writeText "nginx.yaml" cfg.nginxConfig;
-  mongoConfig = pkgs.writeText "mongo.yaml" cfg.mongoConfig;
-  jmxConfig = pkgs.writeText "jmx.yaml" cfg.jmxConfig;
-  processConfig = pkgs.writeText "process.yaml" cfg.processConfig;
-  
   etcfiles =
-    let
-      defaultConfd = import ./dd-agent-defaults.nix;
-    in (map (f: { source = "${pkgs.dd-agent}/agent/conf.d-system/${f}";
-                  target = "dd-agent/conf.d/${f}";
-                }) defaultConfd) ++ [
-      { source = ddConf;
+    map (i: { source = if builtins.hasAttr "config" i
+                       then pkgs.writeText "${i.name}.yaml" i.config
+                       else "${cfg.agent}/agent/conf.d-system/${i.name}.yaml";
+              target = "dd-agent/conf.d/${i.name}.yaml";
+            }
+        ) cfg.integrations ++
+        [ { source = ddConf;
             target = "dd-agent/datadog.conf";
           }
-      { source = diskConfig;
-        target = "dd-agent/conf.d/disk.yaml";
-      }
-      { source = networkConfig;
-        target = "dd-agent/conf.d/network.yaml";
-      } ] ++
-    (optional (cfg.postgresqlConfig != null)
-      { source = postgresqlConfig;
-        target = "dd-agent/conf.d/postgres.yaml";
-      }) ++
-    (optional (cfg.nginxConfig != null)
-      { source = nginxConfig;
-        target = "dd-agent/conf.d/nginx.yaml";
-      }) ++
-    (optional (cfg.mongoConfig != null)
-      { source = mongoConfig;
-        target = "dd-agent/conf.d/mongo.yaml";
-      }) ++
-    (optional (cfg.processConfig != null)
-      { source = processConfig;
-        target = "dd-agent/conf.d/process.yaml";
-      }) ++
-    (optional (cfg.jmxConfig != null)
-      { source = jmxConfig;
-        target = "dd-agent/conf.d/jmx.yaml";
-      });
+        ];
+
+  # restart triggers
+  etcSources = map (i: i.source) etcfiles;
 
 in {
   options.services.dd-agent = {
@@ -139,44 +63,46 @@ in {
       type = types.uniq (types.nullOr types.string);
     };
 
-    postgresqlConfig = mkOption {
-      description = "Datadog PostgreSQL integration configuration";
-      default = null;
-      type = types.uniq (types.nullOr types.string);
+    agent = mkOption {
+      description = "The dd-agent package to use. Useful when overriding the package.";
+      default = pkgs.dd-agent;
+      type = types.package;
     };
 
-    nginxConfig = mkOption {
-      description = "Datadog nginx integration configuration";
-      default = null;
-      type = types.uniq (types.nullOr types.string);
-    };
-    
-    mongoConfig = mkOption {
-      description = "MongoDB integration configuration";
-      default = null;
-      type = types.uniq (types.nullOr types.string);
-    };
-
-    jmxConfig = mkOption {
-      description = "JMX integration configuration";
-      default = null;
-      type = types.uniq (types.nullOr types.string);
-    };
-
-    processConfig = mkOption {
+    integrations = mkOption {
       description = ''
-        Process integration configuration
- 
-        See http://docs.datadoghq.com/integrations/process/
+        Any integrations to use. Default config used if none
+        specified. It is currently up to the user to make sure that
+        the dd-agent package used has all the dependencies chosen
+        integrations require in scope.
+      '';
+      type = types.listOf (types.attrsOf types.string);
+      default = [];
+      example = ''
+        [ { name = "elastic";
+            config = '''
+              init_config:
+
+              instances:
+                - url: http://localhost:9200
+            ''';
+          }
+          { name = "nginx"; }
+          { name = "ntp"; }
+          { name = "network"; }
+        ]
       '';
-      default = null;
-      type = types.uniq (types.nullOr types.string);
     };
 
+    extraDdConfig = mkOption {
+      description = "Extra settings to append to datadog agent config.";
+      default = "";
+      type = types.string;
+    };
   };
 
   config = mkIf cfg.enable {
-    environment.systemPackages = [ pkgs."dd-agent" pkgs.sysstat pkgs.procps ];
+    environment.systemPackages = [ cfg.agent pkgs.sysstat pkgs.procps ];
 
     users.extraUsers.datadog = {
       description = "Datadog Agent User";
@@ -190,46 +116,30 @@ in {
 
     systemd.services.dd-agent = {
       description = "Datadog agent monitor";
-      path = [ pkgs."dd-agent" pkgs.python pkgs.sysstat pkgs.procps ];
+      path = [ cfg.agent pkgs.python pkgs.sysstat pkgs.procps ];
       wantedBy = [ "multi-user.target" ];
       serviceConfig = {
-        ExecStart = "${pkgs.dd-agent}/bin/dd-agent foreground";
+        ExecStart = "${cfg.agent}/bin/dd-agent foreground";
         User = "datadog";
         Group = "datadog";
         Restart = "always";
         RestartSec = 2;
       };
-      restartTriggers = [ pkgs.dd-agent ddConf diskConfig networkConfig postgresqlConfig nginxConfig mongoConfig jmxConfig processConfig ];
+      restartTriggers = [ cfg.agent ddConf ] ++ etcSources;
     };
 
-    systemd.services.dogstatsd = {
-      description = "Datadog statsd";
-      path = [ pkgs."dd-agent" pkgs.python pkgs.procps ];
-      wantedBy = [ "multi-user.target" ];
-      serviceConfig = {
-        ExecStart = "${pkgs.dd-agent}/bin/dogstatsd start";
-        User = "datadog";
-        Group = "datadog";
-        Type = "forking";
-        PIDFile = "/tmp/dogstatsd.pid";
-        Restart = "always";
-        RestartSec = 2;
-      };
-      restartTriggers = [ pkgs.dd-agent ddConf diskConfig networkConfig postgresqlConfig nginxConfig mongoConfig jmxConfig processConfig ];
-    };
-
-    systemd.services.dd-jmxfetch = lib.mkIf (cfg.jmxConfig != null) {
+    systemd.services.dd-jmxfetch = lib.mkIf (builtins.any (i: i.name == "jmx") cfg.integrations) {
       description = "Datadog JMX Fetcher";
-      path = [ pkgs."dd-agent" pkgs.python pkgs.sysstat pkgs.procps pkgs.jdk ];
+      path = [ cfg.agent pkgs.python pkgs.sysstat pkgs.procps pkgs.jdk ];
       wantedBy = [ "multi-user.target" ];
       serviceConfig = {
-        ExecStart = "${pkgs.dd-agent}/bin/dd-jmxfetch";
+        ExecStart = "${cfg.agent}/bin/dd-jmxfetch";
         User = "datadog";
         Group = "datadog";
         Restart = "always";
         RestartSec = 2;
       };
-      restartTriggers = [ pkgs.dd-agent ddConf diskConfig networkConfig postgresqlConfig nginxConfig mongoConfig jmxConfig ];
+      restartTriggers = [ cfg.agent ddConf ] ++ etcSources;
     };
 
     environment.etc = etcfiles;

nixos/modules/services/monitoring/dd-agent/update-dd-agent-defaults

@@ -1,9 +0,0 @@
-#!/usr/bin/env bash
-dd=$(nix-build --no-out-link -A dd-agent ../../../..)
-echo '# Generated using update-dd-agent-default, please re-run after updating dd-agent. DO NOT EDIT MANUALLY.' > dd-agent-defaults.nix
-echo '[' >> dd-agent-defaults.nix
-echo '  "auto_conf"' >> dd-agent-defaults.nix
-for f in $(find $dd/agent/conf.d-system -maxdepth 1 -type f | grep -v '\.example' | sort); do
-  echo "  \"$(basename $f)\"" >> dd-agent-defaults.nix
-done
-echo ']' >> dd-agent-defaults.nix

nixos/modules/services/monitoring/graphite.nix

@@ -488,9 +488,7 @@ in {
             # create index
             ${pkgs.python27Packages.graphite_web}/bin/build-index.sh
 
-            chown graphite:graphite ${cfg.dataDir}
-            chown graphite:graphite ${cfg.dataDir}/whisper
-            chown -R graphite:graphite ${cfg.dataDir}/log
+            chown -R graphite:graphite ${cfg.dataDir}
 
             touch ${dataDir}/db-created
           fi

nixos/modules/services/network-filesystems/glusterfs.nix

@@ -3,7 +3,7 @@
 with lib;
 
 let
-  inherit (pkgs) glusterfs;
+  inherit (pkgs) glusterfs rsync;
 
   cfg = config.services.glusterfs;
 
@@ -50,8 +50,11 @@ in
       after = [ "rpcbind.service" "network.target" "local-fs.target" ];
       before = [ "network-online.target" ];
 
+      # The copying of hooks is due to upstream bug https://bugzilla.redhat.com/show_bug.cgi?id=1452761
       preStart = ''
         install -m 0755 -d /var/log/glusterfs
+        mkdir -p /var/lib/glusterd/hooks/
+        ${rsync}/bin/rsync -a ${glusterfs}/var/lib/glusterd/hooks/ /var/lib/glusterd/hooks/
       '';
 
       serviceConfig = {

nixos/modules/services/network-filesystems/ipfs.nix

@@ -9,7 +9,10 @@ let
 
   ipfsFlags = ''${if cfg.autoMigrate then "--migrate" else ""} ${if cfg.enableGC then "--enable-gc" else ""} ${toString cfg.extraFlags}'';
 
-  pathEnv = { IPFS_PATH = cfg.dataDir; };
+  # Before Version 17.09, ipfs would always use "/var/lib/ipfs/.ipfs" as it's dataDir
+  defaultDataDir = if versionAtLeast config.system.stateVersion "17.09" then
+    "/var/lib/ipfs" else
+    "/var/lib/ipfs/.ipfs";
 
   # Wrapping the ipfs binary with the environment variable IPFS_PATH set to dataDir because we can't set it in the user environment
   wrapped = runCommand "ipfs" { buildInputs = [ makeWrapper ]; } ''
@@ -42,7 +45,7 @@ in
 
       dataDir = mkOption {
         type = types.str;
-        default = "/var/lib/ipfs";
+        default = defaultDataDir;
         description = "The data dir for IPFS";
       };
 
@@ -117,16 +120,15 @@ in
       after = [ "local-fs.target" ];
       before = [ "ipfs.service" "ipfs-offline.service" ];
 
+      environment.IPFS_PATH = cfg.dataDir;
+
       path  = [ pkgs.ipfs pkgs.su pkgs.bash ];
 
       preStart = ''
         install -m 0755 -o ${cfg.user} -g ${cfg.group} -d ${cfg.dataDir}
       '';
-
-      environment = pathEnv;
-
       script =  ''
-        if [[ ! -d ${cfg.dataDir}/.ipfs ]]; then
+        if [[ ! -f ${cfg.dataDir}/config ]]; then
           ${ipfs}/bin/ipfs init ${optionalString cfg.emptyRepo "-e"}
         fi
         ${ipfs}/bin/ipfs --local config Addresses.API ${cfg.apiAddress}
@@ -151,9 +153,9 @@ in
       conflicts = [ "ipfs-offline.service" ];
       wants = [ "ipfs-init.service" ];
 
-      path  = [ pkgs.ipfs ];
+      environment.IPFS_PATH = cfg.dataDir;
 
-      environment = pathEnv;
+      path  = [ pkgs.ipfs ];
 
       serviceConfig = {
         ExecStart = "${ipfs}/bin/ipfs daemon ${ipfsFlags}";
@@ -172,9 +174,9 @@ in
       conflicts = [ "ipfs.service" ];
       wants = [ "ipfs-init.service" ];
 
-      path  = [ pkgs.ipfs ];
+      environment.IPFS_PATH = cfg.dataDir;
 
-      environment = pathEnv;
+      path  = [ pkgs.ipfs ];
 
       serviceConfig = {
         ExecStart = "${ipfs}/bin/ipfs daemon ${ipfsFlags} --offline";

nixos/modules/services/network-filesystems/u9fs.nix

@@ -67,6 +67,7 @@ in
             StandardInput = "socket";
             StandardError = "journal";
             User = cfg.user;
+            AmbientCapabilities = "cap_setuid cap_setgid";
           };
       };
     };

nixos/modules/services/networking/charybdis.nix

@@ -51,6 +51,17 @@ in
         '';
       };
 
+      motd = mkOption {
+        type = types.nullOr types.lines;
+        default = null;
+        description = ''
+          Charybdis MOTD text.
+
+          Charybdis will read its MOTD from /etc/charybdis/ircd.motd .
+          If set, the value of this option will be written to this path.
+        '';
+      };
+
     };
 
   };
@@ -58,8 +69,8 @@ in
 
   ###### implementation
 
-  config = mkIf cfg.enable {
-
+  config = mkIf cfg.enable (lib.mkMerge [
+    {
       users.extraUsers = singleton {
         name = cfg.user;
         description = "Charybdis IRC daemon user";
@@ -88,9 +99,12 @@ in
           ${coreutils}/bin/mkdir -p ${cfg.statedir}
           ${coreutils}/bin/chown ${cfg.user}:${cfg.group} ${cfg.statedir}
         '';
-
       };
 
-  };
+    }
     
+    (mkIf (cfg.motd != null) {
+      environment.etc."charybdis/ircd.motd".text = cfg.motd;
+    })
+  ]);
 }

nixos/modules/services/networking/firewall.nix

@@ -114,14 +114,15 @@ let
     # The "nixos-fw" chain does the actual work.
     ip46tables -N nixos-fw
 
-    # Perform a reverse-path test to refuse spoofers
-    # For now, we just drop, as the raw table doesn't have a log-refuse yet
-    ${optionalString (kernelHasRPFilter && (cfg.checkReversePath != false)) ''
     # Clean up rpfilter rules
     ip46tables -t raw -D PREROUTING -j nixos-fw-rpfilter 2> /dev/null || true
     ip46tables -t raw -F nixos-fw-rpfilter 2> /dev/null || true
-      ip46tables -t raw -N nixos-fw-rpfilter 2> /dev/null || true
+    ip46tables -t raw -X nixos-fw-rpfilter 2> /dev/null || true
 
+    ${optionalString (kernelHasRPFilter && (cfg.checkReversePath != false)) ''
+      # Perform a reverse-path test to refuse spoofers
+      # For now, we just drop, as the raw table doesn't have a log-refuse yet
+      ip46tables -t raw -N nixos-fw-rpfilter 2> /dev/null || true
       ip46tables -t raw -A nixos-fw-rpfilter -m rpfilter ${optionalString (cfg.checkReversePath == "loose") "--loose"} -j RETURN
 
       # Allows this host to act as a DHCPv4 server

nixos/modules/services/networking/i2pd.nix

@@ -212,7 +212,8 @@ in
         type = with types; nullOr int;
         default = null;
         description = ''
-           Set a router bandwidth limit integer in kbps or letters: L (32), O (256), P (2048), X (>9000)
+           Set a router bandwidth limit integer in KBps.
+           If not set, i2pd defaults to 32KBps.
         '';
       };
 

nixos/modules/services/networking/networkmanager.nix

@@ -12,6 +12,7 @@ let
   configFile = writeText "NetworkManager.conf" ''
     [main]
     plugins=keyfile
+    dhcp=${cfg.dhcp}
     dns=${if cfg.useDnsmasq then "dnsmasq" else "default"}
 
     [keyfile]
@@ -21,7 +22,7 @@ let
       ''unmanaged-devices=${lib.concatStringsSep ";" cfg.unmanaged}''}
 
     [logging]
-    level=WARN
+    level=${cfg.logLevel}
 
     [connection]
     ipv6.ip6-privacy=2
@@ -138,6 +139,22 @@ in {
         apply = list: (attrValues cfg.basePackages) ++ list;
       };
 
+      dhcp = mkOption {
+        type = types.enum [ "dhclient" "dhcpcd" "internal" ];
+        default = "dhclient";
+        description = ''
+          Which program (or internal library) should be used for DHCP.
+        '';
+      };
+
+      logLevel = mkOption {
+        type = types.enum [ "OFF" "ERR" "WARN" "INFO" "DEBUG" "TRACE" ];
+        default = "WARN";
+        description = ''
+          Set the default logging verbosity level.
+        '';
+      };
+
       appendNameservers = mkOption {
         type = types.listOf types.str;
         default = [];

nixos/modules/services/networking/tinc.nix

@@ -35,7 +35,8 @@ in
               description = ''
                 The name of the node which is used as an identifier when communicating
                 with the remote nodes in the mesh. If null then the hostname of the system
-                is used.
+                is used to derive a name (note that tinc may replace non-alphanumeric characters in
+                hostnames by underscores).
               '';
             };
 

nixos/modules/services/networking/znc.nix

@@ -53,7 +53,8 @@ let
                   Server = ${net.server} ${lib.optionalString net.useSSL "+"}${toString net.port} ${net.password}
                   ${concatMapStrings (c: "<Chan #${c}>\n</Chan>\n") net.channels}
                   ${lib.optionalString net.hasBitlbeeControlChannel ''
-                    <Chan &bitlbee></Chan>
+                    <Chan &bitlbee>
+                    </Chan>
                   ''}
                   ${net.extraConf}
               </Network>
@@ -279,7 +280,7 @@ in
           example = defaultPassBlock;
           type = types.string;
           description = ''
-            Generate with znc --makepass.
+            Generate with `nix-shell -p znc --command "znc --makepass"`.
             This is the password used to log in to the ZNC web admin interface.
           '';
         };

nixos/modules/services/search/elasticsearch.nix

@@ -5,13 +5,22 @@ with lib;
 let
   cfg = config.services.elasticsearch;
 
+  es5 = builtins.compareVersions (builtins.parseDrvName cfg.package.name).version "5" >= 0;
+
   esConfig = ''
     network.host: ${cfg.listenAddress}
+    cluster.name: ${cfg.cluster_name}
+
+    ${if es5 then ''
+      http.port: ${toString cfg.port}
+      transport.tcp.port: ${toString cfg.tcp_port}
+    '' else ''
       network.port: ${toString cfg.port}
       network.tcp.port: ${toString cfg.tcp_port}
       # TODO: find a way to enable security manager
       security.manager.enabled: false
-    cluster.name: ${cfg.cluster_name}
+    ''}
+
     ${cfg.extraConf}
   '';
 
@@ -19,13 +28,18 @@ let
     name = "elasticsearch-config";
     paths = [
       (pkgs.writeTextDir "elasticsearch.yml" esConfig)
-      (pkgs.writeTextDir "logging.yml" cfg.logging)
+      (if es5 then (pkgs.writeTextDir "log4j2.properties" cfg.logging)
+              else (pkgs.writeTextDir "logging.yml" cfg.logging))
     ];
+    # Elasticsearch 5.x won't start when the scripts directory does not exist
+    postBuild = if es5 then "${pkgs.coreutils}/bin/mkdir -p $out/scripts" else "";
   };
 
   esPlugins = pkgs.buildEnv {
     name = "elasticsearch-plugins";
     paths = cfg.plugins;
+    # Elasticsearch 5.x won't start when the plugins directory does not exist
+    postBuild = if es5 then "${pkgs.coreutils}/bin/mkdir -p $out/plugins" else "";
   };
 
 in {
@@ -85,7 +99,19 @@ in {
 
     logging = mkOption {
       description = "Elasticsearch logging configuration.";
-      default = ''
+      default =
+        if es5 then ''
+          logger.action.name = org.elasticsearch.action
+          logger.action.level = info
+
+          appender.console.type = Console
+          appender.console.name = console
+          appender.console.layout.type = PatternLayout
+          appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] %marker%m%n
+
+          rootLogger.level = info
+          rootLogger.appenderRef.console.ref = console
+        '' else ''
           rootLogger: INFO, console
           logger:
             action: INFO
@@ -112,6 +138,12 @@ in {
       description = "Extra command line options for the elasticsearch launcher.";
       default = [];
       type = types.listOf types.str;
+    };
+
+    extraJavaOptions = mkOption {
+      description = "Extra command line options for Java.";
+      default = [];
+      type = types.listOf types.str;
       example = [ "-Djava.net.preferIPv4Stack=true" ];
     };
 
@@ -133,13 +165,21 @@ in {
       path = [ pkgs.inetutils ];
       environment = {
         ES_HOME = cfg.dataDir;
+        ES_JAVA_OPTS = toString ([ "-Des.path.conf=${configDir}" ] ++ cfg.extraJavaOptions);
       };
       serviceConfig = {
-        ExecStart = "${cfg.package}/bin/elasticsearch -Des.path.conf=${configDir} ${toString cfg.extraCmdLineOptions}";
+        ExecStart = "${cfg.package}/bin/elasticsearch ${toString cfg.extraCmdLineOptions}";
         User = "elasticsearch";
         PermissionsStartOnly = true;
+        LimitNOFILE = "1024000";
       };
       preStart = ''
+        # Only set vm.max_map_count if lower than ES required minimum
+        # This avoids conflict if configured via boot.kernel.sysctl
+        if [ `${pkgs.procps}/bin/sysctl -n vm.max_map_count` -lt 262144 ]; then
+          ${pkgs.procps}/bin/sysctl -w vm.max_map_count=262144
+        fi
+
         mkdir -m 0700 -p ${cfg.dataDir}
 
         # Install plugins

nixos/modules/services/security/hologram-server.nix

@@ -23,6 +23,8 @@ let
     stats  = cfg.statsAddress;
     listen = cfg.listenAddress;
   });
+
+  script = "${pkgs.hologram.bin}/bin/hologram-server --debug --conf ${cfgFile}";
 in {
   options = {
     services.hologram-server = {
@@ -94,9 +96,15 @@ in {
       after       = [ "network.target" ];
       wantedBy    = [ "multi-user.target" ];
 
-      serviceConfig = {
-        ExecStart = "${pkgs.hologram.bin}/bin/hologram-server --debug --conf ${cfgFile}";
+      inherit script;
     };
+
+    docker-containers.hologram-server = {
+      inherit script;
+    };
+
+    trivial-services.hologram-server = {
+      inherit script;
     };
   };
 }

nixos/modules/services/x11/compton.nix

@@ -208,13 +208,13 @@ in {
   config = mkIf cfg.enable {
     systemd.user.services.compton = {
       description = "Compton composite manager";
-      wantedBy = [ "default.target" ];
+      wantedBy = [ "graphical-session.target" ];
+      partOf = [ "graphical-session.target" ];
       serviceConfig = {
         ExecStart = "${cfg.package}/bin/compton --config ${configFile}";
         RestartSec = 3;
         Restart = "always";
       };
-      environment.DISPLAY = ":0";
     };
 
     environment.systemPackages = [ cfg.package ];

nixos/modules/services/x11/desktop-managers/default.nix

@@ -26,6 +26,34 @@ in
 
     services.xserver.desktopManager = {
 
+      wallpaper = {
+        mode = mkOption {
+          type = types.enum [ "center" "fill" "max" "scale" "tile" ];
+          default = "scale";
+          example = "fill";
+          description = ''
+            The file <filename>~/.background-image</filename> is used as a background image.
+            This option specifies the placement of this image onto your desktop.
+
+            Possible values:
+            <literal>center</literal>: Center the image on the background. If it is too small, it will be surrounded by a black border.
+            <literal>fill</literal>: Like <literal>scale</literal>, but preserves aspect ratio by zooming the image until it fits. Either a horizontal or a vertical part of the image will be cut off.
+            <literal>max</literal>: Like <literal>fill</literal>, but scale the image to the maximum size that fits the screen with black borders on one side.
+            <literal>scale</literal>: Fit the file into the background without repeating it, cutting off stuff or using borders. But the aspect ratio is not preserved either.
+            <literal>tile</literal>: Tile (repeat) the image in case it is too small for the screen.
+          '';
+        };
+
+        combineScreens = mkOption {
+          type = types.bool;
+          default = false;
+          description = ''
+            When set to <literal>true</literal> the wallpaper will stretch across all screens.
+            When set to <literal>false</literal> the wallpaper is duplicated to all screens.
+          '';
+        };
+      };
+
       session = mkOption {
         internal = true;
         default = [];
@@ -45,7 +73,7 @@ in
             start = d.start
             + optionalString (needBGCond d) ''
               if [ -e $HOME/.background-image ]; then
-                ${pkgs.feh}/bin/feh --bg-scale $HOME/.background-image
+                ${pkgs.feh}/bin/feh --bg-${cfg.wallpaper.mode} ${optionalString cfg.wallpaper.combineScreens "--no-xinerama"} $HOME/.background-image
               else
                 # Use a solid black background as fallback
                 ${pkgs.xorg.xsetroot}/bin/xsetroot -solid black

nixos/modules/services/x11/display-managers/default.nix

@@ -122,6 +122,9 @@ let
           source ~/.xprofile
       fi
 
+      # Start systemd user services for graphical sessions
+      ${config.systemd.package}/bin/systemctl --user start graphical-session.target
+
       # Allow the user to setup a custom session type.
       if test -x ~/.xsession; then
           exec ~/.xsession
@@ -164,6 +167,9 @@ let
       ''}
 
       test -n "$waitPID" && wait "$waitPID"
+
+      ${config.systemd.package}/bin/systemctl --user stop graphical-session.target
+
       exit 0
     '';
 
@@ -325,6 +331,13 @@ in
 
   config = {
     services.xserver.displayManager.xserverBin = "${xorg.xorgserver.out}/bin/X";
+
+    systemd.user.targets.graphical-session = {
+      unitConfig = {
+        RefuseManualStart = false;
+        StopWhenUnneeded = false;
+      };
+    };
   };
 
   imports = [

nixos/modules/services/x11/redshift.nix

@@ -95,7 +95,8 @@ in {
   config = mkIf cfg.enable {
     systemd.user.services.redshift = {
       description = "Redshift colour temperature adjuster";
-      wantedBy = [ "default.target" ];
+      wantedBy = [ "graphical-session.target" ];
+      partOf = [ "graphical-session.target" ];
       serviceConfig = {
         ExecStart = ''
           ${cfg.package}/bin/redshift \
@@ -107,12 +108,6 @@ in {
         RestartSec = 3;
         Restart = "always";
       };
-      environment = {
-        DISPLAY = ":${toString (
-          let display = config.services.xserver.display;
-          in if display != null then display else 0
-        )}";
-      };
     };
   };
 

nixos/modules/services/x11/unclutter-xfixes.nix

@@ -43,7 +43,8 @@ in {
   config = mkIf cfg.enable {
     systemd.user.services.unclutter-xfixes = {
       description = "unclutter-xfixes";
-      wantedBy = [ "graphical.target" ];
+      wantedBy = [ "graphical-session.target" ];
+      partOf = [ "graphical-session.target" ];
       serviceConfig.ExecStart = ''
         ${cfg.package}/bin/unclutter \
           --timeout ${toString cfg.timeout} \

nixos/modules/services/x11/unclutter.nix

@@ -56,19 +56,17 @@ in {
   config = mkIf cfg.enable {
     systemd.user.services.unclutter = {
       description = "unclutter";
-      wantedBy = [ "default.target" ];
+      wantedBy = [ "graphical-session.target" ];
+      partOf = [ "graphical-session.target" ];
       serviceConfig.ExecStart = ''
         ${cfg.package}/bin/unclutter \
           -idle ${toString cfg.timeout} \
-          -display :${toString (
-             let display = config.services.xserver.display;
-             in if display != null then display else 0
-          )} \
           -jitter ${toString (cfg.threeshold - 1)} \
           ${optionalString cfg.keystroke "-keystroke"} \
           ${concatMapStrings (x: " -"+x) cfg.extraOptions} \
           -not ${concatStringsSep " " cfg.excluded} \
       '';
+      serviceConfig.PassEnvironment = "DISPLAY";
       serviceConfig.RestartSec = 3;
       serviceConfig.Restart = "always";
     };

nixos/modules/services/x11/urxvtd.nix

@@ -21,9 +21,8 @@ in {
     systemd.user = {
       sockets.urxvtd = {
         description = "socket for urxvtd, the urxvt terminal daemon";
-        after = [ "graphical.target" ];
-        wants = [ "graphical.target" ];
-        wantedBy = [ "sockets.target" ];
+        wantedBy = [ "graphical-session.target" ];
+        partOf = [ "graphical-session.target" ];
         socketConfig = {
           ListenStream = "%t/urxvtd-socket";
         };

nixos/modules/services/x11/xbanish.nix

@@ -20,7 +20,8 @@ in {
   config = mkIf cfg.enable {
     systemd.user.services.xbanish = {
       description = "xbanish hides the mouse pointer";
-      wantedBy = [ "default.target" ];
+      wantedBy = [ "graphical-session.target" ];
+      partOf = [ "graphical-session.target" ];
       serviceConfig.ExecStart = ''
         ${pkgs.xbanish}/bin/xbanish ${cfg.arguments}
       '';

nixos/modules/system/boot/systemd-nspawn.nix

@@ -18,7 +18,6 @@ let
     ])
     (assertValueOneOf "Boot" boolValues)
     (assertValueOneOf "ProcessTwo" boolValues)
-    (assertValueOneOf "PrivateUsers" (boolValues ++ [ "pick" ]))
     (assertValueOneOf "NotifyReady" boolValues)
   ];
 
@@ -42,8 +41,7 @@ let
   ];
 
   instanceOptions = {
-    options = {
-
+    options = sharedOptions // {
       execConfig = mkOption {
         default = {};
         example = { Parameters = "/bin/sh"; };
@@ -84,7 +82,8 @@ let
   };
 
   instanceToUnit = name: def:
-    { text = ''
+    let base = {
+      text = ''
         [Exec]
         ${attrsToSection def.execConfig}
 
@@ -94,7 +93,8 @@ let
         [Network]
         ${attrsToSection def.networkConfig}
       '';
-    };
+    } // def;
+    in base // { unit = makeUnit name base; };
 
 in {
 
@@ -110,7 +110,7 @@ in {
 
   config =
     let
-      units = mapAttrs' (n: v: nameValuePair "${n}.nspawn" (instanceToUnit n v)) cfg.instances;
+      units = mapAttrs' (n: v: nameValuePair "${n}.nspawn" (instanceToUnit n v)) cfg;
     in mkIf (cfg != {}) {
 
       environment.etc."systemd/nspawn".source = generateUnits "nspawn" units [] [];

nixos/modules/system/boot/systemd.nix

@@ -396,6 +396,11 @@ let
         '';
     };
 
+  logindHandlerType = types.enum [
+    "ignore" "poweroff" "reboot" "halt" "kexec" "suspend"
+    "hibernate" "hybrid-sleep" "lock"
+  ];
+
 in
 
 {
@@ -595,6 +600,27 @@ in
       '';
     };
 
+    services.logind.lidSwitch = mkOption {
+      default = "suspend";
+      example = "ignore";
+      type = logindHandlerType;
+
+      description = ''
+        Specifies what to be done when the laptop lid is closed.
+      '';
+    };
+
+    services.logind.lidSwitchDocked = mkOption {
+      default = "ignore";
+      example = "suspend";
+      type = logindHandlerType;
+
+      description = ''
+        Specifies what to be done when the laptop lid is closed
+        and another screen is added.
+      '';
+    };
+
     systemd.user.extraConfig = mkOption {
       default = "";
       type = types.lines;
@@ -721,6 +747,8 @@ in
       "systemd/logind.conf".text = ''
         [Login]
         KillUserProcesses=no
+        HandleLidSwitch=${config.services.logind.lidSwitch}
+        HandleLidSwitchDocked=${config.services.logind.lidSwitchDocked}
         ${config.services.logind.extraConfig}
       '';
 

nixos/modules/virtualisation/lxd.nix

@@ -45,7 +45,7 @@ in
         after = [ "systemd-udev-settle.service" ];
 
         # TODO(wkennington): Add lvm2 and thin-provisioning-tools
-        path = with pkgs; [ acl rsync gnutar xz btrfs-progs ];
+        path = with pkgs; [ acl rsync gnutar xz btrfs-progs gzip dnsmasq squashfsTools iproute iptables ];
 
         serviceConfig.ExecStart = "@${pkgs.lxd.bin}/bin/lxd lxd --syslog --group lxd";
         serviceConfig.Type = "simple";

nixos/release.nix

@@ -262,6 +262,7 @@ in rec {
   tests.keystone = callTest tests/keystone.nix {};
   tests.kubernetes = hydraJob (import tests/kubernetes.nix { system = "x86_64-linux"; });
   tests.latestKernel.login = callTest tests/login.nix { latestKernel = true; };
+  tests.ldap = callTest tests/ldap.nix {};
   #tests.lightdm = callTest tests/lightdm.nix {};
   tests.login = callTest tests/login.nix {};
   #tests.logstash = callTest tests/logstash.nix {};

nixos/tests/ldap.nix

@@ -0,0 +1,119 @@
+import ./make-test.nix ({ pkgs, lib, ...} :
+
+let
+
+  dbSuffix = "dc=example,dc=com";
+  dbPath = "/var/db/openldap";
+  dbAdminDn = "cn=admin,${dbSuffix}";
+  dbAdminPwd = "test";
+  serverUri = "ldap:///";
+  ldapUser = "test-ldap-user";
+  ldapUserId = 10000;
+  ldapUserPwd = "test";
+  ldapGroup = "test-ldap-group";
+  ldapGroupId = 10000;
+  setupLdif = pkgs.writeText "test-ldap.ldif" ''
+    dn: ${dbSuffix}
+    dc: ${with lib; let dc = head (splitString "," dbSuffix); dcName = head (tail (splitString "=" dc)); in dcName}
+    o: ${dbSuffix}
+    objectclass: top
+    objectclass: dcObject
+    objectclass: organization
+
+    dn: cn=${ldapUser},${dbSuffix}
+    sn: ${ldapUser}
+    objectClass: person
+    objectClass: posixAccount
+    uid: ${ldapUser}
+    uidNumber: ${toString ldapUserId}
+    gidNumber: ${toString ldapGroupId}
+    homeDirectory: /home/${ldapUser}
+    loginShell: /bin/sh
+    userPassword: ${ldapUserPwd}
+
+    dn: cn=${ldapGroup},${dbSuffix}
+    objectClass: posixGroup
+    gidNumber: ${toString ldapGroupId}
+    memberUid: ${ldapUser}
+  '';
+  mkClient = useDaemon:
+    { config, pkgs, lib, ... }:
+    {
+      virtualisation.memorySize = 256;
+      virtualisation.vlans = [ 1 ];
+      security.pam.services.su.rootOK = lib.mkForce false;
+      users.ldap.enable = true;
+      users.ldap.daemon.enable = useDaemon;
+      users.ldap.loginPam = true;
+      users.ldap.nsswitch = true;
+      users.ldap.server = "ldap://server";
+      users.ldap.base = "${dbSuffix}";
+    };
+
+in
+
+{
+  name = "ldap";
+  meta = with pkgs.stdenv.lib.maintainers; {
+    maintainers = [ montag451 ];
+  };
+
+  nodes = {
+
+    server =
+      { config, pkgs, lib, ... }:
+      {
+        virtualisation.memorySize = 256;
+        virtualisation.vlans = [ 1 ];
+        networking.firewall.allowedTCPPorts = [ 389 ];
+        services.openldap.enable = true;
+        services.openldap.dataDir = dbPath;
+        services.openldap.urlList = [
+          serverUri
+        ];
+        services.openldap.extraConfig = ''
+          include ${pkgs.openldap.out}/etc/schema/core.schema
+          include ${pkgs.openldap.out}/etc/schema/cosine.schema
+          include ${pkgs.openldap.out}/etc/schema/inetorgperson.schema
+          include ${pkgs.openldap.out}/etc/schema/nis.schema
+
+          database mdb
+          suffix ${dbSuffix}
+          rootdn ${dbAdminDn}
+          rootpw ${dbAdminPwd}
+          directory ${dbPath}
+        '';
+      };
+
+    client1 = mkClient true; # use nss_pam_ldapd
+    client2 = mkClient false; # use nss_ldap and pam_ldap
+
+  };
+
+  testScript = ''
+    startAll;
+    $server->waitForUnit("default.target");
+    $client1->waitForUnit("default.target");
+    $client2->waitForUnit("default.target");
+
+    $server->succeed("ldapadd -D '${dbAdminDn}' -w ${dbAdminPwd} -H ${serverUri} -f '${setupLdif}'");
+
+    # NSS tests
+    subtest "nss", sub {
+        $client1->succeed("test \"\$(id -u '${ldapUser}')\" -eq ${toString ldapUserId}");
+        $client1->succeed("test \"\$(id -u -n '${ldapUser}')\" = '${ldapUser}'");
+        $client1->succeed("test \"\$(id -g '${ldapUser}')\" -eq ${toString ldapGroupId}");
+        $client1->succeed("test \"\$(id -g -n '${ldapUser}')\" = '${ldapGroup}'");
+        $client2->succeed("test \"\$(id -u '${ldapUser}')\" -eq ${toString ldapUserId}");
+        $client2->succeed("test \"\$(id -u -n '${ldapUser}')\" = '${ldapUser}'");
+        $client2->succeed("test \"\$(id -g '${ldapUser}')\" -eq ${toString ldapGroupId}");
+        $client2->succeed("test \"\$(id -g -n '${ldapUser}')\" = '${ldapGroup}'");
+    };
+
+    # PAM tests
+    subtest "pam", sub {
+        $client1->succeed("echo ${ldapUserPwd} | su -l '${ldapUser}' -c true");
+        $client2->succeed("echo ${ldapUserPwd} | su -l '${ldapUser}' -c true");
+    };
+  '';
+})

nixos/tests/plasma5.nix

@@ -13,6 +13,21 @@ import ./make-test.nix ({ pkgs, ...} :
     services.xserver.desktopManager.plasma5.enable = true;
     services.xserver.desktopManager.default = "plasma5";
     virtualisation.memorySize = 1024;
+
+    # fontconfig-penultimate-0.3.3 -> 0.3.4 broke OCR apparently, but no idea why.
+    nixpkgs.config.packageOverrides = superPkgs: {
+      fontconfig-penultimate = superPkgs.fontconfig-penultimate.overrideAttrs
+        (_attrs: rec {
+          version = "0.3.3";
+          name = "fontconfig-penultimate-${version}";
+          src = pkgs.fetchFromGitHub {
+            owner = "ttuegel";
+            repo = "fontconfig-penultimate";
+            rev = version;
+            sha256 = "0392lw31jps652dcjazln77ihb6bl7gk201gb7wb9i223avp86w9";
+          };
+        });
+    };
   };
 
   enableOCR = true;

pkgs/applications/altcoins/ethabi.nix

@@ -15,6 +15,8 @@ buildRustPackage rec {
 
   depsSha256 = "1n4rxipna307r4xppb2iaads7kpa3yjv99fimvpn8l0f999ir2rz";
 
+  cargoBuildFlags = ["--features cli"];
+
   meta = {
     description = "Ethereum function call encoding (ABI) utility";
     homepage = https://github.com/ethcore/ethabi/;

pkgs/applications/altcoins/seth.nix

@@ -3,13 +3,13 @@
 
 stdenv.mkDerivation rec {
   name = "seth-${version}";
-  version = "0.5.0";
+  version = "0.5.1";
 
   src = fetchFromGitHub {
     owner = "dapphub";
     repo = "seth";
     rev = "v${version}";
-    sha256 = "0bgygvilhbabb0y9pv9cn8cx7cj513w9is4vh6v69h2czknrjmgz";
+    sha256 = "1qph1gldj24r8l6aswb1w133lrm8zsxmmxl4krjik0a73bm4ghdm";
   };
 
   nativeBuildInputs = [makeWrapper];

pkgs/applications/audio/openmpt123/default.nix

@@ -9,7 +9,7 @@ in stdenv.mkDerivation rec {
     sha256 = "0qp2nnz6pnl1d7yv9hcjyim7q6yax5881k1jxm8jfgjqagmz5k6p";
   };
   buildInputs = [ SDL2 pkgconfig flac libsndfile ];
-  makeFlags = [ "NO_LTDL=1 TEST=0 EXAMPLES=0" ]
+  makeFlags = [ "NO_PULSEAUDIO=1 NO_LTDL=1 TEST=0 EXAMPLES=0" ]
   ++ stdenv.lib.optional (stdenv.isDarwin) "SHARED_SONAME=0";
   installFlags = "PREFIX=\${out}";
 

pkgs/applications/audio/spotify/default.nix

@@ -8,7 +8,7 @@ let
   # Please update the stable branch!
   # Latest version number can be found at:
   # http://repository-origin.spotify.com/pool/non-free/s/spotify-client/
-  version = "1.0.53.758.gde3fc4b2-33";
+  version = "1.0.55.487.g256699aa-16";
 
   deps = [
     alsaLib
@@ -53,7 +53,7 @@ stdenv.mkDerivation {
 
   src = fetchurl {
     url = "https://repository-origin.spotify.com/pool/non-free/s/spotify-client/spotify-client_${version}_amd64.deb";
-    sha256 = "1sh6bv23yx0fcbmf60c2yyi6411ij85k4jalpjlck2w26nfj1b3g";
+    sha256 = "09rhm0jp5prcvyf8zpw4pl422yiy8nkazcjc3lv61ngpznk3n1r0";
   };
 
   buildInputs = [ dpkg makeWrapper ];

pkgs/applications/editors/atom/default.nix

@@ -2,11 +2,11 @@
 
 stdenv.mkDerivation rec {
   name = "atom-${version}";
-  version = "1.17.0";
+  version = "1.17.2";
 
   src = fetchurl {
     url = "https://github.com/atom/atom/releases/download/v${version}/atom-amd64.deb";
-    sha256 = "10m1sww8zkhnhs3frlnd6g3b6f4fimgp0512wcszgqhvlhjbf9ln";
+    sha256 = "05lf9f5c9l111prx7d76cr5h8h340vm7vb8hra5rdrqhjpdvwhhn";
     name = "${name}.deb";
   };
 

pkgs/applications/editors/emacs-modes/elpa-generated.nix

@@ -1100,10 +1100,10 @@
       }) {};
     load-relative = callPackage ({ elpaBuild, fetchurl, lib }: elpaBuild {
         pname = "load-relative";
-        version = "1.2";
+        version = "1.3";
         src = fetchurl {
-          url = "https://elpa.gnu.org/packages/load-relative-1.2.el";
-          sha256 = "0vmfal05hznb10k2y3j9mychi9ra4hxcm6qf7j1r8aw9j7af6riw";
+          url = "https://elpa.gnu.org/packages/load-relative-1.3.el";
+          sha256 = "1hfxb2436jdsi9wfmsv47lkkpa5galjf5q81bqabbsv79rv59dps";
         };
         packageRequires = [];
         meta = {
@@ -1404,10 +1404,10 @@
       }) {};
     org = callPackage ({ elpaBuild, fetchurl, lib }: elpaBuild {
         pname = "org";
-        version = "20170502";
+        version = "20170515";
         src = fetchurl {
-          url = "https://elpa.gnu.org/packages/org-20170502.tar";
-          sha256 = "12inz804j55ycprb2m3ay54d1bhwhjssmn5nrfm7cfklyhfsy27s";
+          url = "https://elpa.gnu.org/packages/org-20170515.tar";
+          sha256 = "0lfapcxil69x1a63cszgq72lqks1z3gpyxw7vcllqlgi7n7a4y6f";
         };
         packageRequires = [];
         meta = {
@@ -1431,10 +1431,10 @@
     other-frame-window = callPackage ({ elpaBuild, emacs, fetchurl, lib }:
     elpaBuild {
         pname = "other-frame-window";
-        version = "1.0.2";
+        version = "1.0.3";
         src = fetchurl {
-          url = "https://elpa.gnu.org/packages/other-frame-window-1.0.2.el";
-          sha256 = "0gr4vn7ld4fx372091wxnzm1rhq6rc4ycim4fwz5bxnpykz83l7d";
+          url = "https://elpa.gnu.org/packages/other-frame-window-1.0.3.el";
+          sha256 = "0vq1zfsdnxdjvmb7lkjyax27kfv0rw0141rd5fjnl6ap9yjwpxkv";
         };
         packageRequires = [ emacs ];
         meta = {
@@ -1550,10 +1550,10 @@
       }) {};
     rainbow-mode = callPackage ({ elpaBuild, fetchurl, lib }: elpaBuild {
         pname = "rainbow-mode";
-        version = "0.12";
+        version = "0.13";
         src = fetchurl {
-          url = "https://elpa.gnu.org/packages/rainbow-mode-0.12.el";
-          sha256 = "10a7qs7fvw4qi4vxj9n56j26gjk61bl79dgz4md1d26slb2j1c04";
+          url = "https://elpa.gnu.org/packages/rainbow-mode-0.13.el";
+          sha256 = "1d3aamx6qgqqpqijwsr02ggwrh67gfink1bir0692alfkm3zdddl";
         };
         packageRequires = [];
         meta = {
@@ -1564,10 +1564,10 @@
     realgud = callPackage ({ cl-lib ? null, elpaBuild, emacs, fetchurl, lib, load-relative, loc-changes, test-simple }:
     elpaBuild {
         pname = "realgud";
-        version = "1.4.3";
+        version = "1.4.4";
         src = fetchurl {
-          url = "https://elpa.gnu.org/packages/realgud-1.4.3.tar";
-          sha256 = "07yfah8kmr60rhrj8y8gs7l85hsbylv26crw3qbqh67ka4ykpj01";
+          url = "https://elpa.gnu.org/packages/realgud-1.4.4.tar";
+          sha256 = "1nc8km339ip90h1j55ahfga03v7x7rh4iycmw6yrxyzir68vwn7c";
         };
         packageRequires = [
           cl-lib
@@ -1730,10 +1730,10 @@
     soap-client = callPackage ({ cl-lib ? null, elpaBuild, fetchurl, lib }:
     elpaBuild {
         pname = "soap-client";
-        version = "3.1.1";
+        version = "3.1.2";
         src = fetchurl {
-          url = "https://elpa.gnu.org/packages/soap-client-3.1.1.tar";
-          sha256 = "0is2923g882farf73dix6ncq3m26yn5j5qr8wz6s0xad04zdbdhk";
+          url = "https://elpa.gnu.org/packages/soap-client-3.1.2.tar";
+          sha256 = "0crfpp2ksqvzf2j3q44qq6z8zh1r10q9kw12cxbmjiih89q01b68";
         };
         packageRequires = [ cl-lib ];
         meta = {
@@ -1850,10 +1850,10 @@
     test-simple = callPackage ({ cl-lib ? null, elpaBuild, fetchurl, lib }:
     elpaBuild {
         pname = "test-simple";
-        version = "1.2.0";
+        version = "1.3.0";
         src = fetchurl {
-          url = "https://elpa.gnu.org/packages/test-simple-1.2.0.el";
-          sha256 = "1j97qrwi3i2kihszsxf3y2cby2bzp8g0zf6jlpdix3dinav8xa3b";
+          url = "https://elpa.gnu.org/packages/test-simple-1.3.0.el";
+          sha256 = "1yd61jc9ds95a5n09052kwc5gasy57g4lxr0jsff040brlyi9czz";
         };
         packageRequires = [ cl-lib ];
         meta = {
@@ -1981,6 +1981,19 @@
           license = lib.licenses.free;
         };
       }) {};
+    vdiff = callPackage ({ elpaBuild, emacs, fetchurl, hydra, lib }: elpaBuild {
+        pname = "vdiff";
+        version = "0.2.3";
+        src = fetchurl {
+          url = "https://elpa.gnu.org/packages/vdiff-0.2.3.el";
+          sha256 = "197wszzhm2kbfvvlg3f0dzfs3lf4536yq5fd67k2rycj421fr9qz";
+        };
+        packageRequires = [ emacs hydra ];
+        meta = {
+          homepage = "https://elpa.gnu.org/packages/vdiff.html";
+          license = lib.licenses.free;
+        };
+      }) {};
     vlf = callPackage ({ elpaBuild, fetchurl, lib }: elpaBuild {
         pname = "vlf";
         version = "1.7";

pkgs/applications/editors/emacs-modes/org-generated.nix

@@ -1,10 +1,10 @@
 { callPackage }: {
     org = callPackage ({ elpaBuild, fetchurl, lib }: elpaBuild {
         pname = "org";
-        version = "20170502";
+        version = "20170515";
         src = fetchurl {
-          url = "http://orgmode.org/elpa/org-20170502.tar";
-          sha256 = "1y5rdf6740z45v75y17yh3a1ivdk5fjrax3hyr11jydyicczk4h1";
+          url = "http://orgmode.org/elpa/org-20170515.tar";
+          sha256 = "04kpi7q1q4r9w4km941cy70q3k9azspw1wdr71if4f8am6frj3d4";
         };
         packageRequires = [];
         meta = {
@@ -14,10 +14,10 @@
       }) {};
     org-plus-contrib = callPackage ({ elpaBuild, fetchurl, lib }: elpaBuild {
         pname = "org-plus-contrib";
-        version = "20170502";
+        version = "20170515";
         src = fetchurl {
-          url = "http://orgmode.org/elpa/org-plus-contrib-20170502.tar";
-          sha256 = "06pr3w11zpns66km27ql3w8qlk6bxaxqx3bmaiwrxykhbf74dib0";
+          url = "http://orgmode.org/elpa/org-plus-contrib-20170515.tar";
+          sha256 = "0jdcxir8wvmdxi0rxnljbhy31yh83n4p0l8jp85fxf5sx0kcc32p";
         };
         packageRequires = [];
         meta = {

pkgs/applications/editors/emacs-modes/proofgeneral/pg.patch

@@ -0,0 +1,16 @@
+diff -r c7d8bfff4c0a bin/proofgeneral
+--- a/bin/proofgeneral	Sat Sep 27 02:25:15 2014 +0100
++++ b/bin/proofgeneral	Sat Sep 27 02:28:16 2014 +0100
+@@ -73,11 +73,7 @@
+ 
+ # Try to find Proof General directory
+ if [ -z "$PGHOME" ] || [ ! -d "$PGHOME" ]; then
+-    # default relative to this script, otherwise PGHOMEDEFAULT
+-    MYDIR="`readlink --canonicalize "$0" | sed -ne 's,/bin/proofgeneral$,,p'`"
+-    if [ -d "$MYDIR/generic" ]; then
+-	PGHOME="$MYDIR"
+-    elif [ -d "$PGHOMEDEFAULT" ]; then
++    if [ -d "$PGHOMEDEFAULT" ]; then
+ 	PGHOME="$PGHOMEDEFAULT"
+     else
+ 	echo "Cannot find the Proof General lisp files: Set PGHOME or use --pghome."

pkgs/applications/editors/jetbrains/default.nix

@@ -305,24 +305,24 @@ in
 
   pycharm-community = buildPycharm rec {
     name = "pycharm-community-${version}";
-    version = "2017.1.2";
+    version = "2017.1.3";
     description = "PyCharm Community Edition";
     license = stdenv.lib.licenses.asl20;
     src = fetchurl {
       url = "https://download.jetbrains.com/python/${name}.tar.gz";
-      sha256 = "03c352lj6vnc7cs5ch8p12i4f95qadnibzbrxmxv5xqglpdrp7g9";
+      sha256 = "06sai589zli5xaggfk4g0j0grbw9mya9qlwabmxh9414qq3bzvbd";
     };
     wmClass = "jetbrains-pycharm-ce";
   };
 
   pycharm-professional = buildPycharm rec {
     name = "pycharm-professional-${version}";
-    version = "2017.1.2";
+    version = "2017.1.3";
     description = "PyCharm Professional Edition";
     license = stdenv.lib.licenses.unfree;
     src = fetchurl {
       url = "https://download.jetbrains.com/python/${name}.tar.gz";
-      sha256 = "0jrrlrkwi6f70nqrrz2vv1wdjpwjbh2in1g658dsbr9gpmkdmy0q";
+      sha256 = "1wzgh83504px7q93h9xkarih2qjchiavgysy4di82q7377s6xd0c";
     };
     wmClass = "jetbrains-pycharm";
   };

pkgs/applications/graphics/ImageMagick/7.0.nix

@@ -12,8 +12,8 @@ let
     else throw "ImageMagick is not supported on this platform.";
 
   cfg = {
-    version = "7.0.4-6";
-    sha256 = "1nm0hjijwhcp6rzcn7zksp2820dxvj4lmblj7kzpzd3s1ds09q0y";
+    version = "7.0.5-7";
+    sha256 = "11k53193az0bvdhp4gz2g9p8fb6r5fr1h74dnfx6ijfnfj80hbgp";
     patches = [];
   };
 in

pkgs/applications/graphics/ImageMagick/default.nix

@@ -12,8 +12,8 @@ let
     else throw "ImageMagick is not supported on this platform.";
 
   cfg = {
-    version = "6.9.7-6";
-    sha256 = "17pc3xz8srb9g5a5gkk6q9sjiss77fgm0wxxfmb5qya4rqivjpzn";
+    version = "6.9.8-6";
+    sha256 = "1sxg2wx3nrzbymh5wcqiv1x401nrz95xkrqgk3x446vx8lq7ln6w";
     patches = [];
   }
     # Freeze version on mingw so we don't need to port the patch too often.

pkgs/applications/graphics/inkscape/default.nix

@@ -26,17 +26,18 @@ stdenv.mkDerivation rec {
   postPatch = ''
     patchShebangs share/extensions
     patchShebangs fix-roff-punct
-  '';
 
     # Python is used at run-time to execute scripts, e.g., those from
     # the "Effects" menu.
-  propagatedBuildInputs = [ python2Env ];
+    substituteInPlace src/extension/implementation/script.cpp \
+      --replace '"python-interpreter", "python"' '"python-interpreter", "${python2Env}/bin/python"'
+  '';
 
   buildInputs = [
     pkgconfig perl perlXMLParser libXft libpng zlib popt boehmgc
     libxml2 libxslt glib gtkmm2 glibmm libsigcxx lcms boost gettext
     makeWrapper intltool gsl poppler imagemagick libwpg librevenge
-    libvisio libcdr libexif automake114x potrace cmake
+    libvisio libcdr libexif automake114x potrace cmake python2Env
   ];
 
   enableParallelBuilding = true;

pkgs/applications/graphics/simple-scan/default.nix

@@ -29,6 +29,17 @@ stdenv.mkDerivation rec {
   postInstall = ''
     mkdir -p $out/share/icons
     mv $out/share/simple-scan/icons/* $out/share/icons/
+    (
+    cd ${gnome3.defaultIconTheme}/share/icons/Adwaita
+    for f in `find . | grep 'scanner\.'`
+    do
+      local outFile="`echo "$out/share/icons/hicolor/$f" | sed \
+        -e 's#/devices/#/apps/#g' \
+        -e 's#scanner\.#simple-scan\.#g'`"
+      mkdir -p "`realpath -m "$outFile/.."`"
+      cp "$f" "$outFile"
+    done
+    )
   '';
 
   enableParallelBuilding = true;

pkgs/applications/misc/dmenu/default.nix

@@ -1,11 +1,11 @@
 { stdenv, fetchurl, libX11, libXinerama, libXft, zlib, patches ? null }:
 
 stdenv.mkDerivation rec {
-  name = "dmenu-4.6";
+  name = "dmenu-4.7";
 
   src = fetchurl {
     url = "http://dl.suckless.org/tools/${name}.tar.gz";
-    sha256 = "1cwnvamqqlgczvd5dv5rsgqbhv8kp0ddjnhmavb3q732i8028yja";
+    sha256 = "1fgph549i0yw62x02jhfc8qmqk2rnzcnm46bxq6jig1cvkw3amm7";
   };
 
   buildInputs = [ libX11 libXinerama zlib libXft ];

pkgs/applications/misc/electrum/default.nix

@@ -2,11 +2,11 @@
 
 python2Packages.buildPythonApplication rec {
   name = "electrum-${version}";
-  version = "2.8.2";
+  version = "2.8.3";
 
   src = fetchurl {
     url = "https://download.electrum.org/${version}/Electrum-${version}.tar.gz";
-    sha256 = "01xphbi7lx64s9380zjfakz5h8blqmxp0ryqlll7px66qpmjn5fq";
+    sha256 = "04jswalydzwm16iismbq1h2shj17iq9sqm0mz8p98sh3dwkacvn1";
   };
 
   propagatedBuildInputs = with python2Packages; [

pkgs/applications/misc/gpxsee/default.nix

@@ -2,13 +2,13 @@
 
 stdenv.mkDerivation rec {
   name = "gpxsee-${version}";
-  version = "4.3";
+  version = "4.8";
 
   src = fetchFromGitHub {
     owner = "tumic0";
     repo = "GPXSee";
     rev = version;
-    sha256 = "15f686frxlrmdvh5cc837kx62g0ihqj4vb87i8433g7l5vqkv3lf";
+    sha256 = "17s1v6b1j7pi0yj554bd0cg14bl854gssp5gj2pl51rxji6zr0wp";
   };
 
   nativeBuildInputs = [ qmakeHook qttools makeQtWrapper ];

pkgs/applications/misc/mupdf/default.nix

@@ -3,26 +3,26 @@
 , libX11, libXcursor, libXrandr, libXinerama, libXext, harfbuzz, mesa }:
 
 stdenv.mkDerivation rec {
-  version = "1.10a";
+  version = "1.11";
   name = "mupdf-${version}";
 
   src = fetchurl {
     url = "http://mupdf.com/downloads/archive/${name}-source.tar.gz";
-    sha256 = "0dm8wcs8i29aibzkqkrn8kcnk4q0kd1v66pg48h5c3qqp4v1zk5a";
+    sha256 = "02phamcchgsmvjnb3ir7r5sssvx9fcrscn297z73b82n1jl79510";
   };
 
   patches = [
     # Compatibility with new openjpeg
     (fetchpatch {
-      name = "mupdf-1.9a-openjpeg-2.1.1.patch";
-      url = "https://git.archlinux.org/svntogit/community.git/plain/mupdf/trunk/0001-mupdf-openjpeg.patch?id=5a28ad0a8999a9234aa7848096041992cc988099";
-      sha256 = "1i24qr4xagyapx4bijjfksj4g3bxz8vs5c2mn61nkm29c63knp75";
+      name = "mupdf-1.11-openjpeg-2.1.1.patch";
+      url = "https://git.archlinux.org/svntogit/community.git/plain/trunk/0001-mupdf-openjpeg.patch?h=packages/mupdf&id=3d997e7ff2ac20c44856ede22760ba6fbca81a5c";
+      sha256 = "1vr12kpzmmfr8pp3scwfhrm5laqwd58xm6vx971c4y8bxy60b2ig";
     })
 
     (fetchurl {
-      name = "CVE-2017-5896.patch";
-      url = "http://git.ghostscript.com/?p=mupdf.git;a=patch;h=2c4e5867ee699b1081527bc6c6ea0e99a35a5c27";
-      sha256 = "14k7x47ifx82sds1c06ibzbmcparfg80719jhgwjk6w1vkh4r693";
+      name = "mupdf-1.11-CVE-2017-6060.patch";
+      url = "http://git.ghostscript.com/?p=mupdf.git;a=blobdiff_plain;f=platform/x11/jstest_main.c;h=f158d9628ed0c0a84e37fe128277679e8334422a;hp=13c3a0a3ba3ff4aae29f6882d23740833c1d842f;hb=06a012a42c9884e3cd653e7826cff1ddec04eb6e;hpb=34e18d127a02146e3415b33c4b67389ce1ddb614";
+      sha256 = "163bllvjrbm0gvjb25lv7b6sih4zr4g4lap3h0cbq8dvpjxx0jfc";
     })
   ];
 
@@ -75,7 +75,7 @@ stdenv.mkDerivation rec {
   meta = with stdenv.lib; {
     homepage = http://mupdf.com;
     repositories.git = git://git.ghostscript.com/mupdf.git;
-    description = "Lightweight PDF viewer and toolkit written in portable C";
+    description = "Lightweight PDF, XPS, and E-book viewer and toolkit written in portable C";
     license = licenses.gpl3Plus;
     maintainers = with maintainers; [ viric vrthra fpletz ];
     platforms = platforms.linux;

pkgs/applications/misc/taskwarrior/default.nix

@@ -16,8 +16,10 @@ stdenv.mkDerivation rec {
   nativeBuildInputs = [ cmake libuuid gnutls ];
 
   postInstall = ''
-    mkdir -p "$out/etc/bash_completion.d"
-    ln -s "../../share/doc/task/scripts/bash/task.sh" "$out/etc/bash_completion.d/"
+    mkdir -p "$out/share/bash-completion/completions"
+    ln -s "../../doc/task/scripts/bash/task.sh" "$out/share/bash-completion/completions/"
+    mkdir -p "$out/etc/fish/completions"
+    ln -s "../../../share/doc/task/scripts/fish/task.fish" "$out/etc/fish/completions/"
   '';
 
   meta = with stdenv.lib; {

pkgs/applications/misc/yakuake/default.nix

@@ -20,13 +20,13 @@
 let
   unwrapped = let
     pname = "yakuake";
-    version = "3.0.2";
+    version = "3.0.3";
   in kdeDerivation rec {
     name = "${pname}-${version}";
 
     src = fetchurl {
       url = "http://download.kde.org/stable/${pname}/${version}/src/${name}.tar.xz";
-      sha256 = "0vcdji1k8d3pz7k6lkw8ighkj94zff2l2cf9v1avf83f4hjyfhg5";
+      sha256 = "ef51aa3325916d352fde17870cf706397e41105103e4c9289cc4032a1b8609a7";
     };
 
     buildInputs = [

pkgs/applications/networking/browsers/chromium/common.nix

@@ -1,4 +1,4 @@
-{ stdenv, ninja, which
+{ stdenv, ninja, which, nodejs
 
 # default dependencies
 , bzip2, flac, speex, libopus
@@ -87,7 +87,7 @@ let
 
     nativeBuildInputs = [
       ninja which python2Packages.python perl pkgconfig
-      python2Packages.ply python2Packages.jinja2
+      python2Packages.ply python2Packages.jinja2 nodejs
     ];
 
     buildInputs = defaultDependencies ++ [
@@ -105,6 +105,7 @@ let
 
     patches = [
       ./patches/nix_plugin_paths_52.patch
+      ./patches/fix-bootstrap-gn.patch
     ] ++ optional (versionOlder version "57.0") ./patches/glibc-2.24.patch
       ++ optional enableWideVine ./patches/widevine.patch;
 
@@ -130,6 +131,9 @@ let
       }' gpu/config/gpu_control_list.cc
 
       patchShebangs .
+      # use our own nodejs
+      mkdir -p third_party/node/linux/node-linux-x64/bin
+      ln -s $(which node) third_party/node/linux/node-linux-x64/bin/node
     '' + optionalString (versionAtLeast version "52.0.0.0") ''
       sed -i -re 's/([^:])\<(isnan *\()/\1std::\2/g' \
         third_party/pdfium/xfa/fxbarcode/utils.h

pkgs/applications/networking/browsers/chromium/patches/fix-bootstrap-gn.patch

@@ -0,0 +1,12 @@
+diff --git a/tools/gn/bootstrap/bootstrap.py b/tools/gn/bootstrap/bootstrap.py
+index 3148ccf..38cfb11 100755
+--- a/tools/gn/bootstrap/bootstrap.py
++++ b/tools/gn/bootstrap/bootstrap.py
+@@ -385,6 +385,7 @@ def write_gn_ninja(path, root_gen_dir, options):
+       'base/base_switches.cc',
+       'base/build_time.cc',
+       'base/callback_internal.cc',
++      'base/callback_helpers.cc',
+       'base/command_line.cc',
+       'base/debug/activity_tracker.cc',
+       'base/debug/alias.cc',

pkgs/applications/networking/browsers/chromium/upstream-info.nix

@@ -1,18 +1,18 @@
 # This file is autogenerated from update.sh in the same directory.
 {
   beta = {
-    sha256 = "1q9iqmq5amzfw03jiw18g1w285b6x2qckn8gc60r5m3xx1hbivv2";
-    sha256bin64 = "1ddhhcydcwwc2pkwm4c8rlr60968zy5vda410g4bwx0v5q7p22q9";
-    version = "58.0.3029.68";
+    sha256 = "1dcad79kfayagqiv85ycla3iv3gc99k0rvnvnpar9hd6x1iv8cfl";
+    sha256bin64 = "0ywf50rfzv1kkfpld62fi5g0kz33an0p03xqf7wkcqi7hild607v";
+    version = "59.0.3071.47";
   };
   dev = {
-    sha256 = "0zvnj9n2p057fxx7n4d1qc0nw34qhlsvrx20fwigq96blamckvd8";
-    sha256bin64 = "1s1r3h7x49bp64lzzphm4jcg7g68l0x7mr3airj3hqii58dvndm0";
-    version = "59.0.3067.0";
+    sha256 = "05kxl938mh41341lh5xsxcqxv9hhx8yn6nkawgg241sfwdx72db8";
+    sha256bin64 = "1mr2615r30zjxa55mdcbfi0k2grgyykzawslmg41aw0jy5hfamal";
+    version = "60.0.3095.5";
   };
   stable = {
-    sha256 = "1xwchazqqx0cs9rd15r80kw6p918zp9libx34qlcj8p5lxq1f0bh";
-    sha256bin64 = "0ggn5rljch36sx0i37qzp6ldcy3ibdj0z9217lqzjq3r7ixsfqja";
-    version = "57.0.2987.133";
+    sha256 = "1zvqim75mlqckvf7awrbyggis71dlkz4gjpfrmfdvydcs8yyyk7j";
+    sha256bin64 = "0vfx2m5zqfvfb6sgd3dg1sji72dzjcd1sf4r6qwhnz38wadys7id";
+    version = "58.0.3029.110";
   };
 }

pkgs/applications/networking/browsers/lynx/default.nix

@@ -1,4 +1,4 @@
-{ stdenv, fetchurl, ncurses, gzip
+{ stdenv, fetchurl, ncurses, gzip, pkgconfig
 , sslSupport ? true, openssl ? null
 }:
 
@@ -13,14 +13,10 @@ stdenv.mkDerivation rec {
     sha256 = "1cqm1i7d209brkrpzaqqf2x951ra3l67dw8x9yg10vz7rpr9441a";
   };
 
-  configureFlags = []
-    ++ stdenv.lib.optionals sslSupport [ "--with-ssl=${openssl.dev}" ];
+  configureFlags = [ "--enable-widec" ] ++ stdenv.lib.optional sslSupport "--with-ssl";
 
-  buildInputs = [ ncurses gzip ];
-
-  crossAttrs = {
-    configureFlags = configureFlags ++ [ "--enable-widec" ];
-  };
+  nativeBuildInputs = stdenv.lib.optional sslSupport pkgconfig;
+  buildInputs = [ ncurses gzip ] ++ stdenv.lib.optional sslSupport openssl.dev;
 
   meta = with stdenv.lib; {
     homepage = http://lynx.isc.org/;

pkgs/applications/networking/browsers/tor-browser-bundle-bin/default.nix

@@ -87,12 +87,18 @@ let
 
   srcs = {
     "x86_64-linux" = fetchurl {
-      url = "https://dist.torproject.org/torbrowser/${version}/tor-browser-linux64-${version}_${lang}.tar.xz";
+      urls = [
+        "https://github.com/TheTorProject/gettorbrowser/releases/download/v${version}/tor-browser-linux64-${version}_${lang}.tar.xz"
+        "https://dist.torproject.org/torbrowser/${version}/tor-browser-linux64-${version}_${lang}.tar.xz"
+      ];
       sha256 = "0jn98arczlgjigpmql1qg5b7izabv4zy4mji6vvcg3b8g1ma108r";
     };
 
     "i686-linux" = fetchurl {
-      url = "https://dist.torproject.org/torbrowser/${version}/tor-browser-linux32-${version}_${lang}.tar.xz";
+      urls = [
+        "https://github.com/TheTorProject/gettorbrowser/releases/download/v${version}/tor-browser-linux32-${version}_${lang}.tar.xz"
+        "https://dist.torproject.org/torbrowser/${version}/tor-browser-linux32-${version}_${lang}.tar.xz"
+      ];
       sha256 = "0micxgkbys0py4bj6csbc8xz4gq0x5v2zirgi38krnm5x5riqj3w";
     };
   };
@@ -105,6 +111,7 @@ stdenv.mkDerivation rec {
   src = srcs."${stdenv.system}" or (throw "unsupported system: ${stdenv.system}");
 
   preferLocalBuild = true;
+  allowSubstitutes = false;
 
   desktopItem = makeDesktopItem {
     name = "torbrowser";

pkgs/applications/networking/cluster/helm/default.nix

@@ -34,6 +34,10 @@ stdenv.mkDerivation {
     tar -xvzf $src
     cp ${arch}/helm $out/bin/${pname}
     chmod +x $out/bin/${pname}
+    mkdir -p $out/share/bash-completion/completions
+    mkdir -p $out/share/zsh/site-functions
+    $out/bin/helm completion bash > $out/share/bash-completion/completions/helm
+    $out/bin/helm completion zsh > $out/share/zsh/site-functions/_helm
   '';
 
   meta = with stdenv.lib; {

pkgs/applications/networking/cluster/kops/default.nix

@@ -2,24 +2,37 @@
 
 buildGoPackage rec {
   name = "kops-${version}";
-  version = "1.4.0";
-  rev = "v${version}";
+  version = "1.6.0";
 
   goPackagePath = "k8s.io/kops";
 
   src = fetchFromGitHub {
-    inherit rev;
+    rev = version;
     owner = "kubernetes";
     repo = "kops";
-    sha256 = "1jwgn7l8c639j5annwymqjdw5mcajwn58y21042jy5lhgdh8pdf5";
+    sha256 = "1z890kjgsdnghg71v4sp7lljvw14dhzr23m2qjmk6wndyssscykr";
   };
 
   buildInputs = [go-bindata];
   subPackages = ["cmd/kops"];
 
+  buildFlagsArray = ''
+    -ldflags=
+        -X k8s.io/kops.Version=${version}
+        -X k8s.io/kops.GitVersion=${version}
+  '';
+
   preBuild = ''
     (cd go/src/k8s.io/kops
-     go-bindata -o upup/models/bindata.go -pkg models -prefix upup/models/ upup/models/...)
+     go-bindata -o upup/models/bindata.go -pkg models -prefix upup/models/ upup/models/...
+     go-bindata -o federation/model/bindata.go -pkg model -prefix federation/model federation/model/...)
+  '';
+
+  postInstall = ''
+    mkdir -p $bin/share/bash-completion/completions
+    mkdir -p $bin/share/zsh/site-functions
+    $bin/bin/kops completion bash > $bin/share/bash-completion/completions/kops
+    $bin/bin/kops completion zsh > $bin/share/zsh/site-functions/_kops
   '';
 
   meta = with stdenv.lib; {

pkgs/applications/networking/cluster/pachyderm/default.nix

@@ -2,7 +2,7 @@
 
 buildGoPackage rec {
   name = "pachyderm-${version}";
-  version = "1.3.0";
+  version = "1.4.6";
   rev = "v${version}";
 
   goPackagePath = "github.com/pachyderm/pachyderm";
@@ -12,7 +12,7 @@ buildGoPackage rec {
     inherit rev;
     owner = "pachyderm";
     repo = "pachyderm";
-    sha256 = "0y25xh6h7p8hg0bzrjlschmz62r6dwh5mrvbnni1hb1pm0w9jb6g";
+    sha256 = "1fivihn9s04lmzdiwg0f05qm708fb14xy81pbc31wxdyjw28m8ns";
   };
 
   meta = with lib; {

pkgs/applications/networking/cluster/terraform/default.nix

@@ -48,13 +48,9 @@ in {
     sha256 = "0ibgpcpvz0bmn3cw60nzsabsrxrbmmym1hv7fx6zmjxiwd68w5gb";
   };
 
-  terraform_0_9_4 = generic {
-    version   = "0.9.4";
-    sha256    = "07vcmjyl0y48hm5lqqzdd51hmrxapvywzbdkg5f3rcqd7dn9c2xs";
-    postPatch = ''
-      rm builtin/providers/dns/data_dns_cname_record_set_test.go
-      rm builtin/providers/vsphere/resource_vsphere_file_test.go
-    '';
+  terraform_0_9_6 = generic {
+    version = "0.9.6";
+    sha256 = "1f6z1zkklzpqgc7akgdz1g306ccmhni5lmg7i6g762n3qai60bnv";
     doCheck = true;
   };
 }

pkgs/applications/networking/dropbox/default.nix

@@ -23,11 +23,11 @@
 let
   # NOTE: When updating, please also update in current stable,
   # as older versions stop working
-  version = "26.4.24";
+  version = "27.4.22";
   sha256 =
     {
-      "x86_64-linux" = "1qzz88d3akbqfk1539w2z0ldyjjscqjqgsvadf9i4xr2y0syfv4y";
-      "i686-linux"   = "12xwmmycrg56xl88k9pqd7mcn0jqi4jijf36vn4fdjvmn7ksskcw";
+      "x86_64-linux" = "1v1q799dj0bffg3s98ifzpsccjaajl221h5ckaqijlmpzr2q5hhh";
+      "i686-linux"   = "1qsw6c0mb0n0djz7xpfixispi9k68kbvf8dc92frpcbcvm0h845y";
     }."${stdenv.system}" or (throw "system ${stdenv.system} not supported");
 
   arch =

pkgs/applications/networking/instant-messengers/dino/default.nix

@@ -13,13 +13,13 @@
  }:
 
 stdenv.mkDerivation rec {
-  name = "dino-unstable-2017-05-11";
+  name = "dino-unstable-2017-05-31";
 
   src = fetchFromGitHub {
     owner = "dino";
     repo = "dino";
-    rev = "b09a056a13de131a4f2f072ffa2f795a0bb2abe7";
-    sha256 = "1dis1kgaqb1925anmxlcy9n722zzyn5wvq8lmczi6h2h3j7wnnmz";
+    rev = "2480c1ec26a8e0ccef3ea76e3c29566221405ffb";
+    sha256 = "0wdjj38gbr2j6yklna3pd8nsfpdfp1j936dy8s49pzayw4pxs2g2";
     fetchSubmodules = true;
   };
 

pkgs/applications/networking/instant-messengers/jackline/default.nix

@@ -3,18 +3,16 @@
 assert stdenv.lib.versionAtLeast ocamlPackages.ocaml.version "4.02.2";
 
 stdenv.mkDerivation rec {
-  version = "2016-11-18";
+  version = "2017-05-21";
   name = "jackline-${version}";
 
   src = fetchFromGitHub {
     owner  = "hannesm";
     repo   = "jackline";
-    rev    = "cab34acab004023911997ec9aee8b00a976af7e4";
-    sha256 = "0h7wdsic4v6ys130w61zvxm5s2vc7y574hn7zby12rq88lhhrjh7";
+    rev    = "0a1e4ea23245633fe23edf09b2309659a1bc3649";
+    sha256 = "1wnmwsp3a5nh3qs4h9grrdsvv0i3p419cfmhrrql3lj2x3ngdw82";
   };
 
-  patches = [ ./uchar.patch ];
-
   buildInputs = with ocamlPackages; [
                   ocaml ocamlbuild findlib topkg ppx_sexp_conv
                   erm_xmpp_0_3 tls nocrypto x509 ocaml_lwt otr astring

pkgs/applications/networking/instant-messengers/jackline/uchar.patch

@@ -1,302 +0,0 @@
-diff --git a/_tags b/_tags
-index 88318d9..b433ee8 100644
---- a/_tags
-+++ b/_tags
-@@ -7,9 +7,11 @@ true : package(sexplib astring)
- 
- <src/xconfig.ml>: package(otr ppx_sexp_conv)
- <src/utils.ml>: package(uutf)
-+<src/muc.ml>: package(uchar)
-+<src/contact.mli>: package(uchar)
- <src/persistency.ml>: package(lwt nocrypto)
- <src/xjid.{ml,mli}>: package(ppx_sexp_conv erm_xmpp)
--<src/user.{ml,mli}>: package(ppx_sexp_conv otr hex ptime ptime.clock.os)
-+<src/user.{ml,mli}>: package(uchar ppx_sexp_conv otr hex ptime ptime.clock.os)
- <src/xmpp_callbacks.ml>: package(erm_xmpp lwt tls tls.lwt ptime)
- <src/xmpp_connection.ml>: package(erm_xmpp lwt tls tls.lwt)
- 
-@@ -18,6 +20,6 @@ true : package(sexplib astring)
- <cli/cli_input.ml>: package(notty lwt erm_xmpp otr)
- <cli/cli_commands.ml>: package(lwt otr erm_xmpp)
- <cli/cli_config.ml>: package(lwt nocrypto otr notty tls.lwt x509)
--<cli/cli_state.ml>: package(hex lwt nocrypto erm_xmpp tls.lwt x509)
-+<cli/cli_state.ml>: package(uchar hex lwt nocrypto erm_xmpp tls.lwt x509)
- 
- <bin/jackline.{ml,byte,native}>: package(erm_xmpp hex lwt notty notty.lwt nocrypto otr sexplib tls tls.lwt ptime ptime.clock.os)
-diff --git a/cli/cli_config.ml b/cli/cli_config.ml
-index 618d655..dac6e68 100644
---- a/cli/cli_config.ml
-+++ b/cli/cli_config.ml
-@@ -34,7 +34,7 @@ let rewrap term above below (prefix, inp, inp2) (width, _) =
-     let height = if col mod width = 0 then succ h else h in
-     (succ (col mod width), height)
-   in
--  Notty_lwt.Term.cursor term (Some (col, row))
-+  Notty_lwt.Term.cursor term (Some (col - 1, row - 1))
- 
- let read_line ?(above = []) ?(prefix = "") ?default ?(below = []) term =
-   let rec go (pre, post) =
-@@ -56,8 +56,8 @@ let read_line ?(above = []) ?(prefix = "") ?default ?(below = []) term =
-       | `Unhandled k ->
-         match k with
-         | `Key (`Enter, []) -> Lwt.return (char_list_to_str (pre @ post))
--        | `Key (`Uchar 0x43, [`Ctrl]) -> Lwt.fail (Invalid_argument "Ctrl-c")
--        | `Key (`Uchar 0x44, [`Ctrl]) -> Lwt.fail (Invalid_argument "Ctrl-d")
-+        | `Key (`Uchar u, [`Ctrl]) when Uchar.to_int u = 0x43 -> Lwt.fail (Invalid_argument "Ctrl-c")
-+        | `Key (`Uchar u, [`Ctrl]) when Uchar.to_int u = 0x44 -> Lwt.fail (Invalid_argument "Ctrl-d")
-         | _ -> go (pre, post)
-   in
-   let pre = Utils.option [] str_to_char_list default in
-@@ -180,7 +180,7 @@ let configure term () =
-     let pw = "Password: " in
-     let chars = match password with
-       | None -> I.string A.empty "will be asked at startup"
--      | Some _ -> I.uchar A.empty 0x2605 5 1
-+      | Some _ -> I.uchar A.empty (Uchar.of_int 0x2605) 5 1
-     in
-     above @ [I.(string A.empty pw <|> chars)]
-   in
-diff --git a/cli/cli_input.ml b/cli/cli_input.ml
-index 34b4288..07488f2 100644
---- a/cli/cli_input.ml
-+++ b/cli/cli_input.ml
-@@ -314,19 +314,19 @@ let read_terminal term mvar input_mvar () =
-           | `Key (`Arrow `Up, []) -> p (fun s -> ok (history s Up)) >>= fun () -> loop ()
-           | `Key (`Arrow `Down, []) -> p (fun s -> ok (history s Down)) >>= fun () -> loop ()
- 
--          | `Key (`Uchar 0x44, [`Ctrl]) (* C-d *) -> p (fun s -> Lwt.return (`Quit s))
-+          | `Key (`Uchar u, [`Ctrl]) when Uchar.to_int u = 0x44 (* C-d *) -> p (fun s -> Lwt.return (`Quit s))
- 
-           (* UI navigation and toggles *)
-           | `Key (`Page `Up, []) -> p (fun s -> ok (navigate_buddy_list s Up)) >>= fun () -> loop ()
-           | `Key (`Page `Down, []) -> p (fun s -> ok (navigate_buddy_list s Down)) >>= fun () -> loop ()
- 
-           | `Key (`Page `Up, [`Ctrl]) -> p (fun s -> ok (navigate_message_buffer s Up)) >>= fun () -> loop ()
--          | `Key (`Uchar 0x50, [`Ctrl]) (* C-p *) -> p (fun s -> ok (navigate_message_buffer s Up)) >>= fun () -> loop ()
-+          | `Key (`Uchar u, [`Ctrl]) when Uchar.to_int u = 0x50 (* C-p *) -> p (fun s -> ok (navigate_message_buffer s Up)) >>= fun () -> loop ()
-           | `Key (`Page `Down, [`Ctrl]) -> p (fun s -> ok (navigate_message_buffer s Down)) >>= fun () -> loop ()
--          | `Key (`Uchar 0x4E, [`Ctrl]) (* C-n *) -> p (fun s -> ok (navigate_message_buffer s Down)) >>= fun () -> loop ()
-+          | `Key (`Uchar u, [`Ctrl]) when Uchar.to_int u = 0x4E (* C-n *) -> p (fun s -> ok (navigate_message_buffer s Down)) >>= fun () -> loop ()
- 
--          | `Key (`Uchar 0x58, [`Ctrl]) (* C-x *) -> p (fun s -> ok (activate_contact s s.last_active_contact)) >>= fun () -> loop ()
--          | `Key (`Uchar 0x51, [`Ctrl]) (* C-q *) ->
-+          | `Key (`Uchar u, [`Ctrl]) when Uchar.to_int u = 0x58 (* C-x *) -> p (fun s -> ok (activate_contact s s.last_active_contact)) >>= fun () -> loop ()
-+          | `Key (`Uchar u, [`Ctrl]) when Uchar.to_int u = 0x51 (* C-q *) ->
-             let handle s =
-               let s = match List.rev s.notifications with
-                 | x::_ -> activate_contact s x
-diff --git a/cli/cli_state.ml b/cli/cli_state.ml
-index 5603cfe..ee320ce 100644
---- a/cli/cli_state.ml
-+++ b/cli/cli_state.ml
-@@ -24,7 +24,7 @@ type connect_v =
-   | Reconnect
-   | Presence of (User.presence * string option * int option)
- 
--type input = int list * int list
-+type input = Uchar.t list * Uchar.t list
- 
- type state = {
-   (* set only initially *)
-diff --git a/cli/cli_support.ml b/cli/cli_support.ml
-index 1c54df6..8275c38 100644
---- a/cli/cli_support.ml
-+++ b/cli/cli_support.ml
-@@ -4,17 +4,17 @@ open Notty
- module Char = struct
-   let hdash a w =
-     if !Utils.unicode then
--      I.uchar a 0x2500 w 1
-+      I.uchar a (Uchar.of_int 0x2500) w 1
-     else
-       I.char a '-' w 1
-   and vdash a h =
-     if !Utils.unicode then
--      I.uchar a 0x2502 1 h
-+      I.uchar a (Uchar.of_int 0x2502) 1 h
-     else
-       I.char a '|' 1 h
-   and star a w =
-     if !Utils.unicode then
--      I.uchar a 0x2605 w 1
-+      I.uchar a (Uchar.of_int 0x2605) w 1
-     else
-       I.char a '*' w 1
- end
-@@ -186,8 +186,8 @@ let v_center left right width =
-   and rw = I.width right
-   in
-   match rw, lw >= width with
--  | 0, true -> (I.hcrop (lw - width + 1) 0 left, width)
--  | 0, false -> (left, succ lw)
-+  | 0, true -> (I.hcrop (lw - width + 1) 0 left, width - 1)
-+  | 0, false -> (left, lw)
-   | _, _ ->
-     if lw + rw >= width then
-       let leftw = min (max (width - rw) (width / 2)) lw in
-@@ -195,11 +195,11 @@ let v_center left right width =
-       let l = I.hcrop (lw - leftw) 0 left
-       and r = I.hcrop 0 (rw - rightw) right
-       in
--      (I.(l <|> r), succ leftw)
-+      (I.(l <|> r), leftw)
-     else
--      (I.(left <|> right), succ lw)
-+      (I.(left <|> right), lw)
- 
--let str_to_char_list str : int list =
-+let str_to_char_list str : Uchar.t list =
-   let open Uutf in
-   List.rev (String.fold_utf_8 (fun acc _ -> function `Uchar i -> i :: acc | `Malformed _ -> acc) [] str)
- 
-@@ -236,22 +236,26 @@ let readline_input = function
-   | k -> `Unhandled k
- 
- let emacs_bindings = function
--  | `Key (`Uchar 0x41, [`Ctrl]) (* C-a *) -> `Ok (fun (pre, post) -> ([], pre @ post))
--  | `Key (`Uchar 0x45, [`Ctrl]) (* C-e *) -> `Ok (fun (pre, post) -> (pre @ post, []))
-+  | `Key (`Uchar u, [`Ctrl]) as k ->
-+    begin match Uchar.to_int u with
-+    | 0x41 (* C-a *) -> `Ok (fun (pre, post) -> ([], pre @ post))
-+    | 0x45 (* C-e *) -> `Ok (fun (pre, post) -> (pre @ post, []))
- 
--  | `Key (`Uchar 0x4b, [`Ctrl]) (* C-k *) -> `Ok (fun (pre, _) -> (pre, []))
--  | `Key (`Uchar 0x55, [`Ctrl]) (* C-u *) -> `Ok (fun (_, post) -> ([], post))
-+    | 0x4b (* C-k *) -> `Ok (fun (pre, _) -> (pre, []))
-+    | 0x55 (* C-u *) -> `Ok (fun (_, post) -> ([], post))
- 
--  | `Key (`Uchar 0x46, [`Ctrl]) (* C-f *) ->
-+    | 0x46 (* C-f *) ->
-     `Ok (fun (pre, post) ->
-         match post with
-         | [] -> (pre, post)
-         | hd::tl -> (pre @ [hd], tl))
--  | `Key (`Uchar 0x42, [`Ctrl]) (* C-b *) ->
-+    | 0x42 (* C-b *) ->
-     `Ok (fun (pre, post) ->
-         match List.rev pre with
-         | [] -> ([], post)
-         | hd::tl -> (List.rev tl, hd :: post))
-+    | _ -> `Unhandled k
-+    end
- 
-   | `Key (`Arrow `Left, [`Ctrl]) ->
-     `Ok (fun (pre, post) ->
-diff --git a/src/contact.mli b/src/contact.mli
-index 6926296..d6c795b 100644
---- a/src/contact.mli
-+++ b/src/contact.mli
-@@ -8,7 +8,7 @@ val bare : contact -> Xjid.bare_jid
- val preserve_messages : contact -> bool
- val expanded : contact -> bool
- val messages : contact -> User.message list
--val input_buffer : contact -> (int list * int list)
-+val input_buffer : contact -> (Uchar.t list * Uchar.t list)
- 
- val readline_history : contact -> string list
- val add_readline_history : contact -> string -> contact
-@@ -18,7 +18,7 @@ val set_history_position : contact -> int -> contact
- val received : contact -> string -> contact
- 
- val expand : contact -> contact
--val set_input_buffer : contact -> (int list * int list) -> contact
-+val set_input_buffer : contact -> (Uchar.t list * Uchar.t list) -> contact
- val set_preserve_messages : contact -> bool -> contact
- 
- val reset : contact -> contact
-diff --git a/src/muc.ml b/src/muc.ml
-index 1c98037..3293541 100644
---- a/src/muc.ml
-+++ b/src/muc.ml
-@@ -132,7 +132,7 @@ type groupchat = {
-   expand : bool ;
-   preserve_messages : bool ;
-   message_history : User.message list ; (* persistent if preserve_messages *)
--  input_buffer : (int list * int list) ;
-+  input_buffer : (Uchar.t list * Uchar.t list) ;
-   readline_history : string list ;
-   history_position : int ;
-   autojoin : bool ;
-diff --git a/src/user.ml b/src/user.ml
-index d039278..42a8c47 100644
---- a/src/user.ml
-+++ b/src/user.ml
-@@ -229,7 +229,7 @@ type user = {
-   properties        : property list ;
-   preserve_messages : bool ;
-   message_history   : message list ; (* persistent if preserve_messages is true *)
--  input_buffer: (int list * int list) ; (* not persistent *)
-+  input_buffer: (Uchar.t list * Uchar.t list) ; (* not persistent *)
-   readline_history  : string list ; (* not persistent *)
-   history_position  : int ; (* not persistent *)
-   otr_fingerprints  : fingerprint list ;
-diff --git a/src/user.mli b/src/user.mli
-index 52b503d..5ce41be 100644
---- a/src/user.mli
-+++ b/src/user.mli
-@@ -118,7 +118,7 @@ type user = {
-   properties        : property list ;
-   preserve_messages : bool ;
-   message_history   : message list ; (* persistent if preserve_messages is true *)
--  input_buffer: (int list * int list) ; (* not persistent *)
-+  input_buffer: (Uchar.t list * Uchar.t list) ; (* not persistent *)
-   readline_history  : string list ; (* not persistent *)
-   history_position  : int ;
-   otr_fingerprints  : fingerprint list ;
-diff --git a/src/utils.ml b/src/utils.ml
-index 0b4a3a7..cd9cb10 100644
---- a/src/utils.ml
-+++ b/src/utils.ml
-@@ -30,31 +30,33 @@ let validate_utf8 txt =
-   let rec loop d buf = match Uutf.decode d with
-     | `Await -> Buffer.contents buf
-     | `End -> Buffer.contents buf
--    | `Malformed _ -> if !unicode then Uutf.Buffer.add_utf_8 buf 0xFFFD; loop d buf
--    | `Uchar 0x000A -> (* newline *) Uutf.Buffer.add_utf_8 buf 0x000A ; loop d buf
--    | `Uchar 0x0009 -> (* tab -> 4 spaces *) Uutf.Buffer.add_utf_8 buf 0x0020 ; Uutf.Buffer.add_utf_8 buf 0x0020 ; Uutf.Buffer.add_utf_8 buf 0x0020 ; Uutf.Buffer.add_utf_8 buf 0x0020 ; loop d buf
--    | `Uchar 0x007F (* DEL *)
-+    | `Malformed _ -> if !unicode then Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0xFFFD); loop d buf
-+    | `Uchar u ->
-+      match Uchar.to_int u with
-+      | 0x000A -> (* newline *) Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0x000A) ; loop d buf
-+      | 0x0009 -> (* tab -> 4 spaces *) Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0x0020) ; Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0x0020) ; Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0x0020) ; Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0x0020) ; loop d buf
-+      | 0x007F (* DEL *)
-     (* See https://en.wikipedia.org/wiki/Unicode_control_characters / https://en.wikipedia.org/wiki/Bi-directional_text *)
--    | `Uchar 0x200E | `Uchar 0x200F (* left-to-right / right-to-left *)
--    | `Uchar 0x202A | `Uchar 0x202D (* left-to-right embedding / override *)
--    | `Uchar 0x202B | `Uchar 0x202E (* right-to-left embedding / override *)
--    | `Uchar 0x202C  (* pop directional format *)
--    | `Uchar 0x2066 | `Uchar 0x2067 (* l-t-r isolate r-t-l isolate *)
--    | `Uchar 0x2068 | `Uchar 0x2069 (* first strong isolate / pop directional isolate *)
--    | `Uchar 0x2028 | `Uchar 0x2029 (* line separator / page separator *) ->
--      if !unicode then Uutf.Buffer.add_utf_8 buf 0xFFFD ; loop d buf
--    | `Uchar x when x < 0x20 ->
-+      | 0x200E | 0x200F (* left-to-right / right-to-left *)
-+      | 0x202A | 0x202D (* left-to-right embedding / override *)
-+      | 0x202B | 0x202E (* right-to-left embedding / override *)
-+      | 0x202C  (* pop directional format *)
-+      | 0x2066 | 0x2067 (* l-t-r isolate r-t-l isolate *)
-+      | 0x2068 | 0x2069 (* first strong isolate / pop directional isolate *)
-+      | 0x2028 | 0x2029 (* line separator / page separator *) ->
-+      if !unicode then Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0xFFFD) ; loop d buf
-+      | x when x < 0x20 ->
-       (* other control characters *)
--      if !unicode then Uutf.Buffer.add_utf_8 buf 0xFFFD ; loop d buf
--    | `Uchar x when x >= 0x0080 && x <= 0x009F ->
-+      if !unicode then Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0xFFFD) ; loop d buf
-+      | x when x >= 0x0080 && x <= 0x009F ->
-       (* ctrl chars used in conjunction with ISO 8859 character sets (C0/C1) *)
--      if !unicode then Uutf.Buffer.add_utf_8 buf 0xFFFD ; loop d buf
-+      if !unicode then Uutf.Buffer.add_utf_8 buf (Uchar.of_int 0xFFFD) ; loop d buf
- 
--    | `Uchar x ->
-+      | x ->
-       let c = if !unicode then x else if x > 0xff then 0x3f else x in
--      Uutf.Buffer.add_utf_8 buf c ; loop d buf
-+      Uutf.Buffer.add_utf_8 buf (Uchar.of_int c) ; loop d buf
-   in
--  let nln = `Readline 0x000A in
-+  let nln = `Readline (Uchar.of_int 0x000A) in
-   loop (Uutf.decoder ~nln ~encoding:`UTF_8 (`String txt)) (Buffer.create (String.length txt))
- 
- let version = "%%VERSION_NUM%%"

pkgs/applications/networking/instant-messengers/qtox/default.nix

@@ -6,13 +6,13 @@
 
 stdenv.mkDerivation rec {
   name = "qtox-${version}";
-  version = "1.10.1";
+  version = "1.10.2";
 
   src = fetchFromGitHub {
     owner  = "tux3";
     repo   = "qTox";
     rev    = "v${version}";
-    sha256 = "1c5y7fwhsq1f6z8208xl1jd6bl1r6k8g0fjqxf0z10373c9395jq";
+    sha256 = "0c2633rc9l73q9qs9hybn11hmlqbwsvih3sf6jk1jp4151k5wp1y";
   };
 
   buildInputs = [

pkgs/applications/networking/instant-messengers/scudcloud/default.nix

@@ -1,14 +1,12 @@
-{ stdenv, fetchgit, python3Packages }:
+{ stdenv, fetchurl, python3Packages }:
 
-python3Packages.buildPythonPackage {
-  name = "scudcloud-1.44";
+let version = "1.58";
+in python3Packages.buildPythonPackage {
+  name = "scudcloud-${version}";
 
-  # Branch 254-port-to-qt5
-  # https://github.com/raelgc/scudcloud/commit/65c304416dfdd5f456fa6f7301432a953d5e12d0
-  src = fetchgit {
-      url = https://github.com/raelgc/scudcloud/;
-      rev = "65c304416dfdd5f456fa6f7301432a953d5e12d0";
-      sha256 = "0h1055y88kldqw31ayqfx9zsksgxsyqd8h0hwnhj80yn3jcx0rp6";
+  src = fetchurl {
+    url = "https://github.com/raelgc/scudcloud/archive/v${version}.tar.gz";
+    sha256 = "1j84qdc2j3dvl1nhrjqm0blc8ww723p9a6hqprkkp8alw77myq1l";
   };
 
   propagatedBuildInputs = with python3Packages; [ pyqt5 dbus-python ];

pkgs/applications/networking/insync/default.nix

@@ -2,10 +2,10 @@
 
 stdenv.mkDerivation rec {
   name = "insync-${version}";
-  version = "1.3.13.36129";
+  version = "1.3.16.36155";
   src = fetchurl {
     url = "http://s.insynchq.com/builds/insync-portable_${version}_amd64.tar.bz2";
-    sha256 = "18d8ww529nvhwcl5k31qmkzb83k9753ics0dw64w202r8vwbm3cd";
+    sha256 = "1gf1qg7mkbcgqhwxkiljmd1w2zvarq6vhxhips3w06bqdyg12210";
   };
 
   buildInputs = [ makeWrapper ];