To achieve reproducible results, `cpio` archive members are added in
sorted order and inodes renumbered.
The `cpio-clean.pl` script is made obsolete by setting mtimes via
`touch` & using `cpio --reproducible`. Suggested by @dezgeg in
https://github.com/NixOS/nixpkgs/pull/21273#issuecomment-268116605.
Note that using `--reproducible` means that initial ramdisk creation now
requires at least `cpio` version 2.12 (released in 2015).
Deprecation warnings should not be used in Nixpkgs because they spam
innocent "nix-env -qa" users with (in this case) dozens of messages
that they can't do anything about.
This also reverts commit 2ca8833383.
I believe this reduces surprises and is actually simpler semantically.
This is important e.g. for relative symlinks when moving both source
and target - now the order of moving won't matter.
Fixes#20723 (a particular instance of the surprise).
This makes the response file handling more consistent with GCC.
For example, a reponse file may contain:
"-Wl,$ORIGIN"
GCC will treat this as a double quoted string and not expand the
variable reference. Previously, cc-wrapper would expand the variable
in the same was as if the string was provided on the command line.
This commit extends fetchFromGitHub with ability to fetch GitHub
repositories with submodules, so we can use the function consistently
with all GitHub repositories.
Note it doesn't change the previous behavior.
Include files are typically in the libc package's `dev` output but previously `-isystem` was looking in `out`, resulting in my cross-compilation environment not finding its include files.
The motivation for this change is the following: As gnu-netcat,
e. g. does not support ipv6, it is not suitable as default netcat.
This commit also fixes all obvious build issues caused by this change.
This is a temporary work-around to fix using grsecurity on NixOS with
the new kernelPackages/kernelPatches machinery.
For whatever reason, when `security.grsecurity.enable = true`, the grsec
patch ends up being applied twice, causing the kernel build to fail.
Until the root cause of this is identified, we hack around it by simply
pruning duplicate patches in the grsec kernel builder.
Closes#19698
`stripHash` documentation states that it prints out the stripped name to
the stdout, but the function stored the value in `strippedName`
instead.
Basically all usages did something like
`$(stripHash $foo | echo $strippedName)` which is just braindamaged.
Fixed the implementation and all invocations.
extraFlagsArray should not be exposed outside of `makeWrapper`, it
should only be possible to set it inside a script supplied via the
`--run` argument.
This ensures that most "trivial" derivations used to build NixOS
configurations no longer depend on GCC. For commands that do invoke
gcc, there is runCommandCC.
This is a standard environment that doesn't contain a C/C++
compiler. This is mostly to prevent trivial builders like runCommand
and substituteAll from pulling in gcc for simple configuration changes
on NixOS.
When building an image with multiple layers, files
already included in an underlying layer are supposed to
be excluded from the current layer. However, some subtleties
in the way filepaths are compared seem to be blocking this.
Specifically:
* tar generates relative filepaths with directories ending in '/'
* find generates absolute filepaths with no trailing slashes on directories
That is, paths extracted from the underlying tarball look like:
nix/store/.../foobar/
whereas the layer being generated uses paths like:
/nix/store/.../foobar
This patch modifies the output of "tar -t" to match the latter format.
They now go to devman, devdoc, or $outputMan, in that order. This is
to prevent cases such as the man-pages package quietly losing its
section 3 pages.
This builds elisp to setup an emacs buffer with the packages given
available. See shlevy/nix-buffer for more information.
Currently only modifies $PATH.
This one was already merged into release-16.09, so let's not have the
stable branch is ahead of master and confuse things. In addition to
that, currently we have an odd situation that master has less things
actually finished building than in staging.
Conflicts:
pkgs/data/documentation/man-pages/default.nix
This was one of the ways to build packages, we are trying
hard to minimize different ways so it's easier for newcomers
to learn only one way.
This also:
- removes texLive (old), fixes#14807
- removed upstream-updater, if that code is still used it should be in
separate repo
- changes a few packages like gitit/mit-scheme to use new texlive
Previously, features.grsecurity wasn't actually set due to a bug in the
grsec builder. We now rely on the generic kernel builder to set features
from kernelPatches.
This reverts commit da68127737, reversing
changes made to 2ba494b728.
These changes are reverted because they cause an incompatibility which
they should not.
"emacsWrapper" replaces emacsWithPackages. In addition to "packagesFun",
emacsWrapper has an optional variable called "execStart". execStart can
be used to append elisp to the default site-start.el script. This is
useful for providing a way to load a user's .emacs.d/init.el
file. "emacsWithPackages" is implemented with emacsWrapper for
convenience and compatability.
Sometimes patches start without a leading prefix. We default to strip
one prefix or path component from patches (-p1) in the patchPhase in
stdenv.
As all patches should therefore be in this format, fetchpatch should
have an option to normalize patch paths. This commit introduces a new
argument to fetchpatch called addPrefixes that adds one patch prefix to
the old and new paths in a patch before putting it into the store.
fetchMavenArtifact downloads a Maven artifact given a group id, an artifact id,
and a version.
Example usage:
org_apache_httpcomponents_httpclient_4_5_2 = fetchMavenArtifact {
groupId = "org.apache.httpcomponents";
artifactId = "httpclient";
version = "4.5.2";
sha256 = "0ms00zc28pwqk83nwwbafhq6p8zci9mrjzbqalpn6v0d80hwdzqd";
# Optionally: repos = [ ... urls to some Maven repos to use ... ];
# Optionally: url, urls - pointing directly to a specific jar url.
};
Now `org_apache_httpcomponents_httpclient_4_5_2.jar` points to the downloaded
JAR file, while `org_apache_httpcomponents_httpclient_4_5_2` refers to a
derivation that when used used in `buildInputs` will be automatically added to
the Java classpath.
downloads.sourceforge.net is the official way to download tarballs from
SourceForge. However, it is reported as unreliable due to SF's weird
load balancing system.
This commit gives the official mirror utmost priority, and will use
other configured mirrors (which may be temporary) as a fallback only
when the official one can't be reached/download fails/hangs.
References: NixOs/nixpkgs#16900
This patch fixes#16614 and #16741.
The first issue was caused by the fact that both `/share` and
`/share/fish/vendor_completions.d` end in the `pathsToLink`. The
`pkgs/build-support/buildenv/builder.pl` creates `/share`, then links
`/share/fish` under `/share` and then tries to create the directory
`/share/fish/vendor_completions.d` and fails because it already exists.
The simplest way to reproduce the issue is to build the next Nix
expression:
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.fish
pkgs.vim
];
pathsToLink = [
"/share"
"/share/fish/vendor_completions.d"
];
}
```
The second issue is more critical and was caused by the fact findFiles
doesn't recurse deep enough. It stops at first unique directory for the
package (e.g., "/share" or even "/") and later the scripts decides it
shouldn't link it as it doesn't match pathsToLink (e.g., "/share/fish"),
so the result is empty.
The test:
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.fish
pkgs.vim
];
pathsToLink = [
"/share/fish/functions"
];
}
```
or
```nix
let pkgs = import <nixpkgs> { };
in pkgs.buildEnv {
name = "buildenv-issue";
paths = [
pkgs.vim
];
pathsToLink = [
"/share"
];
}
```
Prior to this commit, trailing whitespace would be introduced when
modifying '#!' lines with no arguments. For example (whitespace added):
/nix/store/.../foo: interpreter directive changed
from "/bin/bash"
to "/nix/store/...-bash-4.3-p42/bin/bash "
/nix/store/.../bar: interpreter directive changed
from "/bin/baz wef"
to "/nix/store/...-baz wef "
We add a sed command to strip trailing whitespace, so the above commands
would drop the two spaces after "bash", or the one space after "baz wef".
abbradar: fixed commit title
Closes#16785.
This fixes a regression caused by commit f56ab9e
("nix-prefetch-git: Include the date in the machine-readable [...]")
where a couple of directory paths printed by pushd/popd appeared before
the JSON output on stdout (thus breaking it). Fix it by redirecting the
extraneous output to /dev/null.
Reported by Michael Alan Dorman <mdorman@ironicdesign.com>.
stdout, in strict ISO 8601 format.
This will be helpful for automatically updating fetchgit expressions
and the dates in version numbers associated with them.
stripHash uses a global variable to communicate it's computation
results, but it's not necessary. You can just pipe to stdout in a
subshell. A function mostly behaves like just another command.
baseHash() also introduces a suffix-stripping capability since it's
something the users of the function tend to use.
The tests need to expand passed variable and very carefully.
I could see no other easy way than to change single-quoting in
makeWrapper to double-quoting.
The tests now fail with the same problem as on master...
This patch replaces the old grsecurity kernels with a single NixOS
specific grsecurity kernel. This kernel is intended as a general
purpose kernel, tuned for casual desktop use.
Providing only a single kernel may seem like a regression compared to
offering a multitude of flavors. It is impossible, however, to
effectively test and support that many options. This is amplified by
the reality that very few seem to actually use grsecurity on NixOS,
meaning that bugs go unnoticed for long periods of time, simply because
those code paths end up never being exercised. More generally, it is
hopeless to anticipate imagined needs. It is better to start from a
solid foundation and possibly add more flavours on demand.
While the generic kernel is intended to cover a wide range of use cases,
it cannot cover everything. For some, the configuration will be either
too restrictive or too lenient. In those cases, the recommended
solution is to build a custom kernel --- this is *strongly* recommended
for security sensitive deployments.
Building a custom grsec kernel should be as simple as
```nix
linux_grsec_nixos.override {
extraConfig = ''
GRKERNSEC y
PAX y
# and so on ...
'';
}
```
The generic kernel should be usable both as a KVM guest and host. When
running as a host, the kernel assumes hardware virtualisation support.
Virtualisation systems other than KVM are *unsupported*: users of
non-KVM systems are better served by compiling a custom kernel.
Unlike previous Grsecurity kernels, this configuration disables `/proc`
restrictions in favor of `security.hideProcessInformation`.
Known incompatibilities:
- ZFS: can't load spl and zfs kernel modules; claims incompatibility
with KERNEXEC method `or` and RAP; changing to `bts` does not fix the
problem, which implies we'd have to disable RAP as well for ZFS to
work
- `kexec()`: likely incompatible with KERNEXEC (unverified)
- Xen: likely incompatible with KERNEXEC and UDEREF (unverified)
- Virtualbox: likely incompatible with UDEREF (unverified)
Close#15803. This avoids the error:
while setting up the build environment: executing
‘/nix/store/7sb42axk5lrxqz45nldrb2pchlys14s1-bash-4.3-p42/bin/bash’:
Argument list too long
Note: I wanted to make it optional based on buildCommand length,
but that seems pointless as I'm sure it's less performant.
Amended by vcunat:
https://github.com/NixOS/nixpkgs/pull/15803#issuecomment-224841225
This takes another approach at binding FHS directory structure. We
now bind-mount all the root filesystem to directory "/host" in the target tree.
From that we symlink all the directories into the tree if they do not already
exist in FHS structure.
This probably makes `CHROOTENV_EXTRA_BINDS` unnecessary -- its main usecase was
to add bound directories from the host to the sandbox, and we not just symlink
all of them. I plan to get some feedback on its usage and maybe deprecate it.
This also drops old `buildFHSChrootEnv` infrastructure. The main problem with it
is it's very difficult to unmount a recursive-bound directory when mount is not
sandboxed. This problem is a bug even without these changes -- if
you have for example `/home/alice` mounted to somewhere, you wouldn't see
it in `buildFHSChrootEnv` now. With the new directory structure, it's
impossible to use regular bind at all. After some tackling with this I realized
that the fix would be brittle and dangerous (if you don't unmount everything
clearly and proceed to removing the temporary directory, bye-bye fs!). It also
probably doesn't worth it because I haven't heard that someone actually uses it
for a long time, and `buildFHSUserEnv` should cover most cases while being much
more maintainable and safe for the end-user.
1. When multiple versions of the same package are required
$revs is an array.
2. When cargo fetch is run it usually doesn't need a network
connection. But when it does SSL_CERT_FILE isn't set.
find fails when called with an inexistent search path.
That situation may arise when the output is created after by a postFixup hook.
vcunat amended the PR by clarifying one more `return` to `return 0`.
Checkinstall had two problems:
1. when it was called without a version (e.g. with a derivation created
by fetchFromGitHub) it would use `src` as debian version, which caused
dpkg to fail
2. when dpkg failed, it would invoke the pager with the log, which hangs
the build
So now
1. the default version is the dummy `0.0.0`
2. the used pager is `cat`
Source of this change goes back to 2009 and original version of
fetchgit at 205fb0c87e.
The nondeterminism is really caused by changing .git so leave other
files alone as they might be interesting.
Note: this causes a hash mismatch with Hydra's version of Git Plugin
which we should fix to comply.
Move all the dependencies to their own derivation, so that we don't publish all
of them if the wrapper is installed in a profile.
The previous solution just moved them to a custom directory to avoid conflicts,
this refactors that and completely hides them, while preserving the desired
improvement of adding only one directory to each of the emacs search paths
The chroot caps restriction disallows chroot'ed processes from running
any command that requires `CAP_SYS_ADMIN`, breaking `nixos-rebuild`. See
e.g., https://github.com/NixOS/nixpkgs/issues/15293
This significantly weakens chroot protections, but to break
nixos-rebuild out of the box is too severe.
There are no users of it in main tree and recent merge
of multiple outputs branch makes it obsolete for private trees
too.
At the time hook was created, recently merged multiple output
branch was relying on passing flags to autotools to split
outputs, which obviously wasn't working for other build systems
Scatter output was taking different approach where files were
moved out from a build tree based on known paths, which is more
or less what current multiple-outputs.sh hook is able to do too.
A nix specific set of tools for converting icon files
that are not in a freedesktop ready format.
I plan on using these tools for both `keepass` and
`retroarch` packages. It may benifit many other packages.
For every *.{exe,dll} in $output/bin/ we try to find all (potential)
transitive dependencies and symlink those DLLs into $output/bin
so they are found on invocation.
(DLLs are first searched in the directory of the running exe file.)
The links are relative, so relocating whole /nix/store won't break them.
The hook is activated on cygwin and when cross-compiling to mingw.
- the default --docdir is typically DATAROOTDIR/doc/pkgName
- I saw no other way than to employ some magic to guess this `pkgName`
- user can override it by setting $shareDocName
I supplied meta.outputsToInstall automatically in all
mkDerivation products, but some packages still don't use it.
The reported case: jekyll -> bundlerEnv -> buildEnv -> runCommand.
The /nix path in 4d200538 of the layer tar didn't exist for some
packages, such as cacert. This is because cacert just creates an /etc
directory and doesn't depend on any other /nix paths. If we tried
putting this directory in the tar and using overlayfs with it, we'd get
"Invalid argument" when trying to remove the directory.
We now check whether the closure is non-empty before telling tar to
store the /nix directory.
Fixes#14710.
There were two sources of non-determinisim coming into the images. The
first was tar mtimes, the second was pigz/gzip times.
An example image now passes with the --check flag.
