Merge branch 'containers'

Fixes #2105.
This commit is contained in:
Eelco Dolstra 2014-04-10 15:55:51 +02:00
commit d2155649af
17 changed files with 973 additions and 52 deletions

View File

@ -31,8 +31,7 @@ GetOptions("package|p=s" => \$filter,
"maintainer|m=s" => \$maintainer,
"file|f=s" => \$path,
"help" => sub { showHelp() }
)
or die("syntax: $0 ...\n");
) or exit 1;
# Evaluate Nixpkgs into an XML representation.
my $xml = `nix-env -f '$path' -qa '$filter' --xml --meta --drv-path`;

View File

@ -0,0 +1,242 @@
<chapter xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xml:id="ch-containers">
<title>Containers</title>
<para>NixOS allows you to easily run other NixOS instances as
<emphasis>containers</emphasis>. Containers are a light-weight
approach to virtualisation that runs software in the container at the
same speed as in the host system. NixOS containers share the Nix store
of the host, making container creation very efficient.</para>
<warning><para>Currently, NixOS containers are not perfectly isolated
from the host system. This means that a user with root access to the
container can do things that affect the host. So you should not give
container root access to untrusted users.</para></warning>
<para>NixOS containers can be created in two ways: imperatively, using
the command <command>nixos-container</command>, and declaratively, by
specifying them in your <filename>configuration.nix</filename>. The
declarative approach implies that containers get upgraded along with
your host system when you run <command>nixos-rebuild</command>, which
is often not what you want. By contrast, in the imperative approach,
containers are configured and updated independently from the host
system.</para>
<section><title>Imperative container management</title>
<para>Well cover imperative container management using
<command>nixos-container</command> first. You create a container with
identifier <literal>foo</literal> as follows:
<screen>
$ nixos-container create foo
</screen>
This creates the containers root directory in
<filename>/var/lib/containers/foo</filename> and a small configuration
file in <filename>/etc/containers/foo.conf</filename>. It also builds
the containers initial system configuration and stores it in
<filename>/nix/var/nix/profiles/per-container/foo/system</filename>. You
can modify the initial configuration of the container on the command
line. For instance, to create a container that has
<command>sshd</command> running, with the given public key for
<literal>root</literal>:
<screen>
$ nixos-container create foo --config 'services.openssh.enable = true; \
users.extraUsers.root.openssh.authorizedKeys.keys = ["ssh-dss AAAAB3N…"];'
</screen>
</para>
<para>Creating a container does not start it. To start the container,
run:
<screen>
$ nixos-container start foo
</screen>
This command will return as soon as the container has booted and has
reached <literal>multi-user.target</literal>. On the host, the
container runs within a systemd unit called
<literal>container@<replaceable>container-name</replaceable>.service</literal>.
Thus, if something went wrong, you can get status info using
<command>systemctl</command>:
<screen>
$ systemctl status container@foo
</screen>
</para>
<para>If the container has started succesfully, you can log in as
root using the <command>root-login</command> operation:
<screen>
$ nixos-container root-login foo
[root@foo:~]#
</screen>
Note that only root on the host can do this (since there is no
authentication). You can also get a regular login prompt using the
<command>login</command> operation, which is available to all users on
the host:
<screen>
$ nixos-container login foo
foo login: alice
Password: ***
</screen>
With <command>nixos-container run</command>, you can execute arbitrary
commands in the container:
<screen>
$ nixos-container run foo -- uname -a
Linux foo 3.4.82 #1-NixOS SMP Thu Mar 20 14:44:05 UTC 2014 x86_64 GNU/Linux
</screen>
</para>
<para>There are several ways to change the configuration of the
container. First, on the host, you can edit
<literal>/var/lib/container/<replaceable>name</replaceable>/etc/nixos/configuration.nix</literal>,
and run
<screen>
$ nixos-container update foo
</screen>
This will build and activate the new configuration. You can also
specify a new configuration on the command line:
<screen>
$ nixos-container update foo --config 'services.httpd.enable = true; \
services.httpd.adminAddr = "foo@example.org";'
$ curl http://$(nixos-container show-ip foo)/
&lt;!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">…
</screen>
However, note that this will overwrite the containers
<filename>/etc/nixos/configuration.nix</filename>.</para>
<para>Alternatively, you can change the configuration from within the
container itself by running <command>nixos-rebuild switch</command>
inside the container. Note that the container by default does not have
a copy of the NixOS channel, so you should run <command>nix-channel
--update</command> first.</para>
<para>Containers can be stopped and started using
<literal>nixos-container stop</literal> and <literal>nixos-container
start</literal>, respectively, or by using
<command>systemctl</command> on the containers service unit. To
destroy a container, including its file system, do
<screen>
$ nixos-container destroy foo
</screen>
</para>
</section>
<section><title>Declarative container specification</title>
<para>You can also specify containers and their configuration in the
hosts <filename>configuration.nix</filename>. For example, the
following specifies that there shall be a container named
<literal>database</literal> running PostgreSQL:
<programlisting>
containers.database =
{ config =
{ config, pkgs, ... }:
{ services.postgresql.enable = true;
services.postgresql.package = pkgs.postgresql92;
};
};
</programlisting>
If you run <literal>nixos-rebuild switch</literal>, the container will
be built and started. If the container was already running, it will be
updated in place, without rebooting.</para>
<para>By default, declarative containers share the network namespace
of the host, meaning that they can listen on (privileged)
ports. However, they cannot change the network configuration. You can
give a container its own network as follows:
<programlisting>
containers.database =
{ privateNetwork = true;
hostAddress = "192.168.100.10";
localAddress = "192.168.100.11";
};
</programlisting>
This gives the container a private virtual Ethernet interface with IP
address <literal>192.168.100.11</literal>, which is hooked up to a
virtual Ethernet interface on the host with IP address
<literal>192.168.100.10</literal>. (See the next section for details
on container networking.)</para>
<para>To disable the container, just remove it from
<filename>configuration.nix</filename> and run <literal>nixos-rebuild
switch</literal>. Note that this will not delete the root directory of
the container in <literal>/var/lib/containers</literal>.</para>
</section>
<section><title>Networking</title>
<para>When you create a container using <literal>nixos-container
create</literal>, it gets it own private IPv4 address in the range
<literal>10.233.0.0/16</literal>. You can get the containers IPv4
address as follows:
<screen>
$ nixos-container show-ip foo
10.233.4.2
$ ping -c1 10.233.4.2
64 bytes from 10.233.4.2: icmp_seq=1 ttl=64 time=0.106 ms
</screen>
</para>
<para>Networking is implemented using a pair of virtual Ethernet
devices. The network interface in the container is called
<literal>eth0</literal>, while the matching interface in the host is
called <literal>c-<replaceable>container-name</replaceable></literal>
(e.g., <literal>c-foo</literal>). The container has its own network
namespace and the <literal>CAP_NET_ADMIN</literal> capability, so it
can perform arbitrary network configuration such as setting up
firewall rules, without affecting or having access to the hosts
network.</para>
<para>By default, containers cannot talk to the outside network. If
you want that, you should set up Network Address Translation (NAT)
rules on the host to rewrite container traffic to use your external
IP address. This can be accomplished using the following configuration
on the host:
<programlisting>
networking.nat.enable = true;
networking.nat.internalInterfaces = ["c-+"];
networking.nat.externalInterface = "eth0";
</programlisting>
where <literal>eth0</literal> should be replaced with the desired
external interface. Note that <literal>c-+</literal> is a wildcard
that matches all container interfaces.</para>
</section>
</chapter>

View File

@ -54,6 +54,7 @@
<xi:include href="running.xml" />
<!-- <xi:include href="userconfiguration.xml" /> -->
<xi:include href="troubleshooting.xml" />
<xi:include href="containers.xml" />
<xi:include href="development.xml" />
<xi:include href="release-notes.xml" />

View File

@ -28,7 +28,7 @@ in
{
# Provide the NixOS/Nixpkgs sources in /etc/nixos. This is required
# for nixos-install.
boot.postBootCommands =
boot.postBootCommands = mkAfter
''
if ! [ -e /var/lib/nixos/did-channel-init ]; then
echo "unpacking the NixOS/Nixpkgs sources..."

View File

@ -307,6 +307,7 @@
./tasks/scsi-link-power-management.nix
./tasks/swraid.nix
./testing/service-runner.nix
./virtualisation/container-config.nix
./virtualisation/containers.nix
./virtualisation/libvirtd.nix
#./virtualisation/nova.nix

View File

@ -34,8 +34,9 @@ let
# Ignore peth* devices; on Xen, they're renamed physical
# Ethernet cards used for bridging. Likewise for vif* and tap*
# (Xen) and virbr* and vnet* (libvirt).
denyinterfaces ${toString ignoredInterfaces} peth* vif* tap* tun* virbr* vnet* vboxnet*
# (Xen) and virbr* and vnet* (libvirt) and c-* and ctmp-* (NixOS
# containers).
denyinterfaces ${toString ignoredInterfaces} peth* vif* tap* tun* virbr* vnet* vboxnet* c-* ctmp-*
${config.networking.dhcpcd.extraConfig}
'';

View File

@ -10,6 +10,8 @@ let
cfg = config.networking.nat;
dest = if cfg.externalIP == null then "-j MASQUERADE" else "-j SNAT --to-source ${cfg.externalIP}";
in
{
@ -27,14 +29,27 @@ in
'';
};
networking.nat.internalInterfaces = mkOption {
type = types.listOf types.str;
default = [];
example = [ "eth0" ];
description =
''
The interfaces for which to perform NAT. Packets coming from
these interface and destined for the external interface will
be rewritten.
'';
};
networking.nat.internalIPs = mkOption {
type = types.listOf types.str;
example = [ "192.168.1.0/24" ] ;
default = [];
example = [ "192.168.1.0/24" ];
description =
''
The IP address ranges for which to perform NAT. Packets
coming from these networks and destined for the external
interface will be rewritten.
coming from these addresses (on any interface) and destined
for the external interface will be rewritten.
'';
};
@ -80,25 +95,37 @@ in
preStart =
''
iptables -t nat -F PREROUTING
iptables -t nat -F POSTROUTING
iptables -t nat -X
''
+ (concatMapStrings (network:
''
iptables -t nat -A POSTROUTING \
-s ${network} -o ${cfg.externalInterface} \
${if cfg.externalIP == null
then "-j MASQUERADE"
else "-j SNAT --to-source ${cfg.externalIP}"}
''
) cfg.internalIPs) +
''
# We can't match on incoming interface in POSTROUTING, so
# mark packets coming from the external interfaces.
${concatMapStrings (iface: ''
iptables -t nat -A PREROUTING \
-i '${iface}' -j MARK --set-mark 1
'') cfg.internalInterfaces}
# NAT the marked packets.
${optionalString (cfg.internalInterfaces != []) ''
iptables -t nat -A POSTROUTING -m mark --mark 1 \
-o ${cfg.externalInterface} ${dest}
''}
# NAT packets coming from the internal IPs.
${concatMapStrings (range: ''
iptables -t nat -A POSTROUTING \
-s '${range}' -o ${cfg.externalInterface} ${dest}}
'') cfg.internalIPs}
echo 1 > /proc/sys/net/ipv4/ip_forward
'';
postStop =
''
iptables -t nat -F PREROUTING
iptables -t nat -F POSTROUTING
iptables -t nat -X
'';
};
};

View File

@ -621,7 +621,7 @@ in
{ description = "Apache HTTPD";
wantedBy = [ "multi-user.target" ];
requires = [ "keys.target" ];
wants = [ "keys.target" ];
after = [ "network.target" "fs.target" "postgresql.service" "keys.target" ];
path =

View File

@ -26,7 +26,10 @@ EOF
exit 1;
}
die "This is not a NixOS installation (/etc/NIXOS is missing)!\n" unless -f "/etc/NIXOS";
# This is a NixOS installation if it has /etc/NIXOS or a proper
# /etc/os-release.
die "This is not a NixOS installation!\n" unless
-f "/etc/NIXOS" || (read_file("/etc/os-release", err_mode => 'quiet') // "") =~ /ID=nixos/s;
openlog("nixos", "", LOG_USER);
@ -173,7 +176,10 @@ while (my ($unit, $state) = each %{$activePrev}) {
# FIXME: do something?
} else {
my $unitInfo = parseUnit($newUnitFile);
if (!boolIsTrue($unitInfo->{'X-RestartIfChanged'} // "yes")) {
if (boolIsTrue($unitInfo->{'X-ReloadIfChanged'} // "no")) {
write_file($reloadListFile, { append => 1 }, "$unit\n");
}
elsif (!boolIsTrue($unitInfo->{'X-RestartIfChanged'} // "yes")) {
push @unitsToSkip, $unit;
} else {
# If this unit is socket-activated, then stop the
@ -321,7 +327,7 @@ if (scalar @restart > 0) {
# that are symlinks to other units. We shouldn't start both at the
# same time because we'll get a "Failed to add path to set" error from
# systemd.
my @start = unique("default.target", "timers.target", split('\n', read_file($startListFile, err_mode => 'quiet') // ""));
my @start = unique("default.target", "timers.target", "sockets.target", split('\n', read_file($startListFile, err_mode => 'quiet') // ""));
print STDERR "starting the following units: ", join(", ", sort(@start)), "\n";
system("@systemd@/bin/systemctl", "start", "--", @start) == 0 or $res = 4;
unlink($startListFile);

View File

@ -243,6 +243,17 @@ in rec {
'';
};
reloadIfChanged = mkOption {
type = types.bool;
default = false;
description = ''
Whether the service should be reloaded during a NixOS
configuration switch if its definition has changed. If
enabled, the value of <option>restartIfChanged</option> is
ignored.
'';
};
stopIfChanged = mkOption {
type = types.bool;
default = true;

View File

@ -279,7 +279,11 @@ let
[Service]
${let env = cfg.globalEnvironment // def.environment;
in concatMapStrings (n: "Environment=\"${n}=${getAttr n env}\"\n") (attrNames env)}
${optionalString (!def.restartIfChanged) "X-RestartIfChanged=false"}
${if def.reloadIfChanged then ''
X-ReloadIfChanged=true
'' else if !def.restartIfChanged then ''
X-RestartIfChanged=false
'' else ""}
${optionalString (!def.stopIfChanged) "X-StopIfChanged=false"}
${attrsToSection def.serviceConfig}
'';

View File

@ -0,0 +1,103 @@
{ config, pkgs, lib, ... }:
with lib;
{
config = mkIf config.boot.isContainer {
# Provide a login prompt on /var/lib/login.socket. On the host,
# you can connect to it by running socat
# unix:<path-to-container>/var/lib/login.socket -,echo=0,raw.
systemd.sockets.login =
{ description = "Login Socket";
wantedBy = [ "sockets.target" ];
socketConfig =
{ ListenStream = "/var/lib/login.socket";
SocketMode = "0666";
Accept = true;
};
};
systemd.services."login@" =
{ description = "Login %i";
environment.TERM = "linux";
serviceConfig =
{ Type = "simple";
StandardInput = "socket";
ExecStart = "${pkgs.socat}/bin/socat -t0 - exec:${pkgs.shadow}/bin/login,pty,setsid,setpgid,stderr,ctty";
TimeoutStopSec = 1; # FIXME
};
};
# Also provide a root login prompt on /var/lib/root-login.socket
# that doesn't ask for a password. This socket can only be used by
# root on the host.
systemd.sockets.root-login =
{ description = "Root Login Socket";
wantedBy = [ "sockets.target" ];
socketConfig =
{ ListenStream = "/var/lib/root-login.socket";
SocketMode = "0600";
Accept = true;
};
};
systemd.services."root-login@" =
{ description = "Root Login %i";
environment.TERM = "linux";
serviceConfig =
{ Type = "simple";
StandardInput = "socket";
ExecStart = "${pkgs.socat}/bin/socat -t0 - \"exec:${pkgs.shadow}/bin/login -f root,pty,setsid,setpgid,stderr,ctty\"";
TimeoutStopSec = 1; # FIXME
};
};
# Provide a daemon on /var/lib/run-command.socket that reads a
# command from stdin and executes it.
systemd.sockets.run-command =
{ description = "Run Command Socket";
wantedBy = [ "sockets.target" ];
socketConfig =
{ ListenStream = "/var/lib/run-command.socket";
SocketMode = "0600"; # only root can connect
Accept = true;
};
};
systemd.services."run-command@" =
{ description = "Run Command %i";
environment.TERM = "linux";
serviceConfig =
{ Type = "simple";
StandardInput = "socket";
TimeoutStopSec = 1; # FIXME
};
script =
''
#! ${pkgs.stdenv.shell} -e
source /etc/bashrc
read c
eval "command=($c)"
exec "''${command[@]}"
'';
};
systemd.services.container-startup-done =
{ description = "Container Startup Notification";
wantedBy = [ "multi-user.target" ];
after = [ "multi-user.target" ];
script =
''
if [ -p /var/lib/startup-done ]; then
echo done > /var/lib/startup-done
fi
'';
serviceConfig.Type = "oneshot";
serviceConfig.RemainAfterExit = true;
};
};
}

View File

@ -2,6 +2,29 @@
with pkgs.lib;
let
runInNetns = pkgs.stdenv.mkDerivation {
name = "run-in-netns";
unpackPhase = "true";
buildPhase = ''
mkdir -p $out/bin
gcc ${./run-in-netns.c} -o $out/bin/run-in-netns
'';
installPhase = "true";
};
nixos-container = pkgs.substituteAll {
name = "nixos-container";
dir = "bin";
isExecutable = true;
src = ./nixos-container.pl;
perl = "${pkgs.perl}/bin/perl -I${pkgs.perlPackages.FileSlurp}/lib/perl5/site_perl";
inherit (pkgs) socat;
};
in
{
options = {
@ -14,19 +37,12 @@ with pkgs.lib;
'';
};
systemd.containers = mkOption {
containers = mkOption {
type = types.attrsOf (types.submodule (
{ config, options, name, ... }:
{
options = {
root = mkOption {
type = types.path;
description = ''
The root directory of the container.
'';
};
config = mkOption {
description = ''
A specification of the desired configuration of this
@ -45,21 +61,53 @@ with pkgs.lib;
'';
};
privateNetwork = mkOption {
type = types.bool;
default = false;
description = ''
Whether to give the container its own private virtual
Ethernet interface. The interface is called
<literal>eth0</literal>, and is hooked up to the interface
<literal>c-<replaceable>container-name</replaceable></literal>
on the host. If this option is not set, then the
container shares the network interfaces of the host,
and can bind to any port on any interface.
'';
};
hostAddress = mkOption {
type = types.nullOr types.string;
default = null;
example = "10.231.136.1";
description = ''
The IPv4 address assigned to the host interface.
'';
};
localAddress = mkOption {
type = types.nullOr types.string;
default = null;
example = "10.231.136.2";
description = ''
The IPv4 address assigned to <literal>eth0</literal>
in the container.
'';
};
};
config = mkMerge
[ { root = mkDefault "/var/lib/containers/${name}";
}
(mkIf options.config.isDefined {
[ (mkIf options.config.isDefined {
path = (import ../../lib/eval-config.nix {
modules =
let extraConfig =
{ boot.isContainer = true;
security.initialRootPassword = mkDefault "!";
networking.hostName = mkDefault name;
networking.useDHCP = false;
};
in [ extraConfig config.config ];
prefix = [ "systemd" "containers" name ];
prefix = [ "containers" name ];
}).config.system.build.toplevel;
})
];
@ -69,12 +117,10 @@ with pkgs.lib;
example = literalExample
''
{ webserver =
{ root = "/containers/webserver";
path = "/nix/var/nix/profiles/webserver";
{ path = "/nix/var/nix/profiles/webserver";
};
database =
{ root = "/containers/database";
config =
{ config =
{ config, pkgs, ... }:
{ services.postgresql.enable = true;
services.postgresql.package = pkgs.postgresql92;
@ -94,29 +140,96 @@ with pkgs.lib;
};
config = {
config = mkIf (!config.boot.isContainer) {
systemd.services = mapAttrs' (name: container: nameValuePair "container-${name}"
{ description = "Container '${name}'";
systemd.services."container@" =
{ description = "Container '%i'";
wantedBy = [ "multi-user.target" ];
unitConfig.RequiresMountsFor = [ "/var/lib/containers/%i" ];
unitConfig.RequiresMountsFor = [ container.root ];
path = [ pkgs.iproute ];
environment.INSTANCE = "%i";
environment.root = "/var/lib/containers/%i";
preStart =
''
mkdir -p -m 0755 ${container.root}/etc
if ! [ -e ${container.root}/etc/os-release ]; then
touch ${container.root}/etc/os-release
mkdir -p -m 0755 $root/var/lib
# Create a named pipe to get a signal when the container
# has finished booting.
rm -f $root/var/lib/startup-done
mkfifo -m 0600 $root/var/lib/startup-done
'';
script =
''
mkdir -p -m 0755 "$root/etc" "$root/var/lib"
if ! [ -e "$root/etc/os-release" ]; then
touch "$root/etc/os-release"
fi
mkdir -p -m 0755 \
"/nix/var/nix/profiles/per-container/$INSTANCE" \
"/nix/var/nix/gcroots/per-container/$INSTANCE"
SYSTEM_PATH=/nix/var/nix/profiles/system
if [ -f "/etc/containers/$INSTANCE.conf" ]; then
. "/etc/containers/$INSTANCE.conf"
fi
# Cleanup from last time.
ifaceHost=c-$INSTANCE
ifaceCont=ctmp-$INSTANCE
ns=net-$INSTANCE
ip netns del $ns 2> /dev/null || true
ip link del $ifaceHost 2> /dev/null || true
ip link del $ifaceCont 2> /dev/null || true
if [ "$PRIVATE_NETWORK" = 1 ]; then
# Create a pair of virtual ethernet devices. On the host,
# we get c-<container-name, and on the guest, we get
# eth0.
ip link add $ifaceHost type veth peer name $ifaceCont
ip netns add $ns
ip link set $ifaceCont netns $ns
ip netns exec $ns ip link set $ifaceCont name eth0
ip netns exec $ns ip link set dev eth0 up
ip link set dev $ifaceHost up
if [ -n "$HOST_ADDRESS" ]; then
ip addr add $HOST_ADDRESS dev $ifaceHost
ip netns exec $ns ip route add $HOST_ADDRESS dev eth0
ip netns exec $ns ip route add default via $HOST_ADDRESS
fi
if [ -n "$LOCAL_ADDRESS" ]; then
ip netns exec $ns ip addr add $LOCAL_ADDRESS dev eth0
ip route add $LOCAL_ADDRESS dev $ifaceHost
fi
runInNetNs="${runInNetns}/bin/run-in-netns $ns"
extraFlags="--capability=CAP_NET_ADMIN"
fi
exec $runInNetNs ${config.systemd.package}/bin/systemd-nspawn \
-M "$INSTANCE" -D "/var/lib/containers/$INSTANCE" $extraFlags \
--bind-ro=/nix/store \
--bind-ro=/nix/var/nix/db \
--bind-ro=/nix/var/nix/daemon-socket \
--bind="/nix/var/nix/profiles/per-container/$INSTANCE:/nix/var/nix/profiles" \
--bind="/nix/var/nix/gcroots/per-container/$INSTANCE:/nix/var/nix/gcroots" \
"$SYSTEM_PATH/init"
'';
serviceConfig.ExecStart =
"${config.systemd.package}/bin/systemd-nspawn -M ${name} -D ${container.root} --bind-ro=/nix ${container.path}/init";
postStart =
''
# This blocks until the container-startup-done service
# writes something to this pipe. FIXME: it also hangs
# until the start timeout expires if systemd-nspawn exits.
read x < $root/var/lib/startup-done
'';
preStop =
''
pid="$(cat /sys/fs/cgroup/systemd/machine/${name}.nspawn/system/tasks 2> /dev/null)"
pid="$(cat /sys/fs/cgroup/systemd/machine/$INSTANCE.nspawn/system/tasks 2> /dev/null)"
if [ -n "$pid" ]; then
# Send the RTMIN+3 signal, which causes the container
# systemd to start halt.target.
@ -131,7 +244,52 @@ with pkgs.lib;
done
fi
'';
}) config.systemd.containers;
restartIfChanged = false;
#reloadIfChanged = true; # FIXME
serviceConfig.ExecReload = pkgs.writeScript "reload-container"
''
#! ${pkgs.stdenv.shell} -e
SYSTEM_PATH=/nix/var/nix/profiles/system
if [ -f "/etc/containers/$INSTANCE.conf" ]; then
. "/etc/containers/$INSTANCE.conf"
fi
echo $SYSTEM_PATH/bin/switch-to-configuration test | \
${pkgs.socat}/bin/socat unix:$root/var/lib/run-command.socket -
'';
serviceConfig.SyslogIdentifier = "container %i";
};
# Generate a configuration file in /etc/containers for each
# container so that container@.target can get the container
# configuration.
environment.etc = mapAttrs' (name: cfg: nameValuePair "containers/${name}.conf"
{ text =
''
SYSTEM_PATH=${cfg.path}
${optionalString cfg.privateNetwork ''
PRIVATE_NETWORK=1
${optionalString (cfg.hostAddress != null) ''
HOST_ADDRESS=${cfg.hostAddress}
''}
${optionalString (cfg.localAddress != null) ''
LOCAL_ADDRESS=${cfg.localAddress}
''}
''}
'';
}) config.containers;
# FIXME: auto-start containers.
# Generate /etc/hosts entries for the containers.
networking.extraHosts = concatStrings (mapAttrsToList (name: cfg: optionalString (cfg.localAddress != null)
''
${cfg.localAddress} ${name}.containers
'') config.containers);
environment.systemPackages = [ nixos-container ];
};
}

View File

@ -0,0 +1,238 @@
#! @perl@
use strict;
use POSIX;
use File::Path;
use File::Slurp;
use Fcntl ':flock';
use Getopt::Long qw(:config gnu_getopt);
my $socat = '@socat@/bin/socat';
# Parse the command line.
sub showHelp {
print <<EOF;
Usage: nixos-container list
nixos-container create <container-name> [--config <string>] [--ensure-unique-name]
nixos-container destroy <container-name>
nixos-container start <container-name>
nixos-container stop <container-name>
nixos-container login <container-name>
nixos-container root-login <container-name>
nixos-container run <container-name> -- args...
nixos-container set-root-password <container-name> <password>
nixos-container show-ip <container-name>
EOF
exit 0;
}
my $ensureUniqueName = 0;
my $extraConfig = "";
GetOptions(
"help" => sub { showHelp() },
"ensure-unique-name" => \$ensureUniqueName,
"config=s" => \$extraConfig
) or exit 1;
my $action = $ARGV[0] or die "$0: no action specified\n";
# Execute the selected action.
mkpath("/etc/containers", 0, 0755);
mkpath("/var/lib/containers", 0, 0700);
if ($action eq "list") {
foreach my $confFile (glob "/etc/containers/*.conf") {
$confFile =~ /\/([^\/]+).conf$/ or next;
print "$1\n";
}
exit 0;
}
my $containerName = $ARGV[1] or die "$0: no container name specified\n";
$containerName =~ /^[a-zA-Z0-9\-]+$/ or die "$0: invalid container name\n";
sub writeNixOSConfig {
my ($nixosConfigFile) = @_;
my $nixosConfig = <<EOF;
{ config, pkgs, ... }:
with pkgs.lib;
{ boot.isContainer = true;
security.initialRootPassword = mkDefault "!";
networking.hostName = mkDefault "$containerName";
networking.useDHCP = false;
$extraConfig
}
EOF
write_file($nixosConfigFile, $nixosConfig);
}
if ($action eq "create") {
# Acquire an exclusive lock to prevent races with other
# invocations of nixos-container create.
my $lockFN = "/run/lock/nixos-container";
open(my $lock, '>>', $lockFN) or die "$0: opening $lockFN: $!";
flock($lock, LOCK_EX) or die "$0: could not lock $lockFN: $!";
my $confFile = "/etc/containers/$containerName.conf";
my $root = "/var/lib/containers/$containerName";
# Maybe generate a unique name.
if ($ensureUniqueName) {
my $base = $containerName;
for (my $nr = 0; ; $nr++) {
$containerName = "$base-$nr";
$confFile = "/etc/containers/$containerName.conf";
$root = "/var/lib/containers/$containerName";
last unless -e $confFile || -e $root;
}
}
die "$0: container $containerName already exists\n" if -e $confFile;
# Get an unused IP address.
my %usedIPs;
foreach my $confFile2 (glob "/etc/containers/*.conf") {
my $s = read_file($confFile2) or die;
$usedIPs{$1} = 1 if $s =~ /^HOST_ADDRESS=([0-9\.]+)$/m;
$usedIPs{$1} = 1 if $s =~ /^LOCAL_ADDRESS=([0-9\.]+)$/m;
}
my ($ipPrefix, $hostAddress, $localAddress);
for (my $nr = 1; $nr < 255; $nr++) {
$ipPrefix = "10.233.$nr";
$hostAddress = "$ipPrefix.1";
$localAddress = "$ipPrefix.2";
last unless $usedIPs{$hostAddress} || $usedIPs{$localAddress};
$ipPrefix = undef;
}
die "$0: out of IP addresses\n" unless defined $ipPrefix;
my @conf;
push @conf, "PRIVATE_NETWORK=1\n";
push @conf, "HOST_ADDRESS=$hostAddress\n";
push @conf, "LOCAL_ADDRESS=$localAddress\n";
write_file($confFile, \@conf);
close($lock);
print STDERR "host IP is $hostAddress, container IP is $localAddress\n";
mkpath("$root/etc/nixos", 0, 0755);
my $nixosConfigFile = "$root/etc/nixos/configuration.nix";
writeNixOSConfig $nixosConfigFile;
# The per-container directory is restricted to prevent users on
# the host from messing with guest users who happen to have the
# same uid.
my $profileDir = "/nix/var/nix/profiles/per-container";
mkpath($profileDir, 0, 0700);
$profileDir = "$profileDir/$containerName";
mkpath($profileDir, 0, 0755);
system("nix-env", "-p", "$profileDir/system",
"-I", "nixos-config=$nixosConfigFile", "-f", "<nixpkgs/nixos>",
"--set", "-A", "system") == 0
or die "$0: failed to build initial container configuration\n";
print "$containerName\n" if $ensureUniqueName;
exit 0;
}
my $root = "/var/lib/containers/$containerName";
my $profileDir = "/nix/var/nix/profiles/per-container/$containerName";
my $confFile = "/etc/containers/$containerName.conf";
die "$0: container $containerName does not exist\n" if !-e $confFile;
sub isContainerRunning {
my $status = `systemctl show 'container\@$containerName'`;
return $status =~ /ActiveState=active/;
}
sub stopContainer {
system("systemctl", "stop", "container\@$containerName") == 0
or die "$0: failed to stop container\n";
}
if ($action eq "destroy") {
die "$0: cannot destroy declarative container (remove it from your configuration.nix instead)\n"
unless POSIX::access($confFile, &POSIX::W_OK);
stopContainer if isContainerRunning;
rmtree($profileDir) if -e $profileDir;
rmtree($root) if -e $root;
unlink($confFile) or die;
}
elsif ($action eq "start") {
system("systemctl", "start", "container\@$containerName") == 0
or die "$0: failed to start container\n";
}
elsif ($action eq "stop") {
stopContainer;
}
elsif ($action eq "update") {
my $nixosConfigFile = "$root/etc/nixos/configuration.nix";
# FIXME: may want to be more careful about clobbering the existing
# configuration.nix.
writeNixOSConfig $nixosConfigFile if defined $extraConfig;
system("nix-env", "-p", "$profileDir/system",
"-I", "nixos-config=$nixosConfigFile", "-f", "<nixpkgs/nixos>",
"--set", "-A", "system") == 0
or die "$0: failed to build container configuration\n";
if (isContainerRunning) {
print STDERR "reloading container...\n";
system("systemctl", "reload", "container\@$containerName") == 0
or die "$0: failed to reload container\n";
}
}
elsif ($action eq "login") {
exec($socat, "unix:$root/var/lib/login.socket", "-,echo=0,raw");
}
elsif ($action eq "root-login") {
exec($socat, "unix:$root/var/lib/root-login.socket", "-,echo=0,raw");
}
elsif ($action eq "run") {
shift @ARGV; shift @ARGV;
open(SOCAT, "|-", $socat, "unix:$root/var/lib/run-command.socket", "-");
print SOCAT join(' ', map { "'$_'" } @ARGV), "\n";
close(SOCAT);
}
elsif ($action eq "set-root-password") {
# FIXME: don't get password from the command line.
my $password = $ARGV[2] or die "$0: no password given\n";
open(SOCAT, "|-", $socat, "unix:$root/var/lib/run-command.socket", "-");
print SOCAT "passwd\n";
print SOCAT "$password\n";
print SOCAT "$password\n";
close(SOCAT);
}
elsif ($action eq "show-ip") {
my $s = read_file($confFile) or die;
$s =~ /^LOCAL_ADDRESS=([0-9\.]+)$/m or die "$0: cannot get IP address\n";
print "$1\n";
}
else {
die "$0: unknown action $action\n";
}

View File

@ -0,0 +1,50 @@
#define _GNU_SOURCE
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <sched.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/mount.h>
#include <fcntl.h>
#include <linux/limits.h>
int main(int argc, char * * argv)
{
if (argc < 3) {
fprintf(stderr, "%s: missing arguments\n", argv[0]);
return 1;
}
char nsPath[PATH_MAX];
sprintf(nsPath, "/run/netns/%s", argv[1]);
int fd = open(nsPath, O_RDONLY);
if (fd == -1) {
fprintf(stderr, "%s: opening network namespace: %s\n", argv[0], strerror(errno));
return 1;
}
if (setns(fd, CLONE_NEWNET) == -1) {
fprintf(stderr, "%s: setting network namespace: %s\n", argv[0], strerror(errno));
return 1;
}
umount2(nsPath, MNT_DETACH);
if (unlink(nsPath) == -1) {
fprintf(stderr, "%s: unlinking network namespace: %s\n", argv[0], strerror(errno));
return 1;
}
/* FIXME: Remount /sys so that /sys/class/net reflects the
interfaces visible in the network namespace. This requires
bind-mounting /sys/fs/cgroups etc. */
execv(argv[2], argv + 2);
fprintf(stderr, "%s: running command: %s\n", argv[0], strerror(errno));
return 1;
}

View File

@ -0,0 +1,79 @@
# Test for NixOS' container support.
{ pkgs, ... }:
{
machine =
{ config, pkgs, ... }:
{ imports = [ ../modules/installer/cd-dvd/channel.nix ];
virtualisation.writableStore = true;
virtualisation.memorySize = 768;
containers.webserver =
{ privateNetwork = true;
hostAddress = "10.231.136.1";
localAddress = "10.231.136.2";
config =
{ services.httpd.enable = true;
services.httpd.adminAddr = "foo@example.org";
};
};
virtualisation.pathsInNixDB = [ pkgs.stdenv ];
};
testScript =
''
$machine->succeed("nixos-container list") =~ /webserver/;
# Start the webserver container.
$machine->succeed("nixos-container start webserver");
# Since "start" returns after the container has reached
# multi-user.target, we should now be able to access it.
my $ip = $machine->succeed("nixos-container show-ip webserver");
chomp $ip;
$machine->succeed("ping -c1 $ip");
$machine->succeed("curl --fail http://$ip/ > /dev/null");
# Stop the container.
$machine->succeed("nixos-container stop webserver");
$machine->fail("curl --fail --connect-timeout 2 http://$ip/ > /dev/null");
# Make sure we have a NixOS tree (required by nixos-container create).
$machine->succeed("nix-env -qa -A nixos.pkgs.hello >&2");
# Create some containers imperatively.
my $id1 = $machine->succeed("nixos-container create foo --ensure-unique-name");
chomp $id1;
$machine->log("created container $id1");
my $id2 = $machine->succeed("nixos-container create foo --ensure-unique-name");
chomp $id2;
$machine->log("created container $id2");
die if $id1 eq $id2;
my $ip1 = $machine->succeed("nixos-container show-ip $id1");
chomp $ip1;
my $ip2 = $machine->succeed("nixos-container show-ip $id2");
chomp $ip2;
die if $ip1 eq $ip2;
# Start one of them.
$machine->succeed("nixos-container start $id1");
# Execute commands via the root shell.
$machine->succeed("echo uname | nixos-container root-shell $id1") =~ /Linux/;
$machine->succeed("nixos-container set-root-password $id1 foobar");
# Destroy the containers.
$machine->succeed("nixos-container destroy $id1");
$machine->succeed("nixos-container destroy $id2");
# Destroying a declarative container should fail.
$machine->fail("nixos-container destroy webserver");
'';
}

View File

@ -8,6 +8,7 @@ with import ../lib/testing.nix { inherit system minimal; };
{
avahi = makeTest (import ./avahi.nix);
bittorrent = makeTest (import ./bittorrent.nix);
containers = makeTest (import ./containers.nix);
firefox = makeTest (import ./firefox.nix);
firewall = makeTest (import ./firewall.nix);
installer = makeTests (import ./installer.nix);