hercules-ci/arion
1
1
Fork 0
mirror of https://github.com/hercules-ci/arion.git synced 2024-10-26 18:20:14 +03:00
Code Issues Projects Releases Wiki Activity

README: rearrange sections to go: pitch, install, usage

Browse Source 
Inline examples as we can't use imports due to
https://github.com/github/markup/issues/1095
This commit is contained in:
Domen Kožar 2019-04-30 11:40:25 +07:00
parent fbf57b4489
commit f4948f2fde
No known key found for this signature in database
GPG Key ID: C2FFBCAFD2C24246
5 changed files with 274 additions and 211 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

274
README.asciidoc Normal file
View File

@ -0,0 +1,274 @@
== Introduction
Arion is a tool for building and running applications that
consist of multiple docker containers using NixOS modules.
It has special support for docker images that are built with Nix,
for a smooth development experience and improved performance.
It is built on top of https://docs.docker.com/compose/overview/[Docker
Compose], which implements the container orchestration functionality.
Instead of configuring the compositions in YAML files like
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
language to declare the compositions. Because of this, Arion gives you
the ability to declare your deployments, configuration and packaging
in the same language. By replacing multiple tools with a single
language, you decrease your mental load and you can more easily
refactor and maintain your configurations.
Although Arion can be used as a Docker Compose with an improved
configuration front end, there is more to be gained from integrating
with Nix. In particular, the more structured approach of Nix compared
to Dockerfiles allows the following:
* Build components of your image in *parallel, automatically*
* *Share packages between images*, regardless of the order they were
added
* Improve performance by *skipping container
image creation*
* Work with *structured data instead of strings*,
templates and a multitude of expression languages
* Refactor across deployments, configuration and packaging
Arion allows to compose containers with different granuality:
* <<Minimal: Plain command using nixpkgs>>
* <<NixOS: run only one systemd service>>
* <<NixOS: run full OS>>
* <<Docker image from DockerHub>>
== Installation
=== Nix
```bash
$ nix-env -iA arion -f https://github.com/hercules-ci/arion/tarball/master
```
=== NixOS
Add this module to your NixOS configuration:
```nix
{ ... }: {
environment.systemPackages = [ (import (builtins.fetchTarball https://github.com/hercules-ci/arion/tarball/master) {}) ];
virtualisation.docker.enable = true;
users.extraUsers.myuser.extraGroups = ["docker"];
}
```
////
== Not installing: use it in a project
TODO: describe: using nix-shell or in a script, building images as
part of nix-build, pinning, see also todomvc-nix.
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
////
== Usage
Arion is configured declaratively with two files:
=== arion-pkgs.nix
Arion needs `arion-pkgs.nix` to import nixpkgs, it's contents can be as simple as:
```nix
import <nixpkgs> {}
```
or more sophisticated (recommended) setup with https://github.com/nmattia/niv[Niv].
=== arion-compose.nix
Describe containers using NixOS modules. There are a few options:
==== Minimal: Plain command using nixpkgs
`examples/minimal/arion-compose.nix`:
```nix
{ pkgs, ... }:
{
config.docker-compose.services = {
webserver = {
service.useHostStore = true;
service.command = [ "sh" "-c" ''
cd "$$WEB_ROOT"
${pkgs.python3}/bin/python -m http.server
'' ];
service.ports = [
"8000:8000" # host:container
];
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
};
};
}
```
==== NixOS: run only one systemd service
`examples/nixos-unit/arion-compose.nix`:
```nix
{
docker-compose.services.webserver = { config, pkgs, ... }: {
nixos.configuration = {config, pkgs, ...}: {
boot.isContainer = true;
services.nginx.enable = true;
services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
system.build.run-nginx = pkgs.writeScript "run-nginx" ''
#!${pkgs.bash}/bin/bash
PATH='${config.systemd.services.nginx.environment.PATH}'
echo nginx:x:${toString config.users.users.nginx.uid}:${toString config.users.groups.nginx.gid}:nginx web server user:/var/empty:/bin/sh >>/etc/passwd
echo nginx:x:${toString config.users.groups.nginx.gid}:nginx >>/etc/group
${config.systemd.services.nginx.runner}
'';
};
service.command = [ config.nixos.build.run-nginx ];
service.useHostStore = true;
service.ports = [
"8000:80" # host:container
];
};
}
```
==== NixOS: run full OS
`examples/full-nixos/arion-compose.nix`:
```nix
{
docker-compose.services.webserver = { pkgs, ... }: {
nixos.useSystemd = true;
nixos.configuration.boot.tmpOnTmpfs = true;
nixos.configuration.services.nginx.enable = true;
nixos.configuration.services.nginx.virtualHosts.localhost.root = "${pkgs.nix.doc}/share/doc/nix/manual";
service.useHostStore = true;
service.ports = [
"8000:80" # host:container
];
};
}
```
==== Docker image from DockerHub
```nix
{
docker-compose.services.postgres = {
service.image = "postgres:10";
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
};
}
```
=== Run
Start containers and watch their logs:
```bash
$ arion up -d
$ arion logs -f
```
You can go to `examples/*/` and run these commands to give it a quick try.
== A full featured Nix command example
To see how Arion can be used in a project, have a look at
https://github.com/nix-community/todomvc-nix/tree/master/deploy/arion[todomvc-nix].
```bash
$ git clone https://github.com/nix-community/todomvc-nix
$ cd todomvc-nix/deploy/arion
$ arion up
```
== Project Status
This project was born out of a process supervision need for local
development environments while working on
https://www.hercules-ci.com[Hercules CI]. (It was also born out of
ancient Greek deities disguised as horses. More on that later.)
If you do want to use Arion for production environments, you’ll probably
want to either build normal container images or manage garbage
collection roots if you control the deployment host. Neither scenario is
made easier by arion at this time.
Support for other Linux than NixOS is untested.
== How it works
Arion is essentially a thin wrapper around Nix and docker-compose. When
it runs, it does the following:
* Evaluate the configuration using Nix, producing a
`docker-compose.yaml` and a garbage collection root
* Invoke `docker-compose`
* Clean up the garbage collection root
Most of the interesting stuff happens in Arion’s Nix expressions, where
it runs the module system (known from NixOS) and provides the
configuration that makes the Docker Compose file do the things it needs
to do.
One of the more interesting built-in modules is the
link:src/nix/modules/service/host-store.nix[host-store.nix module] which
performs the bind mounts to make the host Nix store available in the
container.
== FAQ
=== Do I need to use Hercules CI?
Nope, it’s just Nix and Docker Compose under the hood.
=== What about garbage collection?
Arion removes the need for garbage collecting docker images, delegating
this task to Nix.
Arion creates a garbage collection root and cleans it up after
completing the command. This means that `arion up` without `-d` is safe
with respect to garbage collection. A deployment that is more serious
than local development must leave a GC root on the deployment host. This
use case is not supported as of now.
=== Why is my container not running latest code?
Restart it with `arion restart <name>` or if you've changed the image rebuild
them using `arion up -d --always-recreate-deps <name>`.
=== What is messing with my environment variables?
Docker Compose performs its own environment variable substitution. This
can be a little annoying in `services.command` for example. Either
reference a script from `pkgs.writeScript` or escape the dollar sign as
`$$`.
=== Why name it ``Arion``?
Arion comes from Greek mythology. Poseidon, the god of ~Docker~ the seas
had his eye on Demeter. Demeter tried to trick him by disguising as a
horse, but Poseidon saw through the deception and they had Arion.
So Arion is a super fast divine horse; the result of some weird mixing.
Also it talks.
(And we feel morally obliged to name our stuff after Greek mythology)

135
README.md
View File

@ -1,135 +0,0 @@
# Run docker-compose without images with Nix
*Wait, what?*
With Arion you can fire up containers without creating images for each
service. It can use a mostly empty image, and makes the host's Nix
store available in the container, allowing the container to run
programs without having to re-package them into a docker image.
Arion is configured using Nix with modules, like those in
NixOS. Similar to `docker-compose` it can therefore combine
configuration from multiple files. For managing the network and
containers it delegates to the `docker-compose` command.
# Project Status
This project was born out of a process supervision need for local
development environments while working
on [Hercules CI](https://www.hercules-ci.com). (It was also born out
of ancient Greek deities disguised as horses. More on that later.)
If you do want to use Arion for production environments, you'll
probably want to either build normal container images or manage
garbage collection roots if you control the deployment host. Neither
scenario is made easier by arion at this time.
Support for other Linux than NixOS is untested.
# Install
Have [Nix](https://nixos.org/nix/) and Docker installed. Ubuntu users:
make sure you've installed via `apt-get install docker.io`. The Docker snap
is broken and apparently unsupported by Canonical.
git clone git@github.com:hercules-ci/arion.git
cd arion
nix-env -iA arion -f .
# Example `arion-compose.nix`
This Nix expression serves the Nix manual at host port 8000 when launched with `arion up`. It is a function from a package set (`pkgs`) to a configuration.
```nix
{ pkgs, ... }:
{
config.docker-compose.services = {
webserver = {
service.useHostStore = true;
# service.depends_on = [ "backend" ];
service.command = [ "sh" "-c" ''
cd "$$WEB_ROOT"
${pkgs.python3}/bin/python -m http.server
'' ];
service.ports = [
"8000:8000" # host:container
];
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
};
# backend = { ... }
};
}
```
The `pkgs` argument comes from a file called `arion-pkgs.nix`. It can be as simple as `import <nixpkgs> {}` to use the Nixpkgs from your `$NIX_PATH`, or you can use it to pin a specific Nixpkgs version.
# A full featured example
To see how Arion can be used in a project, have a look at [todomvc-nix](https://github.com/nix-community/todomvc-nix/tree/master/deploy/arion).
git clone git@github.com:nix-community/todomvc-nix.git
cd todomvc-nix/deploy/arion
arion up
# How it works
Arion is essentially a thin wrapper around Nix and docker-compose.
When it runs, it does the following:
- Evaluate the configuration using Nix, producing a `docker-compose.yaml` and a garbage collection root
- Invoke `docker-compose`
- Clean up the garbage collection root
Most of the interesting stuff happens in Arion's Nix expressions,
where it runs the module system (known from NixOS) and provides the configuration that makes the Docker Compose file do the things it needs to do.
One of the more interesting built-in modules is the [host-store.nix module](src/nix/modules/service/host-store.nix) which performs the bind mounts to make the host Nix store available in the container.
# FAQ
### Do I need to use Hercules CI?
Nope, it's just Nix and Docker Compose under the hood.
### Does Arion support Docker images?
Yes, you can also specify a normal Docker image. For example:
postgres = {
service.image = "postgres:10";
service.volumes = [ "${toString ./.}/postgres-data:/var/lib/postgresql/data" ];
service.environment.POSTGRES_PASSWORD = "mydefaultpass";
};
### What about garbage collection?
Arion removes the need for garbage collecting docker images,
delegating this task to Nix.
Arion creates a garbage collection root and cleans it up after
completing the command. This means that `arion up` without `-d` is
safe with respect to garbage collection. A deployment that is more
serious than local development must leave a GC root on the deployment
host. This use case is not supported as of now.
### What is messing with my environment variables?
Docker Compose performs its own environment variable
substitution. This can be a little annoying in `services.command` for
example. Either reference a script from `pkgs.writeScript` or escape
the dollar sign as `$$`.
### Why "Arion"?
Arion comes from Greek mythology. Poseidon, the god of ~Docker~ the
seas had his eye on Demeter. Demeter tried to trick him by disguising
as a horse, but Poseidon saw through the deception and they had Arion.
So Arion is a super fast divine horse; the result of some weird
mixing. Also it talks.
(And we feel morally obliged to name our stuff after Greek mythology)

39
doc/manual/installation.asciidoc
View File

@ -1,39 +0,0 @@
= Installation
== Imperative installation, bleeding edge
```
git clone git@github.com:hercules-ci/arion.git
cd arion
nix-env -iA arion -f .
```
// TODO: replace the above with something like below
////
== Not installing: use it in a project
TODO: describe: using nix-shell or in a script, building images as
part of nix-build, pinning, see also todomvc-nix.
TODO: exposed Nix functions: arion.build, arion.eval (a bit of IFD)
== Mac or traditional Linux
```
nix-env -iA nixpkgs.arion
```
== NixOS
Add this module to your NixOS configuration:
```
{ pkgs, ... } {
virtualisation.docker.enable = true;
environment.systemPackages = [ pkgs.arion ];
}
```
////

33
doc/manual/introduction.asciidoc
View File

@ -1,33 +0,0 @@
= Introduction
Arion is a tool for building and running applications that
consist of multiple docker containers. It has special support
for docker images that are built with Nix, for a smooth
development experience and improved performance.
It is built on top of https://docs.docker.com/compose/overview/[Docker
Compose], which implements the container orchestration functionality.
Instead of configuring the compositions in YAML files like
`docker-compose.yaml`, Arion uses the https://nixos.org/nix/[Nix]
language to declare the compositions. Because of this, Arion gives you
the ability to declare your deployments, configuration and packaging
in the same language. By replacing multiple tools with a single
language, you decrease your mental load and you can more easily
refactor and maintain your configurations.
Although Arion can be used as a Docker Compose with an improved
configuration front end, there is more to be gained from integrating
with Nix. In particular, the more structured approach of Nix compared
to Dockerfiles allows the following:
* Build components of your image in _parallel, automatically_.
* Share packages between images, regardless of the order they were
added.
* Improve performance by _skipping_ container
image creation.
* Work with _structured data_ instead of strings,
templates and a multitude of expression languages.
* Refactor across deployments, configuration and
packaging.

4
examples/minimal/arion-compose.nix
View File

@ -4,7 +4,6 @@
webserver = {
service.useHostStore = true;
# service.depends_on = [ "backend" ];
service.command = [ "sh" "-c" ''
cd "$$WEB_ROOT"
${pkgs.python3}/bin/python -m http.server
@ -14,8 +13,5 @@
];
service.environment.WEB_ROOT = "${pkgs.nix.doc}/share/doc/nix/manual";
};
# backend = { ... }
};
}