Bors enabled MIT License NixOS 20.09

⚠ Advisory ⚠

DevOS requires the flakes feature available via an experimental branch of nix. Until nix 3.0 is released, this project should be considered unstable, though quite usable as flakes have been maturing well for a while.

Introduction

DevOS grants a simple way to use, deploy and manage NixOS systems for personal and productive use. A sane repository structure is provided, integrating several popular projects like home-manager, devshell, and more.

Stiving for nix first™ solutions with unobstrusive implementations, a flake centric approach is taken for useful conveniences such as automatic source updates.

Skip the indeterminate nature of other systems, and the perceived tedium of bootstrapping Nix. It's easier than you think!

Status: Alpha

A lot of the implementation is less than perfect, and huge redesigns will happen. There are unstable versions (0.x.x) to help users keep track of changes and progress.

Getting Started

Check out the guide to get up and running.

In the Wild

The author maintains his own branch, so you can take inspiration, direction, or make critical comments about the code. 😜

Motivation

NixOS provides an amazing abstraction to manage our environment, but that new power can sometimes bring feelings of overwhelm and confusion. Having a turing complete system can easily lead to unlimited complexity if we do it wrong. Instead, we should have a community consensus on how to manage a NixOS system and its satellite projects, from which best practices can evolve.

The future is declarative! 🎉

Upstream

I'd love to see this in the nix-community should anyone believe its reached a point of maturity to be generally useful, but I'm all for waiting until 1.0#121 to save the cache work, too.

Community Profiles

There are two branches from which to choose: core and community. The community branch builds on core and includes several ready-made profiles for discretionary use.

Every package and NixOS profile declared in community is uploaded to cachix, so everything provided is available without building anything. This is especially useful for the packages that are overridden from master, as without the cache, rebuilds are quite frequent.

Inspiration & Art

License

DevOS is licensed under the MIT License.

Quick Start

The only dependency is nix, so make sure you have it installed.

Get the Template

Here is a snippet that will get you the template without the git history:

nix-shell -p cachix --run "cachix use nrdxp"

nix-shell https://github.com/divnix/devos/archive/core.tar.gz -A shell \
  --run "flk get core"

cd flk

nix-shell

git init
git add .
git commit -m init

This will place you in a new folder named flk with git initialized, and a nix-shell that provides all the dependencies, including the unstable nix version required.

In addition, the binary cache is added for faster deployment.

Notes:
  • You can change core to community in the call to flk get
  • Flakes ignore files that have not been added to git, so be sure to stage new files before building the system.
  • You can choose to simply clone the repo with git if you want to follow upstream changes.

Next Steps:

ISO

Making and writing an installable iso for hosts/NixOS.nix is as simple as:

flk iso NixOS

dd bs=4M if=result/iso/*.iso of=/dev/$your_installation_device \
  status=progress oflag=sync

This works for any file matching hosts/*.nix excluding default.nix.

From NixOS

Generate Configuration

Assuming your happy with your existing partition layout, you can generate a basic NixOS configuration for your system using:

flk up

This will make a new file hosts/up-$(hostname).nix, which you can edit to your liking.

Make sure your i18n.defaultLocale and time.timeZone are set properly for your region. Keep in mind that networking.hostName with be automatically set to the filename of your hosts file, so hosts/my-host.nix will have the hostname my-host.

Now might be a good time to read the docs on suites and profiles and add or create any that you need.

Note:

While the up sub-command is provided as a convenience to quickly set up and install a "fresh" NixOS system on current hardware, committing these files is discouraged.

They are placed in the git staging area automatically because they would be invisible to the flake otherwise, but it is best to move what you need from them directly into a host module of your own making, and commit that instead.

Installation

Once your ready to deploy hosts/my-host.nix:

flk my-host switch

This calls nixos-rebuild with sudo to build and install your configuration.

Notes:

  • Instead of switch, you can pass build, test, boot, etc just as with nixos-rebuild.

  • It is convenient to have the template living at /etc/nixos so you can simply sudo nixos-rebuild switch from anywhere on the system, but it is not required.

Layout

Each of the following sections is a directory in the root of the project serving a singular purpose. Select a chapter to read more about its purpose and usage.

Cachix

The cachix directory simple captures the output of sudo cachix use for the developers personal cache, as well as the nix-community cache. You can easily add your own cache, assuming the template lives in /etc/nixos, by simply running sudo cachix use yourcache.

These caches are only added to the system after a nixos-rebuild switch, so it is recommended to call cachix use divnix before the initial deployment, as it will save a lot of build time.

In the future, users will be able to skip this step once the ability to define the nix.conf within the flake is fully fleshed out upstream.

External Art

When you need to use a module, overlay, or pass a value from one of your inputs to the rest of your NixOS configuration, extern is where you do it.

Modules and overlays are self explanatory, and the specialArgs attribute is used to extend the arguments passed to all NixOS modules, allowing for arbitrary values to be passed from flake inputs to the rest of your configuration.

Home Manager

There is also an hmModules attribute set for pulling home-manager modules in from the outside world:

Declare:

flake.nix:

{
  inputs.doom-emacs.url = "github:vlaci/nix-doom-emacs";
}

extern/default.nix:

with inputs;
{
  hmModules = {
    doom-emacs = doom-emacs.hmModule;
  };
}

Use:

users/nixos/default.nix:

{ hmModules, ... }:
{
  home-manager.users.nixos = {
    imports = [ hmModules.doom-emacs ] ;

    programs.doom-emacs.enable = true;
  };
}

Hosts

Nix flakes contain an output called nixosConfigurations declaring an attribute set of valid NixOS systems. To simplify the management and creation of these hosts, devos automatically imports every .nix file inside this directory to the mentioned attribute set, applying the projects defaults to each. The only hard requirement is that the file contain a valid NixOS module.

As an example, a file hosts/system.nix will be available via the flake output nixosConfigurations.system. You can have as many hosts as you want and all of them will be automatically imported based on their name.

For each host, the configuration automatically sets the networking.hostName attribute to the name of the file minus the .nix extension. This is for convenience, since nixos-rebuild automatically searches for a configuration matching the current systems hostname if one is not specified explicitly.

It is recommended that the host modules only contain configuration information specific to a particular piece of hardware. Anything reusable across machines is best saved for profile modules.

This is a good place to import sets of profiles, called suites, that you intend to use on your machine.

Additionally, this is the perfect place to import anything you might need from the nixos-hardware repository.

Example

hosts/librem.nix:

{ suites, hardware, ... }:
{
  imports = suites.laptop ++ [ hardware.purism-librem-13v3 ];

  boot.loader.systemd-boot.enable = true;
  boot.loader.efi.canTouchEfiVariables = true;

  fileSystems."/" = { device = "/dev/disk/by-label/nixos"; };
}

Lib

The lib directory mirrors the upstream concepts of nixpkgs:./lib, nixpkgs:./nixos/lib and nixpkgs:./pkgs/pkgs-lib, but also occasionally nixpkgs:./pkgs/build-support.

It comes with functions necesary to declutter devos itself, but you are welcome to extend it to your needs.

For example:

  • you want to add a library function that depends on some packages and use it throughout your devos environment: place it into ./lib as if you would place it into nixpkgs:./pkgs/pkgs-lib.

  • you want to add library functions that don't depend on pkgs: place them into ./lib as if you would place them into nixpkgs:./lib.

  • need to try out a newish custom build support: place it here before upstreaming into nixpkgs:./pkgs/build-support.

  • you want to reutilize certain module configuration functions or helpers: place them into ./lib as if you would place them into nixpkgs:./nixos/lib.

Once your library grows, we recoomend you start organizing them into subfolders analogous nixpkgs:

nixpkgsdevos
./lib./lib
./pkgs/pkgs-lib./lib/pkgs-lib
./nixos/lib./lib/nixos-lib
./pkgs/build-support./lib/pkgs-build

Example

lib/nixos-lib/mkCustomI3BindSym/default.nix:

{ pkgs, writers, ... }:
{ name, cmd, workspace, baseKey }:
let
  isWorkspaceEmpty = writers.writePython3 "is-workspace-empty" {
    libraries = [ pkgs.python3Packages.i3ipc ];
  } (builtins.readFile ./is-workspace-empty.py);

  ws = builtins.toString workspace;
in
''

  # ${name}
  #bindsym ${baseKey}+${ws} workspace ${ws}; exec ${cmd}
  bindsym ${baseKey}+${ws} workspace ${ws}; exec bash -c "${isWorkspaceEmpty} && ${cmd}"
''

lib/nixos-lib/mkCustomI3BindSym/is-workspace-empty.py:

# returns 0/1 if current workspace is empty/non-empty

import i3ipc

i3 = i3ipc.Connection()
tree = i3.get_tree()


def current_workspace():
    return tree.find_focused().workspace()


if current_workspace().leaves():
    print("Error current workspace is not empty")
    exit(1)
exit(0)

lib/default.nix:

{ nixos, pkgs, ... }:
# ...
{
  # ...
  mkCustomI3BindSym = pkgs.callPackage ./nixos-lib/mkCustomI3BindSym { };
}

Modules

The modules directory is a replica of nixpkg's NixOS modules , and follows the same semantics. This allows for trivial upstreaming into nixpkgs proper once your module is sufficiently stable.

All modules linked in module-list.nix are automatically exported via nixosModules.<file-basename>, and imported into all hosts.

Note:

This is reserved for declaring brand new module options. If you just want to declare a coherent configuration of already existing and related NixOS options , use profiles instead.

Semantics

In case you've never written a module for nixpkgs before, here is a brief outline of the process.

Declaration

modules/services/service-category/my-service.nix:

{ config, lib, ... }:
let
  cfg = config.services.myService;
in
{
  options.services.myService = {
    enable = lib.mkEnableOption "Description of my new service.";

    # additional options ...
  };

  config = lib.mkIf cfg.enable {
    # implementation ...
  };
}

Import

modules/module-list.nix:

[
  ./services/service-category/my-service.nix
]

Usage

Internal

profiles/profile-category/my-profile.nix:

{ ... }:
{
  services.MyService.enable = true;
}

External

flake.nix:

{
  # inputs omitted

  outputs = { self, devos, nixpkgs, ... }: {
    nixosConfigurations.myConfig = nixpkgs.lib.nixosSystem {
      system = "...";

      modules = [
        devos.nixosModules.my-service
        ({ ... }: {
          services.MyService.enable = true;
        })
      ];
    };
  };
}

Overlays

Writing overlays is a common occurence when using a NixOS system. Therefore, we want to keep the process as simple and straightforward as possible.

Any .nix files declared in this directory will be assumed to be a valid overlay, and will be automatically imported into all hosts, and exported via overlays.<file-basename> as well as packages.<system>.<pkgName> (for valid systems), so all you have to do is write it.

Example

overlays/kakoune.nix:

final: prev: {
  kakoune = prev.kakoune.override {
    configure.plugins = with final.kakounePlugins; [
      (kak-fzf.override { fzf = final.skim; })
      kak-auto-pairs
      kak-buffers
      kak-powerline
      kak-vertical-selection
    ];
  };
}

Overrides

By default, the NixOS systems are based on the latest release. While it is trivial to change this to nixos-unstable or any other branch of nixpkgs by changing the flake url, sometimes all we want is a single package from another branch.

This is what the overrides are for. By default, they are pulled directly from nixpkgs/master, but you can change the override flake input url to nixos-unstable, or even a specific sha revision.

Example

Packages

The override packages are defined as a regular overlay with an extra arguement pkgs. This refers to the packages built from the override flake.

Pulling the manix package from the override flake:

{
  packages = pkgs: final: prev: {
    inherit (pkgs) manix;
  };
}

Modules

You can also pull modules from override. Simply specify their path relative to the nixpkgs modules directory. The old version will be added to disabledModules and the new version imported into the configuration.

Pulling the zsh module from the override flake:

{
  modules = [ "programs/zsh/zsh.nix" ];
}
Note:

Sometimes a modules name will change from one branch to another. This is what the disabledModules list is for. If the module name changes, the old version will not automatically be disabled, so simply put it's old name in this list to disable it.

Packages

Similar to modules, the pkgs directory mirrors the upstream nixpkgs/pkgs, and for the same reason; if you ever want to upstream your package, it's as simple as dropping it into the nixpkgs/pkgs directory.

The only minor difference is that, instead of adding the callPackage call to all-packages.nix, you just add it the the default.nix in this directory, which is defined as a simple overlay.

This overlay is set as the default overlay output attribute for the flake. And all the packages are exported via packages.<system>.<pkg-name>, for all the supported systems listed in the package's meta.platforms attribute.

And, as usual, every package in the overlay is also available to any NixOS host.

Automatic Source Updates

There is the added, but optional, convenience of declaring your sources in pkgs/flake.nix as an input. You can then access them from the srcs package. This allows updates to be managed automatically by simply updating the lock file. No more manually entering sha256 hashes!

As an added bonus, version strings are also generated automatically from either the flake ref, or the date and git revision of the source. For examples, definitely checkout the community branch.

Example

pkgs/development/libraries/libinih/default.nix:

{ stdenv, meson, ninja, lib, srcs, ... }:
let inherit (srcs) libinih; in
stdenv.mkDerivation {
  pname = "libinih";

  # version will resolve to 53, as specified in the final example below
  inherit (libinih) version;

  src = libinih;

  buildInputs = [ meson ninja ];

  # ...
}

pkgs/default.nix:

final: prev: {
  libinih = prev.callPackage ./development/libraries/libinih { };
}

pkgs/flake.nix:

{
  description = "Package sources";

  inputs = {
    libinih.url = "github:benhoyt/inih/r53";
    libinih.flake = false;
  };
}

Profiles

Profiles are a convenient shorthand for the definition of options in contrast to their declaration. They're built into the NixOS module system for a reason: to elegantly provide a clear separation of concerns.

If you need guidance, a community branch is maintained to help get up to speed on their usage.

Constraints

For the sake of consistency, a profile should always be defined in a default.nix containing a nixos module config. A profile's directory is used for quick modularization of interelated bits.

Notes:
  • For declaring module options, there's the modules directory.
  • This directory takes inspiration from upstream .
  • Sticking to a simple spec has refreshing advantages. hercules-ci expects all profiles to be defined in a default.nix, allowing them to be built automatically when added. Congruently, suites expect default.nix to avoid having to manage their paths manually.

Subprofiles

Profiles can also define subprofiles. They follow the same constraints outlined above. A good top level profile should be a high level concern, such as your personal development environment while the subprofiles should be more focused program configurations such as your text editor, and shell configs. This way, you can either pull in the whole development profile, or pick and choose individual programs.

Example

profiles/develop/default.nix:

{
  imports = [ ./zsh ];
  # some generic development concerns ...
}

profiles/develop/zsh/default.nix:

{  ... }:
{
  programs.zsh.enable = true;
  # zsh specific options ...
}

Conclusion

Profiles are the most important concept in DevOS. They allow us to keep our Nix expressions self contained and modular. This way we can maximize reuse across hosts while minimizing boilerplate. Remember, anything machine specific belongs in your host files instead.

Secrets

Secrets are managed using git-crypt so you can keep your flake in a public repository like GitHub without exposing your password or other sensitive data.

By default, everything in the secrets folder is automatically encrypted. Just be sure to run git-crypt init before putting anything in here.

Note:

Currently, there is no mechanism in nix to deploy secrets within the nix/store so, if they end up in the nix/store after deployment, they will be world readable on that machine.

The author of devos intends to implement a workaround for this situation in the near future, but for the time being, simple be aware of this.

Suites

Suites provide a mechanism for users to easily combine and name collecitons of profiles. For good examples, check out the suites defined in the community branch.

In the future, we will use suites as a mechanism for deploying various machine types which don't depend on hardware, such as vm's and containers.

Definition

rec {
  workstation = [ profiles.develop profiles.graphical users.nixos ];
  mobileWS = workstation ++ [ profiles.laptop ];
}

Usage

hosts/my-laptop.nix:

{ suites, ... }:
{
  imports = suites.mobileWS;
}

Testing

Testing is always an important aspect of any software development project, and NixOS offers some incredibly powerful tools to write tests for your configuration, and, optionally, run them in CI.

Lib Tests

You can easily write tests for your own library functions in the tests/lib.nix file and they will be run on every nix flake check or during a CI run.

Unit Tests

Unit tests are can be created from regular derivations, and they can do almost anything you can imagine. By convention, it is best to test your packages during their check phase. All packages and their tests will be built during CI.

Integration Tests

You can write integration tests for one or more NixOS VMs that can, optionally, be networked together, and yes, it's as awesome as it sounds!

Be sure to use the mkTest function, in the tests/default.nix which wraps the official testing-python function to ensure that the system is setup exactly as it is for a bare DevOS system. There are already great resources for learning how to use these tests effectively, including the official docs, a fantastic blog post, and the examples in nixpkgs.

Users

Users are a special case of profiles that define system users and home-manager configurations. For your convenience, home manager is wired in by default so all you have to worry about is declaring your users. For a fully fleshed out example, check out the developers personal branch.

Basic Usage

users/myuser/default.nix:

{ ... }:
{
  users.users.myuser = {
    isNormalUser = true;
  };

  home-manager.users.myuser = {
    programs.mpv.enable = true;
  };
}

External Usage

You can easily use the defined home-manager configurations outside of NixOS using the hmActivations meta-package defined in the flakes legacyPackages output. The flk helper script makes this even easier.

This is great for keeping your environment consistent across Unix systems, including OSX.

From within the projects devshell:

# builds the nixos user defined in the NixOS host
flk home NixOS nixos

# build and activate
flk home NixOS nixos switch

Manually from outside the project:

# build
nix build "github:divnix/devos#hmActivationPackages.NixOS.nixos"

# activate
./result/activate && unlink result

flk command

The devshell for the project incudes a convenient script for managing your system called flk. Each of the following chapters is a reference for one of its subcommands.

Rebuild

Without any of the subcommands, flk acts as a convenient shortcut for nixos-rebuild:

flk NixOS build

Will build hosts/NixOS.nix. You can change out build for switch, test, etc. Any additional arguments are passed through to the call to nixos-rebuild.

Usage

flk help

up

The up subcommand is a simple shortcut for nixos-generate-config that is compatible with devos. There is a short explanation in the the getting started guide.

update

The update subcommand is a simple alias for:

nix flake update --recreate-lock-file --commit-lock-file

As it sounds, this will update your lock file, and commit it.

Updating Package Sources

If you pass an input name then it will only update that input.

For example, you can update any package sources you may have declared in pkgs/flake.nix:

flk update srcs

get

The get subcommand is useful for getting a bare copy of devos without the git history. You can pull either the core or community branches.

Usage

flk get BRANCH DEST-DIR

If DEST-DIR is ommitted, it defaults to ./flk.

ISO

Making and writing an installable iso for hosts/NixOS.nix is as simple as:

flk iso NixOS

dd bs=4M if=result/iso/*.iso of=/dev/$your_installation_device \
  status=progress oflag=sync

This works for any file matching hosts/*.nix excluding default.nix.

install

The install subcommand is a simple convenience for nixos-install, similar to the shortcut for nixos-rebuild, all additional arguments are passed through.

Example

flk install NixOS

This will install hosts/NixOS.nix to /mnt. You can override this directory using standard nixos-install args.

home

The home subcommand is for using your home-manager configurations outside of NixOS, providing an awesome mechanism for keeping your environments synchronized, even when using other systems.

Usage

The users page contains a good usage example.

Integrations

This section explores some of the optional tools included with devos to provide a solution to common concerns such as ci and remote deployment. An effort is made to choose tools that treat nix, and where possible flakes, as first class citizens.

deploy-rs

Deploy-rs is a tool for managing NixOS remote machines. It was chosen for devos after the author experienced some frustrations with the stateful nature of nixops' db. It was also designed from scratch to support flake based deployments, and so is an excellent tool for the job.

By default, all the hosts are also available as deploy-rs nodes, configured with the hostname set to networking.hostName; overridable via the command line.

Usage

Just add your ssh key to the host:

{ ... }:
{
  users.users.${sshUser}.openssh.authorizedKeys.keyFiles = [
    ../secrets/path/to/key.pub
  ];
}

And the private key to your user:

{ ... }:
{
  home-manager.users.${sshUser}.programs.ssh = {
    enable = true;

    matchBlocks = {
      ${host} = {
        host = hostName;
        identityFile = ../secrets/path/to/key;
        extraOptions = { AddKeysToAgent = "yes"; };
      };
    };
  }
}

And run the deployment:

deploy "flk#hostName" --hostname host.example.com
Note:

Your user will need sudo access

Hercules CI

If you start adding your own packages and configurations, you'll probably have at least a few binary artifacts. With hercules we can build every package in our configuration automatically, on every commit. Additionally, we can have it upload all our build artifacts to a binary cache like cachix.

This will work whether your copy is a fork, or a bare template, as long as your repo is hosted on GitHub.

Setup

Just head over to hercules-ci.com to make an account.

Then follow the docs to set up an agent, if you want to deploy to a binary cache (and of course you do), be sure not to skip the binary-caches.json.

Ready to Use

The repo is already set up with the proper nix/ci.nix file, building all declared packages, checks, profiles and shells. So you can see if something breaks, and never build the same package twice!

If you want to get fancy, you could even have hercules deploy your configuration!

Note:

Hercules doesn't have access to anything encrypted in the secrets folder, so none of your secrets will accidentally get pushed to a cache by mistake.

You could pull all your secrets via your user, and then exclude it from allUsers to keep checks passing.

Pull Requests

If making a change to core, or adding a feature, please be sure to update the relevant docs. Each directory contains its own README.md, which will automatically be pulled into the mdbook. The book is rendered on every change, so the docs should always be up to date.

We also use BORS to ensure that all pull requests pass the test suite once at least one review is completed.

Community PRs

While much of your work in this template may be idiosyncratic in nature. Anything that might be generally useful to the broader NixOS community can be synced to the community branch to provide a host of useful NixOS configurations available "out of the box".

Style

If you wish to contribute please follow these guidelines:

  • format your code with nixpkgs-fmt. The default devshell includes a pre-commit hook that does this for you.

  • The commit message follows the same semantics as nixpkgs.

    • You can use a # symbol to specify ambiguities. For example, develop#zsh: <rest of commit message> would tell me that your updating the zsh subprofile living under the develop profile.