nix-community/dream2nix
1
1
Fork 0
mirror of https://github.com/nix-community/dream2nix.git synced 2024-12-23 14:31:55 +03:00
Code Issues Projects Releases Wiki Activity

feat(v1 API): document v1 API examples

Browse Source
This commit is contained in:
DavHau 2023-01-10 14:41:19 +08:00
parent f47ce48d15
commit dc9954aae4
12 changed files with 758 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
docs/src/SUMMARY.md
View File

@ -31,3 +31,16 @@
# Development Roundups
- [April - June 2022](./development-roundups/2022-april-june.md)
- [July - September 2022](./development-roundups/2022-july-september.md)
# WIP
- [v1 API](./v1-api/summary.md)
- [problems of the current dream2nix](./v1-api/problems.md)
- [users of dream2nix](./v1-api/users.md)
- [v1 packaging: project initialization](./v1-api/packaging/nodejs-init-project.md)
- [v1 packaging: workspaces](./v1-api/packaging/nodejs-workspaces.md)
- [v1 packaging: multiple repos](./v1-api/packaging/nodejs-multiple-repos.md)
- [v1 packaging: monorepo](./v1-api/packaging/monorepo.md)
- [v1 consuming: inspect package options](./v1-api/consuming/inspect-options.md)
- [v1 consuming: override packages](./v1-api/consuming/override.md)
- [v1 integrating: lang2nix tool (pure)](./v1-api/integrating/integrate-lang2nix-pure.md)
- [v1 integrating: lang2nix tool (code-gen/impure)](./v1-api/integrating/integrate-lang2nix-impure.md)

28
docs/src/v1-api/consuming/inspect-options.md Normal file
View File

@ -0,0 +1,28 @@
# Inspect the API of a package
Downstream users can inspect the api of any consumed package as well as raw package modules
## Load the dream2nix shell
```shell
nix-shell https://dream2nix.dev -A devShells.default
```
## Get manual of package module
Assuming a package module in `./upstream/my-package.nix`
```shell
$ dream2nix man ./upstream/my-package.nix
```
## Get manual of derivation
Assuming derivations defined via `./upstream/default.nix`
```shell
dream2nix man ./upstream/default.nix -A packages.my-package
```
## Get manual of flake attribute
Assuming derivations defined via a flake on github
```shell
dream2nix man github:user/repo#some-package
```

198
docs/src/v1-api/consuming/override.md Normal file
View File

@ -0,0 +1,198 @@
# Consume and modify dream2nix packages
## Given the following package
`upstream/my-package.nix`
```nix
{config, lib, dream2nix, ...}: {
imports = [
dream2nix.modules.nodejs.mkDerivation
dream2nix.modules.nodejs.package-lock
];
pname = "my-package";
version = "2.0.0";
src = {
type = github;
owner = "my-user";
repo = "my-repo";
ref = config.version;
hash = "sha256-mia90VYv/YTdWNhKpvwvFW9RfbXZJSWhJ+yva6EnLE8=";
};
# declare dependency on python3
deps = {nixpkgs, ...}: {
python3 = nixpkgs.python39;
};
nativeBuildInputs = [
config.deps.python3
];
configurePhase = ''
python3 --version
'';
buildPhase = ''
python3 -c 'print("Hello World!")' > $out
'';
}
```
`upstream/default.nix`
```nix
{
nixpkgs ? import <nixpkgs> {},
dream2nix ?
import
(builtins.fetchTarball "https://dream2nix.dev/tarball/1.0")
{inherit nixpkgs;},
}: {
packages.my-package = dream2nix.eval ./my-package.nix;
}
```
## 1. Override using modules
### 1.1 Define a module for the override
`my-package-override.nix`
```nix
{config, lib, ... }: {
version = "2.1.0";
# No need to re-define other fetcher attributes.
# The module system updates them for us.
src.hash = "sha256-LM5GDNjLcmgZVQEeANWAOO09KppwGaYEzJBjYmuSwys=";
deps = {nixpkgs, ...}: {
# change the python version
python3 = lib.mkForce nixpkgs.python310;
# add a dependency on hello
hello = nixpkgs.hello;
};
# add hello to nativeBuildInputs
# (`oldAttrs.nativeBuildInputs + ...` not needed here)
nativeBuildInputs = [
config.deps.hello
];
# add lines to configurePhase
postConfigure = ''
hello --version
'';
# replace the build phase via mkForce
buildPhase = lib.mkForce "
hello > $out
";
}
```
### 1.2 Apply `my-package-override.nix` via extendModules
Using `extendModules` is simple.
It allows to extend an existing package with another module.
This doesn't require knowledge about the original modules that went into the package.
`./default.nix`
```nix
let
nixpkgs = import <nixpkgs> {};
upstream = import ./upstream {inherit nixpkgs;};
my-package = upstream.packages.my-package;
# The recommended way of modifying a package is using extendModules,
# which uses the module systems merge logic to apply changes.
my-package-extended = my-package.extendModules {
modules = [./my-package-override.nix];
};
in {
inherit my-package-extended;
}
```
### 1.3 Or apply `my-package-override.nix` via dream2nix.eval
This approach is a bit cleaner.
It doesn't introduce a chain of extendModules function calls.
This style also makes it obvious which modules went into the package.
Though, this requires access to the original `my-package.nix` module and knowledge about the `packageSets` that went into it.
`default.nix`
```nix
{
nixpkgs ? import <nixpkgs> {},
dream2nix ?
import
(builtins.fetchTarball "https://dream2nix.dev/tarball/1.0")
{inherit nixpkgs;},
}: let
my-package-extended = dream2nix.eval
{packagetSets = {inherit nixpkgs;};}
[
./upstream/my-package.nix
./my-package-override.nix
];
in {
my-package-extended
}
```
## 2. Override package via `override[Attrs]` functions
It is recommended to use modules for overriding, like described above, but for backward compatibility, `overrideAttrs` and `override` are still supported.
```nix
let
nixpkgs = import <nixpkgs> {};
upstream = import ./upstream {inherit nixpkgs;};
my-package = upstream.packages.my-package;
# Override the package via `override` and `overrideAttrs`
my-package-overridden' = my-package.override
(oldAttrs: {
# change the python version
python3 = nixpkgs.python310;
});
my-package-overridden = my-package-overridden'.overrideAttrs
(oldAttrs: rec {
version = "2.1.0";
src = nixpkgs.fetchFromGithub {
owner = "my-owner";
repo = "my-repo";
ref = version;
hash = "sha256-LM5GDNjLcmgZVQEeANWAOO09KppwGaYEzJBjYmuSwys=";
};
# add hello to nativeBuildInputs
nativeBuildInputs = [
nixpkgs.hello
];
# add lines to configurePhase
postConfigure = ''
hello --version
'';
# replace the build phase
buildPhase = ''
hello > $out
'';
});
in {
inherit my-package-overridden;
}
```

58
docs/src/v1-api/integrating/integrate-lang2nix-impure.md Normal file
View File

@ -0,0 +1,58 @@
# Integrate lang2nix tool (impure/code-gen)
We use [gomod2nix](https://github.com/nix-community/gomod2nix) as an example here to demonstrate creating a dream2nix integration.
Gomod2nix is a nix code generator that requires network access, a great example for an impure dream2nix integration.
`dream2nix/modules/go.gomod2nix.nix`
```nix
{config, lib, dream2nix, system, ...}: rec {
imports = [
# import generic mkDerivation interface, which will add options like:
# - buildInputs
# - nativeBuildInputs
# - ...
dream2nix.modules.mkDerivation-interfaces
# Generic interface for impure lang2nix tools (code generators)
# This provides options like `generateBin` (see below)
dream2nix.modules.integrations.impure
];
options = {
modules = lib.mkOption {
description = "The path to the gomod2nix.toml";
type = lib.types.str;
default = "${config.dream2nix.artifactsLocation}/gomod2nix.toml" ;
};
};
config = {
# Generated code will end up in:
# {repo}/dream2nix/artifacts/{engineName}/{package_identifier}
dream2nix.engineName = "gomod2nix";
# An executable that generates nix code for the given `src`
dream2nix.generateBin = dream2nix.utils.writePureShellScript "gomod2nix-generate.sh"
[
# add gomod2nix tool to PATH
dream2nix.inputs.gomod2nix.packages.${system}.gomod2nix
]
''
targetDir=$1
gomod2nix --dir "${config.src}" --outdir "$targetDir"
'';
# signal that all options should be passed to the final derivation function
argsForward = l.mapAttrs (_: _: true) options;
# the final derivation is built by calling gomod2nix.buildGoApplication
config.final.derivation =
dream2nix.inputs.gomod2nix.lib.${system}.buildGoApplication
config.final.derivation-args;
};
}
```

46
docs/src/v1-api/integrating/integrate-lang2nix-pure.md Normal file
View File

@ -0,0 +1,46 @@
# Integrate lang2nix tool (pure)
We use [crane](https://crane.dev) as an example here to demonstrate creating a dream2nix integration
`dream2nix/modules/rust.crane-buildPackage.nix`
```nix
{config, lib, dream2nix, system, ...}: rec {
imports = [
# import generic mkDerivation interface, which will add options like:
# - buildInputs
# - nativeBuildInputs
# - ...
dream2nix.modules.mkDerivation-interfaces
];
options = {
buildPhaseCargoCommand = lib.mkOption {
description = "A command to run during the derivation's build phase. Pre and post build hooks will automatically be run.";
type = lib.types.nullOr lib.types.str;
default = null;
};
cargoArtifacts = lib.mkOption {
description = "A path (or derivation) which contains an existing cargo target directory, which will be reused at the start of the derivation. Useful for caching incremental cargo builds.";
type = lib.types.nullOr lib.types.str;
default = null;
};
cargoBuildCommand = lib.mkOption {
description = "A cargo invocation to run during the derivation's build phase";
type = lib.types.nullOr lib.types.str;
default = null;
}
# ... more options of crane's buildPackage
};
config = {
# signal that all options should be passed to the final derivation function
argsForward = l.mapAttrs (_: _: true) options;
# the final derivation is built by calling crane.buildPackage
config.final.derivation =
dream2nix.inputs.crane.lib.${system}.buildPackage
config.final.derivation-args;
};
}
```

94
docs/src/v1-api/packaging/monorepo.md Normal file
View File

@ -0,0 +1,94 @@
# build packages in a monorepo
The example mono repo has 3 packages: `nodejs-app`, `python-tool`, `rust-tool`.
The packages `python-tool` and `rust-tool` might or might not be built with dream2nix.
The package `nodejs-app` is built with dream2nix and depends on `python-tool` and `rust-tool`.
## Assuming this repo structure
```
├── default.nix
├── overrides
│ ├── nodejs
│ ├── python
│ └── rust
├── nodejs-app
│ └── default.nix
├── python-tool
│ └── default.nix
└── rust-tool
└── default.nix
```
## Contents of `./nodejs-app/default.nix`
`./nodejs-app/default.nix`
```nix
{config, lib, dream2nix, ...}: {
imports = [
# default module to create a nodejs package
dream2nix.modules.nodejs.mkDerivation
# get package dependencies from package-lock
dream2nix.modules.nodejs.package-lock
];
# Overrides allow to manipulate dependency builds
overrides.local.path = ../overrides/nodejs;
src = ./.;
# include dependencies from nixpkgs and the local monorepo
# see definition of `packageSets` in ../default.nix
deps = {nixpkgs, monorepo, ...} @ packageSets: {
inherit (nixpkgs)
hello
;
inherit (monorepo)
python-tool
rust-tool
;
};
nativeBuildInputs = [
config.deps.hello
config.deps.python-tool
config.deps.rust-tool
];
configurePhase = ''
hello --version
python-tool --version
rust-tool --version
'';
# add more mkDerivation attributes here to customize...
}
```
## Contents of `./default.nix`
`./default.nix`
```nix
{
nixpkgs ? import <nixpkgs> {},
dream2nix ?
import
(builtins.fetchTarball "https://dream2nix.dev/tarball/1.0")
{inherit nixpkgs;},
} @ inputs: let
makePackage = modules: dream2ix.mkDerivation
# Package sets available to each package's `deps` function
{packageSets = {inherit monorepo nixpkgs;};}
modules;
monorepo = {
nodejs-app = makePackage ./nodejs-app;
python-tool = makePackage ./python-tool;
rust-tool = makePackage ./rust-tool;
};
in {
packages = monorepo;
}
```

123
docs/src/v1-api/packaging/nodejs-init-project.md Normal file
View File

@ -0,0 +1,123 @@
# initialize nodejs project + dev shell
## load shell with nodejs + npm
```console tesh-session="next-app" tesh-setup="setup.sh"
$ nix-shell -p https://dream2nix.dev -A devShells.nodejs
```
## create my-app
```console tesh-session="next-app"
npx create-next-app my-app
```
This creates `./my-app/package.json` and more, using `create-next-app` as a helper.
## create `my-app.nix`
`my-app.nix`
```nix
{config, lib, dream2nix, ...}: {
imports = [
# default module to create a nodejs package
dream2nix.modules.nodejs.mkDerivation
# get package dependencies from package-lock
dream2nix.modules.nodejs.package-lock
];
# Allows to manipulate dependency builds
overrides.local.path = ./overrides;
src = ./my-app;
# add more mkDerivation attributes here to customize...
}
```
## create `my-app-shell.nix` for your dev shell
`my-app-shell.nix`
```nix
{config, lib, dream2nix, ...}: {
imports = [
# the default dev shell for nodejs
dream2nix.modules.nodejs.mkShell
# adds dependencies of my-app to the dev shell
dream2nix.modules.nodejs.package-lock
];
src = ./my-app;
# include hello from nixpkgs.
# `deps` is the single source of truth for inputs from the `outside world`.
# `deps` will later allow us to safely override any dependency.
deps = {nixpkgs, ...}: {
inherit (nixpkgs) hello;
};
# add hello from nixpkgs to the dev shell
buildInputs = [
config.deps.hello
]
}
```
## create `default.nix` entry point
`default.nix`
```nix
{
nixpkgs ? import <nixpkgs> {},
dream2nix ?
import
(builtins.fetchTarball "https://dream2nix.dev/tarball/1.0")
{inherit nixpkgs;},
}: {
packages.my-app = dream2nix.eval ./my-app.nix;
devShells.my-app = dream2nix.eval ./my-app-shell.nix;
}
```
## build my-app
```command
nix-build -f ./default.nix -A packages.my-app
```
## create `shell.nix` (used by `nix-shell` command)
`shell.nix`
```nix
(import ./default.nix {}).devShells.my-app
```
Enter the dev shell:
```command
nix-shell
```
all dependencies of my-app are available
## fix build of dependencies via `./overrides/`
Files in `./overrides/` must always be named like the package they apply to.
Example: `./overrides/keytar.nix`
##
```nix
{config, ...}: {
# include dependencies from nixpkgs.
deps = {nixpkgs, ...}: {
inherit (nixpkgs)
libsecret
pkg-config
;
};
# add build time dependencies
nativeBuildInputs = [
config.deps.libsecret
config.deps.pkg-config
];
}
```
Scoped package example: `./overrides/@babel/core.nix`
##
```nix
{config, ...}: {
# ...
}
```

67
docs/src/v1-api/packaging/nodejs-multiple-repos.md Normal file
View File

@ -0,0 +1,67 @@
# handle multiple repos
Assuming that `./repo1` and `./repo2` are separate git repositories.
Both repos have a single package `repo1/my-app` and `repo2/my-tool`.
In order to build `repo1/my-app` we need `repo2/my-tool` as a build time dependency.
The following structure is assumed:
```
├── repo1
│ ├── default.nix
│ └── my-app.nix
└── repo2
├── default.nix
└── my-tool.nix
```
## contents of `repo1/my-app.nix`
`repo1/my-app.nix`
```nix
{config, lib, dream2nix, ...}: {
imports = [
dream2nix.modules.nodejs.mkDerivation
dream2nix.modules.nodejs.package-lock
];
src = ./.;
# include my-tool from repo2
deps = {repo2, ...}: {
inherit (repo2) my-tool;
};
# add my-tool as build time dependency
nativeBuildInputs = [
config.deps.my-tool
];
# use my-tool to build my-app
buildPhase = ''
my-tool build
echo "done building"
'';
}
```
## contents of `repo1/default.nix`
`repo1/default.nix`
```nix
{
pkgs ? import <nixpkgs> {},
dream2nix ?
import
(builtins.fetchTarball "https://dream2nix.dev/tarball/1.0")
{inherit pkgs;},
}: {
packages.my-app = dream2nix.eval
{
packageSets.nixpkgs = pkgs;
# fetchGit could be used here alternatively
packageSets.repo2 = import ../repo2/default.nix {};
}
./my-app.nix;
}
```

82
docs/src/v1-api/packaging/nodejs-workspaces.md Normal file
View File

@ -0,0 +1,82 @@
# build + develop on nodejs workspaces
## assuming a `package.json` with workspaces
`package.json`
```
{
"name": "my-workspaces",
"workspaces": [
"my-tool"
"my-first-app"
"my-second-app"
]
}
```
## define package set via `workspaces.nix`
```nix
{config, lib, dream2nix, ...}: {
imports = [
dream2nix.modules.nodejs.workspaces
dream2nix.modules.nodejs.package-lock
];
src = ./.;
# Allows to manipulate builds of workspace members and their dependencies
overrides.local.path = ./overrides;
}
```
## create `default.nix` entry point
`default.nix`
```nix
{
nixpkgs ? import <nixpkgs> {},
dream2nix ?
import
(builtins.fetchTarball "https://dream2nix.dev/tarball/1.0")
{inherit nixpkgs;},
}: {
packages = {
inherit (dream2nix.lib.mkPackageSet ./workspaces.nix)
my-tool
my-first-app
my-second-app
;
};
}
```
## configure package builds via `./overrides/`
Files in `./overrides/` must always be named like the the package they apply to.
Manipulate my-tool via `./overrides/my-tool.nix`
```nix
{config, ...}: {
# include python from nixpkgs
deps = {nixpkgs, ...}: {
inherit (nixpkgs) python;
};
buildInputs = [
config.deps.python
];
}
```
Manipulate my-first-app via `./overrides/my-first-app.nix`
```nix
{config, ...}: {
# include my-tool from the local workspace
deps = {workspace, ...}: {
inherit (workspace) my-tool;
};
buildInputs = [
config.deps.my-tool
];
}
```

22
docs/src/v1-api/problems.md Normal file
View File

@ -0,0 +1,22 @@
# Problems of the current dream2nix
## Integration of existing lang2nix tools
Until now, integrating existing 2nix solutions into dream2nix was hard because dream2nix imposed standards which are not met by most existing tools.
With v1 we want to lift most of these restrictions to make integration a no-brainer
(see sections [integrate lang2nix tool (pure)](../v1-api/integrate-lang2nix-pure.md) and [integrate lang2ix tool (impure)](../v1-api/integrate-lang2nix-impure.md).
## Tied to flakes
The current api is tied to flakes. The v1 API should not depend on flakes anymore.
## Composability
Composability with the current `makeFlakeOutputs` is bad. Flakes itself aren't nicely composable. Filtering and merging of nested attrsets isn't user friendly.
The v1 api will focus on delivering individual derivations, not flakes.
While we might provide templates, recommendations, and tools for composition, we should not enforce a specific solution onto the user.
## Overridability
The experience of overriding package- and dependency builds was a bit bumpy so far, as the overriding mechanism was built ontop of override functions provided by nixpkgs' `mkDerivation`. The v1 API will make use of the nixos module system instead to handle derivation attributes.
## Discoverability of package options
We want users to be able to inspect the API of an individual package. This will also be made possible by the nixos module system.

16
docs/src/v1-api/summary.md Normal file
View File

@ -0,0 +1,16 @@
# dream2nix v1 API
- [problems of the current dream2nix](../v1-api/problems.md)
- [users of dream2nix](../v1-api/users.md)
- v1 API examples:
- package maintainers:
- [project initialization](../v1-api/packaging/nodejs-init-project.md)
- [workspaces](../v1-api/packaging/nodejs-workspaces.md)
- [multiple repos](../v1-api/packaging/nodejs-multiple-repos.md)
- [monorepo](../v1-api/packaging/monorepo.md)
- consumers:
- [inspect package options](../v1-api/consuming/inspect-options.md)
- [override packages](../v1-api/consuming/override.md)
- integration maintainers:
- [integrate lang2nix tool (pure)](../v1-api/integrating/integrate-lang2nix-pure.md)
- [integrate lang2nix tool (code-gen/impure)](../v1-api/integrating/integrate-lang2nix-impure.md)

11
docs/src/v1-api/users.md Normal file
View File

@ -0,0 +1,11 @@
# The users of dream2nix
The following groups of users are relevant regarding the v1 API design.
## Integration Maintainers (Level1)
People who use dream2nix to maintain language2nix integrations
## Package Maintainers (Level2)
People who use dream2nix to maintain nix derivations for packages
## Consumers (Level3)
People who use and customize packages created via dream2nix