dream2nix/docs/extending-dream2nix.md
Yusuf Bera Ertan 645c6fd98e
refactor: implement a validation system for builders / translators, reorganize files (#155)
* refactor: implement a validation system for builders / translators etc, organize files

* refactor: use seq instead of complicated validation function for validator

* feat: allow adding discoverers, translators and builders via config

* refactor: rework discoverers to use makeSubsystemModules as well

* fix: validate extra modules properly

* feat: support inline modules

* feat: use extra attribute for extending

* feat: make fetchers extensible properly

* fix: add name to extra fetchers

* feat: support list for extra

* docs: add some comment to lib/modules.nix

* fix: get extra module args from extraArgs

* fix: collect all modules instead of only collecting modules for built-in subsystems

* refactor: minor improvements

* refactor: improve how default subsystem modules are declared

* fix: translators and builders are directly under subsystem now

* fix: correct attribute path, remove unused argument

* fix: correct translators attribute paths

* fix: correct file paths and translators attribute paths

* fix: use correct translator attr path in wrapPureTranslator

* fix: update unit tests code

* fix: remove extra paranthesis in unit tests code

* tests: add an extended dream2nix example

* refactor: replace recursiveUpdate usage with normal update op

* tests: fix and extend d2n-extended example

* fix: pass config to d2n instance in wrap pure translator script

* fix: correct toFile usage

* fix: pass config to dlib in more places

* fix: pass config to d2n instance in aggregated hashes cli and gomod2nix translator

* refactor: remove unused extra modules validation, add warning for function modules

* fix: remove non-existent inherited variable

* docs: update translator attr path in contributors guide

* docs: add docs for extending dream2nix

* refactor: comment more code, warn for function modules only if extra is an attrset decl

* docs: fix some typos

* docs: explain some stuff in extending d2n better

* fix: print function modules warning when it is a function

* tests: add a new example that tests adding new subsystem and config.extra as nix file

* tests: use cargo-toml as translator on d2n-extended to potentially catch more bugs

* feat: add ifd warning for builders

* tests: use build-rust-package builder instead of crane builder in d2n-extended to also test it instead of only testing crane builder

* fix(rust/builders): always write the generated Cargo.lock so it doesnt get out of sync with our dream-lock

* fix(rust/builders): delete cargo lock before writing it?

* refactor: also print ifd warnings for translators

* docs: link extending d2n doc in readme, link examples in extending d2n

* docs: example naming (translators.new -> translators.example-translator)

* feat: allow setting nix files for modules declarations (eg. subsystems, subsystems.translators)

* refactor: move IFD warnings to src/lib/builders.nix / translators.nix respectively

* refactor: throw instead of warning if function declarations for modules are used

* refactor: fix throw usage

* refactor: improve modules code

* chore(deps): update nixpkgs

* fix: correct some map usages

* fix: use correct attr path for extra modules

* chore: update examples flake inputs

* style: minor formatting changes
2022-05-29 21:42:47 +02:00

2.2 KiB

Extending dream2nix with external translators, builders etc.

dream2nix can be extended while you are initializing it. This can be done in a few ways. For extending, you need to utilize the config.extra option of the dream2nix config.

Declare extras from a nix file

dream2nix.lib.init {
  # this also works around errors with function modules
  # being declared here, which will be explained later.
  config.extra = ./nix/d2n/extras.nix;
}

this file should look like this:

{dlib, lib, config, ...}:
{
  subsystems.rust.translators.example-translator = ./rust-translator.nix;
  # you can declare modules using functions here
  fetchers.ipfs = {...}: {/* fetcher attrs */};
}

See the d2n-extended-new-subsystem example for a basic example.

Declare extras as an attribute set

dream2nix.lib.init {
  config.extra = {
    subsystems = {
      # add new modules
      ruby.discoverers.default = ./nix/d2n/ruby/discoverer.nix;
      ruby.translators.bundix = ./nix/d2n/ruby/bundix.nix;
      # existing modules can be overridden
      rust.builders.crane = ./nix/d2n/rust/crane.nix;
    };
    # add new fetchers
    fetchers.ipfs = ./nix/d2n/fetchers/ipfs.nix;
    fetchers.gitea = ./nix/d2n/fetchers/gitea.nix;
    # existing fetchers can be overridden
    fetchers.http = ./nix/d2n/fetchers/http-proxied.nix;
  };
}

note: you can't declare modules using functions here. This is because the config may need to be serialized to JSON and passed around in impure situations (mostly impure translators).

See the d2n-extended example for a basic example.

Compose multiple different extras

This allows one to compose multiple extras together with ease. The dream2nixExtras flake output in this case can be any of the approaches explained in this document for declaring config.extra.

dream2nix.lib.init {
  # note: .dream2nixExtras is a hypothetical standardized flake output
  # in practice this can be any flake output.
  config.extra = [
    haskellSubsystemFlake.dream2nixExtras
    crystalSubsystemFlake.dream2nixExtras
    gleamSubsystemFlake.dream2nixExtras
  ];
}