2021-04-27 11:05:03 +03:00
|
|
|
<h1 align="center">
|
|
|
|
<img src="nix-filter.svg" alt="logo" width="200">
|
|
|
|
<br>
|
2021-04-27 11:07:32 +03:00
|
|
|
nix-filter - a small self-contained source filtering lib
|
2021-04-27 11:05:03 +03:00
|
|
|
</h1>
|
2021-04-23 15:16:23 +03:00
|
|
|
|
2021-04-24 17:27:53 +03:00
|
|
|
**STATUS: beta**
|
2021-04-23 15:16:23 +03:00
|
|
|
|
|
|
|
A cool way to include only what you need.
|
|
|
|
|
|
|
|
When using nix within a project, developers often use `src = ./.;` for a
|
|
|
|
project like this:
|
|
|
|
|
|
|
|
```nix
|
|
|
|
{ stdenv }:
|
|
|
|
stdenv.mkDerivation {
|
|
|
|
name = "my-project";
|
|
|
|
src = ./.;
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
This works but has an issue; on each build, nix will copy the whole project
|
|
|
|
source code into the /nix/store. Including the `.git` folder and any temporary
|
|
|
|
files left over by the editor.
|
|
|
|
|
|
|
|
The main workaround is to use either `builtins.fetchGit ./.` or one of the
|
|
|
|
many gitignore filter projects but this is not precise enough. If the
|
|
|
|
project README changes, it should rebuild the project. If the nix code
|
|
|
|
changes, it shouldn't rebuild the project. That's why this project exists. I
|
|
|
|
want total control.
|
|
|
|
|
|
|
|
## Example usage
|
|
|
|
|
|
|
|
```nix
|
|
|
|
{ stdenv, nix-filter }:
|
|
|
|
stdenv.mkDerivation {
|
|
|
|
name = "my-project";
|
|
|
|
src = nix-filter {
|
2021-04-24 14:50:37 +03:00
|
|
|
root = ./.;
|
|
|
|
# If no include is passed, it will include all the paths.
|
2021-04-24 14:43:04 +03:00
|
|
|
include = [
|
2021-04-24 14:50:37 +03:00
|
|
|
# Include the "src" path relative to the root.
|
|
|
|
"src"
|
|
|
|
# Include this specific path. The path must be under the root.
|
|
|
|
./package.json
|
|
|
|
# Include all files with the .js extension
|
|
|
|
(nix-filter.matchExt "js")
|
2021-04-23 15:16:23 +03:00
|
|
|
];
|
|
|
|
|
2021-04-24 14:50:37 +03:00
|
|
|
# Works like include, but the reverse.
|
|
|
|
exclude = [
|
|
|
|
./main.js
|
|
|
|
];
|
2021-04-23 15:16:23 +03:00
|
|
|
};
|
|
|
|
}
|
|
|
|
```
|
2021-04-23 20:19:37 +03:00
|
|
|
|
2021-05-05 11:15:45 +03:00
|
|
|
# Usage
|
2021-04-23 20:19:37 +03:00
|
|
|
|
2021-05-05 11:15:45 +03:00
|
|
|
Import this folder. Eg:
|
|
|
|
|
|
|
|
```nix
|
|
|
|
let
|
|
|
|
nix-filter = import ./path/to/nix-filter;
|
|
|
|
in
|
|
|
|
# ...
|
|
|
|
```
|
|
|
|
|
|
|
|
The top-level is a functor that takes:
|
2021-04-23 20:19:37 +03:00
|
|
|
* `path` of type `path`: pointing to the root of the source to add to the
|
|
|
|
/nix/store.
|
|
|
|
* `name` of type `string` (optional): the name of the derivation (defaults to
|
|
|
|
"source")
|
2021-04-24 14:43:04 +03:00
|
|
|
* `include` of type `list(string|path|matcher)` (optional): a list of patterns to
|
2021-04-23 20:19:37 +03:00
|
|
|
include (defaults to all).
|
2021-04-24 14:43:04 +03:00
|
|
|
* `exclude` of type `list(string|path|matcher)` (options): a list of patterns to
|
2021-04-23 20:19:37 +03:00
|
|
|
exclude (defaults to none).
|
|
|
|
|
2021-04-24 14:43:04 +03:00
|
|
|
The `include` and `exclude` take a matcher, and automatically convert the `string`
|
2021-04-23 20:19:37 +03:00
|
|
|
and `path` types to a matcher.
|
|
|
|
|
|
|
|
The matcher is a function that takes a `path` and `type` and returns `true` if
|
|
|
|
the pattern matches.
|
|
|
|
|
|
|
|
## Builtin matchers
|
|
|
|
|
2021-05-05 11:15:45 +03:00
|
|
|
The functor also contains a number of matchers:
|
|
|
|
|
|
|
|
* `nix-filter.matchExt`: `ext` -> returns a function that matches the given file extension.
|
|
|
|
* `nix-filer.inDirectory`: `directory` -> returns a function that matches a directory and
|
2021-04-29 22:48:37 +03:00
|
|
|
any path inside of it.
|
2021-05-05 11:15:45 +03:00
|
|
|
* `nix-filter.isDirectory`: matches all paths that are directories
|
2021-04-30 02:12:16 +03:00
|
|
|
|
|
|
|
## Combining matchers
|
|
|
|
|
|
|
|
* `and`: `a -> b -> c`
|
|
|
|
combines the result of two matchers into a new matcher.
|
|
|
|
* `or`: `a -> b -> c`
|
|
|
|
combines the result of two matchers into a new matcher.
|
2021-04-23 20:19:37 +03:00
|
|
|
|
2021-05-05 11:15:45 +03:00
|
|
|
# Design notes
|
|
|
|
|
|
|
|
This solution uses `builtins.path { path, name, filter ? path: type: true }`
|
|
|
|
under the hood, which ships with Nix.
|
|
|
|
|
|
|
|
While traversing the filesystem, starting from `path`, it will call `filter`
|
|
|
|
on each file and folder recursively. If the `filter` returns `false` then the
|
|
|
|
file or folder is ignored. If a folder is ignored, it won't recurse into it
|
|
|
|
anymore.
|
|
|
|
|
|
|
|
Because of that, it makes it difficult to implement recursive glob matchers.
|
|
|
|
Something like `**/*.js` would necessarily have to add every folder, to be
|
|
|
|
able to traverse them. And those empty folders will end-up in the output.
|
|
|
|
|
|
|
|
If we want to control rebuild, it's important to have a fixed set of folders.
|
|
|
|
|
|
|
|
One possibility is to use a two-pass system, where first all the folders are
|
|
|
|
being added, and then the empty ones are being filtered out. But all of this
|
|
|
|
happens at Nix evaluation time. Nix evaluation is already slow enough like
|
|
|
|
that.
|
|
|
|
|
|
|
|
That's why nix-filter is asking the users to explicitly list all the folders
|
|
|
|
that they want to add.
|
|
|
|
|
|
|
|
# Future development
|
|
|
|
|
|
|
|
Add more matchers.
|
|
|
|
|
|
|
|
# Related projects
|
2021-04-23 20:19:37 +03:00
|
|
|
|
2021-05-05 11:15:45 +03:00
|
|
|
* nixpkgs' `lib.cleanSourceWith`.
|
|
|
|
* All the git-based solutions. See https://github.com/hercules-ci/gitignore.nix#comparison
|
2021-04-23 20:19:37 +03:00
|
|
|
|
|
|
|
# License
|
|
|
|
|
|
|
|
Copyright (c) 2021 Numtide under the MIT.
|