This enables generating module option documentation.
This commit was genereated by running the following script inside the
repo root dir:
def add_default_text(file)
src = File.read(file)
src2 = src.gsub(/( = mkOption\s+\{[^{]*?)(\n\s+default = )(.*?);$(.*?\})/m) do |str|
pre, defaultVar, default, post = Regexp.last_match.captures
replacement =
if !post.include?('defaultText =')
if default =~ /\bpkgs\b/
defaultText = default.lines.length == 1 ? default : "(See source)"
"#{pre}#{defaultVar}#{default};#{defaultVar.sub('default', 'defaultText')}#{defaultText.inspect};#{post}"
end
end
replacement or str
end
File.write(file, src2) if src2 != src
end
Dir["modules/**/*.nix"].each do |f|
next if File.basename(f) == "nix-bitcoin.nix"
add_default_text f
end
This enables generating module option documentation.
Add `defaultText` to options where the default value can't be
evaluated during metadata generation.
This simplifies integrating a remote bitcoind instance and
makes `bitcoin-cli` work with the remote node.
Add note regarding `whitelistedPort` to docs.
This is simpler and more memory-efficient.
We've also changed other services to use this appraoch.
Also remove unneded `wait_for_unit` in the electrs regtest test.
Split `enforceTor` into `tor.proxy` and `tor.enforce`.
By enabling `tor.proxy` without `tor.enforce`, a service can accept
incoming clearnet connections.
E.g., this allows setting up a Tor-proxied bitcoind node that accepts
RPC connections from LAN.
When NixOS is already running and Tor is restarted due to config
changes, `/var/lib/tor/state` may be present even when Tor has not
yet finished setting up onion services.
This caused the previous version of `onion-addresses` to not wait for
Tor and to skip not yet present onion service files.
`onion-addresses` now waits until each required onion service file
has appeared.
btcpayserver: 1.3.3 -> 1.3.6
electrs: 0.9.2 -> 0.9.3
lnd: 0.13.3-beta -> 0.14.1-beta
nbxplorer: 2.2.16 -> 2.2.18
electrs does not accept command line argument -vv anymore, so we have to use
--log-filter instead.
4a74b7de08 clightning: work around unsupported seccomp syscall (Erik Arvstedt)
38a843d005 clightning: update python pkgs to new version (Erik Arvstedt)
6ad7107ddb update nixpkgs (Erik Arvstedt)
f58d67677e netns-isolation: separate host and netns setup (Erik Arvstedt)
cb6e5ef702 netns-isolation: fix routing issues due to netns restarting (Erik Arvstedt)
7f77147b60 makeShell: minor improvements (Erik Arvstedt)
a5730eb736 makeShell: make the help msg a shell derivation variable (Erik Arvstedt)
Pull request description:
ACKs for top commit:
jonasnick:
ACK 4a74b7de08
Tree-SHA512: 75454b51db6d7ab41590d8579e0a5136e5ac1be78d5c2f547c6ef1982c0de679968879bb9bac57dd66413f59a4659236601ab75414486b0137c7c43d73d22759
- Improves readability
- `netns exec ...` (called via `netnsIptables`) incurs a large
overhead: In addition to netns switching, a mount namespace
is setup and populated with the contents of /etc/netns/<ns>/.
Instead, simply run `nsenter`.
Previously, restarting a service implied restarting its netns.
For unknown reasons, this sometimes caused the netns-local address
to not be routable from the root netns for up to 20 s.
I.e., the service was sometimes unreachable after restarting.
Now the netns is no longer stopped when the service is stopped.
Otherwise liquidd startup fails. This bug was not covered by our tests,
because we're not combining `regtest` with `secure-node`.
But nixbitcoin.org does, which should suffice for now.
This simplifies the host's address configuration.
This also removes unused addresses that are returned when resolving
container hostnames via nss-mymachines:
`getent ahosts nb-test`
There is no security reason why pruning should be enabled and therefore it
surprises users. Turning on pruning in the first place was simply a mistake.
- README: add matrix room
- examples/configuration.nix: explain why bitcoind is enabled by default
- btcpayserver: group lnd service settings
- clightning:
Use public onion port only when the onion service is public
This allows users to enable the onion service while announcing a
non-onion public address.
- netns-isolation: move `readOnly` attr to the top
- tests: use mkDefault to allow for easier overriding
- tests/btcpayserver: test web server response
This allows whitelisting local services without implicitly
whitelisting all inbound onion connections, which would happen when
setting bitcoind/liquidd option `whitelist=localhost`.
Used by electrs and nbxplorer, which requires the unsafe `mempool`
permission.
This removes the module-level dependency from onion-services to
bitcoind.
Due to the `or false` fallback, there's no dependency added in
the reverse direction.
In particular, this allows us to not add a dependency on liquidd in
the following commit.
Whitelisting localhost implicitly whitelists all inbound onion
connections. This prevents banning misbehaving inbound onion peers
and enables message `mempool` which can cause privacy leaks.
Instead, grant `download` as the single bitcoind whitelist permission, which
should be safe for onion peers.
Remove liquidd whitelisting because it doesn't support fine-grained permissions.
After a cursory glance at the nbxplorer code I think that nbxplorer
requires none of the other default whitelist permissions (noban, mempool,
relay).
Details: https://github.com/dgarage/NBXplorer/issues/344
This re-enables onion tagging while still supporting untagged connections.
Onion sockets are not yet supported in the latest liquidd/elements
version 0.18.1.12 available on nixpkgs.
3781a85c9b joinmarket: enable Agora as a third IRC server (nixbitcoin)
ced1637d07 joinmarket: share IRC server definitions between jm and ob-watcher (Erik Arvstedt)
59fc003ebd joinmarket: 0.9.1 -> 0.9.2 (nixbitcoin)
Pull request description:
ACKs for top commit:
erikarvstedt:
ACK 3781a85c9b
Tree-SHA512: 5ec919d2291ecf96fb4ca880f3dbeabff13f2bab71822db893ebbaba1b95463666b098ccc1412a1b56f327a231e10c1f2d47feb0f520fce349ab243d398bf7b4
- `waitfornewblock` was previously not included in the public RPC
whitelist because it's reserved for testing and marked as hidden
in bitcoind.
- electrs changed its verbosity settings. `-vv` is now the best choice
for normal usage.
- bitcoind option `dataDirReadableByGroup` is now unused.
Because it can be valuable for other use cases and implementing
it is intricate, we're keeping it for now.
- test: keep `nc` connection open because otherwise the electrs
RPC server would now close the connection before sending a response.
Previously, Tor was always enabled because `cfg` was always nonempty
(via definitions at `Set sensible defaults for some services`).
Now only enable Tor if there are active onion services.
Also rename var `services` -> `onionServices` to improve readability in
section `Set getPublicAddressCmd ...` where the same name is also used for
option `config.services`.
Previously, the glob (*) returned '*' when no files existed in the
secrets dir, leading to error `chown: cannot access '*'`.
Now `unprocessedFiles` is empty when there are no secrets.
Also remove the unneeded sorting of `unprocessedFiles` and
remove redundant leading zero in the default mode.
- `discover` is automatically disabled by bitcoind because we're
setting `externalip` via the `nix-bitcoin.onionServices` mechanism
- `bech32` is bitcoind's default addresstype
- Add nbxplorer to whitelists.
This is recommended by the nbxplorer docs and guarantees that nbxplorer
can always p2p-connect to bitcoind/liquidd.
- Enable bitcoind/liquidd p2p servers via `listen`.
- bitcoind: Remove obsolete defaultText
- clightning: Fix description
Option `address` can't be used to specify a socket path because it's
used explicitly as an IP address in many places.
- lnd: Break up overlong line
This is required by commit `services: support 0.0.0.0/:: in `address` options`
- nix-bitcoin.nix: Formatting
- secrets: Improve descriptions
Benefits of adding top-level variables for used services:
- Makes it obvious which other services are referenced by a service
- Less code
We already do this in many other places.
These are insignificant, generic options; place them above readonly options.
We already do this in other services.
Also move user/group config to bottom in spark-wallet.
This greatly improves readability and makes it easier to discover options.
This commit was genereated by running the following script inside the
repo root dir:
#!/usr/bin/env ruby
def transform(src)
return false if src.include?('inherit options;')
success = false
options = nil
src.sub!(/^ options.*?^ }.*?;/m) do |match|
options = match
" inherit options;"
end
return false if !options
src.sub!(/^with lib;\s*let\n+/m) do |match|
success = true
<<~EOF
with lib;
let
#{options}
EOF
end
success
end
Dir['modules/**/*.nix'].each do |f|
src = File.read(f)
if transform(src)
puts "Changed file #{f}"
File.write(f, src)
end
end
`generate-secrets` is no longer a monolithic script. Instead, it's
composed of the values of option `nix-bitcoin.generateSecretsCmds`.
This has the following advantages:
- generate-secrets is now extensible by users
- Only secrets of enabled services are generated
- RPC IPs in the `lnd` and `loop` certs are no longer hardcoded.
Secrets are no longer automatically generated when entering nix-shell.
Instead, they are generated before deployment (via `krops-deploy`)
because secrets generation is now dependant on the node configuration.
- btcpayserver: remove unneeded trailing semicolons
- krops/get-sha256:
`tail` is unneeded because `nix-prefetch-url` just outputs a single
line containing the hash.
Now that service uid, gid mappings are included in the backups, along
with the service data dirs, we can remove 'chown -R' for
clightning and liquidd data dirs.
Note that we used 'chown -R' only for these two services, while this
approach would have been relevant for all services with data dirs.
nix-bitcoin.nix is now no longer dependent on clightning.nix and lnd.nix.
Due to condition '!(config.services ? clightning)' lnd.nix still
doesn't depend on clightning.nix.
Also fix the assertion message by renaming clightning.bindPort to clightning.port.
NixOS option `security.hideProcessInformation` for globally restricting
access to /proc has been removed.
Use per-service restrictions via 'ProtectProc' instead.
Rename
`nix-bitcoin.security.hideProcessInformation` to
`nix-bitcoin.security.dbusHideProcessInformation`
because this option now only implements the dbus restriction.
Starting with 0.21.0, bitcoin no longer automatically creates and loads
a default wallet.
This was being ignored because of a JoinMarket issue [1] in CI builds prior
to this version. Now a watch-only Bitcoin Core wallet is created in ExecStartPost.
[1] https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/812
Don't copy bitcoin-rpcpassword-privileged as root, instead run service
with group "bitcoin".
Same effect, less complexity. Note, PoLP still obeyed for joinmarket-ob-watcher.
Due to a possible NixOS bug, this commit has no effect on NixOS 20.09
where `RestrictAddressFamilies` is a no-op.
It's only relevant for NixOS unstable with cgroups v2.
bitcoind+zmq: instead of allowing all address families, only add the required
AF_NETLINK family.
lnd: lnd only runs a zmq client, not a server, therefore it requires
no additional address families.
lightning-pool, clightning-plugin-zmq: add AF_NETLINK.
- Improve readability by using minutes
- set `TimeoutStopSec` like in bitcoin/contrib/init/bitcoind.service.
Stopping bitcoind can exceed the default timeout during IBD.
Keeping the secrets dir read-only is more simple and robust.
- lnd seed mnemonic creation and joinmarket wallet creation can be
run as the regular service user instead of root.
- It is easier to switch to a third-party secrets deployment
method in the future.
Don't create a seed mnemonic for lnd when a wallet exists.
This avoids creating unused mnemonics and helps simplifying
the migration command in `versioning.nix`.
- Fail at evaluation when secrets setup is not configured.
Previously, bitcoind failed at runtime due to the missing secrets target.
- Fail at evaluation when conflicting secrets setup methods are used.
This happens when `secretsSetupMethod` has more than one definition.
With krops or nixops the secrets target is always restarted during
the deployment process.
This previously caused unnecessary restarts of all nix-bitcoin services.
Move this feature from a module preset to a regular option, so that it's
easily discoverable and accessible.
Simplify the implementation of `generateSecrets` by adding it to the
existing `setup-secrets` service script.
Also rename option setup-secrets -> setupSecrets.
This commit fixes an issue with LND, in which if both
nix-bitcoin.onionServices.lnd.public &
services.lnd.restOnionService.enable were enabled, one would try to
create a file named `lnd` and the other would try to create a directory
named `lnd` with a file named `lnd-rest` inside it. This would obiously
cause an error and fail the LND service.
When running as root, use runuser instead of sudo.
As opposed to sudo or doas, runuser is a standalone
binary that needs no external configuration.
Also, it's a bit faster.
Use the following order of definitions for all services:
- assertions
- configuration of other services
- environment.systemPackages
- tmpfiles
- own service
- users
- secrets
Journal entries now look like
`joinmarket-yieldgenerator[9795]: User data location: /var/lib/joinmarket`
instead of
`bash[9795]: User data location: /var/lib/joinmarket`
DynamicUser simplifies services that don't need a persistent uid/gid,
like joinmarket-ob-watcher.
For existing installations the data dir migration to dynamic users
is automatically handled by systemd.
- Fix jm-wallet-seed being globally readable.
- Handle seed extraction failures.
If seed extraction fails, remove the newly created wallet.
This guarantees that wallets always have an accompanying seed.
Systemd's `Description` option is a misnomer (as confessed by `man systemd.unit`):
Its value is used by user-facing tools in place of the unit file name, so this option
could have been more aptly named `label` or `name`.
`Description` should only be set if the unit file name is not sufficient for naming a unit.
This is not the case for our services, except for `systemd.services.nb-netns-bridge`
whose description has been kept.
As an example how this affects users, weird journal lines like
```
nb-test systemd[1]: Starting Run clightningd...
```
are now replaced by
```
nb-test systemd[1]: Starting clightning.service...
```
Previously, onionAddresses definitions in onionServices were of the form
onionAddresses.access.<service> = [<service>];
This caused failures for configurations where a service user name was
overridden or for bitcoind whose default user is 'bitcoin' instead of 'bitcoind'.
Now set the equivalent of:
onionAddresses.access.<actualServiceUser> = [<service>];
Implement this via a new option `onionAddresses.services` to make things more
readable and to work around an infinite recursion error in onionServices.
- enable usage outside of secure-node.nix
- use json as the output format
- show ports
- also show local addresses, which is particularly useful when
netns-isolation is enabled
- only show enabled services
