I've just seen `liquidd` hit the timeout on nixbitcoin.org while
`Loading block index`.
This was probably due to HDD contention while starting services during
boot.
Module authors can now add extra services via option `nix-bitcoin.nodeinfo.services`.
Mark as internal because we're not yet providing a user-friendly, stable API.
cb295d1a16 joinmarket: 0.9.5 -> 0.9.6 (nixbitcoin)
568d728e5c joinmarket: put python-packages in alphabetical order (nixbitcoin)
a3ff1351d9 update nixpkgs (Jonas Nick)
636d508b50 spark-wallet: use nodejs 14 instead of 12 since latter is EOL (Jonas Nick)
Pull request description:
ACKs for top commit:
erikarvstedt:
ACK cb295d1a16
Tree-SHA512: ca124de4d393c3828454599e4d3d3357560a6ae902c569b3edabf395cd6f5d50d8904db9f078f02b70cd0b67d013a263d238e9bc36ba34bc1bb7d9d9fd02a7e9
This makes scripts fail early on request errors.
Previously, curl exited with status 0 when enountering HTTP error status
codes.
`-fsS` equals `--fail --silent --show-error`.
- Move option `rtl.nodes.{lnd,clightning}` -> `rtl.nodes.{lnd,clightning}.enable`
This is required by the next commit.
- Move option `rtl.loop` -> `rtl.nodes.lnd.loop`
- Only enable loop when `nodes.lnd` is enabled
This makes our list definitions mergeable with custom list values
set by users.
Previously, a module error ("value is a string while a list
was expected") was thrown instead.
This commit was partly auto-generated with this script:
#!/usr/bin/env ruby
Dir["**/*.nix"].each do |file|
src = File.read(file)
fixed = src.gsub(/ReadWritePaths *= *(.*?);/) do
"ReadWritePaths = [ #{$1} ];"
end
File.write(file, fixed) if fixed != src
end
Option `services.lnd.restOnionService.package` has been removed.
There's not much use in overriding the [lndconnect pkg](https://github.com/LN-Zap/lndconnect).
- Rename `services.rtl.cl-rest` to `services.clightning-rest`.
`clightning-rest` is generally useful for connecting external REST clients
to clightning.
- Add a dedicated network namespace in netns-isolation.
- Add nodeinfo entry.
- Add datadir (which contains REST auth data) to backups.
This enables generating module option documentation.
This commit was genereated by running the following script inside the
repo root dir:
def add_default_text(file)
src = File.read(file)
src2 = src.gsub(/( = mkOption\s+\{[^{]*?)(\n\s+default = )(.*?);$(.*?\})/m) do |str|
pre, defaultVar, default, post = Regexp.last_match.captures
replacement =
if !post.include?('defaultText =')
if default =~ /\bpkgs\b/
defaultText = default.lines.length == 1 ? default : "(See source)"
"#{pre}#{defaultVar}#{default};#{defaultVar.sub('default', 'defaultText')}#{defaultText.inspect};#{post}"
end
end
replacement or str
end
File.write(file, src2) if src2 != src
end
Dir["modules/**/*.nix"].each do |f|
next if File.basename(f) == "nix-bitcoin.nix"
add_default_text f
end
This enables generating module option documentation.
Add `defaultText` to options where the default value can't be
evaluated during metadata generation.
This simplifies integrating a remote bitcoind instance and
makes `bitcoin-cli` work with the remote node.
Add note regarding `whitelistedPort` to docs.
This is simpler and more memory-efficient.
We've also changed other services to use this appraoch.
Also remove unneded `wait_for_unit` in the electrs regtest test.
Split `enforceTor` into `tor.proxy` and `tor.enforce`.
By enabling `tor.proxy` without `tor.enforce`, a service can accept
incoming clearnet connections.
E.g., this allows setting up a Tor-proxied bitcoind node that accepts
RPC connections from LAN.
When NixOS is already running and Tor is restarted due to config
changes, `/var/lib/tor/state` may be present even when Tor has not
yet finished setting up onion services.
This caused the previous version of `onion-addresses` to not wait for
Tor and to skip not yet present onion service files.
`onion-addresses` now waits until each required onion service file
has appeared.
btcpayserver: 1.3.3 -> 1.3.6
electrs: 0.9.2 -> 0.9.3
lnd: 0.13.3-beta -> 0.14.1-beta
nbxplorer: 2.2.16 -> 2.2.18
electrs does not accept command line argument -vv anymore, so we have to use
--log-filter instead.
4a74b7de08 clightning: work around unsupported seccomp syscall (Erik Arvstedt)
38a843d005 clightning: update python pkgs to new version (Erik Arvstedt)
6ad7107ddb update nixpkgs (Erik Arvstedt)
f58d67677e netns-isolation: separate host and netns setup (Erik Arvstedt)
cb6e5ef702 netns-isolation: fix routing issues due to netns restarting (Erik Arvstedt)
7f77147b60 makeShell: minor improvements (Erik Arvstedt)
a5730eb736 makeShell: make the help msg a shell derivation variable (Erik Arvstedt)
Pull request description:
ACKs for top commit:
jonasnick:
ACK 4a74b7de08
Tree-SHA512: 75454b51db6d7ab41590d8579e0a5136e5ac1be78d5c2f547c6ef1982c0de679968879bb9bac57dd66413f59a4659236601ab75414486b0137c7c43d73d22759
- Improves readability
- `netns exec ...` (called via `netnsIptables`) incurs a large
overhead: In addition to netns switching, a mount namespace
is setup and populated with the contents of /etc/netns/<ns>/.
Instead, simply run `nsenter`.
Previously, restarting a service implied restarting its netns.
For unknown reasons, this sometimes caused the netns-local address
to not be routable from the root netns for up to 20 s.
I.e., the service was sometimes unreachable after restarting.
Now the netns is no longer stopped when the service is stopped.
Otherwise liquidd startup fails. This bug was not covered by our tests,
because we're not combining `regtest` with `secure-node`.
But nixbitcoin.org does, which should suffice for now.
This simplifies the host's address configuration.
This also removes unused addresses that are returned when resolving
container hostnames via nss-mymachines:
`getent ahosts nb-test`
There is no security reason why pruning should be enabled and therefore it
surprises users. Turning on pruning in the first place was simply a mistake.
- README: add matrix room
- examples/configuration.nix: explain why bitcoind is enabled by default
- btcpayserver: group lnd service settings
- clightning:
Use public onion port only when the onion service is public
This allows users to enable the onion service while announcing a
non-onion public address.
- netns-isolation: move `readOnly` attr to the top
- tests: use mkDefault to allow for easier overriding
- tests/btcpayserver: test web server response
This allows whitelisting local services without implicitly
whitelisting all inbound onion connections, which would happen when
setting bitcoind/liquidd option `whitelist=localhost`.
Used by electrs and nbxplorer, which requires the unsafe `mempool`
permission.
This removes the module-level dependency from onion-services to
bitcoind.
Due to the `or false` fallback, there's no dependency added in
the reverse direction.
In particular, this allows us to not add a dependency on liquidd in
the following commit.
Whitelisting localhost implicitly whitelists all inbound onion
connections. This prevents banning misbehaving inbound onion peers
and enables message `mempool` which can cause privacy leaks.
Instead, grant `download` as the single bitcoind whitelist permission, which
should be safe for onion peers.
Remove liquidd whitelisting because it doesn't support fine-grained permissions.
After a cursory glance at the nbxplorer code I think that nbxplorer
requires none of the other default whitelist permissions (noban, mempool,
relay).
Details: https://github.com/dgarage/NBXplorer/issues/344
This re-enables onion tagging while still supporting untagged connections.
Onion sockets are not yet supported in the latest liquidd/elements
version 0.18.1.12 available on nixpkgs.
3781a85c9b joinmarket: enable Agora as a third IRC server (nixbitcoin)
ced1637d07 joinmarket: share IRC server definitions between jm and ob-watcher (Erik Arvstedt)
59fc003ebd joinmarket: 0.9.1 -> 0.9.2 (nixbitcoin)
Pull request description:
ACKs for top commit:
erikarvstedt:
ACK 3781a85c9b
Tree-SHA512: 5ec919d2291ecf96fb4ca880f3dbeabff13f2bab71822db893ebbaba1b95463666b098ccc1412a1b56f327a231e10c1f2d47feb0f520fce349ab243d398bf7b4
- `waitfornewblock` was previously not included in the public RPC
whitelist because it's reserved for testing and marked as hidden
in bitcoind.
- electrs changed its verbosity settings. `-vv` is now the best choice
for normal usage.
- bitcoind option `dataDirReadableByGroup` is now unused.
Because it can be valuable for other use cases and implementing
it is intricate, we're keeping it for now.
- test: keep `nc` connection open because otherwise the electrs
RPC server would now close the connection before sending a response.
Previously, Tor was always enabled because `cfg` was always nonempty
(via definitions at `Set sensible defaults for some services`).
Now only enable Tor if there are active onion services.
Also rename var `services` -> `onionServices` to improve readability in
section `Set getPublicAddressCmd ...` where the same name is also used for
option `config.services`.
Previously, the glob (*) returned '*' when no files existed in the
secrets dir, leading to error `chown: cannot access '*'`.
Now `unprocessedFiles` is empty when there are no secrets.
Also remove the unneeded sorting of `unprocessedFiles` and
remove redundant leading zero in the default mode.
- `discover` is automatically disabled by bitcoind because we're
setting `externalip` via the `nix-bitcoin.onionServices` mechanism
- `bech32` is bitcoind's default addresstype
- Add nbxplorer to whitelists.
This is recommended by the nbxplorer docs and guarantees that nbxplorer
can always p2p-connect to bitcoind/liquidd.
- Enable bitcoind/liquidd p2p servers via `listen`.
- bitcoind: Remove obsolete defaultText
- clightning: Fix description
Option `address` can't be used to specify a socket path because it's
used explicitly as an IP address in many places.
- lnd: Break up overlong line
This is required by commit `services: support 0.0.0.0/:: in `address` options`
- nix-bitcoin.nix: Formatting
- secrets: Improve descriptions
Benefits of adding top-level variables for used services:
- Makes it obvious which other services are referenced by a service
- Less code
We already do this in many other places.
These are insignificant, generic options; place them above readonly options.
We already do this in other services.
Also move user/group config to bottom in spark-wallet.
This greatly improves readability and makes it easier to discover options.
This commit was genereated by running the following script inside the
repo root dir:
#!/usr/bin/env ruby
def transform(src)
return false if src.include?('inherit options;')
success = false
options = nil
src.sub!(/^ options.*?^ }.*?;/m) do |match|
options = match
" inherit options;"
end
return false if !options
src.sub!(/^with lib;\s*let\n+/m) do |match|
success = true
<<~EOF
with lib;
let
#{options}
EOF
end
success
end
Dir['modules/**/*.nix'].each do |f|
src = File.read(f)
if transform(src)
puts "Changed file #{f}"
File.write(f, src)
end
end
`generate-secrets` is no longer a monolithic script. Instead, it's
composed of the values of option `nix-bitcoin.generateSecretsCmds`.
This has the following advantages:
- generate-secrets is now extensible by users
- Only secrets of enabled services are generated
- RPC IPs in the `lnd` and `loop` certs are no longer hardcoded.
Secrets are no longer automatically generated when entering nix-shell.
Instead, they are generated before deployment (via `krops-deploy`)
because secrets generation is now dependant on the node configuration.
- btcpayserver: remove unneeded trailing semicolons
- krops/get-sha256:
`tail` is unneeded because `nix-prefetch-url` just outputs a single
line containing the hash.
Now that service uid, gid mappings are included in the backups, along
with the service data dirs, we can remove 'chown -R' for
clightning and liquidd data dirs.
Note that we used 'chown -R' only for these two services, while this
approach would have been relevant for all services with data dirs.
nix-bitcoin.nix is now no longer dependent on clightning.nix and lnd.nix.
Due to condition '!(config.services ? clightning)' lnd.nix still
doesn't depend on clightning.nix.
Also fix the assertion message by renaming clightning.bindPort to clightning.port.
