Remove absolute reference to xsltproc. This reference breaks cross-compilation: instead of host version builder tries to execute binary for target and fails.
This is useful if your postgresql version is dependant on
`system.stateVersion` and not pinned down manually. Then it's not
necessary to find out which version exactly is in use and define
`package` manually, but just stay with what NixOS provides as default:
$ nix-instantiate -A postgresql
/nix/store/82fzmb77mz2b787dgj7mn4a8i4f6l6sn-postgresql-14.7.drv
$ nix-instantiate -A postgresql_jit
/nix/store/qsjkb72fcrrfpsszrwbsi9q9wgp39m50-postgresql-14.7.drv
$ nix-instantiate -A postgresql.withJIT
/nix/store/qsjkb72fcrrfpsszrwbsi9q9wgp39m50-postgresql-14.7.drv
$ nix-instantiate -A postgresql.withJIT.withoutJIT
/nix/store/82fzmb77mz2b787dgj7mn4a8i4f6l6sn-postgresql-14.7.drv
I.e. you can use postgresql with JIT (for complex queries only[1]) like
this:
services.postgresql = {
enable = true;
enableJIT = true;
};
Performing a new override instead of re-using the `_jit`-variants for
that has the nice property that overlays for the original package apply
to the JIT-enabled variant, i.e.
with import ./. {
overlays = [
(self: super: {
postgresql = super.postgresql.overrideAttrs (_: { fnord = "snens"; });
})
];
};
postgresql.withJIT.fnord
still gives the string `snens` whereas `postgresql_jit` doesn't have the
attribute `fnord` in its derivation.
[1] https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-JIT-ABOVE-COST
Closes#150801
Note: I decided against resuming directly on #150801 because the
conflict was too big (and resolving it seemed too error-prone to me).
Also the `this`-refactoring could be done in an easier manner, i.e. by
exposing JIT attributes with the correct configuration. More on that
below.
This patch creates variants of the `postgresql*`-packages with JIT[1]
support. Please note that a lot of the work was derived from previous
patches filed by other contributors, namely dasJ, andir and abbradar,
hence the co-authored-by tags below.
Effectively, the following things have changed:
* For JIT variants an LLVM-backed stdenv with clang is now used as
suggested by dasJ[2]. We need LLVM and CLang[3] anyways to build the
JIT-part, so no need to mix this up with GCC's stdenv. Also, using the
`dev`-output of LLVM and clang's stdenv for building (and adding llvm
libs as build-inputs) seems more cross friendly to me (which will
become useful when cross-building for JIT-variants will actually be
supported).
* Plugins inherit the build flags from the Makefiles in
`$out/lib/pgxs/src` (e.g. `-Werror=unguarded-availability-new`). Since
some of the flags are clang-specific (and stem from the use of the
CLang stdenv) and don't work on gcc, the stdenv of `pkgs.postgresql`
is passed to the plugins. I.e., plugins for non-JIT variants are built
with a gcc stdenv on Linux and plugins for JIT variants with a clang
stdenv.
Since `plv8` hard-codes `gcc` as `$CC` in its Makefile[4], I marked it
as broken for JIT-variants of postgresql only.
* Added a test-matrix to confirm that JIT works fine on each
`pkgs.postgresql_*_jit` (thanks Andi for the original test in
#124804!).
* For each postgresql version, a new attribute
`postgresql_<version>_jit` (and a corresponding
`postgresqlPackages<version>JitPackages`) are now exposed for better
discoverability and prebuilt artifacts in the binary cache.
* In #150801 the `this`-argument was replaced by an internal recursion.
I decided against this approach because it'd blow up the diff even
more which makes the readability way harder and also harder to revert
this if necessary.
Instead, it is made sure that `this` always points to the correct
variant of `postgresql` and re-using that in an additional
`.override {}`-expression is trivial because the JIT-variant is
exposed in `all-packages.nix`.
* I think the changes are sufficiently big to actually add myself as
maintainer here.
* Added `libxcrypt` to `buildInputs` for versions <v13. While
building things with an LLVM stdenv, these versions complained that
the extern `crypt()` symbol can't be found. Not sure what this is
exactly about, but since we want to switch to libxcrypt for `crypt()`
usage anyways[5] I decided to add it. For >=13 it's not relevant
anymore anyways[6].
* JIT support doesn't work with cross-compilation. It is attempted to
build LLVM-bytecode (`%.bc` is the corresponding `make(1)`-rule) for
each sub-directory in `backend/` for the JIT apparently, but with a
$(CLANG) that can produce binaries for the build, not the host-platform.
I managed to get a cross-build with JIT support working with
`depsBuildBuild = [ llvmPackages.clang ] ++ buildInputs`, but
considering that the resulting LLVM IR isn't platform-independent this
doesn't give you much. In fact, I tried to test the result in a VM-test,
but as soon as JIT was used to optimize a query, postgres would
coredump with `Illegal instruction`.
A common concern of the original approach - with llvm as build input -
was the massive increase of closure size. With the new approach of using
the LLVM stdenv directly and patching out references to the clang drv in
`$out` the effective closure size changes are:
$ nix path-info -Sh $(nix-build -A postgresql_14)
/nix/store/kssxxqycwa3c7kmwmykwxqvspxxa6r1w-postgresql-14.7 306.4M
$ nix path-info -Sh $(nix-build -A postgresql_14_jit)
/nix/store/xc7qmgqrn4h5yr4vmdwy56gs4bmja9ym-postgresql-14.7 689.2M
Most of the increase in closure-size stems from the `lib`-output of
LLVM
$ nix path-info -Sh /nix/store/5r97sbs5j6mw7qnbg8nhnq1gad9973ap-llvm-11.1.0-lib
/nix/store/5r97sbs5j6mw7qnbg8nhnq1gad9973ap-llvm-11.1.0-lib 349.8M
which is why this shouldn't be enabled by default.
While this is quite much because of LLVM, it's still a massive
improvement over the simple approach of adding llvm/clang as
build-inputs and building with `--with-llvm`:
$ nix path-info -Sh $(nix-build -E '
with import ./. {};
postgresql.overrideAttrs ({ configureFlags ? [], buildInputs ? [], ... }: {
configureFlags = configureFlags ++ [ "--with-llvm" ];
buildInputs = buildInputs ++ [ llvm clang ];
})' -j0)
/nix/store/i3bd2r21c6c3428xb4gavjnplfqxn27p-postgresql-14.7 1.6G
Co-authored-by: Andreas Rammhold <andreas@rammhold.de>
Co-authored-by: Janne Heß <janne@hess.ooo>
Co-authored-by: Nikolay Amiantov <ab@fmap.me>
[1] https://www.postgresql.org/docs/current/jit-reason.html
[2] https://github.com/NixOS/nixpkgs/pull/124804#issuecomment-864616931
& https://github.com/NixOS/nixpkgs/pull/150801#issuecomment-1467868321
[3] This fails with the following error otherwise:
```
configure: error: clang not found, but required when compiling --with-llvm, specify with CLANG=
```
[4] https://github.com/plv8/plv8/blob/v3.1.5/Makefile#L14
[5] https://github.com/NixOS/nixpkgs/pull/181764
[6] c45643d618
`bridge` is used by iproute2, so using this name for protonmail-bridge
made it very likely to produce a name "conflict".
Also `bridge` is used in the Makefile by upstream project Makefile but
it apparently is renamed later on when packaged in rpm/deb so even for
coherence purposes it does make sense to revert it back to the name
`protonmail-bridge` that were previously being used.
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
The keyd package already exists, but without a systemd service.
Keyd requires write access to /var/run to create its socket. Currently
the directory it uses can be changed with an environment variable, but
the keyd repo state suggests that this may turn into a compile-time
option. with that set, and some supplementary groups added, we can run
the service under DynamicUser.
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
The single option tries to do too much work, which just ends up confusing people.
So:
- don't force the console font, the kernel can figure this out as of #210205
- don't force the systemd-boot mode, it's an awkward mode that's not supported
on most things and will break flicker-free boot
- add a separate option for the xorg cursor scaling trick and move it under the xorg namespace
- add a general `fonts.optimizeForVeryHighDPI` option that explicitly says what it does
- alias the old option to that
- don't set any of those automatically in nixos-generate-config
the conversion to the markdown-based workflow missed that generating the
manual as docbook also generates the option docs with nixos-specific
wrapper elements that require postprocessing before the document is
real, valid docbook. restore this processing to the full manual.
it's not the prettiest thing, done like this, but we only need it for
one release so it doesn't have to be.
- drop media-session (rip 💀)
- stop trying to let people override default configs, those never got merged correctly
- drop all the complexity arising from having to vendor default config files
- build docs in sandbox as we no longer recurse
This patch fixes two issues:
1. The file in which environment variables are set is inconsistent.
- This file sets them in zprofile when programs.zsh.enable is not
set.
- Zsh module sets them in zshenv when programs.zsh.enable is set.
2. Setting environment variables in zprofile overrides what users set
in .zshenv. See these[1] home-manager[2] issues[3].
/etc/profile is also changed to /etc/set-environment. Here is a
comparison:
Using /etc/profile:
- Pros
- config.environment.shellInit is sourced in all zsh
- Cons
- config.environment.loginShellInit is also sourced in non-login zsh
- config.programs.bash.shellInit is also sourced in all zsh
- config.programs.bash.loginShellInit is also sourced in all zsh
Using /etc/set-environment:
- Pros
- config.programs.bash.shellInit is not sourced in any zsh
- config.programs.bash.loginShellInit is not sourced in any zsh
- Cons
- config.environment.shellInit is not sourced in any zsh
- config.environment.loginShellInit is not sourced in any zsh
[1]: https://github.com/nix-community/home-manager/issues/2751#issuecomment-1048682643
[2]: https://github.com/nix-community/home-manager/issues/2991
[3]: https://github.com/nix-community/home-manager/issues/3681#issuecomment-1436054233
Effectively removes support for the following hashing algorithms
as announced in the NixOS 22.11 release notes:
- bcrypt_x ($2x$)
- sha256crypt ($5$)
- sha1crypt ($sha1$)
- sunmd5 ($md5$)
- md5crypt ($1$)
- nt ($3$)
- bdiscrypt (_)
- bigcrypt (:)
- descrypt (:)
And exposes the crypt scheme ids for enabled algorithms, so they can be
reused for validation in the users-groups module.
- Christmas is over!
- Upstream has changed the name of the target triplet used for the JS
backend from js-unknown-ghcjs to javascript-unknown-ghcjs, since Cabal
calls the architecture "javascript":
6636b67023
Since the triplet is made up anyways, i.e. autoconf does not support
it and Rust uses different triplets for its emscripten backends, we'll
just change it as well.
- Upstream fixed the problem with ar(1) being invoked incorrectly by stage0:
e987e345c8
systemd v253 changelog/NEWS:
https://github.com/systemd/systemd/blob/v253/NEWS
NixOS changes:
0007-hostnamed-localed-timedated-disable-methods-that-cha.patch was
dropped, because systemd gained support to handle read-only /etc.
*-add-rootprefix-to-lookup-dir-paths.patch required some updates too,
as src/basic/def.h moved to src/basic/constants.h.
systemd/systemd#25771 switched p11kit to become
dlopen()'ed, so we need to patch that path.
added a note to the 23.05 release notes to recommend `nixos-rebuild boot`
Co-authored-by: Florian Klink <flokli@flokli.de>
This makes the following work
disabledModules = [ foo.nixosModules.bar ];
even if `bar` is not a path, but rather a module such as
{ key = "/path/to/foo#nixosModules.bar"; config = ...; }
By supporting this, the user will often be able to use the same syntax
for both importing and disabling a module. This is becoming more relevant
because flakes promote the use of attributes to reference modules. Not
all of these modules in flake attributes will be identifiable, but with
the help of a framework such as flake-parts, these attributes can be
guaranteed to be identifiable (by outPath + attribute path).
* k3s: add environmentFile option
Enabling to include secrets through configuration such as 'sops'
* Update nixos/doc/manual/release-notes/rl-2305.section.md
Co-authored-by: Jairo Llopis <973709+yajo@users.noreply.github.com>
Update protonmail-bridge to v3. This also rename the CLI executable from
protonmail-bridge to bridge to be more in line with upstream naming.
Co-authored-by: James Landrein <github@j4m3s.eu>
Signed-off-by: Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@proton.ch>
In general the man pages do not care what OS and manual they are shipped
with, so they don't get to choose the names of them. We were tempted to
do so, as we had inconsistently chosen OS names for the mandoc and
man-db/groff implementations. Since this has been rectified since, we
can just drop this boilerplate from the man pages.
Upstream has officially abandoned the project as of 2021 [0], there's been
no release since 2016, it uses the EoL Qt 4, and alternatives like
KeePassXC exist.
Also move KeePassXC to its own directory -- it doesn't make sense to
have it in KeePassX's folder anymore.
[0]: https://www.keepassx.org/index.html%3Fp=636.html
this reproduces the docbook-generated html manual exactly enough to
appease the compare workflows while we still support both toolchains.
it's also a lot faster than the docbook toolchain, rendering the entire
html manual in about two seconds on this machine (while docbook needs
about 20).
Provide a module to configure Coqui TTS, available as `tts` in nixpkgs
for a few releases already.
The module supports multiple servers in parallel, so multiple languages
and testing scenarios can be covered, without affecting any production
usage.
options processing is pretty slow right now, mostly because the
markdown-it-py parser is pure python (and with performance
pessimizations at that). options parsing *is* embarassingly parallel
though, so we can just fork out all the work to worker processes and
collect the results.
multiprocessing probably has a greater benefit on linux than on darwin
since the worker spawning method darwin uses is less efficient than
fork() on linux. this hasn't been tested on darwin, only on linux, but
if anything darwin will be faster with its preferred method.
these files provided mainly or exclusively xml-related tooling and
support for editing the manual. since docbook is now an implementation
detail (except for still being allowed in option docs, for now) these
tools are no longer necessary, useful, or even functional.
xmlformat.conf is still used by the nixpkgs documentation, so we have to
keep it. there's no reason it can't go live with the nixpkgs docs though.
this adds support for structural includes to nixos-render-docs.
structural includes provide a way to denote the (sub)structure of the
nixos manual in the markdown source files, very similar to how we used
literal docbook blocks before, and are processed by nixos-render-docs
without involvement of xml tooling. this will ultimately allow us to
emit the nixos manual in other formats as well, e.g. html, without going
through docbook at all.
alternatives to this source layout were also considered:
a parallel structure using e.g. toml files that describe the document
tree and links to each part is possible, but much more complicated to
implement than the solution chosen here and makes it harder to follow
which files have what substructure. it also makes it much harder to
include a substructure in the middle of a file.
much the same goes for command-line arguments to the converter, only
that command-lined arguments are even harder to specify correctly and
cannot be reasonably pulled together from many places without involving
another layer of tooling. cli arguments would also mean that the manual
structure would be fixed in default.nix, which is also not ideal.
- inline copySources into single user
- remove `inherit sources` where it's not necessary
- inline generatedSources. this will go away completely soon so we may
as well.
- inline modulesDoc into manual-combined. this too will go away soon.
- inline sources into manual-combined. this too will go away soon.
once we generate the entire manual-combined.xml with a single
nixos-render-docs invocation we will no longer need any options xml
files. likewise we do not need the test options xml in the manpage
build. splitting manpages-combined from manual-combined also allows
these two to run in parallel, slightly improving build times.
man-configuration.xml is the only manpage we build using docbook at this
time, and man-pages.xml includes just this one file. inline
man-configuration.nix into man-pages.xml so we can refer to a single
file to grab the manpage sources later rather than using find-by-suffix.
render all manual chapters to docbook from scratch every time the manual
is built. nixos-render-docs is quick enough at this to not worry about
the cost (needing only about a second), and it means we can remove
md-to-db.sh in the next commit.
no changes to the rendered html manual except for replacements and smartquotes.
we'll soon add another docbook converter that does not emit a section as
a collection of chapters, but sections or chapters on their own. this
should clarify naming a bit before there can be any confusion.
since support for kbd elements was added with explicit intent in #175128
it seems like a good idea to support this in nixos-render-docs instead
of just dropping it in favor of `*F12*` etc. since it's a very rare
thing in the manual and purely presentational it makes sense to use
bracketed spans instead of a new myst role.
the html-elements.lua plugin is now somewhat misnamed, but it'll go away
very soon so we don't want to bother renaming it.
this is a lot easier than adding footnote support for just the one
instance. if a use case for footnotes appears in the future (e.g. if we
wanted to render the nixpkgs manual with nixos-render-docs as well) this
decision should be reevaluated.
nixos-render-docs supports inline anchors, but not ids for blocks. it
seems wise to reserve blocks for special cases that don't have other
syntax already, like admonitions.
pandoc drops .title classes when rendering to docbook, so these are
effectively just paragraphs anyway. without support for including them
in a table of contents the complexity of parsing them in
nixos-render-docs won't be warranted.
the examples for mkPackageOption weren't terminated, leading to pretty
odd nesting of docbook (and thus html) elements. close them properly.
also turn the (likewise unclosed) fenced div containing just an anchor
id and a class that will be silently dropped to an inline anchor while
we're here. we'd have to convert it anyway later.
markdown-it parses deflists slitghtly differently than pandoc does. in
these two cases pandoc would find a deflist item while markdown-it would
not, instead it'd find a lone colon and the rest of the text.
pandoc renders these to multiple docbook paragraphs in a single
definition for the term, not multiple *actual* definitions for the same
term. this is most likely not what is intended here, so let's use
multiple paragraphs instead.
this should be placed before the appendices, not between them. might
even have a good place in the development part, but that's a decision
for another day.
mdoc is just too slow to render on groff, and semantic markup doesn't
help us any for generated pages.
this produces a lot of changes to configuration.nix.5, but only few
rendering changes. most of those seem to be place losing a space where
docbook emitted roff code that did not faithfully represent the input
text, though a few places also gained space where docbook dropped them.
notably we also don't need the compatibility code docbook-xsl emitted
because that problem was fixed over a decade ago.
this will handle block quotes, which the docbook stylesheets turned into
a mess of roff requests that ended up showing up in the output instead
of being processed.
