nixpkgs/pkgs/os-specific/linux/systemd/0019-core-handle-lookup-paths-being-symlinks.patch

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

81 lines
3.7 KiB
Diff
Raw Normal View History

2021-12-12 05:56:45 +03:00
From 169fc6f270ff3e3903a7a31550c964152f9751ec Mon Sep 17 00:00:00 2001
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 16:10:54 +03:00
From: Andreas Rammhold <andreas@rammhold.de>
Date: Wed, 18 Aug 2021 19:10:08 +0200
2021-12-06 17:39:16 +03:00
Subject: [PATCH 19/19] core: handle lookup paths being symlinks
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 16:10:54 +03:00
With a recent change paths leaving the statically known lookup paths
would be treated differently then those that remained within those. That
was done (AFAIK) to consistently handle alias names. Unfortunately that
means that on some distributions, especially those where /etc/ consists
mostly of symlinks, would trigger that new detection for every single
unit in /etc/systemd/system. The reason for that is that the units
directory itself is already a symlink.
---
src/basic/unit-file.c | 33 +++++++++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
diff --git a/src/basic/unit-file.c b/src/basic/unit-file.c
2021-12-12 05:56:45 +03:00
index 30c632dfce..6179100126 100644
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 16:10:54 +03:00
--- a/src/basic/unit-file.c
+++ b/src/basic/unit-file.c
2021-12-12 05:56:45 +03:00
@@ -255,6 +255,7 @@ int unit_file_build_name_map(
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 16:10:54 +03:00
_cleanup_hashmap_free_ Hashmap *ids = NULL, *names = NULL;
_cleanup_set_free_free_ Set *paths = NULL;
+ _cleanup_strv_free_ char **expanded_search_paths = NULL;
uint64_t timestamp_hash;
char **dir;
int r;
2021-12-12 05:56:45 +03:00
@@ -274,6 +275,34 @@ int unit_file_build_name_map(
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 16:10:54 +03:00
return log_oom();
}
+ /* Go over all our search paths, chase their symlinks and store the
+ * result in the expanded_search_paths list.
+ *
+ * This is important for cases where any of the unit directories itself
+ * are symlinks into other directories and would therefore cause all of
+ * the unit files to be recognized as linked units.
+ *
+ * This is important for distributions such as NixOS where most paths
+ * in /etc/ are symlinks to some other location on the filesystem (e.g.
+ * into /nix/store/).
+ */
+ STRV_FOREACH(dir, (char**) lp->search_path) {
+ _cleanup_free_ char *resolved_dir = NULL;
+ r = strv_extend(&expanded_search_paths, *dir);
+ if (r < 0)
+ return log_oom();
+
+ r = chase_symlinks(*dir, NULL, 0, &resolved_dir, NULL);
+ if (r < 0) {
+ if (r != -ENOENT)
+ log_warning_errno(r, "Failed to resolve symlink %s, ignoring: %m", *dir);
+ continue;
+ }
+
+ if (strv_consume(&expanded_search_paths, TAKE_PTR(resolved_dir)) < 0)
+ return log_oom();
+ }
+
STRV_FOREACH(dir, (char**) lp->search_path) {
_cleanup_closedir_ DIR *d = NULL;
2021-12-12 05:56:45 +03:00
@@ -386,11 +415,11 @@ int unit_file_build_name_map(
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 16:10:54 +03:00
continue;
}
- /* Check if the symlink goes outside of our search path.
+ /* Check if the symlink goes outside of our (expanded) search path.
* If yes, it's a linked unit file or mask, and we don't care about the target name.
* Let's just store the link source directly.
* If not, let's verify that it's a good symlink. */
- char *tail = path_startswith_strv(simplified, lp->search_path);
+ char *tail = path_startswith_strv(simplified, expanded_search_paths);
if (!tail) {
log_debug("%s: linked unit file: %s → %s",
__func__, filename, simplified);
--
2021-12-12 05:56:45 +03:00
2.34.0
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 16:10:54 +03:00