Commit Graph

66 Commits

Author SHA1 Message Date
K900
423098c284 nixos/lib/test-driver: drop LegacyStartCommand, clean up create_machine API
We can finally do this now that it's no longer used.
2024-02-27 23:15:59 +03:00
K900
d9a37adc04 lib/test-driver: don't swallow qemu stderr 2024-02-27 23:15:58 +03:00
K900
07e7fcc46f nixos/test-driver: black 2024-01-26 16:27:53 +03:00
Jacek Galowicz
d6318f4a86
Merge pull request #262772 from RaitoBezarius/qemu-vm/wait-for-event
nixos/lib/test-driver: add `wait_for_qmp_event`
2024-01-26 08:27:06 +01:00
Raito Bezarius
1196ae6e6b nixos/lib/test-driver: add wait_for_qmp_event
Adds a function to wait for a new QMP event with a model filter
so that you can expect specific type of events with specific payloads.

e.g. a guest-reset-induced shutdown event.
2024-01-25 20:50:20 +01:00
Someone Serge
392fbc90a8
nixos/lib/test-driver: make wait_for_unit ask for ActiveState only
This is a hotfix for https://github.com/NixOS/nixpkgs/issues/266690,
where `systemctl show $unit` sometimes randomly leaves ActiveState out
2023-11-16 18:25:42 +00:00
Will Fancher
9a0f523372 systemd-stage-1: Enable backdoor in nixos tests 2023-11-09 17:53:48 -05:00
Jacek Galowicz
dda77fcc55
Merge pull request #257535 from RaitoBezarius/vmstate
nixos/lib/test-driver: use QMP API to watch for VM state
2023-10-22 14:22:34 +01:00
Raito Bezarius
f94876a65c nixos/lib/test-driver: wire up QMP client
Now that we have a QMP client, we can wire it up in the test driver.

For now, it is almost completely useless because of the need of a constant "event loop", especially
for event listening.

In the next commits, we will slowly enable more and more usecases.
2023-10-21 13:03:15 +02:00
Carl Dong
e5c2c71280 nixos/nginx: Allow empty port for listen directive
When listening on unix sockets, it doesn't make sense to specify a port
for nginx's listen directive.

Since nginx defaults to port 80 when the port isn't specified (but the
address is), we can change the default for the option to null as well
without changing any behaviour.
2023-10-09 21:16:03 -04:00
Jörg Thalheim
f1450e660f nixos/test-driver: whitelist variable names that don't follow pep8
https://docs.astral.sh/ruff/rules/invalid-argument-name/
2023-09-30 10:31:22 +02:00
Jörg Thalheim
a1f01abe53 nixos/test-driver: apply ruff fixes & suggestions 2023-09-30 10:31:22 +02:00
Raito Bezarius
81becd3c44 nixos/lib/test-driver: reduce spam at boot hangs
Since 008f9f0cd4
 ("nixos/test-driver: actually use the backdoor message to wait for backdoor"),

when boot is still computering, we can get a tons of empty strings in response to the shell.

This is not really useful to print and waste the disk space for any CI system that logs them.

We stop logging chunks whenever they are empty.
2023-09-29 03:45:31 +02:00
Maciej Krüger
03762aa42a
test-driver: revert stderr nullpipe
Removes 2>/dev/null which re-adds stderr output breaking execute
2023-09-22 15:14:13 +02:00
Artturi
7f45760504
Merge pull request #192593 from jlesquembre/test-driver 2023-09-21 00:47:45 +03:00
José Luis Lafuente
c25c10e919 nixos/tests: make wait_for timeouts configurable
While working on #192270, I noticed that only some wait_for_* helper
functions make the timeout configurable. I think we should be able to
customize it in all cases
2023-09-20 21:56:46 +03:00
Maciej Krüger
4729358fa5 nixos/test-driver: do not break if the command writes to stderr
Capturing `stderr` as part of the return `output` could break existing tests.
2023-09-18 17:36:16 +00:00
Bobby Rong
a0c846a05a
nixos/test-driver: format
There is a `black --check --diff ${src}/test_driver` check in
nixos-test-driver's checkPhase so this format change will be needed.
2023-07-30 21:24:49 +08:00
David Arnold
84d9a1e808
nixos/test-driver: log what to do if backdoor service doesn't come oneline 2023-07-29 15:01:58 -05:00
Jacek Galowicz
f59913bad8
Merge pull request #241949 from R-VdP/nixos_test_busybox
nixos/test-driver: use the short form argument to base64 for busybox compatibility.
2023-07-24 18:05:06 +02:00
Jacek Galowicz
846ad444c7 integration test driver: Auto-generate integration test driver's machine
method documentation of nixos docs from python doc strings
2023-07-10 08:51:52 +02:00
Jacek Galowicz
c916884f86 integration test driver: Synchronize integration test driver's docstrings in Machine class methods with nixos documentation 2023-07-09 13:57:07 +02:00
K900
008f9f0cd4 nixos/test-driver: actually use the backdoor message to wait for backdoor
New EDK2 sets up the backdoor port as a serial console, which feeds the test driver
a bunch of boot logs it can safely ignore. Do so by waiting for the message the
backdoor shell prints before doing anything else.
2023-07-08 20:21:50 +03:00
r-vdp
50c922e395
nixos/test-driver: use the short form argument to base64 for busybox compatibility.
Fixes #241938
2023-07-06 17:21:10 +02:00
Raito Bezarius
7088e386ff nixosTests.kexec: reconnect properly after 2nd kexec
By some miracle, before, it was possible to reconnect to the `node1` without
doing any relevant dance.

But now we are direct booting (¿), it seems like we need to do the right things.

This introduces a `check_output` flag for `execute` because we do not want to steal the
messages from the backdoor service as we might execute the kexec too fast compared
to when we will reconnect.

Therefore, we will let the message in the pipe if needed.
2023-06-16 19:43:40 +02:00
Jörg Thalheim
4f992e8da0
Merge pull request #237840 from lilyinstarlight/fix/systemd-initrd-vconsole-test
nixos/tests/systemd-initrd-vconsole: fix test and improve reliability
2023-06-15 19:41:08 +01:00
Lily Foster
f1f0d4fbdd
nixos/test-driver: fix timeout option for wait_for_console_text 2023-06-15 06:47:49 -04:00
Thomas Baggaley
491efa0b3e test-driver: respect timeout in wait_until_fails
- `wait_until_fails` was not passing through its `timeout` argument to
  the internal `retry` function, hence was always using 900 seconds (the
  default timeout for `retry`) rather than the user-specified value.
2023-06-13 05:11:33 +01:00
Alyssa Ross
507ff39251 nixos/test-driver: fix formatting
This caused the test driver to fail to build.

Fixes: 406de94b41 ("nixos/test-driver: add `timeout` option for `wait_for_console_text`")
2023-05-28 12:29:26 +00:00
Jacek Galowicz
b4b45ee6d2
Revert "nixos/lib/test-driver: enable EFI variable reads at runtime" 2023-05-28 13:05:59 +02:00
Jacek Galowicz
d9626034f1
Merge pull request #234513 from NixOS/test-driver/wait_for_console_timeout
nixos/test-driver: add `timeout` option for `wait_for_console_text`
2023-05-28 09:57:00 +02:00
Jacek Galowicz
be62469b22
Merge pull request #229027 from NixOS/qemu-vm/read-efi-var
nixos/lib/test-driver: enable EFI variable reads at runtime
2023-05-28 09:51:29 +02:00
Raito Bezarius
d1104e2109 nixos/test-driver: add timeout option for wait_for_console_text (variant 2) 2023-05-28 00:07:43 +02:00
Raito Bezarius
406de94b41 nixos/test-driver: add timeout option for wait_for_console_text
Previously, `wait_for_console_text` would block indefinitely until there were lines
shown in the buffer.

This is highly annoying when testing for things that can just hang for some reasons.

This introduces a classical timeout mechanism via non-blocking get on the Queue.
2023-05-27 23:55:52 +02:00
Raito Bezarius
f1aee66f92 nixos/lib/test-driver: enable EFI variable reads at runtime
This is useful whenever you want to diagnose the current state of UEFI
variables, to assert that bootloaders or boot programs (systemd-stub)
did their job correctly and set their variables accordingly.

In the future, it can enable inspecting SecureBoot keys also.
2023-05-27 22:44:58 +02:00
Alyssa Ross
e33c2a5e4c nixos/test-driver: add missing spaces to warning 2023-05-27 15:00:00 +00:00
figsoda
202699c918 nixos/tests: fix typos 2023-05-19 22:31:04 -04:00
r-vdp
4147b878bc
nixos-test-driver: include a timeout for the recv call, do not assume sh == bash 2023-04-26 01:05:53 +02:00
Alyssa Ross
27f97953e0
nixos/test-driver: accept non-\w* filenames
What the code was trying to do was helpfully add a directory and
extension if none were specified, but it did this by checking whether
the filename was composed of a very limited character set that didn't
even include dashes.

With this change, the intention of the code is clearer, and I can put
dashes in my screenshot names.
2023-03-22 15:54:06 +00:00
Astro
64a4ae6f37 nixos/test-driver: remove allow_reboot state from Machine, make it a start() kwarg 2023-03-16 01:50:15 +01:00
Astro
9fc85a85d5 nixos/test-driver: implement Machine.reboot() 2023-03-16 01:41:32 +01:00
Astro
95d121e798 nixos/test-driver: fix allow_reboot 2023-03-15 12:48:01 +01:00
rnhmjoj
067d688b16
nixos/test-driver: handle decoding errors in Machine.execute
The output of a command is not guaranteed to be valid UTF-8, so the
decoding can fail raising UnicodeDecodeError. If this happens during a
`succeeds` the check will be erroneously marked failed.

This changes the error handling to the "replace" mode, where invalid
codepoints are replaced with � (REPLACEMENT CHARACTER U+FFFD) and the
decoding can go on.
2023-02-07 08:47:14 +01:00
rnhmjoj
f2929eb949
nixos/test-driver: drop logging from Machine.send_monitor_command
Several machine operations, like `send_chars` and `send_key`, are
implemented by calling `send_monitor_command`, possibly multiple times.
This generates a huge amount of unnecessary noise in the log, because
`send_monitor_command` is a low-level operation and an implementation
detail.

Here's an excerpt from a highlighted log before and afte the change.

Before:

    subtest: Can generate a PGP key
    machine: sending keys 'gpg --gen-key\n'
    machine: sending monitor command: sendkey g
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey g, in 0.00 seconds)
    machine: sending monitor command: sendkey p
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey p, in 0.00 seconds)
    machine: sending monitor command: sendkey g
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey g, in 0.00 seconds)
    machine: sending monitor command: sendkey spc
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey spc, in 0.00 seconds)
    machine: sending monitor command: sendkey 0x0C
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
    machine: sending monitor command: sendkey 0x0C
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
    machine: sending monitor command: sendkey g
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey g, in 0.00 seconds)
    machine: sending monitor command: sendkey e
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey e, in 0.00 seconds)
    machine: sending monitor command: sendkey n
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey n, in 0.00 seconds)
    machine: sending monitor command: sendkey 0x0C
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
    machine: sending monitor command: sendkey k
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey k, in 0.00 seconds)
    machine: sending monitor command: sendkey e
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey e, in 0.00 seconds)
    machine: sending monitor command: sendkey y
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey y, in 0.00 seconds)
    machine: sending monitor command: sendkey ret
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey ret, in 0.00 seconds)
    (finished: sending keys 'gpg --gen-key\n', in 0.15 seconds)

After:

    subtest: Can generate a PGP key
    machine: sending keys 'gpg --gen-key\n'
    (finished: sending keys 'gpg --gen-key\n', in 0.15 seconds)
2023-02-07 08:40:54 +01:00
Jörg Thalheim
29db54c373 nixos/tests: extend shell_interact to accept alternative socat addresses
`shell_interact()` is currently not nice to use.  If you try to cancel
the socat process, it will also break the nixos test. Furthermore
ptpython creates it's own terminal that subprocesses are running in,
which breaks some of the terminal features of socat.
Hence this commit extends `shell_interact` to allow also to connect to
arbitrary servers i.e. tcp servers started by socat.
2023-01-27 13:09:25 +01:00
Jacek Galowicz
869545857f
Merge pull request #208354 from bjornfor/nixos-test-driver-quote-shell-args
nixos/test-driver: quote some shell command lines
2022-12-30 22:41:04 +01:00
Bjørn Forsman
c876f0c146 nixos/test-driver: quote some shell command lines
This makes them robust against paths with whitespace.
2022-12-30 13:19:59 +01:00
Bjørn Forsman
4c45c3f8f2 nixos/test-driver: use ASCII single quotes everywhere
A few places used Unicode U+2018/U+2019 left/right single quotes (but
not always correctly balanced). Let's just use plain ASCII single quotes
everywhere.
2022-12-30 13:00:19 +01:00
Bjørn Forsman
ce5644f658 nixos/test-driver: use f-strings instead of .format()
For readability.

Suggested-by: @tfc
2022-12-29 23:51:35 +01:00
Bjørn Forsman
dba49a43a0 nixos/test-driver: add optional address arg to wait_for_{open,closed}_port
This is useful for testing servers configured to listen on specific
addresses.
2022-12-29 13:50:17 +01:00