In CI / or local testing when you run `serenity.sh tests`, the system
will boot itself in self test mode, and the test runner will be launched
by SystemServer. Previously we were setting up settings for that
environment in the `SystemServer.ini`. This makes reproducing CI
failures a bit confusing, as the system will behavior differently if you
run in self-test mode, vs running `run-tests-and-shutdown.sh` or
`run-tests` manually in a session.
This change moves the settings to `run-tests`, so no matter how you try
to run the test runner, it will always behave the same.
It can sometimes be difficult to tell from the debug.log and test stdout
which test was the last to run before the test runner hangs or exits the
QEMU instance unexpectedly.
Print out a start message before each test is executed, along with a
progress message indicating which test out of how many tests we're about
to run.
If a test run has a lot of tests in it, and they fill up the terminal
buffer, it can be difficult to find out exactly which tests have failed
from your large test run. Make TestRunner print out an optional Vector
of failed test names at the end of the run, and have run-tests add each
failed or crashed test to a Vector it uses for this purpose.
We have a new config argument to add space separated exclude regex'
This is separate from "NotTestsPattern", because these are still Tests,
although they are not supposed to be run by the runner
This also adds the test for a working UserspaceEmulator to the tests run
`using enum` statements will only be supported by the upcoming Clang 13
compiler, so the current code can't be built with the almost-ready Clang
toolchain yet.
TestProcFs expects to be able to stat its stdout and stderr. The new
ProcFS implemetnation properly forwards the symlinks for
/proc/pid/fd/[1,2] to the temporary file that we had unlinked prior to
spawning the process. However, this makes it so that a normal stat on
the symlink to that file fails (as expected). Move the unlink to after
we've waited on the child, so that we know it won't be trying any funny
business with its stdout/stderr anymore.
This test program heavily pulls from the JavaScriptTestRunner/test-js,
but with a twist. Instead of loading JavaScript files into the current
process, constructing a JS environment for them, and executing test
suites/tests directly, run-tests posix_spawns each test file.
Test file stdout is written to a temp file, and only dumped to console
if the test fails or the verbose option is passed to the program. Unlike
test-js, times are always printed for every test executed for better
visibility in CI.
