For a long time, our shutdown procedure has basically been:
- Acquire big process lock.
- Switch framebuffer to Kernel debug console.
- Sync and lock all file systems so that disk caches are flushed and
files are in a good state.
- Use firmware and architecture-specific functionality to perform
hardware shutdown.
This naive and simple shutdown procedure has multiple issues:
- No processes are terminated properly, meaning they cannot perform more
complex cleanup work. If they were in the middle of I/O, for instance,
only the data that already reached the Kernel is written to disk, and
data corruption due to unfinished writes can therefore still occur.
- No file systems are unmounted, meaning that any important unmount work
will never happen. This is important for e.g. Ext2, which has
facilites for detecting improper unmounts (see superblock's s_state
variable) and therefore requires a proper unmount to be performed.
This was also the starting point for this PR, since I wanted to
introduce basic Ext2 file system checking and unmounting.
- No hardware is properly shut down beyond what the system firmware does
on its own.
- Shutdown is performed within the write() call that asked the Kernel to
change its power state. If the shutdown procedure takes longer (i.e.
when it's done properly), this blocks the process causing the shutdown
and prevents any potentially-useful interactions between Kernel and
userland during shutdown.
In essence, current shutdown is a glorified system crash with minimal
file system cleanliness guarantees.
Therefore, this commit is the first step in improving our shutdown
procedure. The new shutdown flow is now as follows:
- From the write() call to the power state SysFS node, a new task is
started, the Power State Switch Task. Its only purpose is to change
the operating system's power state. This task takes over shutdown and
reboot duties, although reboot is not modified in this commit.
- The Power State Switch Task assumes that userland has performed all
shutdown duties it can perform on its own. In particular, it assumes
that all kinds of clean process shutdown have been done, and remaining
processes can be hard-killed without consequence. This is an important
separation of concerns: While this commit does not modify userland, in
the future SystemServer will be responsible for performing proper
shutdown of user processes, including timeouts for stubborn processes
etc.
- As mentioned above, the task hard-kills remaining user processes.
- The task hard-kills all Kernel processes except itself and the
Finalizer Task. Since Kernel processes can delay their own shutdown
indefinitely if they want to, they have plenty opportunity to perform
proper shutdown if necessary. This may become a problem with
non-cooperative Kernel tasks, but as seen two commits earlier, for now
all tasks will cooperate within a few seconds.
- The task waits for the Finalizer Task to clean up all processes.
- The task hard-kills and finalizes the Finalizer Task itself, meaning
that it now is the only remaining process in the system.
- The task syncs and locks all file systems, and then unmounts them. Due
to an unknown refcount bug we currently cannot unmount the root file
system; therefore the task is able to abort the clean unmount if
necessary.
- The task performs platform-dependent hardware shutdown as before.
This commit has multiple remaining issues (or exposed existing ones)
which will need to be addressed in the future but are out of scope for
now:
- Unmounting the root filesystem is impossible due to remaining
references to the inodes /home and /home/anon. I investigated this
very heavily and could not find whoever is holding the last two
references.
- Userland cannot perform proper cleanup, since the Kernel's power state
variable is accessed directly by tools instead of a proper userland
shutdown procedure directed by SystemServer.
The recently introduced Firmware/PowerState procedures are removed
again, since all of the architecture-independent code can live in the
power state switch task. The architecture-specific code is kept,
however.
Once we move to a more proper shutdown procedure, processes other than
the finalizer task must be able to perform cleanup and finalization
duties, not only because the finalizer task itself needs to be cleaned
up by someone. This global variable, mirroring the early boot flags,
allows a future shutdown process to perform cleanup on its own.
Note that while this *could* be considered a weakening in security, the
attack surface is minimal and the results are not dramatic. To exploit
this, an attacker would have to gain a Kernel write primitive to this
global variable (bypassing KASLR among other things) and then gain some
way of calling the relevant functions, all of this only to destroy some
other running process. The same effect can be achieved with LPE which
can often be gained with significantly simpler userspace exploits (e.g.
of setuid binaries).
Since we never check a kernel process's state like a userland process,
it's possible for a kernel process to ignore the fact that someone is
trying to kill it, and continue running. This is not desireable if we
want to properly shutdown all processes, including Kernel ones.
This is correct since unmount doesn't treat bind mounts specially. If we
don't do this, unmounting bind mounts will call
prepare_for_last_unmount() on the guest FS much too early, which will
most likely fail due to a busy file system.
Previously, we started parsing the ELF file again in a completely
different place, and without the partial mapping that we do while
validating.
Instead of doing manual parsing in two places, just capture the
requested stack size right after we validated it.
This resolves the various "implicit truncation from int to a one-bit
wide bit-field changes value from 1 to -1" warnings produced by Clang
16+ when assigning to single-bit bitfields.
The driver would crash if it was unable to find an output route, and
subsequently the destruction of controller did not invoke
`GenericInterruptHandler::will_be_destroyed()` because on the level of
`AudioController`, that method is unavailable.
By decoupling the interrupt handling from the controller, we get a new
refcounted class that correctly cleans up after itself :^)
We used to not care about stopping an audio output stream for Intel HDA
since AudioServer would continuously send new buffers to play. Since
707f5ac150ef858760eb9faa52b9ba80c50c4262 however, that has changed.
Intel HDA now uses interrupts to detect when each buffer was completed
by the device, and uses a simple heuristic to detect whether a buffer
underrun has occurred so it can stop the output stream.
This was tested on Qemu's Intel HDA (Linux x86_64) and a bare metal MSI
Starship/Matisse HD Audio Controller.
This is a preparation before we can create a usable mechanism to use
filesystem-specific mount flags.
To keep some compatibility with userland code, LibC and LibCore mount
functions are kept being usable, but now instead of doing an "atomic"
syscall, they do multiple syscalls to perform the complete procedure of
mounting a filesystem.
The FileBackedFileSystem IntrusiveList in the VFS code is now changed to
be protected by a Mutex, because when we mount a new filesystem, we need
to check if a filesystem is already created for a given source_fd so we
do a scan for that OpenFileDescription in that list. If we fail to find
an already-created filesystem we create a new one and register it in the
list if we successfully mounted it. We use a Mutex because we might need
to initiate disk access during the filesystem creation, which will take
other mutexes in other parts of the kernel, therefore making it not
possible to take a spinlock while doing this.
Otherwise, reading will sometimes fail on the Raspberry Pi.
This is mostly a hack, the spec has some info about how the correct
divisor should be calculated and how we can recover from timeouts.
Namely, we previously forgot to configure the SD Host Controller for
4-bit mode after issuing ACMD6, which caused data transfers to fail on
bare metal.
Instead of using ifdefs to use the correct platform-specific methods, we
can just use the same pattern we use for the microseconds_delay function
which has specific implementations for each Arch CPU subdirectory.
When linking a kernel image, the actual correct and platform-specific
power-state changing methods will be called in Firmware/PowerState.cpp
file.
Since https://reviews.llvm.org/D131441, libc++ must be included before
LibC. As clang includes libc++ as one of the system includes, LibC
must be included after those, and the only correct way to do that is
to install LibC's headers into the sysroot.
Targets that don't link with LibC yet require its headers for one
reason or another must add install_libc_headers as a dependency to
ensure that the correct headers have been (re)installed into the
sysroot.
LibC/stddef.h has been dropped since the built-in stddef.h receives
a higher include priority.
In addition, string.h and wchar.h must
define __CORRECT_ISO_CPP_STRING_H_PROTO and
_LIBCPP_WCHAR_H_HAS_CONST_OVERLOADS respectively in order to tell
libc++ to not try to define methods implemented by LibC.
Once LibC is installed to the sysroot and its conflicts with libc++
are resolved, including LibC headers in such a way will cause errors
with a modern LLVM-based toolchain.
This is needed to avoid including LibC headers in Lagom builds.
Unfortunately, we cannot rely on the build machine to provide a
fully POSIX-compatible ELF header for Lagom builds, so we have to
use our own.
To ensure actual PS2 code is not tied to the i8042 code, we make them
separated in the following ways:
- PS2KeyboardDevice and PS2MouseDevice classes are no longer inheriting
from the IRQHandler class. Instead we have specific IRQHandler derived
class for the i8042 controller implementation, which is used to ensure
that we don't end up mixing PS2 code with low-level interrupt handling
functionality. In the future this means that we could add a driver for
other PS2 controllers that might have only one interrupt handler but
multiple PS2 devices are attached, therefore, making it easier to put
the right propagation flow from the controller driver all the way to
the HID core code.
- A simple abstraction layer is added between the PS2 command set which
devices could use and the actual implementation low-level commands.
This means that the code in PS2MouseDevice and PS2KeyboardDevice
classes is no longer tied to i8042 implementation-specific commands,
so now these objects could send PS2 commands to their PS2 controller
and get a PS2Response which abstracts the given response too.
The HIDController class is removed and instead adding SerialIOController
class. The HIDController class was a mistake - there's no such thing in
real hardware as host controller only for human interface devices
(VirtIO PCI input controller being the exception here, but it could be
technically treated as serial IO controller too).
Instead, we simply add a new abstraction layer - the SerialIO "bus",
which will hold all the code that is related to serial communications
with other devices. A PS2 controller is simply a serial IO controller,
and the Intel 8042 Controller is simply a specific implementation of a
PS2 controller.
Ideally, we would want the audio controller to run a channel at a
device's initial sample rate instead of hardcoding 44.1 KHz. However,
most audio is provided at 44.1 KHz and as long as `Audio::Resampler`
introduces significant audio artifacts, let's set a sensible sample
rate that offers a better experience for most users.
This can be removed after someone implements a higher quality
`Audio::Resampler`.
All code that is related to PC BIOS should not be in the Kernel/Firmware
directory as this directory is for abstracted and platform-agnostic code
like ACPI (and device tree parsing in the future).
This fixes a problem with the aarch64 architecure, as these machines
don't have any PC-BIOS in them so actually trying to access these memory
locations (EBDA, BIOS ROM) does not make any sense, as they're specific
to x86 machines only.
This code is very x86-specific, because Intel introduced the actual
MultiProcessor specification back in 1993, qouted here as a proof:
"The MP specification covers PC/AT-compatible MP platform designs based
on Intel processor architectures and Advanced Programmable Interrupt
Controller (APIC) architectures"
Most of the ACPI static parsing methods (methods that can be called
without initializing a full AML parser) are not tied to any specific
platform or CPU architecture.
The only method that is platform-specific is the one that finds the RSDP
structure. Thus, each CPU architecture/platform needs to implement it.
This means that now aarch64 can implement its own method to find the
ACPI RSDP structure, which would be hooked into the rest of the ACPI
code elegantly, but for now I just added a FIXME and that method returns
empty value of Optional<PhysicalAddress>.
Previously, reads would only be successful for offset 0. For this
reason, the maximum size that could be correctly read from the PCI
expansion ROM SysFS node was limited to the block size, and
subsequent blocks would fail. This commit fixes the computation of
the number of bytes to read.
During receive_tcp_packet(), we now set m_send_window_size for the
socket if it is different from the default.
This removes one FIXME from TCPSocket.h.
These 4 fields were made `Atomic` in
c3f668a758, at which time these were still
accessed unserialized and TOCTOU bugs could happen. Later, in
8ed06ad814, we serialized access to these
fields in a number of helper methods, removing the need for `Atomic`.
Instead of having a single available memory range that encompasses the
whole 0x00000000-0x3EFFFFFF range of physical memory, create a separate
reserved entry for the RAM range used by the VideoCore. This fixes a
crash that happens when we try to allocate physical pages in the GPU's
reserved range.
This will eventually be replaced with parsing the data from the device
tree, but for now, this should solve some of the recurring CI failures.
