Use the new class in HID code, because all other HID device controllers
will be using this class as their parent class.
Hence, we no longer keep a reference to any PS/2 device in HIDManagement
and rely on HIDController derived classes to do this for us.
It also means that we removed another instance of a LockRefPtr, which
is designated to be removed and is replaced by the better pattern of
SpinlockProtected<RefPtr<>> instead.
The definitions were being defined already by `BootInfo.h` and that was
being included here via transitive includes. The extern definitions of
the variables do not have the `READONLY_AFTER_INIT` attribute in
`BootInfo.h`. This causes conflicting definitions of the same variable.
The `READONLY_AFTER_INIT` specifier is not needed for extern variables
as it only effects their linkage, not their actual use, so just use the
versions in `BootInfo.h` instead of re-declaring.
Specifically this commit implements two setters set_userspace_sp and
set_ip in RegisterState.h, and also adds a stack pointer getter (sp) in
ThreadRegisters.h. Contributed by konrad, thanks for that.
Setting the page table base register (ttbr0_el1) is not enough, and will
not flush the TLB caches, in contrary with x86_64 where setting the CR3
register will actually flush the caches. This commit adds the necessary
code to properly flush the TLB caches when context switching. This
commit also changes Processor::flush_tlb_local to use the vmalle1
variant, as previously we would be flushing the tlb's of all the cores
in the inner-shareable domain.
This replaces the previous owning address space pointer. This commit
should not change any of the existing functionality, but it lays down
the groundwork needed to let us properly access the region table under
the address space spinlock during page fault handling.
- Instead of taking the first new thread as an out-parameter, we now
bundle the process and its first thread in a struct and use that
as the return value.
- Make all Process factory functions return ErrorOr. Use this to convert
some places to more TRY().
- Drop the "try_" prefix on Process factory functions.
This matches x86_64's behaviour in common_trap_exit. (called from
thread_context_first_enter)
Currently thread_context_first_enter is only called when creating new
processes from scratch, in which case this doesn't change the actual
behaviour. But once thread_context_first_enter is called as part of
execve support, this will ensure the Thread's m_current_trap is set
correctly to the new trap frame.
The details of the specific interrupt bits that must be turned on are
irrelevant to the sys$execve implementation. Abstract it away to the
Processor implementations using the InterruptsState enum.
Forked processes already have an existing value for the link register,
which we can't overwrite. But since they're forked the original link
register value that points to exit_kernel_thread was already saved
somewhere on the stack, so it's ok not to set it.
Storage controllers are initialized during init and are never modified.
NonnullRefPtr can be safely used instead of the NonnullLockRefPtr. This
also fixes one of the UB issue that was there when using an NVMe device
because of NonnullLockRefPtr.
We can add proper locking when we need to modify the storage controllers
after init.
This is done with 2 major steps:
1. Remove JailManagement singleton and use a structure that resembles
what we have with the Process object. This is required later for the
second step in this commit, but on its own, is a major change that
removes this clunky singleton that had no real usage by itself.
2. Use IntrusiveLists to keep references to Process objects in the same
Jail so it will be much more straightforward to iterate on this kind
of objects when needed. Previously we locked the entire Process list
and we did a simple pointer comparison to check if the checked
Process we iterate on is in the same Jail or not, which required
taking multiple Spinlocks in a very clumsy and heavyweight way.
Since the ProcFS doesn't hold many global objects within it, the need
for a fully-structured design of backing components and a registry like
with the SysFS is no longer true.
To acommodate this, let's remove all backing store and components of the
ProcFS, so now it resembles what we had in the early days of ProcFS in
the project - a mostly-static filesystem, with very small amount of
kmalloc allocations needed.
We still use the inode index mechanism to understand the role of each
inode, but this is done in a much "static"ier way than before.
Even though we currently build all of Userland and the Kernel with the
-mstrict-align flag, the compiler will still emit unaligned memory
accesses. To work around this, we disable the check for now. See
https://github.com/SerenityOS/serenity/issues/17516 for the relevant
issue.
This commit adds Processor::set_thread_specific_data, and this function
is used to factor out architecture specific implementation of setting
the thread specific data. This function is implemented for
aarch64 and x86_64, and the callsites are changed to use this function
instead.
This adds the necessary code to init.cpp to be able to execute the first
userspace process. To do this, first the filesystem code is initialized,
which will use the ramdisk embedded into the kernel image. Then the
first userspace process, /bin/SystemServer is executed. :^)
The ramdisk code is used as it is useful for the bring-up of the aarch64
port, however once the kernel has support for better ram-based
filesystems, the ramdisk code will be removed again.
This sets up the correct ThreadRegisters state when a process is
exec'ed, which happens when the first userspace application is executed.
Also changes Processor.cpp to get the stack pointer from the
ThreadRegisters.
This allows the function to be called from other translation units, in
particular this allows the CrashHandler.cpp file to be shared between
aarch64 and x86_64.
Setting the kernel_load_base variable caused backtracking to regress, so
to have proper backtracing the calculation of the symbol address in
KSyms.cpp needs to keep into account that the aarch64 kernel is linked
at a high virtual memory address.
When we execute in userspace, the exception level is EL0, so to handle
exceptions, such as interrupts, and syscalls, we need to add handlers to
vector_table.S. For now we only support running userspace applications
in AArch64 mode, so this commit only adds the handlers for that mode.
To detect instruction aborts, a helper to Registers.h is added, and used
in Interrupts.cpp. Additionally, the PageFault class gets a setter to
set the PageFaults m_is_instruction_fetch bool, and is also used in
Interrupts.cpp.
This patch removes the x86 mechanism for calling syscalls, favoring
the more modern syscall instruction. It also moves architecture
dependent code from functions that are meant to be architecture
agnostic therefore paving the way for adding more architectures.
This will cause page faults to be generated. Since the previous commits
introduced the handling of page faults, we can now actually correctly
handle page faults.
The code in PageDirectory.cpp now keeps track of the registered page
directories, and actually sets the TTBR0_EL1 to the page table base of
the currently executing thread. When context switching, we now also
change the TTBR0_EL1 to the page table base of the thread that we
context switch into.
The handling of page tables is very architecture specific, so belongs
in the Arch directory. Some parts were already architecture-specific,
however this commit moves the rest of the PageDirectory class into the
Arch directory.
While we're here the aarch64/PageDirectory.{h,cpp} files are updated to
be aarch64 specific, by renaming some members and removing x86_64
specific code.
The class used to look at the x86_64 specific exception code to figure
out what kind of page fault happend, however this refactor allows
aarch64 to use the same class.
Various places in the kernel were manually checking the cs register for
x86_64, however to share this with aarch64 a function in RegisterState
is added, and the call-sites are updated. While we're here the
PreviousMode enum is renamed to ExecutionMode.
Until now the kernel was always executing with SP_EL0, as this made the
initial dropping to EL1 a bit easier. This commit changes this behaviour
to use the corresponding SP_ELx for each exception level.
To make sure that the execution of the C++ code can continue, the
current stack pointer is copied into the corresponding SP_ELx just
before dropping an exception level.
There are now 2 separate classes for almost the same object type:
- EnumerableDeviceIdentifier, which is used in the enumeration code for
all PCI host controller classes. This is allowed to be moved and
copied, as it doesn't support ref-counting.
- DeviceIdentifier, which inherits from EnumerableDeviceIdentifier. This
class uses ref-counting, and is not allowed to be copied. It has a
spinlock member in its structure to allow safely executing complicated
IO sequences on a PCI device and its space configuration.
There's a static method that allows a quick conversion from
EnumerableDeviceIdentifier to DeviceIdentifier while creating a
NonnullRefPtr out of it.
The reason for doing this is for the sake of integrity and reliablity of
the system in 2 places:
- Ensure that "complicated" tasks that rely on manipulating PCI device
registers are done in a safe manner. For example, determining a PCI
BAR space size requires multiple read and writes to the same register,
and if another CPU tries to do something else with our selected
register, then the result will be a catastrophe.
- Allow the PCI API to have a united form around a shared object which
actually holds much more data than the PCI::Address structure. This is
fundamental if we want to do certain types of optimizations, and be
able to support more features of the PCI bus in the foreseeable
future.
This patch already has several implications:
- All PCI::Device(s) hold a reference to a DeviceIdentifier structure
being given originally from the PCI::Access singleton. This means that
all instances of DeviceIdentifier structures are located in one place,
and all references are pointing to that location. This ensures that
locking the operation spinlock will take effect in all the appropriate
places.
- We no longer support adding PCI host controllers and then immediately
allow for enumerating it with a lambda function. It was found that
this method is extremely broken and too much complicated to work
reliably with the new paradigm being introduced in this patch. This
means that for Volume Management Devices (Intel VMD devices), we
simply first enumerate the PCI bus for such devices in the storage
code, and if we find a device, we attach it in the PCI::Access method
which will scan for devices behind that bridge and will add new
DeviceIdentifier(s) objects to its internal Vector. Afterwards, we
just continue as usual with scanning for actual storage controllers,
so we will find a corresponding NVMe controllers if there were any
behind that VMD bridge.
There’s similar RDRAND register (encoded as ‘s3_3_c2_c4_0ʼ) to be
added if needed. RNG CPU feature on Aarch64 guarantees existence of both
RDSEED and RDRAND registers simultaneously—in contrast to x86-64, where
respective instructions are independent of each other.
This is the same address that the x86_64 kernel runs at, and allows us
to run the kernel at a high virtual memory address. Since we now run
completely in high virtual memory, we can also unmap the identity
mapping. Additionally some changes in MMU.cpp are required to
successfully boot.
Since we link the kernel at a high virtual memory address, the addresses
of global variables are also at virtual addresses. To be able to access
them without the MMU enabled, we have to subtract the
KERNEL_MAPPING_BASE.
This is a separate file that behaves similar to the Prekernel for
x86_64, and makes sure the CPU is dropped to EL1, the MMU is enabled,
and makes sure the CPU is running in high virtual memory. This code then
jumps to the usual init function of the kernel.
As the kernel is now linked at high address in virtual memory, we cannot
use absolute addresses as they refer to high addresses in virtual
memory. At this point in the boot process we are still running with the
MMU off, so we have to make sure the accesses are using physical memory
addresses.
This function will be used once the kernel runs in high virtual memory
to unmap the identity mapping as userspace will later on use this memory
range instead.
And use it the code that will be part of the early boot process.
The PANIC macro and dbgln functions cannot be used as it accesses global
variables, which in the early boot process do not work, since the MMU is
not yet enabled.
In the upcoming commits, we'll change the kernel to run at a virtual
address in high memory. This commit prepares for that by making sure the
kernel and mmio are mapped into high virtual memory.
A lot of interrupt numbers are initialized with the unhandled interrupt
handler. Whenever a new handler is registered on one of these
interrupts, the old handler is unregistered first. Let's not be verbose
about this since it is perfectly normal.
Following registers accessors are updated and put in use:
* ID_AA64ISAR0_EL1, Instruction Set Attribute Register 0
Accessors for following registers are added and put in use:
* ID_AA64ISAR1_EL1, Instruction Set Attribute Register 1
* ID_AA64ISAR2_EL1, Instruction Set Attribute Register 2
* ID_AA64MMFR1_EL1, AArch64 Memory Model Feature Register 1
* ID_AA64MMFR2_EL1, AArch64 Memory Model Feature Register 2
* ID_AA64MMFR3_EL1, AArch64 Memory Model Feature Register 3
* ID_AA64MMFR4_EL1, AArch64 Memory Model Feature Register 4
* ID_AA64PFR0_EL1, AArch64 Processor Feature Register 0
* ID_AA64PFR1_EL1, AArch64 Processor Feature Register 1
* ID_AA64PFR2_EL1, AArch64 Processor Feature Register 2
* ID_AA64ZFR0_EL1, AArch64 SVE Feature ID register 0
* ID_AA64SMFR0_EL1, AArch64 SME Feature ID register 0
* ID_AA64DFR0_EL1, AArch64 Debug Feature Register 0
* ID_AA64DFR1_EL1, AArch64 Debug Feature Register 1
Additionally, there are few CPU features detected with
* TCR_EL1, Translation Control Register
but detection mechanism using it (for LPA/LPA2) is probably wrong as
this is control register, not a id register, and needs further work.
Finally, following registers are provided. Former one is already used,
while latter is given for future use:
* MIDR_EL1, Main ID Register
* AIDR_EL1, Auxiliary ID Register
Settled for `cpu_feature_to_name` as that naming is more descriptive
and similarly named `cpu_feature_to_description` function will be
provided for Aarch64.
This seems to work perfectly OK on my ICH7 test machine and also it
works on QEMU, so it is probably OK to restore this.
This will ensure we always get scan code set 1 input, because we enable
scan code set 2 and PS/2 translation on the first (keyboard) port.
The setting of scan code set sequence is removed, as it's buggy and
could lead the controller to fail immediately when doing self-test
afterwards. We will restore it when we understand how to do so safely.
Allow the user to determine a preferred detection path with a new kernel
command line argument. The defualt option is to check i8042 presence
with an ACPI check and if necessary - an "aggressive" test to determine
i8042 existence in the system.
Also, keep the i8042 controller pointer on the stack, so don't assign
m_i8042_controller member pointer if it does not exist.
A virtual method named device_name() was added to
Kernel::PCI to support logging the PCI::Device name
and address using dmesgln_pci. Previously, PCI::Device
did not store the device name.
All devices inheriting from PCI::Device now use dmesgln_pci where
they previously used dmesgln.
These instances were detected by searching for files that include
AK/Memory.h, but don't match the regex:
\\b(fast_u32_copy|fast_u32_fill|secure_zero|timing_safe_compare)\\b
This regex is pessimistic, so there might be more files that don't
actually use any memory function.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
AK/Concepts.h, but don't match the regex:
\\b(AnyString|Arithmetic|ArrayLike|DerivedFrom|Enum|FallibleFunction|Flo
atingPoint|Fundamental|HashCompatible|Indexable|Integral|IterableContain
er|IteratorFunction|IteratorPairWith|OneOf|OneOfIgnoringCV|SameAs|Signed
|SpecializationOf|Unsigned|VoidFunction)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use any concepts.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
AK/StdLibExtras.h, but don't match the regex:
\\b(abs|AK_REPLACED_STD_NAMESPACE|array_size|ceil_div|clamp|exchange|for
ward|is_constant_evaluated|is_power_of_two|max|min|mix|move|_RawPtr|RawP
tr|round_up_to_power_of_two|swap|to_underlying)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use any "extra stdlib" functions.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
These instances were detected by searching for files that include
AK/Format.h, but don't match the regex:
\\b(CheckedFormatString|critical_dmesgln|dbgln|dbgln_if|dmesgln|FormatBu
ilder|__FormatIfSupported|FormatIfSupported|FormatParser|FormatString|Fo
rmattable|Formatter|__format_value|HasFormatter|max_format_arguments|out
|outln|set_debug_enabled|StandardFormatter|TypeErasedFormatParams|TypeEr
asedParameter|VariadicFormatParams|v_critical_dmesgln|vdbgln|vdmesgln|vf
ormat|vout|warn|warnln|warnln_if)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use any formatting functions.
Observe that this revealed that Userland/Libraries/LibC/signal.cpp is
missing an include.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
This step would ideally not have been necessary (increases amount of
refactoring and templates necessary, which in turn increases build
times), but it gives us a couple of nice properties:
- SpinlockProtected inside Singleton (a very common combination) can now
obtain any lock rank just via the template parameter. It was not
previously possible to do this with SingletonInstanceCreator magic.
- SpinlockProtected's lock rank is now mandatory; this is the majority
of cases and allows us to see where we're still missing proper ranks.
- The type already informs us what lock rank a lock has, which aids code
readability and (possibly, if gdb cooperates) lock mismatch debugging.
- The rank of a lock can no longer be dynamic, which is not something we
wanted in the first place (or made use of). Locks randomly changing
their rank sounds like a disaster waiting to happen.
- In some places, we might be able to statically check that locks are
taken in the right order (with the right lock rank checking
implementation) as rank information is fully statically known.
This refactoring even more exposes the fact that Mutex has no lock rank
capabilites, which is not fixed here.
These instances were detected by searching for files that include
Array.h, but don't match the regex:
\\b(Array(?!\.h>)|iota_array|integer_sequence_generate_array)\\b
These are the three symbols defined by Array.h.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
If a page fault occurs while interrupts are disabled, we were wrongly
enabling interrupts right away in the page fault handler.
Instead, we should only do this if interrupts were enabled when the
page fault occurred.
The hand-written assembly does not compile under Clang due to register
size mismatches. Using a loop is slower (~6 instructions on O2 as
opposed to 2 with hand-written assembly), but using the pause
instruction makes this more efficient even under TCG.
For pause we use isb sy which will put the processor to sleep while the
pipeline is being flushed. This instruction is also used by Rust in spin
loops and found to be more efficient, as well as being a rough
equivalent to the x86 pause instruction which we also use here.
For wait_check we use yield, which is a hinted nop that is faster to
execute, and I leave a FIXME for processing SMP messages once we support
SMP.
These two changes probably make spin loops work on aarch64 :^)
This commit changes the init.cpp file to start and initialize the
Scheduler, and actually runs init_stage2. To show that it actually
works, another thread is spawned and executed simultaneously, by context
switching between the two!
This requires two new functions, context_first_init and
restore_context_and_eret. With this code in place, we are now running
the first idle thread! :^)
This changes the stack pointer to the initial_thread stack pointer, and
pushes two pointers onto the stack that point to the initial_thread. The
function then jumps to the ip of the initial_thread, which will be
thread_context_first_enter, and hangs there because that function is not
yet implemented.
This does not handle everything correctly yet, such as setting the
correct state for running userspace applications, however this should be
enough to get kernel scheduling to work.
These are architecture-specific anyway, so they belong in the Arch
directory. This commit also adds ThreadRegisters::set_initial_state to
factor out the logic in Thread.cpp.
I can't think of a reason why copying the Processor class makes sense,
so lets make sure it's not possible to do it by accident by declaring
the copy constructor as deleted.
This removes the x86 specific hlt instruction from the scheduler, and
allows us to run the scheduler code for aarch64 by implementing
Processor::wait_for_interrupt for aarch64.
This file does not contain any architecture specific implementations,
so we can move it to the Kernel base directory. Also update the relevant
include paths.
We are actually storing tpidr_el0, as can be seen in vector_table.S, but
the RegisterState.h incorrectly had tpidr_el1. This will probably save
some annoying debugging later on.
