The purpose of init() is to get multi-tasking up and running. We don't
want to do anything in init() that doesn't advance that goal.
This patch moves some things from init() to init_stage2(), and adds a
comment block explaining the split.
In contrast to the previous patchset that was reverted, this time we use
a "special" method to access a file with block size of 512 bytes (like
a harddrive essentially).
This new subsystem includes better abstractions of how time will be
handled in the OS. We take advantage of the existing RTC timer to aid
in keeping time synchronized. This is standing in contrast to how we
handled time-keeping in the kernel, where the PIT was responsible for
that function in addition to update the scheduler about ticks.
With that new advantage, we can easily change the ticking dynamically
and still keep the time synchronized.
In the process context, we no longer use a fixed declaration of
TICKS_PER_SECOND, but we call the TimeManagement singleton class to
provide us the right value. This allows us to use dynamic ticking in
the future, a feature known as tickless kernel.
The scheduler no longer does by himself the calculation of real time
(Unix time), and just calls the TimeManagment singleton class to provide
the value.
Also, we can use 2 new boot arguments:
- the "time" boot argument accpets either the value "modern", or
"legacy". If "modern" is specified, the time management subsystem will
try to setup HPET. Otherwise, for "legacy" value, the time subsystem
will revert to use the PIT & RTC, leaving HPET disabled.
If this boot argument is not specified, the default pattern is to try
to setup HPET.
- the "hpet" boot argumet accepts either the value "periodic" or
"nonperiodic". If "periodic" is specified, the HPET will scan for
periodic timers, and will assert if none are found. If only one is
found, that timer will be assigned for the time-keeping task. If more
than one is found, both time-keeping task & scheduler-ticking task
will be assigned to periodic timers.
If this boot argument is not specified, the default pattern is to try
to scan for HPET periodic timers. This boot argument has no effect if
HPET is disabled.
In hardware context, PIT & RealTimeClock classes are merely inheriting
from the HardwareTimer class, and they allow to use the old i8254 (PIT)
and RTC devices, managing them via IO ports. By default, the RTC will be
programmed to a frequency of 1024Hz. The PIT will be programmed to a
frequency close to 1000Hz.
About HPET, depending if we need to scan for periodic timers or not,
we try to set a frequency close to 1000Hz for the time-keeping timer
and scheduler-ticking timer. Also, if possible, we try to enable the
Legacy replacement feature of the HPET. This feature if exists,
instructs the chipset to disconnect both i8254 (PIT) and RTC.
This behavior is observable on QEMU, and was verified against the source
code:
ce967e2f33
The HPETComparator class is inheriting from HardwareTimer class, and is
responsible for an individual HPET comparator, which is essentially a
timer. Therefore, it needs to call the singleton HPET class to perform
HPET-related operations.
The new abstraction of Hardware timers brings an opportunity of more new
features in the foreseeable future. For example, we can change the
callback function of each hardware timer, thus it makes it possible to
swap missions between hardware timers, or to allow to use a hardware
timer for other temporary missions (e.g. calibrating the LAPIC timer,
measuring the CPU frequency, etc).
Also, duplicate data in dbg() and klog() calls were removed.
In addition, leakage of virtual address to kernel log is prevented.
This is done by replacing kprintf() calls to dbg() calls with the
leaked data instead.
Also, other kprintf() calls were replaced with klog().
Now the ACPI & PCI code is more safer, because we don't use raw pointers
or references to objects or data that are located in the physical
address space, so an accidental dereference cannot happen easily.
Instead, we use the PhysicalAddress class to represent those addresses.
This was only used by HashTable::dump() which I used when doing the
first HashTable implementation. Removing this allows us to also remove
most includes of <AK/kstdio.h>.
We add this feature together with the VMWareBackdoor class.
VMWareBackdoor class is responsible for enabling the vmmouse, and then
controlling it from the PS2 mouse IRQ handler.
Before putting itself back on the wait queue, the finalizer task will
now check if there's more work to do, and if so, do it first. :^)
This patch also puts a bunch of process/thread debug logging behind
PROCESS_DEBUG and THREAD_DEBUG since it was unbearable to debug this
stuff with all the spam.
System components that need an IRQ handling are now inheriting the
InterruptHandler class.
In addition to that, the initialization process of PATAChannel was
changed to fit the changes.
PATAChannel, E1000NetworkAdapter and RTL8139NetworkAdapter are now
inheriting from PCI::Device instead of InterruptHandler directly.
We don't need to have this method anymore. It was a hack that was used
in many components in the system but currently we use better methods to
create virtual memory mappings. To prevent any further use of this
method it's best to just remove it completely.
Also, the APIC code is disabled for now since it doesn't help booting
the system, and is broken since it relies on identity mapping to exist
in the first 1MB. Any call to the APIC code will result in assertion
failed.
In addition to that, the name of the method which is responsible to
create an identity mapping between 1MB to 2MB was changed, to be more
precise about its purpose.
As suggested by Joshua, this commit adds the 2-clause BSD license as a
comment block to the top of every source file.
For the first pass, I've just added myself for simplicity. I encourage
everyone to add themselves as copyright holders of any file they've
added or modified in some significant way. If I've added myself in
error somewhere, feel free to replace it with the appropriate copyright
holder instead.
Going forward, all new source files should include a license header.
The kernel and its static data structures are no longer identity-mapped
in the bottom 8MB of the address space, but instead move above 3GB.
The first 8MB above 3GB are pseudo-identity-mapped to the bottom 8MB of
the physical address space. But things don't have to stay this way!
Thanks to Jesse who made an earlier attempt at this, it was really easy
to get device drivers working once the page tables were in place! :^)
Fixes#734.
Also, PCI Initializer dismiss() now deletes the object correctly, and
the PCI initialization process no longer use the DMI decoder to
determine if PCI is supported.
grub configuration files include an entry to boot the OS without
ACPI support.
The chroot() syscall now allows the superuser to isolate a process into
a specific subtree of the filesystem. This is not strictly permanent,
as it is also possible for a superuser to break *out* of a chroot, but
it is a useful mechanism for isolating unprivileged processes.
The VFS now uses the current process's root_directory() as the root for
path resolution purposes. The root directory is stored as an uncached
Custody in the Process object.
We now have these API's in <Kernel/Random.h>:
- get_fast_random_bytes(u8* buffer, size_t buffer_size)
- get_good_random_bytes(u8* buffer, size_t buffer_size)
- get_fast_random<T>()
- get_good_random<T>()
Internally they both use x86 RDRAND if available, otherwise they fall
back to the same LCG we had in RandomDevice all along.
The main purpose of this patch is to give kernel code a way to better
express its needs for random data.
Randomness is something that will require a lot more work, but this is
hopefully a step in the right direction.
The new PCI subsystem is initialized during runtime.
PCI::Initializer is supposed to be called during early boot, to
perform a few tests, and initialize the proper configuration space
access mechanism. Kernel boot parameters can be specified by a user to
determine what tests will occur, to aid debugging on problematic
machines.
After that, PCI::Initializer should be dismissed.
PCI::IOAccess is a class that is derived from PCI::Access
class and implements PCI configuration space access mechanism via x86
IO ports.
PCI::MMIOAccess is a class that is derived from PCI::Access
and implements PCI configurtaion space access mechanism via memory
access.
The new PCI subsystem also supports determination of IO/MMIO space
needed by a device by checking a given BAR.
In addition, Every device or component that use the PCI subsystem has
changed to match the last changes.
This prevents code running outside of kernel mode from using the
following instructions:
* SGDT - Store Global Descriptor Table
* SIDT - Store Interrupt Descriptor Table
* SLDT - Store Local Descriptor Table
* SMSW - Store Machine Status Word
* STR - Store Task Register
There's no need for userspace to be able to use these instructions so
let's just disable them to prevent information leakage.
We now refuse to boot on machines that don't support PAE since all
of our paging code depends on it.
Also let's only enable SSE and PGE support if the CPU advertises it.
Threads now have numeric priorities with a base priority in the 1-99
range.
Whenever a runnable thread is *not* scheduled, its effective priority
is incremented by 1. This is tracked in Thread::m_extra_priority.
The effective priority of a thread is m_priority + m_extra_priority.
When a runnable thread *is* scheduled, its m_extra_priority is reset to
zero and the effective priority returns to base.
This means that lower-priority threads will always eventually get
scheduled to run, once its effective priority becomes high enough to
exceed the base priority of threads "above" it.
The previous values for ThreadPriority (Low, Normal and High) are now
replaced as follows:
Low -> 10
Normal -> 30
High -> 50
In other words, it will take 20 ticks for a "Low" priority thread to
get to "Normal" effective priority, and another 20 to reach "High".
This is not perfect, and I've used some quite naive data structures,
but I think the mechanism will allow us to build various new and
interesting optimizations, and we can figure out better data structures
later on. :^)
The idea of all processes reliably having a main thread was nice in
some ways, but cumbersome in others. More importantly, it didn't match
up with POSIX thread semantics, so let's move away from it.
This thread gets rid of Process::main_thread() and you now we just have
a bunch of Thread objects floating around each Process.
When the finalizer nukes the last Thread in a Process, it will also
tear down the Process.
There's a bunch of more things to fix around this, but this is where we
get started :^)
Also added an option in the run script to force PIO operation mode with
the IDE controller.
In addition, we're no longer limited to PIIX3 and PIIX4 chipsets for DMA
Instead of the big ugly switch statement, build a lookup table using
the syscall enumeration macro.
This greatly simplifies the syscall implementation. :^)
Scheduling priority is now set at the thread level instead of at the
process level.
This is a step towards allowing processes to set different priorities
for threads. There's no userspace API for that yet, since only the main
thread's priority is affected by sched_setparam().
Oops, we were creating these and then throwing them away. They will
get instantiated a bit later, when we bring up the mounts in /etc/fstab
from userspace.
Add text.startup to the .text block, add .ctors as well.
Use them in init.cpp to call global constructors after
gtd and idt init. That way any funky constructors should be ok.
Also defines some Itanium C++ ABI methods that probably shouldn't be,
but without them the linker gets very angry.
If the code ever actually tries to use __dso_handle or call
__cxa_atexit, there's bigger problems with the kernel.
Bit of a hack would be an understatement but hey. It works :)
Also added a script to handle creation of GPT partitioned disk (with
GRUB config file). Block limit will be used to disallow potential access
to other partitions.
This is a freelist allocator with static size classes that works as a
complement to the generic kmalloc(). It's a lot faster than kmalloc()
since allocation just means popping from the freelist.
It's also significantly more compact when there are a lot of objects
smaller than the minimum kmalloc chunk size (32 bytes.)
This patch enables it for the Region and PhysicalPage classes.
In the PhysicalPage (8 bytes) case, it's a huge improvement since we
no longer waste 75% of the storage allocated.
There are also a number of ways this can be improved, so let's keep
working on it going forward.
If we receive an IRQ while the idle task is running, prevent it from
re-halting the CPU after the IRQ handler returns.
Instead have the idle task yield to the scheduler, so we can see if
the IRQ has unblocked something.
By setting up the devices in init() and looping over the registered
network adapters in NetworkTask_main, we can remove the remaining
hard-coded adapter references from the network code.
This also assigns IPs according to the default range supplied by QEMU
in its slirp networking mode.
This implements a very basic VGA device using the information provided
to us by the bootloader in the multiboot header. This allows Serenity to
boot to the desktop on basically any halfway modern system.
The complication is around /proc/sys/ variables, which were attached
to inodes. Now they're their own thing, and the corresponding inodes
are lazily created (as all other ProcFS inodes are) and simply refer
to them by index.
This should probably call out to a login program at some point. Right now
it just puts a root terminal on tty{1,2,3}.
Remember not to leave your Serenity workstation unattended!
serial_debug will output all the kprintf and dbgprintf data to COM1 at
8-N-1 57600 baud. this is particularly useful for debugging the boot
process on live hardware.
Note: it must be the first parameter in the boot cmdline.
The previous implementation of the PIIX3/4 PATA/IDE channel driver only
supported a single drive, as the object model was wrong (the channel
inherits the IRQ, not the disk drive itself). This fixes it by 'attaching'
two `PATADiskDevices` to a `PATAChannel`, which makes more sense.
The reading/writing code is presented as is, which violates the spec
outlined by Seagate in the linked datasheet. That spec is rather old,
so it might not be 100% up to date, though may cause issues on real
hardware, so until we can actually test it, this will suffice.
And use this to return EINTR in various places; some of which we were
not handling properly before.
This might expose a few bugs in userspace, but should be more compatible
with other POSIX systems, and is certainly a little cleaner.
A basic Floppy Disk Controller device driver for any system later than (and including) the IBM AT. The driver is based on the documentation supplied by QEMU, which is the datasheet for the Intel 82078 Floppy Disk controller (found here: https://wiki.qemu.org/images/f/f0/29047403.pdf)
Naturally, floppy disks are a _very_ outdated storage medium, however, as Serenity is a throwback to aesthetic 90s computing, it's a definite must have. Not to mention that there are still a lot of floppy disks around, with countless petabytes of software on them, so it would be nice if people could create images of said disks with serenity.
The code for this is mostly clean. however there are a LOT of values specified in the datasheet, so some of them might be wrong, not to mention that the actual specification itself is rather dirt and seemingly hacked together.
I'm also only supporting 3.5" floppy disks, without PIO polling (DMA only), so if you want anything more/less than 1.44MB HD Floppys, you'll have to do it yourself.
Also add an AudioServer that (right now) doesn't do much.
It tries to open, parse, and play a wav file. In the future, it can do more.
My general thinking here here is that /dev/audio will be "owned" by AudioServer,
and we'll do mixing in software before passing buffers off to the kernel
to play, but we have to start somewhere.
This is obviously more readable. If we ever run into a situation where
ref count churn is actually causing trouble in the future, we can deal with
it then. For now, let's keep it simple. :^)
The IDE Disk Controller driver has been extended to allow the secondary device on the channel to be initialised and used. A test as to whether this is working (for anyone interested) is to modify `init.cpp:87` to `auto dev_hd0 = IDEDiskDevice::create(IdeDiskDevice::DeviceType::SLAVE);`. The kernel will fail to boot, as there is no disk attached to CHANNEL 1's slave. This was born out of the fact that my FAT driver can't be tested as easily without creating a partition on `hda`.
This introduces very basic handling of the kernel command line to choose
the root filesystem at startup. Given that we currently only support a
single IDE hard drive, it's hard-coded to look for `/dev/hda` at the start
of the argument.
If there is nothing following this, or if the parameter is empty,
init_stage2 will try to load the ext2 filesystem from the start of the
device. This is intended to be the default behaviour when running
development builds, as it is faster to set up and doesn't require a
working grub installation.
If `/dev/hda` is followed by a number, init_stage2 will try to read an MBR
partition header from the drive, then load the requested partition. It
will reject non-numeric trailing data, and anything outside of partitions
one through four.
This implements a passthrough disk driver that translates the read/write
block addresses by a fixed offset. This could form the basis of MBR
partition support if we were to parse the MBR table at boot and create that
OffsetDiskDevice dynamically, rather than seeking to a fixed offset.
This also introduces a dependency in the form of grub. You'll need to have
32-bit grub binaries installed to build the project now.
As a bonus, divorcing Serenity from qemu's kernel loading means we can now
*technically* boot on real hardware. It just... doesn't get very far yet.
If you write the `_disk_image` file to an IDE hard drive and boot it in a
machine that supports all the basic PC hardware, it *will* start loading
the kernel.
Define the multiboot info struct properly so we don't have to grab at byte
offsets in the memory access checker code. Also print kernel command line
in init().
This is pretty shaky still, but the basic idea is that you subclass GModel
and return true for editable indices. The table view also needs to have its
editable flag set.
The old bootloader was hilariously complicated, requiring a floppy disk with
the kernel on it, and a hard drive with the file system. This patch removes
the floppy disk from the equation and replaces it with a multiboot header.
This means the kernel can now be booted with qemu-system-i386 -kernel kernel
The scheduler now operates on threads, rather than on processes.
Each process has a main thread, and can have any number of additional
threads. The process exits when the main thread exits.
This patch doesn't actually spawn any additional threads, it merely
does all the plumbing needed to make it possible. :^)
We now talk to the lookup server over a local socket and it does the lookup
on our behalf. Including some retry logic, which is nice, because it seems
like DNS requests disappear in the ether pretty damn often where I am.
This is a monster patch that required changing a whole bunch of things.
There are performance and stability issues all over the place, but it works.
Pretty cool, I have to admit :^)
Currently you can only mmap the entire framebuffer.
Using this when starting up the WindowServer gets us yet another step
closer towards it moving into userspace. :^)
Let GButton have an optional icon (GraphicsBitmap) that gets rendered in the
middle of the button if present.
Also add GraphicsBitmap::load_from_file() which allows mmap'ed RGBA32 files.
I wrote a little program to take "raw" files from GIMP and swizzle them into
the correct byte order.
For now, the WindowServer process will run with high priority,
while the Finalizer process will run with low priority.
Everyone else gets to be "normal".
At the moment, priority simply determines the size of your time slices.
Instead of processes themselves getting scheduled to finish dying,
let's have a Finalizer process that wakes up whenever someone is dying.
This way we can do all kinds of lock-taking in process cleanup without
risking reentering the scheduler.
Instead of cowboy-calling the VESA BIOS in the bootloader, find the emulator
VGA adapter by scanning the PCI bus. Then set up the desired video mode by
sending device commands.
Now the filesystem is generated on-the-fly instead of manually adding and
removing inodes as processes spawn and die.
The code is convoluted and bloated as I wrote it while sleepless. However,
it's still vastly better than the old ProcFS, so I'm committing it.
I also added /proc/PID/fd/N symlinks for each of a process's open fd's.
This required a fair bit of plumbing. The CharacterDevice::close() virtual
will now be closed by ~FileDescriptor(), allowing device implementations to
do custom cleanup at that point.
One big problem remains: if the master PTY is closed before the slave PTY,
we go into crashy land.
You can now open as many PTY pairs as you like. Well, it's actually capped
at 8 for now, but it's just a constant and trivial to change.
Unregistering a PTY pair is untested because I didn't want to start
mucking with that in Terminal right now.
The system can finally idle without burning CPU. :^)
There are some issues with scheduling making the mouse cursor sloppy
and unresponsive that need to be dealt with.
When you open /dev/ptmx, you get a file descriptor pointing to one of the
available MasterPTY's. If none are available, you get an EBUSY.
This makes it possible to open multiple (up to 4) Terminals. :^)
To support this, I also added a CharacterDevice::open() that gets control
when VFS is opening a CharacterDevice. This is useful when we want to return
a custom FileDescriptor like we do here.
Userspace programs can now open /dev/gui_events and read a stream of GUI_Event
structs one at a time.
I was stuck on a stupid problem where we'd reenter Scheduler::yield() due to
having one of the has_data_available_for_reading() implementations using locks.
Process page directories can now actually be freed. This could definitely
be implemented in a nicer, less wasteful way, but this works for now.
The spawn stress test can now run for a lot longer but eventually dies
due to kmalloc running out of memory.
Also use a simple array of { dword, const char* } for the KSyms and put the
whole shebang in kmalloc_eternal() memory. This was a fugly source of
kmalloc perma-frag.
Pass the file name in a stack-allocated buffer instead of using an AK::String
when iterating directories. This dramatically reduces the amount of cycles
spent traversing the filesystem.
- Process::exec() needs to restore the original paging scope when called
on a non-current process.
- Add missing InterruptDisabler guards around g_processes access.
- Only flush the TLB when modifying the active page tables.
sys$fork() now clones all writable regions with per-page COW bits.
The pages are then mapped read-only and we handle a PF by COWing the pages.
This is quite delightful. Obviously there's lots of work to do still,
and it needs better data structures, but the general concept works.
We no longer disable interrupts around the whole affair.
Since MM manages per-process data structures, this works quite smoothly now.
Only procfs had to be tweaked with an InterruptDisabler.
I'm still playing around with finding a style that I like.
This is starting to feel pleasing to the eye. I guess this is how long
it took me to break free from the habit of my previous Qt/WK coding style.
I spent some time stuck on a problem where processes would clobber each
other's stacks. Took me a moment to figure out that their stacks
were allocated in the sub-4MB linear address range which is shared
between all processes. Oops!
We now make three VirtualConsoles at boot: tty0, tty1, and tty2.
We launch an instance of /bin/sh in each one.
You switch between them with Alt+1/2/3
How very very cool :^)
The SpinLock was all backwards and didn't actually work. Fixing it exposed
how wrong most of the locking here is.
I need to come up with a better granularity here.
I also added a generator cache to FileHandle. This way, multiple
reads to a generated file (i.e in a synthfs) can transparently
handle multiple calls to read() without the contents changing
between calls.
The cache is discarded at EOF (or when the FileHandle is destroyed.)
FileHandle gets a hasDataAvailableForRead() getter.
If this returns true in sys$read(), the task will block(BlockedRead) + yield.
The fd blocked on is stored in Task::m_fdBlockedOnRead.
The scheduler then looks at the state of that fd during the unblock phase.
This makes "sh" restful. :^)
There's still some problem with the kernel not surviving the colonel task
getting scheduled. I need to figure that out and fix it.
It's implemented as a separate process. How cute is that.
Tasks now have a current working directory. Spawned tasks inherit their
parent task's working directory.
Currently everyone just uses "/" as there's no way to chdir().
Add a separate lock to protect the VFS. I think this might be a good idea.
I'm not sure it's a good approach though. I'll fiddle with it as I go along.
It's really fun to figure out all these things on my own.
- Turn Keyboard into a CharacterDevice (85,1) at /dev/keyboard.
- Implement MM::unmapRegionsForTask() and MM::unmapRegion()
- Save SS correctly on interrupt.
- Add a simple Spawn syscall for launching another process.
- Move a bunch of IO syscall debug output behind DEBUG_IO.
- Have ASSERT do a "cli" immediately when failing.
This makes the output look proper every time.
- Implement a bunch of syscalls in LibC.
- Add a simple shell ("sh"). All it can do now is read a line
of text from /dev/keyboard and then try launching the specified
executable by calling spawn().
There are definitely bugs in here, but we're moving on forward.
