LadybirdBrowser/ladybird
1
1
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-11-11 01:06:01 +03:00
Code Issues Packages Projects Releases Wiki Activity
37253ebcae
ladybird/Kernel/Syscalls/mmap.cpp

641 lines
22 KiB
C++
Raw Normal View History

Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
/*
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
* Copyright (c) 2018-2021, Andreas Kling <kling@serenityos.org>
Kernel: munmap multiple regions at a time This implements a fallback to munmap that unmaps multiple regions at a time, with splitting some when needed. The way it is implemented is possibly not optimal, due to it searching without looking into the cache
2021-02-24 19:00:58 +03:00
* Copyright (c) 2021, Leon Albrecht <leon2002.la@gmail.com>
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 11:24:48 +03:00
* SPDX-License-Identifier: BSD-2-Clause
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
*/
Kernel+LibELF: Support initializing values of TLS data Previously, TLS data was always zero-initialized. To support initializing the values of TLS data, sys$allocate_tls now receives a buffer with the desired initial data, and copies it to the master TLS region of the process. The DynamicLinker gathers the initial TLS image and passes it to sys$allocate_tls. We also now require the size passed to sys$allocate_tls to be page-aligned, to make things easier. Note that this doesn't waste memory as the TLS data has to be allocated in separate pages anyway.
2021-04-24 11:30:20 +03:00
#include <Kernel/Arch/x86/SmapDisabler.h>
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
#include <Kernel/FileSystem/FileDescription.h>
Kernel+Profiler: Capture metadata about all profiled processes The perfcore file format was previously limited to a single process since the pid/executable/regions data was top-level in the JSON. This patch moves the process-specific data into a top-level array named "processes" and we now add entries for each process that has been sampled during the profile run. This makes it possible to see samples from multiple threads when viewing a perfcore file with Profiler. This is extremely cool! :^)
2021-03-02 21:01:02 +03:00
#include <Kernel/PerformanceEventBuffer.h>
Kernel: Add PerformanceManager static class, move perf event APIs there The current method of emitting performance events requires a bit of boiler plate at every invocation, as well as having to ignore the return code which isn't used outside of the perf event syscall. This change attempts to clean that up by exposing high level API's that can be used around the code base.
2021-05-07 08:29:19 +03:00
#include <Kernel/PerformanceManager.h>
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
#include <Kernel/Process.h>
Kernel: sys$mmap PAGE_ROUND_UP size before calling allocate_randomized (#5154) `allocate_randomized` assert an already sanitized size but `mmap` were just forwarding whatever the process asked so it was possible to trigger a kernel panic from an unpriviliged process just by asking some randomly placed memory and a size non alligned with the page size. This fixes this issue by rounding up to the next page size before calling `allocate_randomized`. Fixes #5149
2021-01-29 00:36:20 +03:00
#include <Kernel/VM/MemoryManager.h>
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
#include <Kernel/VM/PageDirectory.h>
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
#include <Kernel/VM/PrivateInodeVMObject.h>
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
#include <Kernel/VM/Region.h>
#include <Kernel/VM/SharedInodeVMObject.h>
#include <LibC/limits.h>
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
#include <LibELF/Validation.h>
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
namespace Kernel {
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
static bool should_make_executable_exception_for_dynamic_loader(bool make_readable, bool make_writable, bool make_executable, const Region& region)
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
{
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
// Normally we don't allow W -> X transitions, but we have to make an exception
// for the dynamic loader, which needs to do this after performing text relocations.
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
// FIXME: Investigate whether we could get rid of all text relocations entirely.
// The exception is only made if all the following criteria is fulfilled:
// The region must be RW
if (!(region.is_readable() && region.is_writable() && !region.is_executable()))
return false;
// The region wants to become RX
if (!(make_readable && !make_writable && make_executable))
return false;
// The region is backed by a file
if (!region.vmobject().is_inode())
return false;
// The file mapping is private, not shared (no relocations in a shared mapping!)
if (!region.vmobject().is_private_inode())
return false;
Kernel: Read the ELF header from the inode rather than the mapped pages Reading from the mapping doesn't work when the text segment has a non-zero offset because in that case the first mapped page doesn't contain the ELF header.
2021-04-13 20:29:34 +03:00
auto& inode_vm = static_cast<const InodeVMObject&>(region.vmobject());
auto& inode = inode_vm.inode();
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
Elf32_Ehdr header;
Kernel: Read the ELF header from the inode rather than the mapped pages Reading from the mapping doesn't work when the text segment has a non-zero offset because in that case the first mapped page doesn't contain the ELF header.
2021-04-13 20:29:34 +03:00
auto buffer = UserOrKernelBuffer::for_kernel_buffer((u8*)&header);
Kernel: Change Inode::{read/write}_bytes interface to KResultOr<ssize_t> The error handling in all these cases was still using the old style negative values to indicate errors. We have a nicer solution for this now with KResultOr<T>. This change switches the interface and then all implementers to use the new style.
2021-05-02 00:29:39 +03:00
auto result = inode.read_bytes(0, sizeof(header), buffer, nullptr);
if (result.is_error() || result.value() != sizeof(header))
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
return false;
// The file is a valid ELF binary
if (!ELF::validate_elf_header(header, inode.size()))
return false;
// The file is an ELF shared object
if (header.e_type != ET_DYN)
return false;
// FIXME: Are there any additional checks/validations we could do here?
return true;
}
Kernel: Remove "has made executable exception for dynamic loader" flag As Idan pointed out, this flag is actually not needed, since we don't allow transitioning from previously-executable to writable anyway.
2021-01-30 12:06:52 +03:00
static bool validate_mmap_prot(int prot, bool map_stack, bool map_anonymous, const Region* region = nullptr)
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
{
bool make_readable = prot & PROT_READ;
bool make_writable = prot & PROT_WRITE;
bool make_executable = prot & PROT_EXEC;
Kernel: Disallow mapping anonymous memory as executable This adds another layer of defense against introducing new code into a running process. The only permitted way of doing so is by mmapping an open file with PROT_READ | PROT_EXEC. This does make any future JIT implementations slightly more complicated but I think it's a worthwhile trade-off at this point. :^)
2021-01-29 16:46:53 +03:00
if (map_anonymous && make_executable)
return false;
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
if (make_writable && make_executable)
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
return false;
if (map_stack) {
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
if (make_executable)
return false;
if (!make_readable || !make_writable)
return false;
}
if (region) {
if (make_writable && region->has_been_executable())
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
return false;
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
if (make_executable && region->has_been_writable()) {
Kernel: Remove "has made executable exception for dynamic loader" flag As Idan pointed out, this flag is actually not needed, since we don't allow transitioning from previously-executable to writable anyway.
2021-01-30 12:06:52 +03:00
if (should_make_executable_exception_for_dynamic_loader(make_readable, make_writable, make_executable, *region))
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
return true;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
return false;
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
}
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
return true;
}
static bool validate_inode_mmap_prot(const Process& process, int prot, const Inode& inode, bool map_shared)
{
auto metadata = inode.metadata();
if ((prot & PROT_READ) && !metadata.may_read(process))
return false;
if (map_shared) {
// FIXME: What about readonly filesystem mounts? We cannot make a
// decision here without knowing the mount flags, so we would need to
// keep a Custody or something from mmap time.
if ((prot & PROT_WRITE) && !metadata.may_write(process))
return false;
InterruptDisabler disabler;
Kernel: Hold InodeVMObject reference while inspecting it in sys$mmap()
2020-12-29 15:32:08 +03:00
if (auto shared_vmobject = inode.shared_vmobject()) {
if ((prot & PROT_EXEC) && shared_vmobject->writable_mappings())
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
return false;
Kernel: Hold InodeVMObject reference while inspecting it in sys$mmap()
2020-12-29 15:32:08 +03:00
if ((prot & PROT_WRITE) && shared_vmobject->executable_mappings())
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
return false;
}
}
return true;
}
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
KResultOr<FlatPtr> Process::sys$mmap(Userspace<const Syscall::SC_mmap_params*> user_params)
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
{
REQUIRE_PROMISE(stdio);
Syscall::SC_mmap_params params;
Kernel: Make copy_to/from_user safe and remove unnecessary checks Since the CPU already does almost all necessary validation steps for us, we don't really need to attempt to do this. Doing it ourselves doesn't really work very reliably, because we'd have to account for other processors modifying virtual memory, and we'd have to account for e.g. pages not being able to be allocated due to insufficient resources. So change the copy_to/from_user (and associated helper functions) to use the new safe_memcpy, which will return whether it succeeded or not. The only manual validation step needed (which the CPU can't perform for us) is making sure the pointers provided by user mode aren't pointing to kernel mappings. To make it easier to read/write from/to either kernel or user mode data add the UserOrKernelBuffer helper class, which will internally either use copy_from/to_user or directly memcpy, or pass the data through directly using a temporary buffer on the stack. Last but not least we need to keep syscall params trivial as we need to copy them from/to user mode using copy_from/to_user.
2020-09-12 06:11:07 +03:00
if (!copy_from_user(&params, user_params))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Use Userspace<T> in sys${munmap,mprotect,madvise,msyscall}()
2021-03-01 17:53:33 +03:00
FlatPtr addr = params.addr;
Kernel: Refactor storage stack with u64 as mmap offset
2021-03-19 00:57:25 +03:00
auto size = params.size;
auto alignment = params.alignment;
auto prot = params.prot;
auto flags = params.flags;
auto fd = params.fd;
auto offset = params.offset;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Add "prot_exec" pledge promise and require it for PROT_EXEC This prevents sys$mmap() and sys$mprotect() from creating executable memory mappings in pledged programs that don't have this promise. Note that the dynamic loader runs before pledging happens, so it's unaffected by this.
2021-01-29 20:50:27 +03:00
if (prot & PROT_EXEC) {
REQUIRE_PROMISE(prot_exec);
}
Kernel: Add "map_fixed" pledge promise This is a new promise that guards access to mmap() with MAP_FIXED. Fixed-address mappings are rarely used, but can be useful if you are trying to groom the process address space for malicious purposes. None of our programs need this at the moment, as the only user of MAP_FIXED is DynamicLoader, but the fixed mappings are constructed before the process has had a chance to pledge anything.
2021-02-21 03:08:48 +03:00
if (prot & MAP_FIXED) {
REQUIRE_PROMISE(map_fixed);
}
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (alignment & ~PAGE_MASK)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Assert if rounding-up-to-page-size would wrap around to 0 If we try to align a number above 0xfffff000 to the next multiple of the page size (4 KiB), it would wrap around to 0. This is most likely never what we want, so let's assert if that happens.
2021-02-14 11:57:19 +03:00
if (page_round_up_would_wrap(size))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Assert if rounding-up-to-page-size would wrap around to 0 If we try to align a number above 0xfffff000 to the next multiple of the page size (4 KiB), it would wrap around to 0. This is most likely never what we want, so let's assert if that happens.
2021-02-14 11:57:19 +03:00
if (!is_user_range(VirtualAddress(addr), page_round_up(size)))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Use KString for Region names Replace the AK::String used for Region::m_name with a KString. This seems beneficial across the board, but as a specific data point, it reduces time spent in sys$set_mmap_name() by ~50% on test-js. :^)
2021-05-28 10:33:14 +03:00
OwnPtr<KString> name;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (params.name.characters) {
if (params.name.length > PATH_MAX)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENAMETOOLONG;
Kernel: Use KString for Region names Replace the AK::String used for Region::m_name with a KString. This seems beneficial across the board, but as a specific data point, it reduces time spent in sys$set_mmap_name() by ~50% on test-js. :^)
2021-05-28 10:33:14 +03:00
auto name_or_error = try_copy_kstring_from_user(params.name);
if (name_or_error.is_error())
return name_or_error.error();
name = name_or_error.release_value();
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
if (size == 0)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if ((FlatPtr)addr & ~PAGE_MASK)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
bool map_shared = flags & MAP_SHARED;
bool map_anonymous = flags & MAP_ANONYMOUS;
bool map_private = flags & MAP_PRIVATE;
bool map_stack = flags & MAP_STACK;
bool map_fixed = flags & MAP_FIXED;
Kernel: Add MAP_NORESERVE support to mmap Rather than lazily committing regions by default, we now commit the entire region unless MAP_NORESERVE is specified. This solves random crashes in low-memory situations where e.g. the malloc heap allocated memory, but using pages that haven't been used before triggers a crash when no more physical memory is available. Use this flag to create large regions without actually committing the backing memory. madvise() can be used to commit arbitrary areas of such regions after creating them.
2020-09-04 00:06:25 +03:00
bool map_noreserve = flags & MAP_NORESERVE;
Kernel+LibC: Add MAP_RANDOMIZED flag for sys$mmap() This can be used to request random VM placement instead of the highly predictable regular mmap(nullptr, ...) VM allocation strategy. It will soon be used to implement ASLR in the dynamic loader. :^)
2021-01-28 16:55:06 +03:00
bool map_randomized = flags & MAP_RANDOMIZED;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (map_shared && map_private)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (!map_shared && !map_private)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Prevent mmap-ing as both fixed and randomized
2021-01-29 02:28:27 +03:00
if (map_fixed && map_randomized)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Prevent mmap-ing as both fixed and randomized
2021-01-29 02:28:27 +03:00
Kernel: Fix mix-up between MAP_STACK/MAP_ANONYMOUS in prot validation
2021-01-30 12:30:17 +03:00
if (!validate_mmap_prot(prot, map_stack, map_anonymous))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (map_stack && (!map_private || !map_anonymous))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Region* region = nullptr;
Kernel+LibC: Add MAP_RANDOMIZED flag for sys$mmap() This can be used to request random VM placement instead of the highly predictable regular mmap(nullptr, ...) VM allocation strategy. It will soon be used to implement ASLR in the dynamic loader. :^)
2021-01-28 16:55:06 +03:00
Optional<Range> range;
if (map_randomized) {
Kernel: Assert if rounding-up-to-page-size would wrap around to 0 If we try to align a number above 0xfffff000 to the next multiple of the page size (4 KiB), it would wrap around to 0. This is most likely never what we want, so let's assert if that happens.
2021-02-14 11:57:19 +03:00
range = space().page_directory().range_allocator().allocate_randomized(page_round_up(size), alignment);
Kernel+LibC: Add MAP_RANDOMIZED flag for sys$mmap() This can be used to request random VM placement instead of the highly predictable regular mmap(nullptr, ...) VM allocation strategy. It will soon be used to implement ASLR in the dynamic loader. :^)
2021-01-28 16:55:06 +03:00
} else {
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
range = space().allocate_range(VirtualAddress(addr), size, alignment);
Kernel+LibC: Add MAP_RANDOMIZED flag for sys$mmap() This can be used to request random VM placement instead of the highly predictable regular mmap(nullptr, ...) VM allocation strategy. It will soon be used to implement ASLR in the dynamic loader. :^)
2021-01-28 16:55:06 +03:00
if (!range.has_value()) {
if (addr && !map_fixed) {
// If there's an address but MAP_FIXED wasn't specified, the address is just a hint.
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
range = space().allocate_range({}, size, alignment);
Kernel+LibC: Add MAP_RANDOMIZED flag for sys$mmap() This can be used to request random VM placement instead of the highly predictable regular mmap(nullptr, ...) VM allocation strategy. It will soon be used to implement ASLR in the dynamic loader. :^)
2021-01-28 16:55:06 +03:00
}
Kernel: sys$mmap() without MAP_FIXED should consider address a hint If we can't use that specific address, it's still okay to put it anywhere else in VM.
2021-01-27 22:48:38 +03:00
}
}
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel+LibC: Add MAP_RANDOMIZED flag for sys$mmap() This can be used to request random VM placement instead of the highly predictable regular mmap(nullptr, ...) VM allocation strategy. It will soon be used to implement ASLR in the dynamic loader. :^)
2021-01-28 16:55:06 +03:00
if (!range.has_value())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENOMEM;
Kernel+LibC: Add MAP_RANDOMIZED flag for sys$mmap() This can be used to request random VM placement instead of the highly predictable regular mmap(nullptr, ...) VM allocation strategy. It will soon be used to implement ASLR in the dynamic loader. :^)
2021-01-28 16:55:06 +03:00
Kernel: Remove MAP_PURGEABLE from mmap This brings mmap more in line with other operating systems. Prior to this, it was impossible to request memory that was definitely committed, instead MAP_PURGEABLE would provide a region that was not actually purgeable, but also not fully committed, which meant that using such memory still could cause crashes when the underlying pages could no longer be allocated. This fixes some random crashes in low-memory situations where non-volatile memory is mapped (e.g. malloc, tls, Gfx::Bitmap, etc) but when a page in these regions is first accessed, there is insufficient physical memory available to commit a new page.
2020-09-04 20:47:49 +03:00
if (map_anonymous) {
Revert "Kernel: Allocate shared memory regions immediately" This reverts commit fe6b3f99d1f544715098182ce5bed71d67f266cf.
2021-01-02 20:16:39 +03:00
auto strategy = map_noreserve ? AllocationStrategy::None : AllocationStrategy::Reserve;
Kernel: Use KString for Region names Replace the AK::String used for Region::m_name with a KString. This seems beneficial across the board, but as a specific data point, it reduces time spent in sys$set_mmap_name() by ~50% on test-js. :^)
2021-05-28 10:33:14 +03:00
auto region_or_error = space().allocate_region(range.value(), {}, prot, strategy);
Kernel: Make Process::allocate_region*() return KResultOr<Region*> This allows region allocation to return specific errors and we don't have to assume every failure is an ENOMEM.
2021-01-15 19:27:52 +03:00
if (region_or_error.is_error())
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return region_or_error.error().error();
Kernel: Make Process::allocate_region*() return KResultOr<Region*> This allows region allocation to return specific errors and we don't have to assume every failure is an ENOMEM.
2021-01-15 19:27:52 +03:00
region = region_or_error.value();
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
} else {
if (offset < 0)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (static_cast<size_t>(offset) & ~PAGE_MASK)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
auto description = file_description(fd);
if (!description)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EBADF;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (description->is_directory())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENODEV;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
// Require read access even when read protection is not requested.
if (!description->is_readable())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EACCES;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (map_shared) {
if ((prot & PROT_WRITE) && !description->is_writable())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EACCES;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
if (description->inode()) {
if (!validate_inode_mmap_prot(*this, prot, *description->inode(), map_shared))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EACCES;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Hoist VM range allocation up to sys$mmap() itself Instead of letting each File subclass do range allocation in their mmap() override, do it up front in sys$mmap(). This makes us honor alignment requests for file-backed memory mappings and simplifies the code somwhat.
2021-01-25 16:52:36 +03:00
Kernel: Refactor storage stack with u64 as mmap offset
2021-03-19 00:57:25 +03:00
auto region_or_error = description->mmap(*this, range.value(), static_cast<u64>(offset), prot, map_shared);
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (region_or_error.is_error())
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return region_or_error.error().error();
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
region = region_or_error.value();
}
if (!region)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENOMEM;
Kernel+Profiler: Improve profiling subsystem This turns the perfcore format into more a log than it was before, which lets us properly log process, thread and region creation/destruction. This also makes it unnecessary to dump the process' regions every time it is scheduled like we did before. Incidentally this also fixes 'profile -c' because we previously ended up incorrectly dumping the parent's region map into the profile data. Log-based mmap support enables profiling shared libraries which are loaded at runtime, e.g. via dlopen(). This enables profiling both the parent and child process for programs which use execve(). Previously we'd discard the profiling data for the old process. The Profiler tool has been updated to not treat thread IDs as process IDs anymore. This enables support for processes with more than one thread. Also, there's a new widget to filter which process should be displayed.
2021-04-26 00:42:36 +03:00
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
region->set_mmap(true);
if (map_shared)
region->set_shared(true);
if (map_stack)
region->set_stack(true);
Kernel: Use KString for Region names Replace the AK::String used for Region::m_name with a KString. This seems beneficial across the board, but as a specific data point, it reduces time spent in sys$set_mmap_name() by ~50% on test-js. :^)
2021-05-28 10:33:14 +03:00
region->set_name(move(name));
Kernel: Add PerformanceManager static class, move perf event APIs there The current method of emitting performance events requires a bit of boiler plate at every invocation, as well as having to ignore the return code which isn't used outside of the perf event syscall. This change attempts to clean that up by exposing high level API's that can be used around the code base.
2021-05-07 08:29:19 +03:00
PerformanceManager::add_mmap_perf_event(*this, *region);
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return region->vaddr().get();
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
static KResultOr<Range> expand_range_to_page_boundaries(FlatPtr address, size_t size)
{
if (page_round_up_would_wrap(size))
return EINVAL;
if ((address + size) < address)
return EINVAL;
if (page_round_up_would_wrap(address + size))
return EINVAL;
auto base = VirtualAddress { address }.page_base();
auto end = page_round_up(address + size);
return Range { base, end - base.get() };
}
Kernel: Use Userspace<T> in sys${munmap,mprotect,madvise,msyscall}()
2021-03-01 17:53:33 +03:00
KResultOr<int> Process::sys$mprotect(Userspace<void*> addr, size_t size, int prot)
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
{
REQUIRE_PROMISE(stdio);
Kernel: Add "prot_exec" pledge promise and require it for PROT_EXEC This prevents sys$mmap() and sys$mprotect() from creating executable memory mappings in pledged programs that don't have this promise. Note that the dynamic loader runs before pledging happens, so it's unaffected by this.
2021-01-29 20:50:27 +03:00
if (prot & PROT_EXEC) {
REQUIRE_PROMISE(prot_exec);
}
Kernel: Use Userspace<T> in sys${munmap,mprotect,madvise,msyscall}()
2021-03-01 17:53:33 +03:00
auto range_or_error = expand_range_to_page_boundaries(addr, size);
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
if (range_or_error.is_error())
return range_or_error.error();
Kernel: Round up ranges to page size multiples in munmap and mprotect This prevents passing bad inputs to RangeAllocator who then asserts. Found by fuzz-syscalls. :^)
2021-02-13 02:47:47 +03:00
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
auto range_to_mprotect = range_or_error.value();
Kernel: Round up ranges to page size multiples in munmap and mprotect This prevents passing bad inputs to RangeAllocator who then asserts. Found by fuzz-syscalls. :^)
2021-02-13 02:47:47 +03:00
if (!range_to_mprotect.size())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Round up ranges to page size multiples in munmap and mprotect This prevents passing bad inputs to RangeAllocator who then asserts. Found by fuzz-syscalls. :^)
2021-02-13 02:47:47 +03:00
if (!is_user_range(range_to_mprotect))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
if (auto* whole_region = space().find_region_from_range(range_to_mprotect)) {
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (!whole_region->is_mmap())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Remove "has made executable exception for dynamic loader" flag As Idan pointed out, this flag is actually not needed, since we don't allow transitioning from previously-executable to writable anyway.
2021-01-30 12:06:52 +03:00
if (!validate_mmap_prot(prot, whole_region->is_stack(), whole_region->vmobject().is_anonymous(), whole_region))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (whole_region->access() == prot_to_region_access_flags(prot))
return 0;
if (whole_region->vmobject().is_inode()
&& !validate_inode_mmap_prot(*this, prot, static_cast<const InodeVMObject&>(whole_region->vmobject()).inode(), whole_region->is_shared())) {
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EACCES;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
whole_region->set_readable(prot & PROT_READ);
whole_region->set_writable(prot & PROT_WRITE);
whole_region->set_executable(prot & PROT_EXEC);
Kernel: Enforce W^X more strictly (like PaX MPROTECT) This patch adds enforcement of two new rules: - Memory that was previously writable cannot become executable - Memory that was previously executable cannot become writable Unfortunately we have to make an exception for text relocations in the dynamic loader. Since those necessitate writing into a private copy of library code, we allow programs to transition from RW to RX under very specific conditions. See the implementation of sys$mprotect()'s should_make_executable_exception_for_dynamic_loader() for details.
2021-01-29 16:38:49 +03:00
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
whole_region->remap();
return 0;
}
// Check if we can carve out the desired range from an existing region
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
if (auto* old_region = space().find_region_containing(range_to_mprotect)) {
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (!old_region->is_mmap())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Remove "has made executable exception for dynamic loader" flag As Idan pointed out, this flag is actually not needed, since we don't allow transitioning from previously-executable to writable anyway.
2021-01-30 12:06:52 +03:00
if (!validate_mmap_prot(prot, old_region->is_stack(), old_region->vmobject().is_anonymous(), old_region))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (old_region->access() == prot_to_region_access_flags(prot))
return 0;
if (old_region->vmobject().is_inode()
&& !validate_inode_mmap_prot(*this, prot, static_cast<const InodeVMObject&>(old_region->vmobject()).inode(), old_region->is_shared())) {
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EACCES;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Remove old region from process' regions vector before splitting This does not affect functionality right now, but it means that the regions vector will now never have any overlapping regions, which will allow the use of balance binary search trees instead of a vector in the future. (since they require keys to be exclusive)
2021-04-07 02:17:05 +03:00
// Remove the old region from our regions tree, since were going to add another region
// with the exact same start address, but dont deallocate it yet
auto region = space().take_region(*old_region);
VERIFY(region);
// Unmap the old region here, specifying that we *don't* want the VM deallocated.
region->unmap(Region::ShouldDeallocateVirtualMemoryRange::No);
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
// This vector is the region(s) adjacent to our range.
// We need to allocate a new region for the range we wanted to change permission bits on.
Kernel: Remove old region from process' regions vector before splitting This does not affect functionality right now, but it means that the regions vector will now never have any overlapping regions, which will allow the use of balance binary search trees instead of a vector in the future. (since they require keys to be exclusive)
2021-04-07 02:17:05 +03:00
auto adjacent_regions = space().split_region_around_range(*region, range_to_mprotect);
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Remove old region from process' regions vector before splitting This does not affect functionality right now, but it means that the regions vector will now never have any overlapping regions, which will allow the use of balance binary search trees instead of a vector in the future. (since they require keys to be exclusive)
2021-04-07 02:17:05 +03:00
size_t new_range_offset_in_vmobject = region->offset_in_vmobject() + (range_to_mprotect.base().get() - region->range().base().get());
auto& new_region = space().allocate_split_region(*region, range_to_mprotect, new_range_offset_in_vmobject);
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
new_region.set_readable(prot & PROT_READ);
new_region.set_writable(prot & PROT_WRITE);
new_region.set_executable(prot & PROT_EXEC);
// Map the new regions using our page directory (they were just allocated and don't have one).
for (auto* adjacent_region : adjacent_regions) {
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
adjacent_region->map(space().page_directory());
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
new_region.map(space().page_directory());
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
return 0;
}
Kernel: Implement mprotect for multiple Regions
2021-04-01 23:04:04 +03:00
if (const auto& regions = space().find_regions_intersecting(range_to_mprotect); regions.size()) {
size_t full_size_found = 0;
// first check before doing anything
for (const auto* region : regions) {
if (!region->is_mmap())
return EPERM;
if (!validate_mmap_prot(prot, region->is_stack(), region->vmobject().is_anonymous(), region))
return EINVAL;
if (region->access() == prot_to_region_access_flags(prot))
return 0;
if (region->vmobject().is_inode()
&& !validate_inode_mmap_prot(*this, prot, static_cast<const InodeVMObject&>(region->vmobject()).inode(), region->is_shared())) {
return EACCES;
}
full_size_found += region->range().intersect(range_to_mprotect).size();
}
if (full_size_found != range_to_mprotect.size())
return ENOMEM;
// then do all the other stuff
for (auto* old_region : regions) {
const auto intersection_to_mprotect = range_to_mprotect.intersect(old_region->range());
// full sub region
if (intersection_to_mprotect == old_region->range()) {
old_region->set_readable(prot & PROT_READ);
old_region->set_writable(prot & PROT_WRITE);
old_region->set_executable(prot & PROT_EXEC);
old_region->remap();
continue;
}
// Remove the old region from our regions tree, since were going to add another region
// with the exact same start address, but dont deallocate it yet
auto region = space().take_region(*old_region);
VERIFY(region);
// Unmap the old region here, specifying that we *don't* want the VM deallocated.
region->unmap(Region::ShouldDeallocateVirtualMemoryRange::No);
// This vector is the region(s) adjacent to our range.
// We need to allocate a new region for the range we wanted to change permission bits on.
auto adjacent_regions = space().split_region_around_range(*old_region, intersection_to_mprotect);
// there should only be one
VERIFY(adjacent_regions.size() == 1);
size_t new_range_offset_in_vmobject = old_region->offset_in_vmobject() + (intersection_to_mprotect.base().get() - old_region->range().base().get());
auto& new_region = space().allocate_split_region(*region, intersection_to_mprotect, new_range_offset_in_vmobject);
new_region.set_readable(prot & PROT_READ);
new_region.set_writable(prot & PROT_WRITE);
new_region.set_executable(prot & PROT_EXEC);
// Map the new region using our page directory (they were just allocated and don't have one) if any.
if (adjacent_regions.size())
adjacent_regions[0]->map(space().page_directory());
new_region.map(space().page_directory());
}
return 0;
}
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Use Userspace<T> in sys${munmap,mprotect,madvise,msyscall}()
2021-03-01 17:53:33 +03:00
KResultOr<int> Process::sys$madvise(Userspace<void*> address, size_t size, int advice)
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
{
REQUIRE_PROMISE(stdio);
Kernel: Use Userspace<T> in sys${munmap,mprotect,madvise,msyscall}()
2021-03-01 17:53:33 +03:00
auto range_or_error = expand_range_to_page_boundaries(address, size);
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
if (range_or_error.is_error())
return range_or_error.error();
Kernel: Assert if rounding-up-to-page-size would wrap around to 0 If we try to align a number above 0xfffff000 to the next multiple of the page size (4 KiB), it would wrap around to 0. This is most likely never what we want, so let's assert if that happens.
2021-02-14 11:57:19 +03:00
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
auto range_to_madvise = range_or_error.value();
Kernel: Round up ranges to page size multiples in munmap and mprotect This prevents passing bad inputs to RangeAllocator who then asserts. Found by fuzz-syscalls. :^)
2021-02-13 02:47:47 +03:00
if (!range_to_madvise.size())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Round up ranges to page size multiples in munmap and mprotect This prevents passing bad inputs to RangeAllocator who then asserts. Found by fuzz-syscalls. :^)
2021-02-13 02:47:47 +03:00
if (!is_user_range(range_to_madvise))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Round up ranges to page size multiples in munmap and mprotect This prevents passing bad inputs to RangeAllocator who then asserts. Found by fuzz-syscalls. :^)
2021-02-13 02:47:47 +03:00
auto* region = space().find_region_from_range(range_to_madvise);
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (!region)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (!region->is_mmap())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Memory purging improvements This adds the ability for a Region to define volatile/nonvolatile areas within mapped memory using madvise(). This also means that memory purging takes into account all views of the PurgeableVMObject and only purges memory that is not needed by all of them. When calling madvise() to change an area to nonvolatile memory, return whether memory from that area was purged. At that time also try to remap all memory that is requested to be nonvolatile, and if insufficient pages are available notify the caller of that fact.
2020-09-03 07:57:09 +03:00
bool set_volatile = advice & MADV_SET_VOLATILE;
bool set_nonvolatile = advice & MADV_SET_NONVOLATILE;
if (set_volatile && set_nonvolatile)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Memory purging improvements This adds the ability for a Region to define volatile/nonvolatile areas within mapped memory using madvise(). This also means that memory purging takes into account all views of the PurgeableVMObject and only purges memory that is not needed by all of them. When calling madvise() to change an area to nonvolatile memory, return whether memory from that area was purged. At that time also try to remap all memory that is requested to be nonvolatile, and if insufficient pages are available notify the caller of that fact.
2020-09-03 07:57:09 +03:00
if (set_volatile || set_nonvolatile) {
Kernel: Merge PurgeableVMObject into AnonymousVMObject This implements memory commitments and lazy-allocation of committed memory.
2020-09-06 00:52:14 +03:00
if (!region->vmobject().is_anonymous())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Memory purging improvements This adds the ability for a Region to define volatile/nonvolatile areas within mapped memory using madvise(). This also means that memory purging takes into account all views of the PurgeableVMObject and only purges memory that is not needed by all of them. When calling madvise() to change an area to nonvolatile memory, return whether memory from that area was purged. At that time also try to remap all memory that is requested to be nonvolatile, and if insufficient pages are available notify the caller of that fact.
2020-09-03 07:57:09 +03:00
bool was_purged = false;
switch (region->set_volatile(VirtualAddress(address), size, set_volatile, was_purged)) {
case Region::SetVolatileError::Success:
break;
case Region::SetVolatileError::NotPurgeable:
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Memory purging improvements This adds the ability for a Region to define volatile/nonvolatile areas within mapped memory using madvise(). This also means that memory purging takes into account all views of the PurgeableVMObject and only purges memory that is not needed by all of them. When calling madvise() to change an area to nonvolatile memory, return whether memory from that area was purged. At that time also try to remap all memory that is requested to be nonvolatile, and if insufficient pages are available notify the caller of that fact.
2020-09-03 07:57:09 +03:00
case Region::SetVolatileError::OutOfMemory:
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENOMEM;
Kernel: Memory purging improvements This adds the ability for a Region to define volatile/nonvolatile areas within mapped memory using madvise(). This also means that memory purging takes into account all views of the PurgeableVMObject and only purges memory that is not needed by all of them. When calling madvise() to change an area to nonvolatile memory, return whether memory from that area was purged. At that time also try to remap all memory that is requested to be nonvolatile, and if insufficient pages are available notify the caller of that fact.
2020-09-03 07:57:09 +03:00
}
if (set_nonvolatile)
return was_purged ? 1 : 0;
return 0;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
if (advice & MADV_GET_VOLATILE) {
Kernel: Merge PurgeableVMObject into AnonymousVMObject This implements memory commitments and lazy-allocation of committed memory.
2020-09-06 00:52:14 +03:00
if (!region->vmobject().is_anonymous())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Memory purging improvements This adds the ability for a Region to define volatile/nonvolatile areas within mapped memory using madvise(). This also means that memory purging takes into account all views of the PurgeableVMObject and only purges memory that is not needed by all of them. When calling madvise() to change an area to nonvolatile memory, return whether memory from that area was purged. At that time also try to remap all memory that is requested to be nonvolatile, and if insufficient pages are available notify the caller of that fact.
2020-09-03 07:57:09 +03:00
return region->is_volatile(VirtualAddress(address), size) ? 0 : 1;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
KResultOr<int> Process::sys$set_mmap_name(Userspace<const Syscall::SC_set_mmap_name_params*> user_params)
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
{
REQUIRE_PROMISE(stdio);
Syscall::SC_set_mmap_name_params params;
Kernel: Make copy_to/from_user safe and remove unnecessary checks Since the CPU already does almost all necessary validation steps for us, we don't really need to attempt to do this. Doing it ourselves doesn't really work very reliably, because we'd have to account for other processors modifying virtual memory, and we'd have to account for e.g. pages not being able to be allocated due to insufficient resources. So change the copy_to/from_user (and associated helper functions) to use the new safe_memcpy, which will return whether it succeeded or not. The only manual validation step needed (which the CPU can't perform for us) is making sure the pointers provided by user mode aren't pointing to kernel mappings. To make it easier to read/write from/to either kernel or user mode data add the UserOrKernelBuffer helper class, which will internally either use copy_from/to_user or directly memcpy, or pass the data through directly using a temporary buffer on the stack. Last but not least we need to keep syscall params trivial as we need to copy them from/to user mode using copy_from/to_user.
2020-09-12 06:11:07 +03:00
if (!copy_from_user(&params, user_params))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (params.name.length > PATH_MAX)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENAMETOOLONG;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Use KString for Region names Replace the AK::String used for Region::m_name with a KString. This seems beneficial across the board, but as a specific data point, it reduces time spent in sys$set_mmap_name() by ~50% on test-js. :^)
2021-05-28 10:33:14 +03:00
auto name_or_error = try_copy_kstring_from_user(params.name);
if (name_or_error.is_error())
return name_or_error.error();
auto name = name_or_error.release_value();
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
auto range_or_error = expand_range_to_page_boundaries((FlatPtr)params.addr, params.size);
if (range_or_error.is_error())
return range_or_error.error();
auto range = range_or_error.value();
auto* region = space().find_region_from_range(range);
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (!region)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
if (!region->is_mmap())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Add PerformanceManager static class, move perf event APIs there The current method of emitting performance events requires a bit of boiler plate at every invocation, as well as having to ignore the return code which isn't used outside of the perf event syscall. This change attempts to clean that up by exposing high level API's that can be used around the code base.
2021-05-07 08:29:19 +03:00
Kernel: Make copy_to/from_user safe and remove unnecessary checks Since the CPU already does almost all necessary validation steps for us, we don't really need to attempt to do this. Doing it ourselves doesn't really work very reliably, because we'd have to account for other processors modifying virtual memory, and we'd have to account for e.g. pages not being able to be allocated due to insufficient resources. So change the copy_to/from_user (and associated helper functions) to use the new safe_memcpy, which will return whether it succeeded or not. The only manual validation step needed (which the CPU can't perform for us) is making sure the pointers provided by user mode aren't pointing to kernel mappings. To make it easier to read/write from/to either kernel or user mode data add the UserOrKernelBuffer helper class, which will internally either use copy_from/to_user or directly memcpy, or pass the data through directly using a temporary buffer on the stack. Last but not least we need to keep syscall params trivial as we need to copy them from/to user mode using copy_from/to_user.
2020-09-12 06:11:07 +03:00
region->set_name(move(name));
Kernel: Add PerformanceManager static class, move perf event APIs there The current method of emitting performance events requires a bit of boiler plate at every invocation, as well as having to ignore the return code which isn't used outside of the perf event syscall. This change attempts to clean that up by exposing high level API's that can be used around the code base.
2021-05-07 08:29:19 +03:00
PerformanceManager::add_mmap_perf_event(*this, *region);
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
return 0;
}
Kernel: Use Userspace<T> in sys${munmap,mprotect,madvise,msyscall}()
2021-03-01 17:53:33 +03:00
KResultOr<int> Process::sys$munmap(Userspace<void*> addr, size_t size)
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
{
REQUIRE_PROMISE(stdio);
Kernel: Move sys$munmap functionality into a helper method
2021-05-28 12:03:21 +03:00
auto result = space().unmap_mmap_range(VirtualAddress { addr }, size);
if (result.is_error())
return result;
Kernel: munmap multiple regions at a time This implements a fallback to munmap that unmaps multiple regions at a time, with splitting some when needed. The way it is implemented is possibly not optimal, due to it searching without looking into the cache
2021-02-24 19:00:58 +03:00
return 0;
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
KResultOr<FlatPtr> Process::sys$mremap(Userspace<const Syscall::SC_mremap_params*> user_params)
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
{
REQUIRE_PROMISE(stdio);
Kernel: Round old address/size in sys$mremap() to page size multiples Found by fuzz-syscalls. :^)
2021-02-14 15:14:25 +03:00
Syscall::SC_mremap_params params {};
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
if (!copy_from_user(&params, user_params))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
auto range_or_error = expand_range_to_page_boundaries((FlatPtr)params.old_address, params.old_size);
if (range_or_error.is_error())
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return range_or_error.error().error();
Kernel: Round old address/size in sys$mremap() to page size multiples Found by fuzz-syscalls. :^)
2021-02-14 15:14:25 +03:00
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
auto old_range = range_or_error.value();
Kernel: Round old address/size in sys$mremap() to page size multiples Found by fuzz-syscalls. :^)
2021-02-14 15:14:25 +03:00
Kernel: Factor out mmap & friends range expansion to a helper function sys$mmap() and related syscalls must pad to the nearest page boundary below the base address *and* above the end address of the specified range. Since we have to do this in many places, let's make a helper.
2021-02-18 20:04:58 +03:00
auto* old_region = space().find_region_from_range(old_range);
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
if (!old_region)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
if (!old_region->is_mmap())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
Kernel: Remove MAP_PURGEABLE from mmap This brings mmap more in line with other operating systems. Prior to this, it was impossible to request memory that was definitely committed, instead MAP_PURGEABLE would provide a region that was not actually purgeable, but also not fully committed, which meant that using such memory still could cause crashes when the underlying pages could no longer be allocated. This fixes some random crashes in low-memory situations where non-volatile memory is mapped (e.g. malloc, tls, Gfx::Bitmap, etc) but when a page in these regions is first accessed, there is insufficient physical memory available to commit a new page.
2020-09-04 20:47:49 +03:00
if (old_region->vmobject().is_shared_inode() && params.flags & MAP_PRIVATE && !(params.flags & (MAP_ANONYMOUS | MAP_NORESERVE))) {
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
auto range = old_region->range();
auto old_prot = region_access_flags_to_prot(old_region->access());
Kernel: Make sure the offset stays the same when using mremap() When using mmap() on a file with a non-zero offset subsequent calls to mremap() would incorrectly reset the offset to zero.
2021-04-13 20:27:49 +03:00
auto old_offset = old_region->offset_in_vmobject();
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
NonnullRefPtr inode = static_cast<SharedInodeVMObject&>(old_region->vmobject()).inode();
Kernel: Remove allocate_region() functions that don't take a Range Let's force callers to provide a VM range when allocating a region. This makes ENOMEM error handling more visible and removes implicit VM allocation which felt a bit magical.
2021-01-26 16:13:57 +03:00
Kernel: Make PrivateInodeVMObject factory APIs OOM safe
2021-05-28 13:18:07 +03:00
auto new_vmobject = PrivateInodeVMObject::create_with_inode(inode);
if (!new_vmobject)
return ENOMEM;
Kernel: Fix use-after-free in sys$mremap Now that Region::name() has been changed to return a StringView we can't rely on keeping a copy of the region's name past the region's destruction just by holding a copy of the StringView.
2021-06-02 12:58:56 +03:00
auto old_name = old_region->take_name();
Kernel: Remove allocate_region() functions that don't take a Range Let's force callers to provide a VM range when allocating a region. This makes ENOMEM error handling more visible and removes implicit VM allocation which felt a bit magical.
2021-01-26 16:13:57 +03:00
// Unmap without deallocating the VM range since we're going to reuse it.
old_region->unmap(Region::ShouldDeallocateVirtualMemoryRange::No);
Kernel: Remove old region from process' regions vector before splitting This does not affect functionality right now, but it means that the regions vector will now never have any overlapping regions, which will allow the use of balance binary search trees instead of a vector in the future. (since they require keys to be exclusive)
2021-04-07 02:17:05 +03:00
bool success = space().deallocate_region(*old_region);
VERIFY(success);
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
Kernel: Fix use-after-free in sys$mremap Now that Region::name() has been changed to return a StringView we can't rely on keeping a copy of the region's name past the region's destruction just by holding a copy of the StringView.
2021-06-02 12:58:56 +03:00
auto new_region_or_error = space().allocate_region_with_vmobject(range, new_vmobject.release_nonnull(), old_offset, old_name->view(), old_prot, false);
Kernel: Make Process::allocate_region*() return KResultOr<Region*> This allows region allocation to return specific errors and we don't have to assume every failure is an ENOMEM.
2021-01-15 19:27:52 +03:00
if (new_region_or_error.is_error())
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return new_region_or_error.error().error();
Kernel: Make Process::allocate_region*() return KResultOr<Region*> This allows region allocation to return specific errors and we don't have to assume every failure is an ENOMEM.
2021-01-15 19:27:52 +03:00
auto& new_region = *new_region_or_error.value();
new_region.set_mmap(true);
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return new_region.vaddr().get();
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
}
dbgln("sys$mremap: Unimplemented remap request (flags={})", params.flags);
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENOTIMPL;
Kernel+LibC: Add a very limited sys$mremap() implementation This syscall can currently only remap a shared file-backed mapping into a private file-backed mapping.
2020-12-29 04:11:47 +03:00
}
Kernel+LibELF: Support initializing values of TLS data Previously, TLS data was always zero-initialized. To support initializing the values of TLS data, sys$allocate_tls now receives a buffer with the desired initial data, and copies it to the master TLS region of the process. The DynamicLinker gathers the initial TLS image and passes it to sys$allocate_tls. We also now require the size passed to sys$allocate_tls to be page-aligned, to make things easier. Note that this doesn't waste memory as the TLS data has to be allocated in separate pages anyway.
2021-04-24 11:30:20 +03:00
KResultOr<FlatPtr> Process::sys$allocate_tls(Userspace<const char*> initial_data, size_t size)
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
{
REQUIRE_PROMISE(stdio);
Kernel+LibELF: Support initializing values of TLS data Previously, TLS data was always zero-initialized. To support initializing the values of TLS data, sys$allocate_tls now receives a buffer with the desired initial data, and copies it to the master TLS region of the process. The DynamicLinker gathers the initial TLS image and passes it to sys$allocate_tls. We also now require the size passed to sys$allocate_tls to be page-aligned, to make things easier. Note that this doesn't waste memory as the TLS data has to be allocated in separate pages anyway.
2021-04-24 11:30:20 +03:00
if (!size || size % PAGE_SIZE != 0)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
if (!m_master_tls_region.is_null())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EEXIST;
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
if (thread_count() != 1)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
Thread* main_thread = nullptr;
for_each_thread([&main_thread](auto& thread) {
main_thread = &thread;
return IterationDecision::Break;
});
Everywhere: Rename ASSERT => VERIFY (...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED) Since all of these checks are done in release builds as well, let's rename them to VERIFY to prevent confusion, as everyone is used to assertions being compiled out in release. We can introduce a new ASSERT macro that is specifically for debug checks, but I'm doing this wholesale conversion first since we've accumulated thousands of these already, and it's not immediately obvious which ones are suitable for ASSERT.
2021-02-23 22:42:32 +03:00
VERIFY(main_thread);
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
auto range = space().allocate_range({}, size);
Kernel: Remove Range "valid" state and use Optional<Range> instead It's easier to understand VM ranges if they are always valid. We can simply use an empty Optional<Range> to encode absence when needed.
2021-01-27 23:01:45 +03:00
if (!range.has_value())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return ENOMEM;
Kernel: Remove allocate_region() functions that don't take a Range Let's force callers to provide a VM range when allocating a region. This makes ENOMEM error handling more visible and removes implicit VM allocation which felt a bit magical.
2021-01-26 16:13:57 +03:00
Kernel: Give a name to the Master TLS region allocation
2021-04-23 18:59:30 +03:00
auto region_or_error = space().allocate_region(range.value(), String("Master TLS"), PROT_READ | PROT_WRITE);
Kernel: Make Process::allocate_region*() return KResultOr<Region*> This allows region allocation to return specific errors and we don't have to assume every failure is an ENOMEM.
2021-01-15 19:27:52 +03:00
if (region_or_error.is_error())
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return region_or_error.error().error();
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
Kernel: Make Process::allocate_region*() return KResultOr<Region*> This allows region allocation to return specific errors and we don't have to assume every failure is an ENOMEM.
2021-01-15 19:27:52 +03:00
m_master_tls_region = region_or_error.value()->make_weak_ptr();
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
m_master_tls_size = size;
m_master_tls_alignment = PAGE_SIZE;
Kernel+LibELF: Support initializing values of TLS data Previously, TLS data was always zero-initialized. To support initializing the values of TLS data, sys$allocate_tls now receives a buffer with the desired initial data, and copies it to the master TLS region of the process. The DynamicLinker gathers the initial TLS image and passes it to sys$allocate_tls. We also now require the size passed to sys$allocate_tls to be page-aligned, to make things easier. Note that this doesn't waste memory as the TLS data has to be allocated in separate pages anyway.
2021-04-24 11:30:20 +03:00
{
Kernel::SmapDisabler disabler;
void* fault_at;
if (!Kernel::safe_memcpy((char*)m_master_tls_region.unsafe_ptr()->vaddr().as_ptr(), (char*)initial_data.ptr(), size, fault_at))
return EFAULT;
}
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
auto tsr_result = main_thread->make_thread_specific_region({});
if (tsr_result.is_error())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
auto& tls_descriptor = Processor::current().get_gdt_entry(GDT_SELECTOR_TLS);
Kernel: Tighten some typing in Arch/i386/CPU.h Use more appropriate types for some things.
2021-02-25 12:29:41 +03:00
tls_descriptor.set_base(main_thread->thread_specific_data());
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
tls_descriptor.set_limit(main_thread->thread_specific_region_size());
Kernel: Take some baby steps towards x86_64 Make more of the kernel compile in 64-bit mode, and make some things pointer-size-agnostic (by using FlatPtr.) There's a lot of work to do here before the kernel will even compile.
2021-02-25 18:18:36 +03:00
return m_master_tls_region.unsafe_ptr()->vaddr().get();
Kernel: Support TLS allocation from userspace This adds an allocate_tls syscall through which a userspace process can request the allocation of a TLS region with a given size. This will be used by the dynamic loader to allocate TLS for the main executable & its libraries.
2020-10-10 12:17:07 +03:00
}
Kernel: Use Userspace<T> in sys${munmap,mprotect,madvise,msyscall}()
2021-03-01 17:53:33 +03:00
KResultOr<int> Process::sys$msyscall(Userspace<void*> address)
Kernel: Add a way to specify which memory regions can make syscalls This patch adds sys$msyscall() which is loosely based on an OpenBSD mechanism for preventing syscalls from non-blessed memory regions. It works similarly to pledge and unveil, you can call it as many times as you like, and when you're finished, you call it with a null pointer and it will stop accepting new regions from then on. If a syscall later happens and doesn't originate from one of the previously blessed regions, the kernel will simply crash the process.
2021-02-02 21:56:11 +03:00
{
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
if (space().enforces_syscall_regions())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EPERM;
Kernel: Add a way to specify which memory regions can make syscalls This patch adds sys$msyscall() which is loosely based on an OpenBSD mechanism for preventing syscalls from non-blessed memory regions. It works similarly to pledge and unveil, you can call it as many times as you like, and when you're finished, you call it with a null pointer and it will stop accepting new regions from then on. If a syscall later happens and doesn't originate from one of the previously blessed regions, the kernel will simply crash the process.
2021-02-02 21:56:11 +03:00
if (!address) {
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
space().set_enforces_syscall_regions(true);
Kernel: Add a way to specify which memory regions can make syscalls This patch adds sys$msyscall() which is loosely based on an OpenBSD mechanism for preventing syscalls from non-blessed memory regions. It works similarly to pledge and unveil, you can call it as many times as you like, and when you're finished, you call it with a null pointer and it will stop accepting new regions from then on. If a syscall later happens and doesn't originate from one of the previously blessed regions, the kernel will simply crash the process.
2021-02-02 21:56:11 +03:00
return 0;
}
Kernel: Make sys$msyscall() EFAULT on non-user address Fixes #5361.
2021-02-16 13:32:00 +03:00
if (!is_user_address(VirtualAddress { address }))
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EFAULT;
Kernel: Make sys$msyscall() EFAULT on non-user address Fixes #5361.
2021-02-16 13:32:00 +03:00
Kernel: Factor address space management out of the Process class This patch adds Space, a class representing a process's address space. - Each Process has a Space. - The Space owns the PageDirectory and all Regions in the Process. This allows us to reorganize sys$execve() so that it constructs and populates a new Space fully before committing to it. Previously, we would construct the new address space while still running in the old one, and encountering an error meant we had to do tedious and error-prone rollback. Those problems are now gone, replaced by what's hopefully a set of much smaller problems and missing cleanups. :^)
2021-02-08 17:45:40 +03:00
auto* region = space().find_region_containing(Range { VirtualAddress { address }, 1 });
Kernel: Add a way to specify which memory regions can make syscalls This patch adds sys$msyscall() which is loosely based on an OpenBSD mechanism for preventing syscalls from non-blessed memory regions. It works similarly to pledge and unveil, you can call it as many times as you like, and when you're finished, you call it with a null pointer and it will stop accepting new regions from then on. If a syscall later happens and doesn't originate from one of the previously blessed regions, the kernel will simply crash the process.
2021-02-02 21:56:11 +03:00
if (!region)
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Add a way to specify which memory regions can make syscalls This patch adds sys$msyscall() which is loosely based on an OpenBSD mechanism for preventing syscalls from non-blessed memory regions. It works similarly to pledge and unveil, you can call it as many times as you like, and when you're finished, you call it with a null pointer and it will stop accepting new regions from then on. If a syscall later happens and doesn't originate from one of the previously blessed regions, the kernel will simply crash the process.
2021-02-02 21:56:11 +03:00
Kernel: Don't allow sys$msyscall() on non-mmap regions
2021-02-02 22:16:13 +03:00
if (!region->is_mmap())
Kernel: Make all syscall functions return KResultOr<T> This makes it a lot easier to return errors since we no longer have to worry about negating EFOO errors and can just return them flat.
2021-03-01 15:49:16 +03:00
return EINVAL;
Kernel: Don't allow sys$msyscall() on non-mmap regions
2021-02-02 22:16:13 +03:00
Kernel: Add a way to specify which memory regions can make syscalls This patch adds sys$msyscall() which is loosely based on an OpenBSD mechanism for preventing syscalls from non-blessed memory regions. It works similarly to pledge and unveil, you can call it as many times as you like, and when you're finished, you call it with a null pointer and it will stop accepting new regions from then on. If a syscall later happens and doesn't originate from one of the previously blessed regions, the kernel will simply crash the process.
2021-02-02 21:56:11 +03:00
region->set_syscall_region(true);
return 0;
}
Kernel: Move syscall implementations out of Process.cpp This is something I've been meaning to do for a long time, and here we finally go. This patch moves all sys$foo functions out of Process.cpp and into files in Kernel/Syscalls/. It's not exactly one syscall per file (although it could be, but I got a bit tired of the repetitive work here..) This makes hacking on individual syscalls a lot less painful since you don't have to rebuild nearly as much code every time. I'm also hopeful that this makes it easier to understand individual syscalls. :^)
2020-07-31 00:38:15 +03:00
}
Reference in New Issue Copy Permalink