This should make stuff like placement new work correctly when building
outside of Serenity. This stuff is a bit delicate due to the weirdly
staged toolchain build at the moment. Hopefully we can unify this stuff
in the future.
This allows RefPtr to be stored in a HashTable<RefPtr<T>> :^)
It's unfortunate about the const_casts. We'll need to fix HashMap::get
to play nice with non-const Traits<T>::PeekType at some point.
Weakable objects ended up with differing memory layouts in some ports
since they don't build with the DEBUG macro defined.
Instead of forcing ports to define DEBUG, just put this behind a custom
WEAKABLE_DEBUG macro and leave it always-on for now.
You can now #include <AK/Forward.h> to get most of the AK types as
forward declarations.
Header dependency explosion is one of the main contributors to compile
times at the moment, so this is a step towards smaller include graphs.
This was only used by HashTable::dump() which I used when doing the
first HashTable implementation. Removing this allows us to also remove
most includes of <AK/kstdio.h>.
Since BufferStream is about creating specific binary stream formats,
let's not have a flaky type like size_t in there. Instead, clients of
BufferStream can cast their size_t to the binary size they want to use.
Account for this in IPCCompiler by making String lengths always 32-bit.
Now that we're trying to be more portable, we can't only rely on using
i32/u32 and i64/u64 since different systems have different combinations
of int/long/long long and unsigned/unsigned long/unsigned long long.
This implementation uses the new helper method of Bitmap called
find_longest_range_of_unset_bits. This method looks for the biggest
range of contiguous bits unset in the bitmap and returns the start of
the range back to the caller.
Trying to make_weak_ptr() on something that has begun destruction is
very unlikely to be what you want. Let's assert if that scenario comes
up so we can catch it immediately.
This changes copyright holder to myself for the source code files that I've
created or have (almost) completely rewritten. Not included are the files
that were significantly changed by others even though it was me who originally
created them (think HtmlView), or the many other files I've contributed code to.
When using dbg() in the kernel, the output is automatically prefixed
with [Process(PID:TID)]. This makes it a lot easier to understand which
thread is generating the output.
This patch also cleans up some common logging messages and removes the
now-unnecessary "dbg() << *current << ..." pattern.
Previously, when deallocating a range of VM, we would sort and merge
the range list. This was quite slow for large processes.
This patch optimizes VM deallocation in the following ways:
- Use binary search instead of linear scan to find the place to insert
the deallocated range.
- Insert at the right place immediately, removing the need to sort.
- Merge the inserted range with any adjacent range(s) in-line instead
of doing a separate merge pass into a list copy.
- Add Traits<Range> to inform Vector that Range objects are trivial
and can be moved using memmove().
I've also added an assertion that deallocated ranges are actually part
of the RangeAllocator's initial address range.
I've benchmarked this using g++ to compile Kernel/Process.cpp.
With these changes, compilation goes from ~41 sec to ~35 sec.
The generic swap() is not able to swap a NonnullRefPtr with itself,
due to its use of a temporary and NonnullRefPtr asserting when trying
to move() from an already move()'d instance.
Given the following situation:
struct Object : public RefCounted<Object> {
RefPtr<Object> parent;
}
NonnullRefPtr<Object> object = get_some_object();
object = *object->parent;
We would previously crash if 'object' was the only strongly referencing
pointer to 'parent'. This happened because NonnullRefPtr would unref
the outgoing pointee before reffing the incoming pointee.
This patch fixes that by implementing NonnullRefPtr assignments using
pointer swaps, just like RefPtr already did.
As suggested by Joshua, this commit adds the 2-clause BSD license as a
comment block to the top of every source file.
For the first pass, I've just added myself for simplicity. I encourage
everyone to add themselves as copyright holders of any file they've
added or modified in some significant way. If I've added myself in
error somewhere, feel free to replace it with the appropriate copyright
holder instead.
Going forward, all new source files should include a license header.
It was possible to craft a custom ELF executable that when symbolicated
would cause the kernel to read from user-controlled addresses anywhere
in memory. You could then fetch this memory via /proc/PID/stack
We fix this by making ELFImage hand out StringView rather than raw
const char* for symbol names. In case a symbol offset is outside the
ELF image, you get a null StringView. :^)
Test: Kernel/elf-symbolication-kernel-read-exploit.cpp
This removes an item at an index without preserving the sort order of
the Vector.
This enables constant-time removal from unsorted Vectors, as it avoids
shifting all of the entries following the removed one.
If the last character was the separator and keep_empty is true, the
previous if statement would have already appended the last empty part,
so no need to do this again.
This was even more problematic, because the result of split_view() is
expected to consist of true substrings that are usable with the
StringView::substring_view_starting_*_substring() methods, not of
equal strings located elsewhere.
Fixes https://github.com/SerenityOS/serenity/issues/970
See https://github.com/SerenityOS/serenity/pull/938
This was tripping up CObject which interprets timer ID 0 as "no timer".
Once we got ID 0 assigned, it was impossible to turn it off and it
would fire on every event loop iteration, causing CPU churn.
This variant of get() returns a const JsonValue* instead of a JsonValue
and can be used when you want to peek into a JsonObject's member fields
without making copies.
Lock each directory before entering it so when using -j, the same
dependency isn't built more than once at a time.
This doesn't get full -j parallelism though, since one make child
will be sitting idle waiting for flock to receive its lock and
continue making (which should then do nothing since it will have
been built already). Unfortunately there's not much that can be
done to fix that since it can't proceed until its dependency is
built by another make process.
Allow everything to be built from the top level directory with just
'make', cleaned with 'make clean', and installed with 'make
install'. Also support these in any particular subdirectory.
Specifying 'make VERBOSE=1' will print each ld/g++/etc. command as
it runs.
Kernel and early host tools (IPCCompiler, etc.) are built as
object.host.o so that they don't conflict with other things built
with the cross-compiler.
Using int was a mistake. This patch changes String, StringImpl,
StringView and StringBuilder to use size_t instead of int for lengths.
Obviously a lot of code needs to change as a result of this.
This patch reduces the O(n) tab completion to something like O(log(n)).
The cache is just a sorted vector of strings and we binary search it to
get a string matching our input, and then check the surrounding strings
to see if we need to remove any characters. Also we no longer stat each
file every time.
Also added an #include in BinarySearch since it was using size_t. Oops.
If `export` is called, we recache. Need to implement the `hash` builtin
for when an executable has been added to a directory in PATH.
This is a special specifier that does not output anything to the stream,
but saves the number of already output chars to the provided pointer.
This is apparently used by GNU Nano.
binary_search takes a haystack, a size, a needle and a compare function.
The compare function should return negative if a < b, positive if a > b
and 0 if a == b. The "sane default" compare function is integral_compare
which implements this with subtraction a - b.
binary_search returns a pointer to a matching element, NOT necessarily
the FIRST matching element. It returns a nullptr if the element was not
found.
This patch includes tests for binary_search.
It's missing query string parsing from new URLs, but you can set the
query string programmatically, and it will be part of the URL when
serialized through to_string().
We were forgetting to adopt the WeakLink, causing a reference leak.
This ended up costing us one allocation per exec(), with this stack:
kmalloc_impl()
Inode::set_vmo()
InodeVMObject::create_with_inode()
Process::do_exec()
Process::exec()
Process::sys$execve()
This was a pain to track down, in the end I caught it by dumping out
every live kmalloc pointer between runs and diffing the sets. Then it
was just a matter of matching the pointer to a call stack and looking
at what went wrong. :^)
I'll be reconstructing parts of the VisualBuilder application here and
then we can retire VisualBuilder entirely once all the functionality
is available in HackStudio.
Since NonnullRefPtr and NonnullOwnPtr cannot be null, it is pointless
to convert them to a bool, since it would always be true.
This patch makes it an error to null-check one of these pointers.
It's too dang frustrating that we actually crash whenever we hit some
unimplemented printf specifier. Let's just log the whole format string
and carry on as best we can.
Add dedicated internal types for Int64 and UnsignedInt64. This makes it
a bit more straightforward to work with 64-bit numbers (instead of just
implicitly storing them as doubles.)
This is just a wrapper around strstr() for now. There are many better
ways to search for a string within a string, but I'm just adding a nice
API at the moment. :^)
Previously we would not run destructors for items in a CircularQueue,
which would lead to memory leaks.
This patch fixes that, and also adds a basic unit test for the class.
ELFLoader::layout() had a "failed" variable that was never set. This
patch checks the return value of each hook (alloc/map section and tls)
and fails the load if they return null.
I also needed to patch Process so that the alloc_section_hook and
map_section_hook actually return nullptr when allocating a region fails.
Fixes#664 :)
This class inherits from CircularQueue and adds the ability dequeue
from the end of the queue using dequeue_end().
Note that I had to make some of CircularQueue's fields protected to
properly implement dequeue_end.
This kind of thing is a bit annoying. On Serenity, size_t is the same
size as u32, but not the same type. Because of "long" or whatever.
This patch makes String not complain about duplicate overloads.
The former allows you to inspect the string while it's being built.
It's an explicit method rather than `operator StringView()` because
you must remember you can only look at it in between modifications;
appending to the StringBuilder invalidates the StringView.
The latter lets you clear the state of a StringBuilder explicitly, to
start from an empty string again.
This simplifies the ownership model and makes Region easier to reason
about. Userspace Regions are now primarily kept by Process::m_regions.
Kernel Regions are kept in various OwnPtr<Regions>'s.
Regions now only ever get unmapped when they are destroyed.
`AK::String` can now be reversed via AK::String::reverse(). This makes
life a lot easier for functions like `itoa()`, where the output
ends up being backwards. Very much not like the normal STL
(which requires an `std::reverse` object) way of doing things.
A call to reverse returns a new `AK::String` so as to not upset any
of the possible references to the same `StringImpl` shared between
Strings.
The old implementation tried to move forward as long as the current
byte looks like a UTF-8 character continuation byte (has its two
most significant bits set to 10). This is correct as long as we assume
the string is actually valid UTF-8, which we do (we also have a separate
method that can check whether it is the case).
We can't, however, assume that the data after the end of our string
is also valid UTF-8 (in fact, we're not even allowed to look at data
outside out string, but it happens to a valid memory region most of
the time). If the byte after the end of our string also has its most
significant bits set to 10, we would move one byte forward, and then
fail the m_length > 0 assertion.
One way to fix this would be to add a length check inside the loop
condition. The other one, implemented in this commit, is to reimplement
the whole function in terms of decode_first_byte(), which gives us
the length as encoded in the first byte. This also brings it more
in line with the other functions around it that do UTF-8 decoding.
This patch adds support for TLS according to the x86 System V ABI.
Each thread gets a thread-specific memory region, and the GS segment
register always points _to a pointer_ to the thread-specific memory.
In other words, to access thread-local variables, userspace programs
start by dereferencing the pointer at [gs:0].
The Process keeps a master copy of the TLS segment that new threads
should use, and when a new thread is created, they get a copy of it.
It's basically whatever the PT_TLS program header in the ELF says.
This was a workaround to be able to build on case-insensitive file
systems where it might get confused about <string.h> vs <String.h>.
Let's just not support building that way, so String.h can have an
objectively nicer name. :^)
Passing these through the generic JsonValue path was causing us to
instantiate temporary JsonValues that incurred a heap allocation.
This avoids that by adding specialized overloads for string types.
Right now if we encounter an unknown character, printf (and its related
functions) fail in a really bad way, where they forget to pull things off
the stack. This usually leads to a crash somewhere else, which is hard to
debug.
This patch makes printf abort as soon as it encounters a formatting
character that it can't handle. This is not the optimal solution, but it
is an improvement for debugging.
Utf8View wraps a StringView and implements begin() and end() that
return a Utf8CodepointIterator, which parses UTF-8-encoded Unicode
codepoints and returns them as 32-bit integers.
This is the first step towards supporting emojis in Serenity ^)
https://github.com/SerenityOS/serenity/issues/490
When printing hex numbers, we were printing the wrong thing sometimes. This
was because we were dividing the digit to print by 15 instead of 16. Also,
dividing by 16 is the same as shifting four bits to the right, which is a
bit closer to our actual intention in this case, so let's use a shift
instead.
This way, primitive JsonValue serialization is still handled by
JsonValue::serialize(), but JsonArray and JsonObject serialization
always goes through serializer classes. This is no less efficient
if you have the whole JSON in memory already.
