Since ByteBuffer is a Buffer, it should allow us to overwrite parts of
it that we have allocated.
This comes in useful in handling unsequenced writes like handling
fragmented ip packets :^)
With relative filenames in the executable code, the executable is basically not
relocatable. This makes out-of-source builds unneccesery hard. This patchset moves
the relative link into the filesystem: that can be handled much easier :^)
This patchsets adds the semantic check of two values. One first approach
was to compare the (generated) json strings of the two values. This works
out in the most cases, but not with numbers, where "1.0" and "1" in JSON
format are semantically the same. Therefore, this patch adds deep (recursive)
check of two JsonValues.
This adds a replace functionality that replaces a string that contains
occurences of a "needle" by a "replacement" value. With "all_occurences"
enabled, all occurences are being replaced, otherwise only the first
occurence is being replaced.
* Add double number to object serializer
* Handle negative double numbers correctly
* Handle \r and \n in quoted strings independently
This improves the situation when keys contain \r or \n that currently
has the effect that "a\rkey" and "a\nkey" in an JSON object are the
same key value.
This patchset allows double numbers to be printed with the printf function.
The fraction will always be printed as 6 digit number. This can be improved :^)
This commit adds a basic implementation of
the ptrace syscall, which allows one process
(the tracer) to control another process (the tracee).
While a process is being traced, it is stopped whenever a signal is
received (other than SIGCONT).
The tracer can start tracing another thread with PT_ATTACH,
which causes the tracee to stop.
From there, the tracer can use PT_CONTINUE
to continue the execution of the tracee,
or use other request codes (which haven't been implemented yet)
to modify the state of the tracee.
Additional request codes are PT_SYSCALL, which causes the tracee to
continue exection but stop at the next entry or exit from a syscall,
and PT_GETREGS which fethces the last saved register set of the tracee
(can be used to inspect syscall arguments and return value).
A special request code is PT_TRACE_ME, which is issued by the tracee
and causes it to stop when it calls execve and wait for the
tracer to attach.
This patch adds the parsing of double values to the JSON parser.
There is another char buffer that get's filled when a "." is present
in the number parsing. When number finished, a divider is calculated
to transform the number behind the "." to the actual fraction value.
FlyString is a flyweight string class that wraps a RefPtr<StringImpl>
known to be unique among the set of FlyStrings. The class is very
unoptimized at the moment.
When to use FlyString:
- When you want O(1) string comparison
- When you want to deduplicate a lot of identical strings
When not to use FlyString:
- For strings that don't need either of the above features
- For strings that are likely to be unique
Move the "fast memcpy" stuff out of StdLibExtras.h and into Memory.h.
This will break a ton of things that were relying on StdLibExtras.h
to include a bunch of other headers. Fix will follow immediately after.
This makes it possible to include StdLibExtras.h from Types.h, which is
the main point of this exercise.
We can use __builtin_memset() without including <string.h>.
This is pretty neat, as it will allow us to reduce the header deps
of AK templates a bit, if applied consistently.
Note that this is an enabling change for an upcoming #include removal.
This was causing some obvious-in-hindsight but hard to spot bugs where
we'd implicitly convert the bool to an integer type and carry on with
the number 1 instead of the actual value().
Instead of set(const JsonValue&) and set(JsonValue&&), just do
set(JsonValue) and let callers move() if they want. This removes some
ambiguity and the compiler is smart enough to optimize it anyway.
Now it actually defaults to "a < b" comparison, instead of forcing you
to provide a trivial less-than comparator. Also you can pass in any
collection type that has .begin() and .end() and we'll sort it for you.
Add an extra out-parameter to shbuf_get() that receives the size of the
shared buffer. That way we don't need to make a separate syscall to
get the size, which we always did immediately after.
This feels a lot more consistent and Unixy:
create_shared_buffer() => shbuf_create()
share_buffer_with() => shbuf_allow_pid()
share_buffer_globally() => shbuf_allow_all()
get_shared_buffer() => shbuf_get()
release_shared_buffer() => shbuf_release()
seal_shared_buffer() => shbuf_seal()
get_shared_buffer_size() => shbuf_get_size()
Also, "shared_buffer_id" is shortened to "shbuf_id" all around.
This commit replaces SinglyLinkedListIterator::universal_end() with an
empty SinglyLinkedListIterator(). Piano needs this in order to
initialize a member array of iterators without 84 lines of
universal_end().
This should make stuff like placement new work correctly when building
outside of Serenity. This stuff is a bit delicate due to the weirdly
staged toolchain build at the moment. Hopefully we can unify this stuff
in the future.
This allows RefPtr to be stored in a HashTable<RefPtr<T>> :^)
It's unfortunate about the const_casts. We'll need to fix HashMap::get
to play nice with non-const Traits<T>::PeekType at some point.
Weakable objects ended up with differing memory layouts in some ports
since they don't build with the DEBUG macro defined.
Instead of forcing ports to define DEBUG, just put this behind a custom
WEAKABLE_DEBUG macro and leave it always-on for now.
You can now #include <AK/Forward.h> to get most of the AK types as
forward declarations.
Header dependency explosion is one of the main contributors to compile
times at the moment, so this is a step towards smaller include graphs.
This was only used by HashTable::dump() which I used when doing the
first HashTable implementation. Removing this allows us to also remove
most includes of <AK/kstdio.h>.
Since BufferStream is about creating specific binary stream formats,
let's not have a flaky type like size_t in there. Instead, clients of
BufferStream can cast their size_t to the binary size they want to use.
Account for this in IPCCompiler by making String lengths always 32-bit.
Now that we're trying to be more portable, we can't only rely on using
i32/u32 and i64/u64 since different systems have different combinations
of int/long/long long and unsigned/unsigned long/unsigned long long.
This implementation uses the new helper method of Bitmap called
find_longest_range_of_unset_bits. This method looks for the biggest
range of contiguous bits unset in the bitmap and returns the start of
the range back to the caller.
Trying to make_weak_ptr() on something that has begun destruction is
very unlikely to be what you want. Let's assert if that scenario comes
up so we can catch it immediately.
This changes copyright holder to myself for the source code files that I've
created or have (almost) completely rewritten. Not included are the files
that were significantly changed by others even though it was me who originally
created them (think HtmlView), or the many other files I've contributed code to.
When using dbg() in the kernel, the output is automatically prefixed
with [Process(PID:TID)]. This makes it a lot easier to understand which
thread is generating the output.
This patch also cleans up some common logging messages and removes the
now-unnecessary "dbg() << *current << ..." pattern.
Previously, when deallocating a range of VM, we would sort and merge
the range list. This was quite slow for large processes.
This patch optimizes VM deallocation in the following ways:
- Use binary search instead of linear scan to find the place to insert
the deallocated range.
- Insert at the right place immediately, removing the need to sort.
- Merge the inserted range with any adjacent range(s) in-line instead
of doing a separate merge pass into a list copy.
- Add Traits<Range> to inform Vector that Range objects are trivial
and can be moved using memmove().
I've also added an assertion that deallocated ranges are actually part
of the RangeAllocator's initial address range.
I've benchmarked this using g++ to compile Kernel/Process.cpp.
With these changes, compilation goes from ~41 sec to ~35 sec.
The generic swap() is not able to swap a NonnullRefPtr with itself,
due to its use of a temporary and NonnullRefPtr asserting when trying
to move() from an already move()'d instance.
Given the following situation:
struct Object : public RefCounted<Object> {
RefPtr<Object> parent;
}
NonnullRefPtr<Object> object = get_some_object();
object = *object->parent;
We would previously crash if 'object' was the only strongly referencing
pointer to 'parent'. This happened because NonnullRefPtr would unref
the outgoing pointee before reffing the incoming pointee.
This patch fixes that by implementing NonnullRefPtr assignments using
pointer swaps, just like RefPtr already did.
As suggested by Joshua, this commit adds the 2-clause BSD license as a
comment block to the top of every source file.
For the first pass, I've just added myself for simplicity. I encourage
everyone to add themselves as copyright holders of any file they've
added or modified in some significant way. If I've added myself in
error somewhere, feel free to replace it with the appropriate copyright
holder instead.
Going forward, all new source files should include a license header.
It was possible to craft a custom ELF executable that when symbolicated
would cause the kernel to read from user-controlled addresses anywhere
in memory. You could then fetch this memory via /proc/PID/stack
We fix this by making ELFImage hand out StringView rather than raw
const char* for symbol names. In case a symbol offset is outside the
ELF image, you get a null StringView. :^)
Test: Kernel/elf-symbolication-kernel-read-exploit.cpp
This removes an item at an index without preserving the sort order of
the Vector.
This enables constant-time removal from unsorted Vectors, as it avoids
shifting all of the entries following the removed one.
If the last character was the separator and keep_empty is true, the
previous if statement would have already appended the last empty part,
so no need to do this again.
This was even more problematic, because the result of split_view() is
expected to consist of true substrings that are usable with the
StringView::substring_view_starting_*_substring() methods, not of
equal strings located elsewhere.
Fixes https://github.com/SerenityOS/serenity/issues/970
See https://github.com/SerenityOS/serenity/pull/938
This was tripping up CObject which interprets timer ID 0 as "no timer".
Once we got ID 0 assigned, it was impossible to turn it off and it
would fire on every event loop iteration, causing CPU churn.
This variant of get() returns a const JsonValue* instead of a JsonValue
and can be used when you want to peek into a JsonObject's member fields
without making copies.
Lock each directory before entering it so when using -j, the same
dependency isn't built more than once at a time.
This doesn't get full -j parallelism though, since one make child
will be sitting idle waiting for flock to receive its lock and
continue making (which should then do nothing since it will have
been built already). Unfortunately there's not much that can be
done to fix that since it can't proceed until its dependency is
built by another make process.
Allow everything to be built from the top level directory with just
'make', cleaned with 'make clean', and installed with 'make
install'. Also support these in any particular subdirectory.
Specifying 'make VERBOSE=1' will print each ld/g++/etc. command as
it runs.
Kernel and early host tools (IPCCompiler, etc.) are built as
object.host.o so that they don't conflict with other things built
with the cross-compiler.
Using int was a mistake. This patch changes String, StringImpl,
StringView and StringBuilder to use size_t instead of int for lengths.
Obviously a lot of code needs to change as a result of this.
This patch reduces the O(n) tab completion to something like O(log(n)).
The cache is just a sorted vector of strings and we binary search it to
get a string matching our input, and then check the surrounding strings
to see if we need to remove any characters. Also we no longer stat each
file every time.
Also added an #include in BinarySearch since it was using size_t. Oops.
If `export` is called, we recache. Need to implement the `hash` builtin
for when an executable has been added to a directory in PATH.
This is a special specifier that does not output anything to the stream,
but saves the number of already output chars to the provided pointer.
This is apparently used by GNU Nano.
binary_search takes a haystack, a size, a needle and a compare function.
The compare function should return negative if a < b, positive if a > b
and 0 if a == b. The "sane default" compare function is integral_compare
which implements this with subtraction a - b.
binary_search returns a pointer to a matching element, NOT necessarily
the FIRST matching element. It returns a nullptr if the element was not
found.
This patch includes tests for binary_search.
It's missing query string parsing from new URLs, but you can set the
query string programmatically, and it will be part of the URL when
serialized through to_string().
We were forgetting to adopt the WeakLink, causing a reference leak.
This ended up costing us one allocation per exec(), with this stack:
kmalloc_impl()
Inode::set_vmo()
InodeVMObject::create_with_inode()
Process::do_exec()
Process::exec()
Process::sys$execve()
This was a pain to track down, in the end I caught it by dumping out
every live kmalloc pointer between runs and diffing the sets. Then it
was just a matter of matching the pointer to a call stack and looking
at what went wrong. :^)
I'll be reconstructing parts of the VisualBuilder application here and
then we can retire VisualBuilder entirely once all the functionality
is available in HackStudio.
Since NonnullRefPtr and NonnullOwnPtr cannot be null, it is pointless
to convert them to a bool, since it would always be true.
This patch makes it an error to null-check one of these pointers.
It's too dang frustrating that we actually crash whenever we hit some
unimplemented printf specifier. Let's just log the whole format string
and carry on as best we can.
Add dedicated internal types for Int64 and UnsignedInt64. This makes it
a bit more straightforward to work with 64-bit numbers (instead of just
implicitly storing them as doubles.)
This is just a wrapper around strstr() for now. There are many better
ways to search for a string within a string, but I'm just adding a nice
API at the moment. :^)
Previously we would not run destructors for items in a CircularQueue,
which would lead to memory leaks.
This patch fixes that, and also adds a basic unit test for the class.
