Commit Graph

448 Commits

Author SHA1 Message Date
Andreas Kling
216e21a1fa AK: Convert AK::Format formatting helpers to returning ErrorOr<void>
This isn't a complete conversion to ErrorOr<void>, but a good chunk.
The end goal here is to propagate buffer allocation failures to the
caller, and allow the use of TRY() with formatting functions.
2021-11-17 00:21:13 +01:00
Andreas Kling
587f9af960 AK: Make JSON parser return ErrorOr<JsonValue> (instead of Optional)
Also add slightly richer parse errors now that we can include a string
literal with returned errors.

This will allow us to use TRY() when working with JSON data.
2021-11-17 00:21:10 +01:00
Linus Groh
58c6a156bf LibCrypto: Fix subtracting two negative SignedBigIntegers
Currently, we get the following results

    -1 - -2 = -1
    -2 - -1 =  1

Correct would be:

    -1 - -2 =  1
    -2 - -1 = -1

This was already attempted to be fixed in 7ed8970, but that change was
incorrect. This directly translates to LibJS BigInts having the same
incorrect behavior - it even was tested.
2021-11-16 10:06:53 +00:00
Tim Schumacher
8417c0fb1e Tests/LibRegex: Add tests for line end anchors in PosixBasic 2021-11-13 15:06:52 +03:30
Andreas Kling
09780ba7a9 Tests/LibGfx: Actually test image decoders in TestImageDecoder
We were passing raw Gfx::Bitmap objects into the various image decoders
instead of encoded image data. This made all of them fail, but the test
expectations were set up in a way that aligned with this outcome.

With this patch, we now test the codecs for real. Except ICO, since we
don't have an ICO file handy. That's a FIXME.
2021-11-11 11:20:58 +01:00
Andreas Kling
8b1108e485 Everywhere: Pass AK::StringView by value 2021-11-11 01:27:46 +01:00
Andreas Kling
a15ed8743d AK: Make ByteBuffer::try_* functions return ErrorOr<void>
Same as Vector, ByteBuffer now also signals allocation failure by
returning an ENOMEM Error instead of a bool, allowing us to use the
TRY() and MUST() patterns.
2021-11-10 21:58:58 +01:00
Andreas Kling
5f7d008791 AK+Everywhere: Stop including Vector.h from StringView.h
Preparation for using Error.h from Vector.h. This required moving some
things out of line.
2021-11-10 21:58:58 +01:00
Jan de Visser
3425730294 LibSQL: Implement table joins
This patch introduces table joins. It uses a pretty dumb algorithm-
starting with a singleton '__unity__' row consisting of a single boolean
value, a cartesian product of all tables in the 'FROM' clause is built.
This cartesian product is then filtered through the 'WHERE' clause,
again without any smarts just using brute force.

This patch required a bunch of busy work to allow for example the
ColumnNameExpression having to deal with multiple tables potentially
having columns with the same name.
2021-11-10 14:47:49 +01:00
Jan de Visser
1c50e9aadc LibSQL: Add current statement to the ExecutionContext
Because SQL is the craptastic language that it is, sometimes expressions
need to know details about the calling statement. For example the tables
in the 'FROM' clause may be needed to determine which columns are
referenced in 'WHERE' expressions. So the current statement is added
to the ExecutionContext and a new 'execute' overload on Statement is
created which takes the Database and the Statement and builds an
ExecutionContaxt from those.
2021-11-10 14:47:49 +01:00
Jan de Visser
7ea54db430 LibSQL: Add 'schema' and 'table' to TupleElementDescriptor
These are needed to distinguish columns from different tables with the
same column name in one and the same (joined) Tuple. Not quite happy
yet with this API; I think some sort of hierarchical structure would be
better but we'll burn that bridge when we get there :^)
2021-11-10 14:47:49 +01:00
Timothy Flynn
357c97dfa8 LibUnicode: Parse the CLDR's defaultContent.json locale list
This file contains the list of locales which default to their parent
locale's values. In the core CLDR dataset, these locales have their own
files, but they are empty (except for identity data). For example:

https://github.com/unicode-org/cldr/blob/main/common/main/en_US.xml

In the JSON export, these files are excluded, so we currently are not
recognizing these locales just by iterating the locale files.

This is a prerequisite for upgrading to CLDR version 40. One of these
default-content locales is the popular "en-US" locale, which defaults to
"en" values. We were previously inferring the existence of this locale
from the "en-US-POSIX" locale (many implementations, including ours,
strip variants such as POSIX). However, v40 removes the "en-US-POSIX"
locale entirely, meaning that without this change, we wouldn't know that
"en-US" exists (we would default to "en").

For more detail on this and other v40 changes, see:
https://cldr.unicode.org/index/downloads/cldr-40#h.nssoo2lq3cba
2021-11-09 20:44:52 +01:00
Andreas Kling
a7f1f1c34b LibCore: Use ErrorOr<T> for Core::File::open() 2021-11-08 00:35:27 +01:00
Andreas Kling
0de33b3d6c LibGfx: Use ErrorOr<T> for Bitmap::try_create()
Another one that was used in a fajillion places.
2021-11-08 00:35:27 +01:00
Andreas Kling
a54be656ae LibRegex: Don't push LibRegex's "Error" into the global namespace 2021-11-08 00:35:27 +01:00
Idan Horowitz
390a04a985 LibJS: Convert the GetValue AO to ThrowCompletionOr 2021-11-02 19:48:35 +01:00
Idan Horowitz
d73b258874 LibJS: Convert reference deletion to ThrowCompletionOr 2021-11-02 19:48:35 +01:00
Ben Wiederhake
f20a42e871 Kernel: Write test that crashes ProcFS 2021-10-31 18:44:12 +01:00
Idan Horowitz
10b93506ad Tests: Convert test-wasm functions to ThrowCompletionOr 2021-10-31 18:20:37 +02:00
Daniel Bertalan
fed9cb5d2d AK+Tests: Fix formatting of infinity and NaN values
When I added this code in 1472f6d, I forgot to add tests for it. That's
why I didn't realize that the values were appended to the wrong
FormatBuilder object, so an empty string was returned instead of the
expected "nan"/"inf". This made debugging some FPU issues with the
ScummVM port significantly more difficult.
2021-10-31 12:15:34 +01:00
Ali Mohammad Pur
ac856cb965 LibRegex: Don't ignore empty alternatives in append_alternation()
Doing so would cause patterns like `(a|)` to not match the empty string.
2021-10-29 15:57:59 +02:00
Liav A
8554952690 Kernel + WindowServer: Re-define the interface to framebuffer devices
We create a base class called GenericFramebufferDevice, which defines
all the virtual functions that must be implemented by a
FramebufferDevice. Then, we make the VirtIO FramebufferDevice and other
FramebufferDevice implementations inherit from it.
The most important consequence of rearranging the classes is that we now
have one IOCTL method, so all drivers should be committed to not
override the IOCTL method or make their own IOCTLs of FramebufferDevice.
All graphical IOCTLs are known to all FramebufferDevices, and it's up to
the specific implementation whether to support them or discard them (so
we require extensive usage of KResult and KResultOr, together with
virtual characteristic functions).
As a result, the interface is much cleaner and understandable to read.
2021-10-27 07:57:44 +03:00
Jan de Visser
7496f17620 LibSQL Tests: Add tests for SELECT ... WHERE ... 2021-10-25 12:59:42 +02:00
Jelle Raaijmakers
a44978b9b0 LibC: Fix %n conversion specifier in scanf() format
Also add a test to prevent this from happening again. There were two
bugs:

* The number of bytes just after processing the last value was written,
  instead of the number of bytes after skipping remaining whitespace.
  Confirmed by testing against GNU's `scanf()` since the man page
  leaves something to be desired.

* The number of bytes was written to the wrong variable argument; i.e.
  the first argument was overwritten.
2021-10-24 22:43:27 -07:00
Jelle Raaijmakers
00f36fc5ae Tests: Print full 32-byte range of values in TestScanf
We are trying to show 8 u32 values, each of which needs at most 8
hexadecimal characters to be shown entirely.
2021-10-24 22:43:27 -07:00
Jelle Raaijmakers
e71e9de61f Tests: Reword 'output' to 'return value' in TestScanf 2021-10-24 22:43:27 -07:00
Jelle Raaijmakers
e3f17401cb Tests: Use correct argument count for value conformance in TestScanf 2021-10-24 22:43:27 -07:00
Tim Schumacher
65bcee3c62 Tests: Only test truthiness for iswctype 2021-10-24 22:40:11 -07:00
Tim Schumacher
9e3e4a692d Tests: Use proper comparison macros for wctype 2021-10-24 22:40:11 -07:00
Ben Wiederhake
cb868cfa41 AK+Everywhere: Make Base64 decoding fallible 2021-10-23 19:16:40 +01:00
Ben Wiederhake
3bf1f7ae87 AK: Don't crash on invalid Base64 input
In the long-term, we should probably have a way to signal decoding
failure. For now, it should suffice to at least not crash. This is
particularly relevant because apparently this can be triggered while
parsing a PEM certificate, which happens during every TLS connection.

Found by OSS Fuzz
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=38979
2021-10-23 19:16:40 +01:00
Tim Schumacher
79bcfa967b LibC: Fix up mblen 2021-10-22 13:28:56 -07:00
Tim Schumacher
8df6955838 LibC: Fix up mbtowc
One more proper implementation and one less FIXME.
2021-10-22 13:28:56 -07:00
Liav A
cf0dbc9069 Tests: Add a unit test to ensure the /dev/mem device works correctly
To ensure everything works as expected, a unit test was added with
multiple scenarios.
This binary has to have the SetUID flag, and we also bind-mount the
/usr/Tests directory to allow running of SetUID binaries.
2021-10-22 13:13:00 +02:00
Tim Schumacher
89afd4d063 LibC: Implement mbsnrtowcs 2021-10-21 23:57:32 -07:00
Tim Schumacher
552ae77f0d LibC: Implement wcsnrtombs 2021-10-21 23:57:32 -07:00
Tim Schumacher
e618602433 LibC: Implement mbrlen 2021-10-21 23:47:20 -07:00
Idan Horowitz
44555eb50a LibJS: Convert test-js/test-web/test-wasm to ThrowCompletionOr 2021-10-20 12:27:19 +01:00
Idan Horowitz
40eb3a39d4 LibJS: Rename define_native_function => define_old_native_function
This method will eventually be removed once all native functions are
converted to ThrowCompletionOr
2021-10-20 12:27:19 +01:00
Idan Horowitz
20163c0584 LibJS: Add ThrowCompletionOr versions of the JS native function macros
The old versions were renamed to JS_DECLARE_OLD_NATIVE_FUNCTION and
JS_DEFINE_OLD_NATIVE_FUNCTION, and will be eventually removed once all
native functions were converted to the new format.
2021-10-20 12:27:19 +01:00
Daniel Bertalan
48687c3fbb Tests: Remove Clang workaround from TestSourceLocation
Clang 13 now correctly handles `__builtin_FILE()` and
`-ffile-prefix-map` being specified together, so this test should fully
pass.
2021-10-17 17:09:58 +01:00
Daniel Bertalan
13e6d9d71a LibC: Implement wcslcpy 2021-10-17 17:09:58 +01:00
Idan Horowitz
1639ed7e0a LibJS: Convert to_double() to ThrowCompletionOr 2021-10-17 12:12:35 +01:00
Idan Horowitz
df181809fd LibJS: Convert to_bigint_int64() to ThrowCompletionOr 2021-10-17 12:12:35 +01:00
Tim Schumacher
420bdccf0b LibC: Implement mbsrtowcs 2021-10-15 21:50:19 -07:00
Tim Schumacher
b0babd062e LibC: Implement wcsrtombs 2021-10-15 21:50:19 -07:00
Daniel Bertalan
c8367df746 LibC: Implement wcrtomb
This function converts a single wide character into its multibyte
representation (UTF-8 in our case). It is called from libc++'s
`std::basic_ostream<wchar_t>::flush`, which gets called at program exit
from a global destructor in order to flush `std::wcout`.
2021-10-15 21:50:19 -07:00
Tim Schumacher
4b423a5ec7 LibC: Implement twalk 2021-10-15 21:50:19 -07:00
Tim Schumacher
7448626bae LibC: Implement tfind and tsearch 2021-10-15 21:50:19 -07:00
Linus Groh
52976bfac6 LibJS: Convert to_object() to ThrowCompletionOr 2021-10-13 09:55:10 +01:00
Linus Groh
4d8912a92b LibJS: Convert to_string() to ThrowCompletionOr
Also update get_function_name() to use ThrowCompletionOr, but this is
not a standard AO and should be refactored out of existence eventually.
2021-10-13 09:55:10 +01:00
Ben Wiederhake
50ad294527 AK: Implement a way to resolve relative paths lexically 2021-10-10 15:18:55 -07:00
Nico Weber
96666f3209 Tests: Fix -Wunreachable-code warnings from clang 2021-10-08 23:33:46 +02:00
Andreas Kling
fc36f830ae Tests: Disable LibThreading detach tests for now
These tests don't pass (at least not without KVM) at the moment,
so let's not run them on CI.
2021-10-06 19:21:35 +02:00
Junior Rantila
1552873275 Tests: Add LibThreading to CMakeLists.txt 2021-10-06 19:05:39 +02:00
Rodrigo Tobar
4b091a7cc2 LibELF: Fix dynamic linking of dlopen()-ed libs
Consider the situation where two shared libraries libA and libB, both
depending (as in having a NEEDED dtag) on libC. libA is first
dlopen()-ed, which produces libC to be mapped and linked. When libB is
dlopen()-ed the DynamicLinker would re-map and re-link libC though,
causing any previous references to its old location to be invalid. And
if libA's PLT has been patched to point to libC's symbols, then any
further invocations to libA will cause the code to jump to a virtual
address that isn't mapped anymore, therefore causing a crash. This
situation was reported in #10014, although the setup was more convolved
in the ticket.

This commit fixes the issue by distinguishing between a main program
loading being performed by Loader.so, and a dlopen() call. The main
difference between these two cases is that in the former the
s_globals_objects maps is always empty, while in the latter it might
already contain dependencies for the library being dlopen()-ed. Hence,
when collecting dependencies to map and link, dlopen() should skip those
that are present in the global map to avoid the issue described above.

With this patch the original issue seen in #10014 is gone, with all
python3 modules (so far) loading correctly.

A unit test reproducing a simplified issue is also included in this
commit. The unit test includes the building of two dynamic libraries A
and B with both depending on libline.so (and B also depending on A); the
test then dlopen()s libA, invokes one its function, then does the same
with libB.
2021-10-06 12:33:21 +02:00
Mahmoud Mandour
0e5b2c923d LibSQL: Add an INSERT without column names test
This adds a passing test of an insert statement that contains no column
names and assumes full tuple input
2021-10-04 15:51:48 +02:00
Mahmoud Mandour
235573f7ba LibSQL: Test INSERT statement with wrong number of values 2021-10-04 15:51:48 +02:00
Mahmoud Mandour
4df85840c3 LibSQL: Test INSERT statement with wrong data types 2021-10-04 15:51:48 +02:00
Mahmoud Mandour
6065383e05 LibSQL: Check NoError individually in execution tests
This enables tests to check that a statement is erroneous, we could not
do so by only calling `execute()` since it asserted that no errors
occurred.
2021-10-04 15:51:48 +02:00
Tim Schumacher
7af7fc8c16 Everywhere: Fix more Copyright header inconsistencies 2021-10-04 11:10:09 +01:00
Ali Mohammad Pur
8f722302d9 LibRegex: Use a match table for character classes
Generate a sorted, compressed series of ranges in a match table for
character classes, and use a binary search to find the matches.
This is about a 3-4x speedup for character class match performance. :^)
2021-10-03 19:16:36 +02:00
davidot
ac2c3a73b1 LibJS: Add a specific test for invalid unicode characters in the lexer
Also fixes that it tried to make substrings past the end of the source
if we overran the source length.
2021-10-03 17:42:05 +02:00
Tim Schumacher
4302e7ac26 Tests: Add tests for mbrtowc 2021-10-03 11:13:50 +00:00
Tim Schumacher
67a579aab0 AK: Add a basic formatter for wchar_t 2021-10-03 11:13:50 +00:00
Tim Schumacher
e7f99edefa Tests: Add a test for mbsinit 2021-10-03 11:13:50 +00:00
Tim Schumacher
05b283f552 LibC: Implement wmemmove 2021-10-03 05:28:51 +00:00
Tim Schumacher
fa1208edfd LibC: Implement wmemset 2021-10-03 05:28:51 +00:00
Tim Schumacher
485c0ef691 LibC: Implement wmemcpy 2021-10-03 05:28:51 +00:00
Tim Schumacher
0ca1df4dc6 LibC: Implement wmemchr 2021-10-03 05:28:51 +00:00
Tim Schumacher
5ac2e84264 LibC: Implement wcsstr 2021-10-03 05:28:51 +00:00
Tim Schumacher
1b078f87b7 LibC: Implement wcspbrk 2021-10-03 05:28:51 +00:00
Liav A
4974727dbb Kernel: Move x86 IO instructions code into the x86 specific folder 2021-10-01 12:27:20 +02:00
Nico Weber
841a5fe81b Tests: Fix typos 2021-10-01 01:33:43 +01:00
davidot
ce3f29a135 LibJS + test-js: Get results from the global object directly
This is as the spec would require you to do it and necessary for changes
to come in the following commits.
2021-09-30 08:16:32 +01:00
Rodrigo Tobar
a8fae3d2c8 LibPthread: Add first test cases for RWlock 2021-09-28 18:36:20 +03:30
Andreas Kling
f67648f872 LibWeb: Rename HTMLDocumentParser => HTMLParser 2021-09-25 23:36:43 +02:00
Ben Wiederhake
743470c8f2 AK: Introduce ability to default-initialize a Variant
I noticed that Variant<Empty, …> is a somewhat common pattern while
working on #10080, and this will simplify a few use-cases. :^)
2021-09-21 04:22:52 +04:30
Ali Mohammad Pur
436693c0c9 LibTLS: Use a setter for on_tls_ready_to_write with some more smarts
The callback should be called as soon as the connection is established,
and if we actually set the callback when it already is, we expect it to
be called immediately.
2021-09-19 21:10:23 +04:30
thankyouverycool
41ce6d0cd0 Tests: Conform font tests to new font format 2021-09-19 00:58:59 +02:00
Andreas Kling
1be4cbd639 AK: Make Utf8View constructors inline and remove C string constructor
Using StringView instead of C strings is basically always preferable.
The only reason to use a C string is because you are calling a C API.
2021-09-18 19:54:24 +02:00
Tim Schumacher
b8c756a53a LibC: Primitively implement wcscoll
At the moment, sorting like LC_COLLATE=C would do is better than
nothing.
2021-09-18 02:57:56 +00:00
Tim Schumacher
42031f026a LibC: Implement towctrans 2021-09-17 22:59:51 +00:00
Tim Schumacher
8c7b566629 LibC: Implement iswctype 2021-09-17 22:59:51 +00:00
Tim Schumacher
ff0ab8b9a9 LibC: Implement wctrans 2021-09-17 22:59:51 +00:00
Tim Schumacher
7b17230d7a LibC: Implement wctype 2021-09-17 22:59:51 +00:00
Ben Wiederhake
5f0f0ac413 crash: Don't test for qemu-unsupported feature
See #10042 for details. In short: qemu doesn't seem to implement that
feature, therefore the test correctly fails. However, that does not help
us, so we skip that test.
2021-09-16 20:51:24 +00:00
Ben Wiederhake
c680ef0a09 crash: Run automatically during CI 2021-09-16 20:51:24 +00:00
Andrew Kaster
6e7cc40b18 Meta: Add Meta/CMake to the CMAKE_MODULE_PATH for Serenity and Lagom
This makes it so we don't need to specify the full path to all the
helper scripts we include() from different places in the codebase and
feels a lot cleaner.
2021-09-15 19:04:52 +04:30
Ali Mohammad Pur
741886a4c4 LibRegex: Make the optimizer understand references and capture groups
Otherwise the fork in patterns like `(1+)\1` would be (incorrectly)
optimized away.
2021-09-15 15:52:28 +04:30
Andreas Kling
0a09eaf3a1 LibJS+LibTest: Use JS::Script and JS::SourceTextModule in test-js
Instead of creating a Parser and Lexer manually in test-js, we now
use either JS::Script::parse() or JS::SourceTextModule::parse()
to load tests.
2021-09-14 21:41:51 +02:00
Ali Mohammad Pur
ccb53c64e9 AK: Add an abstraction over multiple disjoint buffers
DisjointChunks<T> provides a nice interface over multiple sequential
Vector<T>'s, allowing the user to iterate over/index into/slice from
said buffers as if they were a single contiguous buffer.
To work with views on such objects, DisjointSpans<T> is provided, which
has the same behaviour but does not own the underlying objects.
2021-09-14 21:33:15 +04:30
Idan Horowitz
d6cfa34667 AK: Make URL::m_port an Optional<u16>, Expose raw port getter
Our current way of signalling a missing port with m_port == 0 was
lacking, as 0 is a valid port number in URLs.
2021-09-14 00:14:45 +02:00
Ali Mohammad Pur
246ab432ff LibRegex: Add a basic optimization pass
This currently tries to convert forking loops to atomic groups, and
unify the left side of alternations.
2021-09-13 14:38:53 +04:30
Timothy Flynn
470262c8ab LibJS: Use ErrorType::NotAnObjectOfType instead of NotA 2021-09-12 00:16:39 +02:00
Timothy Flynn
c59b97043e LibWeb: Use ErrorType::NotAnObjectOfType instead of NotA 2021-09-12 00:16:39 +02:00
Idan Horowitz
6704961c82 AK: Replace the mutable String::replace API with an immutable version
This removes the awkward String::replace API which was the only String
API which mutated the String and replaces it with a new immutable
version that returns a new String with the replacements applied. This
also fixes a couple of UAFs that were caused by the use of this API.

As an optimization an equivalent StringView::replace API was also added
to remove an unnecessary String allocations in the format of:
`String { view }.replace(...);`
2021-09-11 20:36:43 +03:00
Ben Wiederhake
2e4ec891da Everywhere: Fix format-vulnerabilities
Command used:
grep -Pirn '(out|warn)ln\((?!["\)]|format,|stderr,|stdout,|output, ")' \
     AK Kernel/ Tests/ Userland/
(Plus some manual reviewing.)

Let's pick ArgsParser as an example:
    outln(file, m_general_help);
This will fail at runtime if the general help happens to contain braces.

Even if this transformation turns out to be unnecessary in a place or
two, this way the code is "more obviously" correct.
2021-09-11 15:16:26 +01:00
Brian Gianforcaro
a4efaa7b47 Tests/Kernel: Fix test after off-by-one fix in Memory::is_user_range()
Commit 890c647e0f fixed an off-by-one bug, so the mapping of the page
at the very end of the user address space now works correctly.

This change adjusts the test so cover the corner cases the original
version was designed too.validate.
2021-09-11 04:15:16 +00:00
Ali Mohammad Pur
14c8373eb0 AK+Kernel: Reduce the number of template parameters of IntrusiveRBTree
This makes the user-facing type only take the node member pointer, and
lets the compiler figure out the other needed types from that.
2021-09-10 18:05:46 +03:00