Commit Graph

672 Commits

Author SHA1 Message Date
Timothy Flynn
32a2a02489 LibUnicode: Fix typo in listPatterns.json parsing method 2021-09-08 21:08:48 +01:00
Idan Horowitz
24ed8511dd CI+Meta: Add Signed-off-by tag filter to the commit message linter 2021-09-07 20:46:34 +01:00
Idan Horowitz
f56bdd2bb7 Meta: Don't immediately continue in serenity.sh's gdb option
This allows us to step through the initial boot instructions.
2021-09-07 12:58:46 +02:00
Idan Horowitz
783a58dbc7 Toolchain: Build aarch64-gdb for cross-debugging on x86 2021-09-07 12:58:46 +02:00
Timothy Flynn
4ad2159812 LibUnicode: Remove Unicode locale variants from CLDR path names
There's only a couple of cases like this, but there are some locale
paths in the CLDR that contain variants. For example, there isn't a
en-US path, but there is a en-US-POSIX path. This interferes with the
operation to search for locales by name. The algorithm is such that
searching for en-US will not result in en-US-POSIX being found. To
resolve this, we should remove variants from the locale name.
2021-09-06 23:49:56 +01:00
Timothy Flynn
3f64a14e06 LibUnicode: Parse and generate the Unicode locale list patterns dataset
This data informs consumers how to join lists of values. For example,
in en-US, the list ["a", "b", "c"] formatted to a string should become
"a, b, and c".
2021-09-06 23:49:56 +01:00
Timothy Flynn
9cd986d8c0 LibUnicode: Extract cldr-misc dataset from CLDR database 2021-09-06 23:49:56 +01:00
Nico Weber
f1d1418256 Meta: Unbreak serenity.sh run aarch64 after 918f7cb4a8 2021-09-06 21:57:59 +01:00
Timothy Flynn
077a693de6 LibUnicode: Sort special casing array by locale specificity
This is to simply the Default Case Conversion implementation. Otherwise,
the implementation would need to determine which special casing rule to
apply, instead of just picking the first match.
2021-09-06 15:24:27 +01:00
Timothy Flynn
91db61ae8d LibUnicode: Generate canonical combining class in Unicode data
Will be used by special casing rules.
2021-09-06 15:24:27 +01:00
Ali Mohammad Pur
d20fc922c5 Meta: Correct misuse of ByteBuffer::resize() as grow() in FuzziliJS 2021-09-06 01:53:26 +02:00
Ali Mohammad Pur
97e97bccab Everywhere: Make ByteBuffer::{create_*,copy}() OOM-safe 2021-09-06 01:53:26 +02:00
Tom
918f7cb4a8 Meta: Force legacy xAPIC when using 8 cores or less
This is merely so that we exercise the legacy xAPIC code when we don't
really need x2APIC.
2021-09-04 22:22:58 +02:00
Timothy Flynn
e6a2ab1202 LibUnicode: Generate an implementation of the Add Likely Subtags method 2021-09-04 13:51:40 +01:00
Timothy Flynn
28ae63177e LibUnicode: Generate the entire locale likely-subtags dataset
The amount of aliases in the likely-subtags dataset is quite large, so
this also needed to change the way the data is generated. Otherwise, the
compiler would complain about the size of the generated code.

Previously, a static method was generated that would effectively parse
the dataset into a HashMap of Unicode::LanguageID at runtime. We now
perform that parsing at generation-time, and instead generate an Array
of a structure similar to Unicode::LanguageID (we cannot use the same
structure because it contains String and Optional, which cannot be used
at compile-time).
2021-09-04 13:51:40 +01:00
Liav A
8cb4fcdbd8 Meta: Add q35grub option in run.sh
This option is similar to the qgrub option, but instead of starting a
QEMU PIIX4 machine, it starts a QEMU Q35 machine, booting a grub image
disk within it.
2021-09-04 12:54:00 +02:00
Tom
77953a937d Meta: Add the ability to specify clang with serenity.sh
This enables maintaining gcc and clang builds side-by-side.
2021-09-03 23:12:17 +02:00
Luke Wilde
ed97ee902b LibWeb: Add support for IDL static functions 2021-09-03 23:11:58 +02:00
Luke Wilde
1d8f8ea5b1 LibWeb: Add initial support for AbortController and AbortSignal
The DOM specification says that the primary use case for these is to
give Promises abort semantics. It is also a prerequisite for Fetch,
as it is used to make Fetch abortable.
a
2021-09-02 09:12:17 +02:00
Timothy Flynn
1fbc5dba08 LibUnicode: Generate Unicode locale likely subtag data
CLDR contains a set of likely subtag data where, given a locale, you can
resolve what is the most likely language, script, or territory of that
locale. This data is needed for resolving territory aliases. These
aliases might contain multiple territories, and we need to resolve which
of those territories is most likely correct for a locale.

Note that the likely subtag data is quite huge (a few thousand entries).
As an optimization encouraged by the spec, we only generate the smallest
subset of this data that we actually need (about 150 entries).
2021-09-01 14:14:47 +01:00
Timothy Flynn
9ae7ac4c87 LibUnicode: Generate complex Unicode locale alias matching
Most alias substitutions are "simple", meaning that alias matching is
done by examining a single locale subtag. However, there are a handful
of "complex" aliases where matching is done by examining multiple
subtags. For example, the variant subtag "lojban" causes the locale
"art-lojban" to be canonicalized to "jbo", but only when the language
subtag is "art" (i.e. this should not occur for the locale "en-lojban").

This generates a method to perform complex alias matching.
2021-09-01 14:14:47 +01:00
Timothy Flynn
9b118f1f06 LibUnicode: Generate Unicode locale alias data
CLDR contains a set of aliases for languages, territories, etc. that no
longer are meant to be used (e.g. due to deprecation). For example, the
language "aam" is deprecated and should be canonicalized as "aas".
2021-09-01 14:14:47 +01:00
Timothy Flynn
caf5b6fa6f LibUnicode: Extract cldr-core dataset from CLDR database 2021-09-01 14:14:47 +01:00
Peter Elliott
8d2c04821f Tests: Test LibMarkdown against commonmark test suite
TestCommonmark runs the CommonMark test suite
(https://spec.commonmark.org/0.30/spec.json) against LibMarkdown.
Currently 44/652 tests pass.
2021-08-31 16:53:51 +02:00
Ali Mohammad Pur
1465b11b58 Meta: Implement support for the "unlinkable" Wasm spectest assertion 2021-08-30 22:47:02 +02:00
Ali Mohammad Pur
e93c740df5 Meta: Generate bigints for i64 values in LibWasm test suite files 2021-08-30 22:47:02 +02:00
Andreas Kling
57ac8ff1fd Meta: Update lint-ports.py for the SerenityPorts move 2021-08-30 20:49:37 +02:00
Brian Gianforcaro
619200774b CMake: Add custom target to build only the generated sources
This is needed so all headers and files exist on disk, so that
the sonar cloud analyzer can find them when executing the compilation
commands contained in compile_commands.json, without actually building.

Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2021-08-30 16:44:16 +02:00
Linus Groh
747af8a031 Meta: Remove duplicate valid target check from serenity.sh 2021-08-29 13:50:07 +01:00
Linus Groh
2b70f32876 Meta: Mention aarch64 as a supported TARGET in serenity.sh's help 2021-08-29 13:48:35 +01:00
Nico Weber
ce98aa567c Meta: Don't pass net -device flags in aarch64 buils in run.sh 2021-08-28 21:51:30 +01:00
Nico Weber
ad480505af Meta: Introduce SERENITY_MACHINE variable in run.sh
Set it to all the `-m`, `-display`, `-device` flags for Intel,
and to just "-M raspi3" for aarch64 builds.
2021-08-28 21:51:30 +01:00
Nico Weber
4f3c0fa659 Meta: Disable spice in aarch64 buils in run.sh 2021-08-28 21:51:30 +01:00
Nico Weber
230ea3754d Meta: Run qemu-system-aarch64 in aarch64 builds in run.sh 2021-08-28 21:51:30 +01:00
Nico Weber
00c5e68b11 Meta: Disable kvm for aarch64 in run.sh 2021-08-28 21:51:30 +01:00
Nico Weber
23bc752c5d Meta: Make build-root-filesystem.sh omit libgcc_so.so in aarch64 builds 2021-08-28 21:51:30 +01:00
Andrew Kaster
72de228695 Kernel: Verify interrupts are disabled when interacting with Mutexes
This should help prevent deadlocks where a thread blocks on a Mutex
while interrupts are disabled, and makes it impossible for the holder of
the Mutex to make forward progress because it cannot be scheduled in.

Hide it behind a new debug macro LOCK_IN_CRITICAL_DEBUG for now, because
Ext2FS takes a series of Mutexes from the page fault handler, which
executes with interrupts disabled.
2021-08-28 20:53:38 +02:00
Nico Weber
7052f403c8 Toolchain: Enough to make rebuild-toolchain aarch64 work
The gcc patch might not be completely correct, but at least the
toolchain completes building.
2021-08-28 14:43:07 +01:00
Linus Groh
822566ff1a Meta: Let serenity.sh's TARGET default to SERENITY_ARCH, not i686
This environment variable is already widely used across our build
scripts and tooling, so serenity.sh should respect it as well.
It still uses i686 as the fallback.
2021-08-28 11:20:52 +01:00
Andrew Kaster
834be1a895 Meta: Remove write-only-on-difference script
It's now been replaced with "${CMAKE_PROGRAM}" -E copy_if_different in
all previous usage locations.
2021-08-28 08:44:17 +01:00
Andrew Kaster
63956b36d0 Everywhere: Move all host tools into the Lagom/Tools subdirectory
This allows us to remove all the add_subdirectory calls from the top
level CMakeLists.txt that referred to targets linking LagomCore.

Segregating the host tools and Serenity targets helps us get to a place
where the main Serenity build can simply use a CMake toolchain file
rather than swapping all the compiler/sysroot variables after building
host libraries and tools.
2021-08-28 08:44:17 +01:00
Andrew Kaster
e88761b2b9 Meta+LibUnicode: Move unicode_data helper to Meta/CMake
Moving this helper CMake file to the centralized Meta/CMake folder helps
to get a better grasp on what extra files are required for the build,
and what files are generated.

While we're at it, don't use add_compile_definitions for
ENABLE_UNICODE_DATA, which only needs to be seen by LibUnicode sources.
2021-08-28 08:44:17 +01:00
Andrew Kaster
47471c0ec2 Meta: Prefer SerenityOS_SOURCE_DIR to CMAKE_SOURCE_DIR
By using SerenityOS_SOURCE_DIR we can make custom targets and commands
agnostic to the actual location of the root CMakeLists directory.
All we care about is the root of the SerenityOS project.
2021-08-28 08:44:17 +01:00
Andrew Kaster
20e904d87c Meta: Move code generator helpers to their own CMake helper file
compile_gml, compile_ipc, and generate_state_machine all use host
tools to generate sources for the target build. As part of trying to
organize host tools into a common area, let's move these helper rules to
a common file that we can add other host tools to later. And, keep the
host tool helpers separate from the CMake target helpers for apps and
libraries.
2021-08-28 08:44:17 +01:00
Andrew Kaster
a83847f8bf Meta: Move components CMake helpers to their own file
It's hard to follow how all the functions in the utils.cmake helper file
flow together, so let's move the pieces that are related to each other
into specialized helpers. First up: all the ConfigureComponents related
properties and functions.
2021-08-28 08:44:17 +01:00
kleines Filmröllchen
2dc614127e Meta: Add System32 to the PATH so that reg.exe is always found
On my machine (c), /mnt/c/Windows/System32 is not on the PATH by
default. This causes reg.exe to fail, which is responsible for detecting
the presence of QEMU. By putting this path on the PATH on WSL, it will
always work regardless of the specific PATH configuration, and QEMU is
always detected.
2021-08-27 21:24:25 +01:00
Timothy Flynn
9663525542 CI: Ensure unzip is installed on the builders
The CLDR database comes in a .zip file.
2021-08-26 23:40:23 +02:00
Timothy Flynn
3127454642 CI: Skip Unicode caches for Fuzzer build
There is a bit of a race here between the Fuzzer and non-Fuzzer Lagom
builds. If the Unicode caches are empty, and the Fuzzer build completes
first, then the UCD and CLDR directories will be empty or won't exist.
Skip handling the Unicode caches for this build.
2021-08-26 22:04:09 +01:00
Timothy Flynn
e883792fd4 CI: Cache downloaded Unicode CLDR database 2021-08-26 22:04:09 +01:00
Andreas Kling
c915174563 Userland: Remove IRC Client
The IRC Client application made some sense while our main communication
hub was an IRC channel. Now that we've moved on, IRC is just a random
protocol with no particular relevance to this project.

This also has the benefit of removing one major client of the single-
process Web::InProcessWebView class.
2021-08-24 16:37:28 +02:00