Commit Graph

605 Commits

Author SHA1 Message Date
sin-ack
1d2bde2403 Meta: Add new-project.sh :^)
This script will instantiate a HackStudio template into a project on the
host. It currently supports all templates used by HackStudio.

To avoid having to maintain compatibility between other shells and the
Serenity shell in the postcreate scripts, we build the Serenity shell
with Lagom and use that to run the script.
2021-08-13 21:12:17 +04:30
Jean-Baptiste Boric
f641cc6470 Meta: Tune default QEMU disk size
Having lots of small files in Base/ may require more inodes in the
ext2 filesystem than the format utility sets aside by default. Let's
make a more educated guess since we have a rough idea of how many
inodes we need by counting files and directories.
2021-08-13 08:04:59 +02:00
Ali Mohammad Pur
f0e7e5bbe8 Meta: Un-escape escaped strings when generating Wasm tests 2021-08-12 21:03:53 +02:00
Ali Mohammad Pur
799471d16f Meta: Don't roundtrip floats for i64/i32 hex literals in wasm tests 2021-08-12 21:03:53 +02:00
sin-ack
a871a2040b Meta: Properly quote some CMake variables in serenity_component
This probably isn't all of them, because I'm no CMake expert. :^)
It does however allow "/bin/false" to build now.
2021-08-12 07:48:50 +02:00
Tom
e3e52a4e61 Meta: Add SERENITY_CPUS environment variable to control number of CPUs 2021-08-09 21:02:23 +02:00
Daniel Bertalan
13e3df41de Meta: Add Clang support to the CMake build scripts 2021-08-08 10:55:36 +02:00
Daniel Bertalan
15e217ea68 Toolchain: Add LLVM patch and script for building it
This contains all the bits and pieces necessary to build a Clang binary
that will correctly compile SerenityOS.

I had some trouble with getting LLVM building with a single command, so
for now, I decided to build each LLVM component in a separate command
invocation. In the future, we can also make the main llvm build step
architecture-independent, but that would come with extra work to make
library and include paths work.

The binutils build invocation and related boilerplate is duplicated
because we only use `objdump` from GNU binutils in the Clang toolchain,
so most features can be disabled.
2021-08-08 10:55:36 +02:00
Gunnar Beutner
2fec891d8e Meta: Allow attaching GDB to the QEMU instance on the WSL2 host 2021-08-08 01:46:41 +02:00
Gunnar Beutner
45f2e9f3cf Meta: Don't use x86_64 QEMU for the i686 kernel
This seemed like a good idea at the time to avoid an unnecessary
dependency on qemu-system-i386. However this makes debugging the
kernel with GDB more difficult because GDB assumes that the QEMU
architectures matches the kernel architecture.
2021-08-08 01:46:41 +02:00
Andrew Kaster
891bc81f30 Meta: Remove syslog.h from clang-format exclusion list
Before Libraries was moved to Userland/Libraries syslog.h had a bunch
of manually aligned defines and array initializations.

Andreas seems to have formatted the file with clang-format as part of
that file move. Since syslog.h is now properly formatted, we don't
need to exclude it from the linter list.
2021-08-07 15:36:51 +02:00
sin-ack
0d468f2282 Kernel: Implement a ISO 9660 filesystem reader :^)
This commit implements the ISO 9660 filesystem as specified in ECMA 119.
Currently, it only supports the base specification and Joliet or Rock
Ridge support is not present. The filesystem will normalize all
filenames to be lowercase (same as Linux).

The filesystem can be mounted directly from a file. Loop devices are
currently not supported by SerenityOS.

Special thanks to Lubrsi for testing on real hardware and providing
profiling help.

Co-Authored-By: Luke <luke.wilde@live.co.uk>
2021-08-07 15:21:58 +02:00
Liav A
02c72de090 Meta: Remove legacy hardware components from Q35 machine
As this is a test machine I use personally to test "modern" hardware
setups, it feels quite comfortable to not care too much about VGA with
this type of machine.
Also, we don't actively use the IDE controller on this machine type, so
let's just remove it :^)
2021-08-07 11:49:07 +02:00
sin-ack
bb609cee7f Meta: Add run-local.sh
This allows one to set their desired parameters for run.sh without the
need to set them in every terminal session or add it to the user account
shell files. If a run-local.sh file exists at the repository root and is
executable, it will be sourced. The file can contain any variables that
are expected to be set in run.sh.
2021-08-07 11:47:07 +02:00
Mandar Kulkarni
566702ffa2 Meta: Fix wrong path for AK Tests in check-ak-test-files.sh 2021-08-05 19:58:55 +02:00
Gunnar Beutner
d9d2287a11 Meta: Detect nested KVM support for WSL2 and use that when available
This allows running QEMU inside WSL2 for hosts which have nested KVM
and WSLg support (e.g. Windows 11).

Running QEMU inside the WSL2 VM is slightly slower than running QEMU
on Windows, probably because of how WSLg handles screen updates.
2021-08-04 21:15:41 +02:00
Gunnar Beutner
723c89af67 Meta: Automatically enable WHPX when possible 2021-08-04 21:15:41 +02:00
Gunnar Beutner
5445155dba Meta: Auto-detect where QEMU is installed on Windows 2021-08-04 21:15:41 +02:00
Liav A
cca1498e09 Meta: Make the Q35 machine more realistic
Although it is nice to test the system without too many devices, in
reality bare metal hardware is far more complex than the default skeleon
that QEMU provides. As a preparation of supporting more devices, we
need to ensure we are capable of at least booting on complex hardware
setups without easily-observable problems. Later on, this can be the
foundations of testing new drivers :^)
2021-08-03 21:08:01 +02:00
Gunnar Beutner
31655c9486 Meta: Prefer to use the 64-bit QEMU binary if that's available
Some users might not have qemu-system-i386 installed.
2021-08-03 13:59:57 +01:00
Gunnar Beutner
b082488e94 Meta: Prefer to use -machine pcspk-audiodev for QEMU >=5.1
This gets rid of the following warning message from QEMU on startup:

qemu-system-i386: warning: '-soundhw pcspk' is deprecated, please set a
backend using '-machine pcspk-audiodev=<name>' instead

Fixes #4093.
2021-08-03 13:59:57 +01:00
Gunnar Beutner
74af43ed97 Meta: Improve WSL detection for the run.sh script
For users who use a custom kernel with WSL our previous method of
detecting WSL doesn't work. This new check instead detects WSL by
checking if the wslpath utility is available.
2021-08-02 11:47:29 +01:00
Andrew Kaster
40b0767d88 Meta: Add BUILD_SHARED_LIBS option for Lagom builds
This standard CMake option controls whether add_library() calls will
use STATIC or SHARED by default. The flag is set to on by default
since that's what we want for normal CI jobs and local builds and the
test262 runner, but disabled for oss-fuzz builds.

This should finally fix the oss-fuzz build after it was broken in #9017

oss-fuzz un-breakage was verified by running the following commands in
the oss-fuzz repo:

python infra/helper.py build_image serenity
python infra/helper.py build_fuzzers --sanitizer address --engine afl \
    --architecture x86_64 serenity /path/to/local/checkout/Meta/Lagom
python infra/helper.py check_build --sanitizer address --engine afl \
    --architecture x86_64 serenity
2021-08-02 09:05:28 +02:00
x-yl
8e8b1c8333 Meta: Explicitly specify the disk format in the QEMU options
Otherwise we're getting this warning:

WARNING: Image format was not specified for '_disk_image' and probing
         guessed raw. Automatically detecting the format is dangerous
         for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
2021-08-02 00:35:22 +02:00
Gunnar Beutner
c73c40a21d Meta: Fix compatibility with QEMU 5.x
QEMU 5 doesn't support -machine pcspk-audiodev so we need to fall back
to using -soundhw for that.
2021-08-02 00:35:22 +02:00
Sam Atkins
e54531244f LibWeb: Define proper debug symbols for CSS Parser and Tokenizer
You can now turn debug logging for them on using `CSS_PARSER_DEBUG` and
`CSS_TOKENIZER_DEBUG`.
2021-07-31 00:18:11 +02:00
Gunnar Beutner
8b2e76b838 Meta: Make audiodev detection more robust
This fixes audio on Windows as well as Linux systems which don't have
SDL.

Fixes #8903.
2021-07-30 23:51:26 +02:00
Gunnar Beutner
dff8b3d2d9 Meta: Make rsync a hard dependency and remove the fallback code
Previously we'd fall back to using cp if rsync wasn't available. Not
only is this considerably slower it also breaks when some of the files
in the target directory are symlinks because cp tries to dereference
them.

Fixes #8672.
2021-07-30 23:50:29 +02:00
Timothy Flynn
d485cf29d7 LibRegex+LibUnicode: Begin implementing Unicode property escapes
This supports some binary property matching. It does not support any
properties not yet parsed by LibUnicode, nor does it support value
matching (such as Script_Extensions=Latin).
2021-07-30 21:26:31 +01:00
Brian Gianforcaro
a4dc1eb330 Lagom: Fix oss-fuzz build error due to CMake typo
The OSS-Fuzz build was failing with the following error:

 /usr/bin/ld: cannot find -l{}
2021-07-30 20:28:59 +01:00
Andrew Kaster
32d076ef54 Lagom: Add proper install rules
Create the proper export files to allow Lagom to be a well-behaved
ExternalProject, based on the example project from the cmake-init
project generator here:

https://github.com/friendlyanon/cmake-init-shared-static
2021-07-29 21:46:25 +01:00
Linus Groh
7392a73066 Lagom: Exclude GMLAutocompleteProvider.cpp from GML library sources 2021-07-29 21:46:25 +01:00
Andrew Kaster
b6e5117f65 Lagom: Change to shared library build for all Lagom code
Split the Lagom build into shared libraries to match the Serenity build.

This reduces the cognitive load when trying to edit the Lagom CMakeLists
significantly. It also reduces the amount of source files that must be
compiled to run each test or host program significantly.

Also re-organize all the build rules into sections. And reorganize the
CMakeLists file in general.
2021-07-29 21:46:25 +01:00
Andrew Kaster
de2bf3f333 Meta/LibTest: Build object libraries for test main files
By using the power of object libraries and $<TARGET_OBJECTS> we can make
sure to only build TestMain.cpp and JavaScriptTestRunnerMain.cpp once.

Previously we built these cpp files into object files once for every
single test executable. This change reduces the number of total compile
jobs in a Serenity target build by around 100.
2021-07-29 21:46:25 +01:00
Andrew Kaster
af9be6e093 LibTTF/LibGfx: Remove circular dependency by merging LibTTF into LibGfx
LibTTF has a concrete dependency on LibGfx for things like Gfx::Bitmap,
and LibGfx has a concrete dependency in the TTF::Font class in
Gfx::FontDatabase. This circular dependency works fine for Serenity and
Lagom Linux builds of the two libraries. It also works fine for static
library builds on Lagom macOS builds.

However, future changes will make Lagom use shared libraries, and
circular library dependencies are not tolerated in macOS.
2021-07-29 21:46:25 +01:00
Andreas Kling
b3d27c2340 LibGUI: Add GUI_HOVER_DEBUG runtime debugging flag (environment)
You can now see the outline of GUI widgets when hovering them.
For example:

    $ export GUI_HOVER_DEBUG=1
    $ FileManager

Then move the mouse around in the file manager. :^)
2021-07-28 01:01:02 +02:00
Karol Kosek
220dd28b02 Meta: Ignore everything after the git cut line in a lint-commit.sh hook
I have set up a commit.verbose variable in my git config,
which shows the patch diff on bottom of the commit message.

Unfortunately the character limit was also applied to the diff,
which meant that I got a false-positive lint error almost every time.
2021-07-27 22:35:49 +01:00
Idan Horowitz
350fb0e3db Meta: Bring lint-commit.sh up to date with the CI commit linter
This was missing 2 of the recently added checks. Also added a reminder
in the CI linter to update the Meta (commit hook) version.
2021-07-27 20:41:49 +01:00
x-yl
4bdd62939a Meta: Set -audiodev explictly
Unfortunately seems like QEMU tries to use SPICE for audio so we need to
explicitly set -audiodev to use either SDL or coreaudio (depending on
platorm)
2021-07-27 19:15:37 +02:00
Gunnar Beutner
57417a3d6e Kernel: Support loading the kernel at almost arbitrary virtual addresses
This enables further work on implementing KASLR by adding relocation
support to the pre-kernel and updating the kernel to be less dependent
on specific virtual memory layouts.
2021-07-27 13:15:16 +02:00
Timothy Flynn
98d8274040 Meta: Add LibUnicode (and its tests) to Lagom
This is primarily to allow using LibUnicode within LibJS and its REPL.

Note: this seems to be the first time that a Lagom dependency requires
generated source files. For this to work, some of Lagom's CMakeLists.txt
commands needed to be re-organized to include the CMake files that fetch
and parse UnicodeData.txt. The paths required to invoke the generator
also differ depending on what is currently building (SerenityOS vs.
Lagom as part of the Serenity build vs. a standalone Lagom build).
2021-07-26 17:03:55 +01:00
Ali Mohammad Pur
466e2a2fb7 Meta: Don't reboot on triple-fault in CI mode
Instead, just make QEMU quit immediately.
2021-07-26 02:29:25 +04:30
Thomas Wagenveld
2002b7e2e3 Meta/run.sh: Allow for overriding of QEMU ethernet device type
You can set the SERENITY_ETHERNET_DEVICE_TYPE environment variable to
pick another device type (i.e. ne2k_pci). Defaults to e1000 as before.
2021-07-24 21:28:22 +02:00
Luke
7e7c65abb6 Lagom/Fuzzers: Add fuzzer for the quoted printable decoder 2021-07-24 20:11:28 +04:30
Gunnar Beutner
cc0914ae58 Meta: Add Homebrew paths for macOS on M1 to the PATH env variable 2021-07-24 14:04:13 +02:00
Andreas Kling
3f9e018d9a CrashDaemon: Remove BACKTRACE_DEBUG debugging code
This thing seems to work fine, no need to hang on to old debug code.
2021-07-22 23:34:33 +02:00
Gunnar Beutner
9031ed0b84 Meta: Add a new qextlinux target for the run.sh script
This allows testing the extlinux image with QEMU.
2021-07-20 15:12:19 +02:00
Gunnar Beutner
696e15fcd8 Meta: Update extlinux config for the recent prekernel changes 2021-07-20 15:12:19 +02:00
Gunnar Beutner
6a45ebe282 Meta: Fix syslinux detection on Arch Linux 2021-07-20 15:12:19 +02:00
Gunnar Beutner
e3b7ae0c77 Meta: Remove unused bootloader_test target
Due to other changes this is now just a copy of the q35 target.
2021-07-18 22:08:03 +02:00