Commit Graph

238 Commits

Author SHA1 Message Date
kleines Filmröllchen
47b6030347 Meta: Replace run.sh by run.py
A Python script is much easier to maintain than the organically grown
variable mess that was run.sh.

For now, the script inherits most environment variable modifiability
from the shell script, but this is not a requirement.

While porting this script, a couple of improvements have been made:
- Spaces (especially in paths) cannot break most arguments anymore.
  Exceptions are environment variables specifying multiple arguments on
  purpose, these should be replaced in the future anyways.
- Force control over virtualization is now possible with
  SERENITY_VIRTUALIZATION_SUPPORT. If set to 0, this variable was
  sometimes ignored before.
- Handling Windows native QEMU is much more robust. Multiple incorrect
  checks for WSL, but not Windows native QEMU, were used before. This
  would also allow disabling native Windows QEMU much more easily in the
  future, which is necessary for GDB.
- Various machine types had wrong or outdated arguments, such as qn.

Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2023-12-15 00:11:50 +01:00
Daniel Bertalan
1f747b9132 CMake: Use CMAKE_POSITION_INDEPENDENT_CODE instead of explicit -fpic
This makes CMake pass `-fpie` instead of `-fpic` to the compiler when
building the Kernel and userland *executables*. This allows the compiler
to make certain optimizations based on the fact that the code will be
used in an executable, such as not having to emit `.localalias` symbols.
This leads to a 450 KiB decrease in the size of the Kernel binary.
2023-09-18 10:26:42 +02:00
Daniel Bertalan
ae21002cb5 Toolchain+Ports: Update GCC to 13.2.0
This is a minor bugfix release, which to my knowledge contains nothing
of importance to us. However, there is one QoL change to our patches.

We no longer force `-fpic` in the compiler driver, and instead use the
`--enable-default-pie` configure option to generate position-independent
code suitable for executables. For building shared libraries, the
`-fpic` flag must be specified explicitly.
2023-09-18 10:26:42 +02:00
Daniel Bertalan
44365074fe CMake: Add UNDEFINED_BEHAVIOR_IS_FATAL configure option
This is mainly intended for use on CI, as UBSan instrumentation results
in a serious load and execution time penalty there. See the previous
commit for more details.

With this enabled, the size of LibWeb, built for x86-64 with Clang 17 as
of 0b91d36a is reduced as follows:

      FILE SIZE        VM SIZE
   --------------  --------------
     +18% +2.99Mi  [ = ]       0    .debug_info
     +14%  +758Ki  [ = ]       0    .debug_addr
    +2.6% +7.92Ki  [ = ]       0    .debug_abbrev
    +129% +2.66Ki  [ = ]       0    [Unmapped]
    -0.2%    -208  -0.2%    -208    .plt
    -0.2%    -312  -0.2%    -312    .rela.plt
    -0.1%    -336  -0.1%    -336    .dynsym
    -0.0%    -647  -0.0%    -513    [8 Others]
    -0.1% -1.14Ki  -0.1% -1.14Ki    .dynstr
   -20.1% -53.5Ki -20.1% -53.5Ki    .eh_frame_hdr
    -7.2% -56.8Ki  [ = ]       0    .debug_str_offsets
    -7.1%  -156Ki  [ = ]       0    .debug_str
   -15.0%  -160Ki  [ = ]       0    .symtab
   -63.6%  -245Ki -63.6%  -245Ki    .relr.dyn
   -25.4%  -357Ki -25.4%  -357Ki    .eh_frame
   -27.7% -1.09Mi  [ = ]       0    .strtab
   -59.3% -10.0Mi  [ = ]       0    .debug_rnglists
   -41.3% -11.0Mi  [ = ]       0    .debug_line
   -70.0% -12.0Mi -70.0% -12.0Mi    .rodata
   -65.2% -15.1Mi -65.2% -15.1Mi    .data
   -53.0% -15.7Mi -53.0% -15.7Mi    .text
   -41.7% -62.1Mi -57.7% -43.4Mi    TOTAL
2023-08-13 05:14:07 +02:00
Daniel Bertalan
055d2b6c8a CMake: Enable RELR relocations for Clang OR x86-64
While LLD and mold support RELR "packed" relocations on all
architectures, the BFD linker currently only implements them on x86-64
and POWER.

This fixes two issues:
- The Kernel had it enabled even for AArch64 + GCC, which led to the
  following being printed: `warning: -z pack-relative-relocs ignored`.
- The userland always had it disabled, even in the supported AArch64 +
  Clang/mold scenarios.
2023-08-12 19:39:00 +02:00
implicitfield
941d68ac2d Toolchain+Ports: Update LLVM to 16.0.6 2023-06-27 12:40:38 +02:00
implicitfield
5dfe2eb389 Everywhere: Resolve conflicts with LibC and libc++
Since https://reviews.llvm.org/D131441, libc++ must be included before
LibC. As clang includes libc++ as one of the system includes, LibC
must be included after those, and the only correct way to do that is
to install LibC's headers into the sysroot.

Targets that don't link with LibC yet require its headers for one
reason or another must add install_libc_headers as a dependency to
ensure that the correct headers have been (re)installed into the
sysroot.

LibC/stddef.h has been dropped since the built-in stddef.h receives
a higher include priority.

In addition, string.h and wchar.h must
define __CORRECT_ISO_CPP_STRING_H_PROTO and
_LIBCPP_WCHAR_H_HAS_CONST_OVERLOADS respectively in order to tell
libc++ to not try to define methods implemented by LibC.
2023-06-27 12:40:38 +02:00
Daniel Bertalan
07918b79b7 Toolchain+Ports: Update GCC to 13.1.0
This is a mostly straight-forward rebase of our patches on top of
13.1.0. The spec files needed a change, as GCC no longer supports STABS
debug information, but we were building GCC with support for it.

Highlights of this release include static `operator()`, The Equality
Operator You Are Looking For and extended `constexpr` support.
2023-05-15 07:00:29 +02:00
Timothy Flynn
593a4a4232 Meta: Install PCI and USB ID files directly into /res
The install() command used by 1e36d54493
installs the provided file into the *directory* named by the DESTINATION
parameter. So if we ask it to install pci.ids to /res/pci.ids, the final
destination will be /res/pci.ids/pci.ids.
2023-04-01 08:48:28 -04:00
Sam Atkins
1e36d54493 Meta: Make sure directories for pci.ids and usb.ids exist
`file(COPY_FILE ...)` fails if the target directory does not already
exist, whereas `install(FILES ... DESTINATION ...)` happily creates all
the required directories.
2023-03-15 12:49:49 -04:00
Timothy Flynn
c11860bdcd Meta: Download pci.ids and usb.ids to Build/caches
This ensures we only need to download these files once for all build
configurations. We similarly download the UCD, CLDR, and TZDB to this
cache directory as well.
2023-03-15 11:26:34 +01:00
Nico Weber
3cff36b7ab Meta+CMake: Remove "image" ninja target in favor of "qemu-image"
"image" was an alias for "qemu-image".

I want to add an `image` userland utility, which clashes with that
shortname.

So remove the existing "image" target. It was just an alias for
"qemu-image".

If you use serenity.sh to build, nothing changes. This only affects you
if you run ninja manually -- you now have to say `ninja qemu-image` to
build the disk image.
2023-03-12 01:48:56 +00:00
Andrew Kaster
733a318709 Meta+CMake: Extract Wasm spec tests into the binary directory
Clean up the Wasm spec tests CMake rules to extract and compile the wat
files into wasm files in the LibWasm binary directory instead of its
source directory. Also make the rules more robust to missing host tools,
and use more CMake install rules for the test files rather than relying
on build-root-filesystem.sh. Add some FIXMEs for later, we really
shouldn't be doing installation of test files into /home/anon at the
build-root-filesystem stage in $CURRENT_YEAR. Tests go in /usr/Tests
2022-12-14 20:29:43 +03:30
Andrew Kaster
16d4791212 CMake: Pass NO_POLICY_SCOPE to options cmake helpers
Tell CMake to not create a new policy scope for the
(lagom|serenity|common)_options.cmake helpers, and lets us set common
policies for both projects in common_options.cmake that actually apply
to the rest of the project, instead of just common_options.cmake itself.
2022-12-14 12:43:12 +01:00
Andrew Kaster
d9e76254a8 CMake: Bump cmake_minimum_required to 3.25
The SerenityOS platform files we upstreamed were first released in
version 3.25.0
2022-12-13 11:36:14 +01:00
Jan200101
10d40af167 Meta: Separate ccache setup into its own component 2022-12-12 21:34:09 -07:00
Tim Schumacher
e5e7ea90b1 Toolchain: Update LLVM to 15.0.3 2022-10-24 15:33:58 +02:00
Gunnar Beutner
3065577b54 Everywhere: Enable building userspace and test binaries on AARCH64
Surely this will just work once the kernel boots, right?
2022-10-14 13:01:13 +02:00
Gunnar Beutner
a650c74b27 AK+Toolchain: Make char and wchar_t behave on AARCH64
By default char and wchar_t are unsigned on AARCH64. This fixes a
bunch of related compiler errors.
2022-10-14 13:01:13 +02:00
MacDue
72605e854f Meta: Remove unused variables in CMakeLists.txt
Both USB_IDS_PATH and PCI_IDS_PATH are now unused so can be safely
removed.
2022-10-08 18:01:52 +01:00
MacDue
2ba3229ac6 Meta: Fix install path of usb.ids 2022-10-08 18:01:52 +01:00
Tim Schumacher
1b25c38b8b Everywhere: Fully remove the separate LibM directory 2022-09-16 16:09:19 +00:00
Tim Schumacher
f45238db0f Toolchain: Update to LLVM 15.0.0 2022-09-16 05:39:28 +00:00
Diego Iastrubni
629c77440b Meta: Download uncompressed files instead of *.gz
Problem: cmake cannot handle gzip files (see
https://gitlab.kitware.com/cmake/cmake/-/issues/23054 for more
details).

Instead of downloading the compressed (*.gz) USB and PCI ids,
we not download the raw uncompressed files. The sizes we "loose"
due to downloading such files are meaningless.

This are the file sizes:
```
diego@debian:~/$ ls -lh pci.ids{,.gz} usb.ids{,.gz}
-rw-r--r-- 1 diego diego 1.3M Aug  7 04:15 pci.ids
-rw-r--r-- 1 diego diego 300K Aug  7 04:15 pci.ids.gz
-rw-r--r-- 1 diego diego 700K May 20 22:34 usb.ids
-rw-r--r-- 1 diego diego 245K May 20 22:34 usb.ids.gz
```
2022-09-09 10:31:12 +01:00
Diego Iastrubni
8b30b69dac Meta: Use CMake functions to extract files
Newer cmake's have internal functions to un-compress files. These
functions will work on pure windows - as well as linux. This
eliminates the need to search for external tools (TAR,GZIP,ZIP) - and
helps fixing #9866.

In order to finally fix #9866 we need to decide to bump the cmake
version requirements and remove the checks. If we demand a newer cmake
version, we will loose Ubuntu 20.04 as a build target - as it ships
with CMake 3.16.

For now - we keep compatibility with CMake 3.16 - and only if CMake
3.18 as been found - we use its new functionality.
2022-09-09 10:31:12 +01:00
Andrew Kaster
5ab3fcf710 Meta: Update jakt build support for fully bootstrapped compiler
Remove the Corrosion dependency, and use the now-builtin
add_jakt_executable function from the Jakt install rules to build our
example application.

By using find_package(Jakt), we now have to set ENABLE_JAKT manually on
both serenity and Lagom at the same time, so the preferred method to do
this for now is:

    cmake -B Build/superbuild<arch><toolchain> \
          -S Meta/CMake/Superbuild \
          -DENABLE_JAKT=ON \
          -DJAKT_SOURCE_DIR=/path/to/jakt

Where omitting JAKT_SOURCE_DIR will still pull from the main branch of
SerenityOS/jakt. This can be done after runing Meta/serenity.sh run.
2022-09-09 11:23:42 +02:00
kleines Filmröllchen
c91511b883 Meta+Tests: Allow running FLAC spec tests
The FLAC "spec tests", or rather the test suite by xiph that exercises
weird FLAC features and edge cases, can be found at
https://github.com/ietf-wg-cellar/flac-test-files and is a good
challenge for our FLAC decoder to become more spec compliant. Running
these tests is similar to LibWasm spec tests, you need to pass
INCLUDE_FLAC_SPEC_TESTS to CMake.

As of integrating these tests, 23 out of 63 fail. :yakplus:
2022-09-02 23:54:50 +01:00
demostanis
c56cbf8027 CMake: Quote all CMAKE_COMMAND occurences
Building might fail if the cmake command path contains
whitespace. See https://stackoverflow.com/a/35853080.
2022-09-02 23:34:47 +01:00
Brian Gianforcaro
7ac3dffe9e Toolchain: Update gcc to version 12.2.0 2022-08-24 22:24:34 +00:00
Timothy Flynn
d86b25c460 Meta: Move downloading of emoji-test.txt to unicode_data.cmake
The current emoji_txt.cmake does not handle download errors (which were
a common source of issues in the build problems channel) or Unicode
versioning. These are both handled by unicode_data.cmake. Move the
download to unicode_data.cmake so that we can more easily handle next
month's Unicode 15 release.
2022-08-22 16:00:29 +01:00
Ryan Liptak
221d9089e9 Meta: Generate emoji.txt at build time from Unicode's emoji-test.txt
Instead of manually updating emoji.txt whenever new emoji are added,
we use Unicode's emoji-test.txt to generate emoji.txt on each build,
including only the emojis that Serenity supports at that time.

By using emoji-test.txt, we can also include all forms of each emoji
(fully-qualified, minimally-qualified, and unqualified) which can be
helpful when double-checking how certain forms are handled.
2022-08-22 08:42:54 +01:00
Liav A
622e3b06ce Meta: Put USB and PCI IDs in the right folder in the Build environment
Not sure why that happens and how it worked until now, but we need to be
more precise about the location of PCI and USB IDs when decompressing
them while building the OS.
2022-07-28 04:25:07 -07:00
Tim Schumacher
28061cf94d Everywhere: Fully remove the separate LibPthread directory 2022-07-19 11:00:35 +01:00
Tim Schumacher
656528f483 Meta: Prefix manually created directories with the staging prefix
In commit 02e8f29560 we started exporting
the `CMAKE_INSTALL_*DIR` variables without prefix in order to make
better use of the actual `PREFIX` settings.

However, commands like `file(MAKE_DIRECTORY ...)` don't understand the
GNUInstallDirs way of building paths, so we ended up creating
directories in our main project directory by accident.

Fix that by manually adding the correct prefix onto the path.
2022-07-10 16:55:29 +02:00
Andrew Kaster
02e8f29560 Meta: Use CMAKE_INSTALL_FOODIR variables instead of hardcoding usr/foo
In preparation for future refactoring of Lagom, let's use the variables
from GNUInstallDirs as much as possible for the helper macros and other
scripts used by the main build already.
2022-07-06 14:24:23 +02:00
Andrew Kaster
ca42da23c2 Meta+Userland: Add jakt as an optional Lagom Tool
We can now use ENABLE_JAKT to pull jakt as a host tool and use it to
pre-process .jakt files into .cpp files for use in serenity applications
2022-05-23 23:05:45 +02:00
Linus Groh
ee721978f6 Meta: Move compiler flags into standalone CMake files
This way we can have all of them in a single place, similar to how we
structure options added via the serenity_option() macro.
2022-05-14 17:53:06 +02:00
Daniel Bertalan
6cf260c7af Toolchain: Upgrade to GCC 12.1.0
This release brings support for various C++23 constructs like `if
consteval` and multidimensional subscript operators. Vectorization is
now enabled for O2 too, and `-ftrivial-auto-var-init` has been added
which can help us find and prevent security issues coming from
uninitialized variables.

Toolchain/Patches/gcc.patch is now significanly smaller as some unused,
autoconf-generated code has been removed.
2022-05-12 13:12:37 +02:00
Jelle Raaijmakers
a8949f4eca AK: Install generated Debug.h into /usr/include
Ports like SDL2 can depend on headers that try to include `Debug.h`.
2022-04-29 18:05:42 +02:00
Daniel Bertalan
01b31d9858 Toolchain+Ports: Update LLVM to 14.0.1
Besides a version bump, the following changes have been made to our
toolchain infrastructure:
- LLVM/Clang is now built with -march=native if the host compiler
  supports it. An exception to this is CI, as the toolchain cache is
  shared among many different machines there.
- The LLVM tarball is not re-extracted if the hash of the applied
  patches doesn't differ.
- The patches have been split up into atomic chunks.
- Port-specific patches have been integrated into the main patches,
  which will aid in the work towards self-hosting.
- <sysroot>/usr/local/lib is now appended to the linker's search path by
  default.
- --pack-dyn-relocs=relr is appended to the linker command line by
  default, meaning ports take advantage of RELR relocations without any
  patches or additional compiler flags.

The formatting of LLVM port's package.sh has been bothering me, so I
also indented the arguments to the CMake invocation.
2022-04-23 10:43:32 -07:00
Brian Gianforcaro
66e7ac1954 Meta: Error out on find_program errors with CMake less than 3.18
We have seen some cases where the build fails for folks, and they are
missing unzip/tar/gzip etc. We can catch some of these in CMake itself,
so lets make sure to handle that uniformly across the build system.

The REQUIRED flag to `find_program` was only added on in CMake 3.18 and
above, so we can't rely on that to actually halt the program execution.
2022-03-19 15:01:22 -07:00
Itamar
b6f358689c CMake: Modify include path when building from Hack Studio
With regular builds, the generated IPC headers exist inside the Build
directory. The path Userland/Services under the build directory is
added to the include path.

For in-system builds the IPC headers are installed at /usr/include/.
To support this, we add /usr/include/Userland/Services to the build path
when building from Hack Studio.

Co-Authored-By: Andrew Kaster <akaster@serenityos.org>
2022-03-19 22:02:44 +01:00
Daniel Bertalan
ee9125fa9a Meta: Use correct variable for checking if the mold linker is used
This variable was originally called USE_MOLD_LINKER, but it was changed
to ENABLE_MOLD_LINKER during review to be consistent with other
configuration options. I branched off the commits that added RELR
support before this change, and I failed to update the variable name
there.
2022-02-19 14:21:52 +02:00
czapek1337
64ff8af074 Meta: Add support for the Limine bootloader 2022-02-14 11:52:07 +01:00
Daniel Bertalan
ba5bbde7ee Meta: Enable RELR relocations
Also add a check to serenity.sh to ensure that the toolchain is new
enough for this feature to work.
2022-02-11 18:07:53 +01:00
Brian Gianforcaro
ee61739e0a Meta: Add install-native-partition CMake target installing to a real FS
While playing around with getting serenity to run on my main desktop
machine I wanted a way of easily updating my physical serenity
partition.

To use it you just need to:
- Create and format your local partition to ext4
- Set `SERENITY_TARGET_INSTALL_PARTITION` to the partition /dev path.
- Run the `install-native-partition` build target.

Example:

    $ export SERENITY_TARGET_INSTALL_PARTITION=/dev/nvme1n1p3
    $ cd serenity/Build/x86_64
    $ ninja install-native-partition
2022-02-04 12:44:50 +01:00
Timothy Flynn
6f8e89a905 Meta: Download USB and PCI ID data with fallible download function 2022-01-26 00:22:53 +00:00
Daniel Bertalan
4055c393fc Toolchain: Add support for building the userland with the mold linker
This commit adds support for building the SerenityOS userland with the
new [mold linker].

This is not enabled by default yet; to link using mold, run the
`Toolchain/BuildMold.sh` script to build the latest release of mold, and
set the `ENABLE_MOLD_LINKER` CMake variable to ON. This option relies on
toolchain support that has been added just recently, so you might need
to rebuild your toolchain for mold to work.

[mold linker]: https://github.com/rui314/mold
2022-01-24 10:41:47 +00:00
Itamar
f464ae84ce CMake: Add HACKSTUDIO_BUILD option for building from Hack Studio
If this option is set, we will not build all components.
Instead, we include an external CMake file passed in via a variable
named HACKSTUDIO_BUILD_CMAKE_FILE.

This will be used to build serenity components from Hack Studio.
2022-01-12 14:55:19 +01:00
Daniel Bertalan
4a428ba252 Meta: Do not allow undefined symbols in executables and shared objects
The `--allow-shlib-undefined` option is a bit of a misnomer. It actually
controls whether we should be allowed to have undefined references after
symbols from all dependencies have been resolved, so it applies both to
shared libraries and executables.

LLD defaults to allowing undefined references in shared libraries, but
not in executables. Previously, we had to disable this check for
executables too, as it caused a build failure due to the
LibC-LibPthread-libc++ and the LibCore-LibCrypto circular dependencies.

Now that those have been resolved, we can enable this warning, in the
hopes that it will prevent us from introducing circular libraries and
missing dependencies that might cause unexpected breakage.
2022-01-09 20:40:22 +01:00