Commit Graph

160 Commits

Author SHA1 Message Date
Timothy Flynn
7ea55c883b Toolchain: Make Kernel/API headers available to Clang toolchain build
Copied from 9b79867909.
2021-08-17 13:31:43 +01:00
Andreas Kling
9b79867909 Toolchain: Make Kernel/API/ headers available during toolchain build 2021-08-14 20:25:56 +02:00
Daniel Bertalan
657fbc1e6c Toolchain: Remove static LLVM libraries
We link against these dynamically anyways, so having them around is not
useful. Removing them frees precious storage space on CI.
2021-08-08 23:54:00 +02:00
Daniel Bertalan
d3595477d8 Toolchain: Build libLLVM as a dynamic library
This library is used by virtually all executables in the Clang
toolchain. By default, it is linked statically, which leads to huge
file sizes and us running out of artifact storage disk space on CI.
2021-08-08 23:54:00 +02:00
Gunnar Beutner
a29c18f969 Toolchain+Ports: Let -nostdlib make us not link against libgcc_s
For now this is not a mandatory toolchain rebuild.
2021-08-08 16:41:51 +02:00
Daniel Bertalan
9809e685c1 Toolchain: Add caching to BuildClang.sh for CI 2021-08-08 10:55:36 +02:00
Daniel Bertalan
15e217ea68 Toolchain: Add LLVM patch and script for building it
This contains all the bits and pieces necessary to build a Clang binary
that will correctly compile SerenityOS.

I had some trouble with getting LLVM building with a single command, so
for now, I decided to build each LLVM component in a separate command
invocation. In the future, we can also make the main llvm build step
architecture-independent, but that would come with extra work to make
library and include paths work.

The binutils build invocation and related boilerplate is duplicated
because we only use `objdump` from GNU binutils in the Clang toolchain,
so most features can be disabled.
2021-08-08 10:55:36 +02:00
Linus Groh
a613e00caa Ports: Change Python's auth_type to sha256 2021-08-03 21:01:07 +01:00
Gunnar Beutner
5df8c274bb Toolchain+Ports: Fix building ports on x86_64
Fixes #8377.
2021-07-28 21:28:31 +02:00
Gunnar Beutner
9c431ae7ef Toolchain: Update GCC to version 11.2.0 2021-07-28 21:28:31 +02:00
Gunnar Beutner
cfeffbe524 Toolchain: Update binutils to version 2.37 2021-07-28 21:28:31 +02:00
Gunnar Beutner
12bfc7b3fc Toolchain+Ports: Ignore -arch on macOS M1 hosts
CMake specifies -arch arm64 for our toolchain. Unfortunately that's an
option GCC only understands when built for macOS. This causes the build
to fail.

I haven't been able to get CMake to not specify that option so this adds
a dummy option to GCC.
2021-07-24 14:04:13 +02:00
Gunnar Beutner
9bc3ad75b4 Toolchain: Use correct variable when deleting the QEMU tarball 2021-07-10 11:13:56 +01:00
modmuss50
036f2a14ed Toolchain: Fix building toolchain on arm64 macs
This patch is based off:
https://github.com/osx-cross/homebrew-avr/pull/248 and
https://github.com/riscv/riscv-gnu-toolchain/issues/800
2021-07-08 16:07:14 +02:00
Gunnar Beutner
d86275d271 Kernel+Toolchain: Remove the kernel-specific toolchain
This is no longer necessary now that the kernel doesn't use libsupc++
anymore.
2021-07-06 19:08:22 +02:00
Gunnar Beutner
128e504de6 Meta: Change the QEMU binary directory to Toolchain/Local/qemu
Previously we'd place the QEMU binaries into the architecture-specific
toolchain directory. This is a problem because the BuildIt.sh script
clears those directories which also removes the QEMU binaries users
may have built earlier. Also, the QEMU binaries are not specific to
the target architecture.
2021-07-03 12:06:20 +02:00
kleines Filmröllchen
7ff363127b Toolchain: Allow containerized work with Serenity via Docker
Docker is a nice way of doing build automation, or just
containerizing builds for increased safety and isolating unstable
packages. The old Dockerfile in the toolchain did not satisfy these
needs. The new Dockerfile is known to run successfully on Docker
version 20.10.7. It clones the SerenityOS repo and builds the
toolchain. In this way, it is intended to be a starting point for other
Docker images that can e.g. run builds. For example, one can simply run
this docker image as-is, exec a shell in it and run a build there.
2021-06-29 15:56:02 +02:00
Idan Horowitz
5b909a94e7 Toolchain: Build the x86_64 target in addition to i386 in BuildQemu.sh 2021-06-26 17:59:54 +02:00
Jelle Raaijmakers
3067ee9c4c Toolchain: Add gettext as a dependency to Dockerfile
We need `msgfmt` inside of the `gettext` package in order to build the
git port.
2021-06-07 22:46:53 +02:00
Jelle Raaijmakers
6af9d87258 Toolchain: Add ccache to Dockerfile
Following up on 2d38d56e, we were missing this in our Dockerfile.
2021-06-07 19:22:03 +02:00
Gunnar Beutner
988dfa7f33 Toolchain+Ports: Fix building binutils on FreeBSD
This imports the upstream patch from
https://sourceware.org/bugzilla/show_bug.cgi?id=27382

Fixes #7407.
2021-06-07 09:51:12 +02:00
Jelle Raaijmakers
68f0170bc6 Toolchain: Add ImageMagick to Dockerfile 2021-06-04 12:54:27 +02:00
Hediadyoin1
6fe7d4d7b8 Toolchain: Use gcc's ar
the vanilla versions might not handle all things, that gcc can do;
For example is lto not really supported by the vanilla versions
source:
https://gcc.gnu.org/wiki/LinkTimeOptimizationFAQ
2021-06-03 00:24:06 +01:00
Oleg Kosenkov
839aad6e5b
Toolchain: Set CMAKE_AR to the right executable 2021-06-02 23:25:44 +01:00
Gunnar Beutner
7dedf09f3d Toolchain: Remove dependency check for texinfo
Turns out we don't need that after all.
2021-05-30 23:36:10 +01:00
Gunnar Beutner
259822493f Toolchain: Check whether required tools and libraries are available
Rather than having the toolchain build fail half-way through we should
check whether the user has installed all the required tools and
libraries early on.
2021-05-30 13:06:28 +01:00
Gunnar Beutner
afbab621aa Toolchain: Use set -o pipefail to the toolchain build script
Previously the buildstep function would obscure error codes because
the return value of the function was the exit code for the sed command
which caused us to continue execution even though one of the build
steps had failed.

With set -o pipefail the return value of the buildstep function is
the real command's exit code.
2021-05-27 08:58:37 +01:00
Idan Horowitz
bcfde43849 Toolchain: Replace the -march=native flag with -mtune=native
This ensures inter-machine compatibility by not emitting any processor
specific instructions. This fixes the issue raised by the non AVX-512
supporting GitHub actions runners.
2021-05-21 15:23:07 +02:00
Brian Gianforcaro
17ff895e1c Toolchain: Enable native host optimizations when building toolchain
-march=native specializes the binaries for the CPU features available on
the CPU the binary is being compiled on. This matches the needs of the
Toolchain, as it's always built and used on that machine only.

This should be safe for the github actions VMs as well, as they all run
on a standard VM SKU in "the cloud".

I saw small but notable improvements in end-2-end build times in my
local testing. Each compilation unit is on average around a second
faster on my Intel(R) Core(TM) i7-8705G CPU @ 3.10GHz.
2021-05-21 10:04:49 +01:00
Gunnar Beutner
b223233b15 Toolchain+Ports: Skip link tests for libstdc++v3
This makes stdlib.h and stdio.h functions available in the std
namespace for C++.

libstdc++v3's link tests can fail if you don't have an up-to-date
build directory, for example:

1. Have libc with missing _Exit symbol because you haven't done
   a build since that was added.

2. Run toolchain rebuild. libstdc++v3's configure script will
   realize that it can do link tests in general but will fail
   later on when it tries to link a program that tests for _Exit.

Even though this is a toolchain patch this does not necessarily
require rebuilding the toolchain right away. This is only required
once we start using any of these new members in the std namespace,
e.g. for ports.
2021-05-09 15:35:01 +02:00
Gunnar Beutner
1889006b90 Toolchain: Make -nodefaultlibs not link against libgcc_s
This fixes the -nodefaultlibs flag for gcc which previously
linked against libgcc_s anyway. Even though this is a toolchain
patch we don't need to rebuild the toolchain right away.
2021-05-07 15:35:50 +02:00
Brian Gianforcaro
8b856bd5a1 Toolchain: Fix expansion bugs and make BuildIt.sh shellcheck compliant
BuildIt.sh had a bunch of SC2086 errors, where we were not quoting
variables in variable expansions. The logic being:

    Quoting variables prevents word splitting and glob expansion,
    and prevents the script from breaking when input contains spaces,
    line feeds, glob characters and such.

    Reference: https://github.com/koalaman/shellcheck/wiki/SC2086

As bcoles noticed in #6772, shellcheck actually found a real bug here,
where the user's build directory included spaces.

Close: #6772
2021-05-05 21:26:37 +02:00
Brian Gianforcaro
f558a44610 Toolchain: Make BuildFuseExt2.sh shellcheck compliant
BuildFuseExt2.sh was saying it should be run under /bin/sh but it is
using bash extensions like pushd/popd, ${BASH_SOURCE[0]}, etc. So just
run it under bash to avoid any potential issues.
2021-05-05 21:26:37 +02:00
Brian Gianforcaro
b4927a753d Toolchain: Make BuildPython.sh shellcheck compliant
Shellcheck is unable to source non-literal includes,
so inform shellcheck to just ignore this include.
2021-05-05 21:26:37 +02:00
Gunnar Beutner
d685db6eb6 Toolchain: Enable building all code with -fPIC
Ordinarily this would force the compiler to not inline certain
symbols and call them via the PLT instead. To counteract this
I've also added -fno-semantic-interposition which disables
ELF symbol interposition. Our dynamic loader doesn't support
this anyway and we might even consider not implementing this
at all.

Even though this is a toolchain change this doesn't require
rebuilding the toolchain unless you're planning to build
for the x86_64 arch.
2021-05-03 08:42:39 +02:00
Brian Gianforcaro
5acac54085 Toolchain: Upgrade to QEMU 6.0 release 2021-04-30 08:36:50 +02:00
Gunnar Beutner
19064eaaca Toolchain: Update the binutils patch file to match the port
Previously the toolchain's binutils would not have been able to
build binaries on 32-bit host systems (not that this would be
much of an issue nowadays) because one of the #ifdefs was in
the wrong place.

I moved the #ifdef in the port's patch and this now updates
the toolchain's patch file to match the port's patch.
2021-04-29 10:33:44 +02:00
Gunnar Beutner
8cd62b5780 Toolchain+Ports: Update GCC to version 11.1.0 2021-04-29 10:33:44 +02:00
Brian Gianforcaro
d19180433f Toolchain: Update QEMU to 6.0.0-rc5
Changes since rc4:

0cef06d187: Update version for v6.0.0-rc5 release
5351fb7cb2: hw/block/nvme: fix invalid msix exclusive uninit
ffa090bc56: target/s390x: fix s390_probe_access to check PAGE_WRITE_ORG
bc38e31b4e: net: check the existence of peer before trying to pad
2021-04-28 09:43:42 +02:00
Brian Gianforcaro
2ef93a3c07 Build: Use variables when concatenating Toolchain paths.
Make this stuff a bit easier to maintain by using the
root level variables to build up the Toolchain paths.

Also leave a note for future editors of BuildIt.sh to
give them warning about the other changes they'll need
to make.
2021-04-27 13:07:04 +02:00
Jelle Raaijmakers
03569f9686 Toolchain/Dockerfile: Add rsync and unzip
Both utilies are used in the .port_include.sh file.
2021-04-27 08:58:29 +02:00
Brian Gianforcaro
1399048662 Toolchain: Update QEMU to 6.0-rc4 2021-04-26 09:10:53 +02:00
Gunnar Beutner
e6953d14d8 Toolchain: Update cmake platform definition
This fixes building cmake and other ports which use cmake
to detect whether we have -ldl.
2021-04-25 10:14:50 +02:00
Jelle Raaijmakers
7b6954afe3 Toolchain/Dockerfile: Update to Ubuntu 21.04, add git 2021-04-22 12:30:02 +02:00
Gunnar Beutner
d71f64b97f Toolchain: Build GCC with optimizations 2021-04-20 23:23:23 +02:00
Gunnar Beutner
75d41657d5 Toolchain+Ports: Enable threads for gcc
This enables POSIX threads for GCC and makes the -pthread
argument available.
2021-04-20 21:08:17 +02:00
Panagiotis Vasilopoulos
e45e0eeb47 Everywhere: Replace SERENITY_ROOT with SERENITY_SOURCE_DIR 2021-04-20 15:27:52 +02:00
xackus
6e2f2cd8b1 Toolchain: Don't produce debug symbols
Previously debug symbols were produced and then stripped.
2021-04-19 09:49:14 +02:00
Emanuele Torre
685556ae84 Toolchain/BuildIt.sh: let's have colours on macOS too :^)
MacOS's sed doesn't support "\x1b" expansion unlike GNU sed, but we can
still use bash's $'\x1b' to expand it.
2021-04-18 19:00:49 +02:00
Gunnar Beutner
05d140e7bd Toolchain: Add missing buildstep calls 2021-04-18 15:51:56 +02:00