Commit Graph

72 Commits

Author SHA1 Message Date
Andrew Kaster
49b07cda12 CI: Bump Android NDK version used in CI to 25.2.9519653
The previous version is no longer able to compile Lagom

Clang version 14.0.1 in NDK 24.0.8215888 crashes trying to build
Core::Process
2023-03-15 20:36:22 -06:00
Nico Weber
3cff36b7ab Meta+CMake: Remove "image" ninja target in favor of "qemu-image"
"image" was an alias for "qemu-image".

I want to add an `image` userland utility, which clashes with that
shortname.

So remove the existing "image" target. It was just an alias for
"qemu-image".

If you use serenity.sh to build, nothing changes. This only affects you
if you run ninja manually -- you now have to say `ninja qemu-image` to
build the disk image.
2023-03-12 01:48:56 +00:00
Andrew Kaster
b996e15d9d CI: Remove stale dependence on Toolchain.yml from nightly job
We removed this file in 0a7d0362ea but
forgot to update the nightly job.
2023-03-11 05:36:14 -07:00
Andrew Kaster
3d33217d60 Ladybird+CI: Move layout_test.sh test runner from CI yml into CMake
We should be able to run this locally, as long as ENABLE_LAGOM_LADYBIRD
is true, or if building ladybird from the ladybird source directory.

This removes a special case from the Lagom CI yml file.
2023-03-10 23:01:55 +00:00
Andrew Kaster
35b07048ef CI: Make sure to brew update in macOS Azure setup
This picks up any upstream brew changes that haven't been pulled into
the default Azure VMs. For example, the ffmpeg dependency of qt is
currently broken in the 8 day old brew commit they are using.
2023-02-24 02:39:26 -07:00
Timothy Flynn
49474d8718 Revert "Meta: Automatically select best apt mirror"
This reverts commit b0606d90f0.

This seems to prevent libegl-mesa0 from being installed (which for some
reason isn't failing the Azure jobs - the failure seen later is that
ccache is not installed).
2023-02-09 14:57:27 +00:00
MacDue
b0606d90f0 Meta: Automatically select best apt mirror
This is an attempt to avoid issues with the azure ubuntu apt mirror
by using apt-spy2 to select the best mirror before installing
dependencies.
2023-02-07 13:22:17 +00:00
Andrew Kaster
8f70e365f0 Meta+CI: Disable Ladybird for fuzzer, compiler explorer and Android
And move it after the declaration of headless-browser, since WebDriver
depends on headless-browser.
2023-02-02 05:35:44 -07:00
Timothy Flynn
9f9b8e7273 CI: Move running LibWeb layout tests to Azure
The current config on GitHub Actions does not use ccache, so it takes
quite a while to build. Instead, let's just run these tests on Azure
where we already build Ladybird and have ccache enabled. This also lets
us sanitize LibWeb on both Linux and macOS.

The script changes here are to A) handle differences between Azure and
GitHub Actions and B) to support running on macOS.
2023-02-01 14:04:44 +00:00
Timothy Flynn
0a7d0362ea CI: Remove extraneous toolchain job from Azure CI
This was useful when building both i686 and x86_64 SerenityOS targets as
we could use a single toolchain build for both targets. But now all this
extra job does is create the opportunity for the toolchain to need to be
built twice (i.e. if the pipelines are backed up and the toolchain cache
is busted between these jobs while the x86_64 step is waiting for a VM).
2022-12-28 15:26:12 -05:00
Liav A
55f17fff36 Meta: Remove i686 target 2022-12-28 11:53:41 +01:00
Andrew Kaster
4ae2a54f3d CI: Bump macOS Azure runners to macos-12 tag 2022-12-25 07:58:58 -07:00
Andrew Kaster
03edff1f39 CI: Enable ladybird builds in Lagom CI 2022-12-25 07:58:58 -07:00
Timothy Flynn
2334b4cebd Meta: Move UCD/CLDR/TZDB downloaded artifacts to Build/caches
They currently reside under Build/<arch>, meaning that they would be
redownloaded for each architecture/toolchain build combo. Move them to a
location that can be re-used for all builds.
2022-12-24 09:46:28 -05:00
Linus Groh
8639d8bc21 Meta: Switch to clang-format-15 as the standard formatter
The two major changes noticeable on the SerenityOS codebase are:
- Much improved support for const placement, clang-format-14 ignored
  our east-const configuration in various places
- Different formatting for requires clauses, now breaking them onto
  their own line, which helps with readability a bit

Current versions of CLion also ship LLVM 15, so the built-in formatting
now matches CI formatting again :^)
2022-12-03 23:52:23 +00:00
Andrew Kaster
1adc3530b2 CI: Install gcc-12 for Android nightly CI job
This *also* got missed in the gcc-12 update, because we weren't
installing an explicit gcc version prior. Hopefully that's the last of
the long tail of issues from that migration!
2022-11-26 09:40:07 +01:00
Andrew Kaster
a2cd61d19d CI: Use gcc-12 in nightly Android host lagom build stage 2022-11-11 10:26:55 +01:00
Linus Groh
bc2ebcadc0 Everywhere: Require version >= 12 for GCC host compiler
So far we've gotten away with using GCC 11 for Lagom and to compile the
toolchain, but via #15795 we discovered a compiler bug that has been
fixed in the latest version but would error the build with CI's GCC 11.
Time for an upgrade :^)

We already use ubuntu-22.04 images in most places, so this is pretty
straightforward. The only exception is Idan's self-hosted runner, which
uses Ubuntu Focal. LibJS should build fine with GCC 11, still.
2022-10-25 23:15:51 +01:00
Tim Schumacher
1faca15193 Toolchain: Move the binutils patch into its own folder
This makes binutils compatible with `./package.sh dev`, so that we can
regenerate patches more easily, and neatly stack (temporary) patches on
top.
2022-10-01 18:43:29 +01:00
kleines Filmröllchen
a5b72577e6 Meta: Enable full region dumping on CI 2022-09-24 14:22:09 +02:00
Andrew Kaster
cca54e9535 CI: Remove Android NDK Cache
This actually never worked as a cache, the build has failed since it
tried to use it. This should restore the Android nightly build.
2022-09-22 11:07:21 -04:00
Timothy Flynn
3dd11a007f CI: Fix typo preventing the CLDR cache from being pulled 2022-09-14 08:21:03 -04:00
Timothy Flynn
89d1813b5d LibUnicode: Move CLDR data generators to a LibLocale subfolder
To prepare for placing all CLDR generated data in a new library,
LibLocale, this moves the code generators for the CLDR data to the
LibLocale subfolder.
2022-09-05 14:37:16 -04:00
Timothy Flynn
b9fc9aeba1 CI: Bust the macOS Lagom ccache and reduce its size
It currently takes upwards of 40 minutes to download the ccache on macOS
and often errors-out near the end. Change the cache version to bust it
so we can start anew. Reduce its max size to 2 GB (a clean build is ~0.9
GB, so this allows just over 2 clean builds to be cached).
2022-08-23 19:03:56 +01:00
Linus Groh
ae264c9143 CI: Install newer Bash via homebrew on macOS
The recently added generate-emoji-txt.sh script uses a Bash 4
substitution feature, causing CI to fail as macOS ships an ancient Bash
3.x. We'll want to use a more recent version anyway, so let's do that
instead of updading the script to older syntax.
2022-08-23 13:25:39 +01:00
Liav A
6164729d06 Everywhere: Get rid of the fbdev kernel boot argument remainders 2022-08-14 01:03:23 +01:00
Timothy Flynn
407a88ebf6 CI: Enable downloading the Azure remote data cache for Fuzzer builds
This cache was disabled in 3127454 because it wasn't needed and there
was a race between the builders for this cache. Then commit 0c95d99
started fuzzing the generated Unicode / TZDB data. Since then, we've
been pulling this data from the live servers instead of Azure's cache.
2022-08-09 20:06:34 -04:00
Timothy Flynn
5c0f1d5b8a CI: Add a restoration key for Azure's remote data caches
We do a similar trick for the compiler cache. This allows each builder
to separately push their local data cache (if it changed) while pulling
a shared cache, without the race outlined in commit 3127454. This is
needed for a subsequent commit which will enable this cache for Fuzzer
builds.
2022-08-09 20:06:34 -04:00
Andrew Kaster
2aaaee6744 CI+Lagom: Add Lagom Android CI for arm64-v8a on NDK 24 with API level 30
This will let us validate that we aren't breaking any library compile
steps for arm64.
2022-07-21 16:37:15 +02:00
Tim Schumacher
28061cf94d Everywhere: Fully remove the separate LibPthread directory 2022-07-19 11:00:35 +01:00
Andrew Kaster
9c2211f246 Meta: Add Android cross-compile support to Lagom
This is a start to properly letting us cross-compile Lagom where both
the Tools and the BUILD_LAGOM=ON build are using Lagom CMakeLists.

The initial cut allows an Android build to succeed, more or less.
But there are issues with namespace clashes when using FetchContent with
this approach.
2022-07-19 10:44:02 +01:00
Nathan Wallace
2f1029e7c4 Meta+CI: Upgrade to ubuntu-22.04
This commit upgrades Github Actions workers to ubuntu-22.04

As part of that change, we (currently) no longer need the backports
nor toolchain-r/test PPAs, because ubuntu-22.04 include
recent-enough version of QEMU and gcc
2022-05-24 06:30:57 -04:00
Daniel Bertalan
d0edf2627c Toolchain+Ports: Split the GCC patches
This shouldn't cause any breaking changes, so a toolchain rebuild is not
required.

As per Hendiadyoin's request, math errno is disabled by default, which
should enable some extra compiler optimizations in LibGL and LibSoftGPU
code that uses math functions heavily.

Co-Authored-By: Ali Mohammad Pur <mpfard@serenityos.org>
2022-05-19 20:17:10 +04:30
Andrew Kaster
28241f25dc CI: Ensure that Azure nightly pipeline doesn't run on CI or PR triggers
GitHub YAML pipelines have both of these on by default, so we need to
explicitly disable them.
2022-05-07 20:49:13 +02:00
Andrew Kaster
d0e002d122 CI: Use proper paths to template files in nightly-pipeline.yml
Azure paths are relative to the pipeline file.
Addtionally, pipeline stages can't have spaces or parentheses in them
2022-05-07 20:38:18 +02:00
Andrew Kaster
44bcd7c7aa CI: Add x86_64 Clang Coverage pipeline in Azure
Add a job to the Azure pipelines to run tests with coverage enabled, and
aggregate the test results in a folder of html pages showing the
coverage results overall, and per-file.

Future work is needed to take the published pipeline artifact for the
coverage results and display them somewhere interesting.
2022-05-02 01:46:18 +02:00
Daniel Bertalan
57c6829249 CI: Update the path to our LLVM patches
The LLVM patch has been broken up into smaller commits and moved to a
separate directory. CI should look at this new location to determine if
the toolchain needs to be rebuilt.
2022-04-23 10:43:32 -07:00
Idan Horowitz
852ae6c195 Meta: Switch to clang-format-14 as the standard formatter
Now that clang-format-14 ubuntu packages are available, it's time to
finally upgrade our clang-format version. This version brings with it
a bunch of useful features with const-placement being the most notable.
These will be enabled in the following commits.
2022-04-01 21:24:45 +01:00
Tim Schumacher
e3519b8e5c Meta: Rename Fuzzer flags to ENABLE_FUZZERS_{LIBFUZZER,OSSFUZZ} 2022-03-31 22:11:04 -07:00
Andrew Kaster
0c95d9962c Lagom: Add two-stage build for Fuzzers to enable fuzzing generated code
This allows us to fuzz the generated unicode and timezone database
helpers, and to fuzz things like LibJS using Fuzzilli to get proper
coverage of our unicode handling code.

Update the Azure CI to use the new two-stage build as well, and cleanup
some unused CMake options there.
2022-02-20 19:04:59 +00:00
Idan Horowitz
ab14abc40f Meta: Actually run gml-format in CI
Third time's a charm.
2022-02-15 19:33:46 +02:00
Idan Horowitz
bc38155b9c Meta: Increase Azure CI timeout for Lagom builds 2022-02-15 18:02:54 +02:00
Idan Horowitz
6be75bd5e4 Meta: Use clang-13 instead of clang-12 in Azure CI
This should hopefully resolve a clang ICE seen in PR #12523.
2022-02-14 23:55:23 +02:00
Idan Horowitz
0a93bf5e7b Meta: Actually run gml-format in CI
Since gml-format is part of Lagom, it must be added to the post-lagom
linters section, or else it won't ever actually run.
2022-02-13 02:36:35 +02:00
Idan Horowitz
024ee0a99a CI: Fail Lagom linters step as soon as any linter fails
This prevents linters from covering the failure of previous linters by
overwriting the error code that Azure reads at the end of the step.
2022-01-22 21:40:09 +02:00
Timothy Flynn
6a4c1e4b7b CI: Disallow test failures on macOS Lagom :^) 2022-01-14 22:39:06 +01:00
Daniel Bertalan
162606f358 Toolchain+CI: Link LLVM with LLD if available
According to most benchmarks, LLD is faster than GNU ld and the macOS
linker, so let's use it if possible in order to speed up the toolchain
build.
2022-01-10 09:55:45 +03:30
Daniel Bertalan
d21c38a72d CI: Hash only relevant files for the key of the compiler cache
In the last few commits, a second patch was added to the LLVM toolchain,
and it no longer uses our binutils patch. This commit changes the CI
cache keys accordingly, in order to prevent unnecessary rebuilds of both
toolchains when only one is changed.

The Clang toolchain's cache now only takes into account patches that
begin with `llvm`, and the GNU toolchain excludes those from the hash
calculation. We now also hash the two CMake cache files that we use for
building LLVM and its runtime libraries.
2022-01-10 09:55:45 +03:30
Timothy Flynn
41f4a5050c CI: Add a cache for the IANA time zone database files 2022-01-08 12:45:34 +01:00
Timothy Flynn
3938a2cca7 CI: Extract toolchain ccache in the Serenity build stage
The toolchain is built in a previous stage, but once the Serenity stage
has begun, we have to re-pull the toolchain from the Azure cache. There
is a timing window where a cache-busting change can be commited between
these steps; to alleviate the affect this has, pull the toolchain ccache
so that the build only takes a few minutes instead of a couple hours.
2022-01-07 18:16:30 -08:00