Commit Graph

194 Commits

Author SHA1 Message Date
Itamar
bbedd320b5 Toolchain+LibC: Fix usage of crt files
We now configure the gcc spec files to use a different crt files for
static & PIE binaries.

This relieves us from the need to explicitly specify the desired crt0
file in cmake scripts.
2020-12-24 21:46:35 +01:00
Lenny Maiorani
1db5276c05 Meta: Run all lint checks and report failures together
Problem:
- The first lint check that fails results in all subsequent checks not
  being run.

Solution:
- Run all the lint checks aggregating the number of failures.
- Return a non-0 exit code if any have failed.
2020-12-24 21:00:35 +01:00
Lenny Maiorani
d46de3aeb4 Meta: Verify all AK test files are listed in CMake
Problem:
- It is possible for a new test file to be added to the `AK/Tests`
  directory without being added to the corresponding
  `CMakeLists.txt`. This results in the tests not being run.

Solution:
- As part of CI linting, verify that all the `AK/Tests/Test*.cpp`
  files are mentioned in the `CMakeLists.txt`.
2020-12-24 21:00:10 +01:00
Lenny Maiorani
e4ce485309 CMake: Decouple cmake utility functions from top-level CMakeLists.txt
Problem:
- These utility functions are only used in `AK`, but are being defined
  in the top-level. This clutters the top-level.

Solution:
- Move the utility functions to `Meta/CMake/utils.cmake` and include
  where needed.
- Also, move `all_the_debug_macros.cmake` into `Meta/CMake` directory
  to consolidate the location of `*.cmake` script files.
2020-12-24 11:02:04 +01:00
Lenny Maiorani
cef6b7b2e4 CMake: Use built-in add_compile_definitions for *_DEBUG macros
Problem:
- Modifying CXXFLAGS directly is an old CMake style.
- The giant and ever-growing list of `*_DEBUG` macros clutters the
  top-level CMakeLists.txt.

Solution:
- Use the more current `add_compile_definitions` function.
- Sort all the debug options so that they are easy to view.
- Move the `*_DEBUG` macros to their own file which can be included
  directly.
2020-12-22 21:01:51 +01:00
Ben Wiederhake
32210802ac Meta: Use the correct clang-format
After all the work of determining the correct name for clang-format-10,
we forgot to actually use it. How silly!
2020-12-22 20:41:35 +01:00
Liav A
b781495e74 Build: Create device files according to the new major-minor numbers 2020-12-21 00:19:21 +01:00
William Marlow
39364bdda4 Build: Embed application icons directly in the executables.
New serenity_app() targets can be defined which allows application
icons to be emedded directly into the executable. The embedded
icons will then be used when creating an icon for that file in
LibGUI.
2020-12-21 00:12:59 +01:00
Lenny Maiorani
765936ebae
Everywhere: Switch from (void) to [[maybe_unused]] (#4473)
Problem:
- `(void)` simply casts the expression to void. This is understood to
  indicate that it is ignored, but this is really a compiler trick to
  get the compiler to not generate a warning.

Solution:
- Use the `[[maybe_unused]]` attribute to indicate the value is unused.

Note:
- Functions taking a `(void)` argument list have also been changed to
  `()` because this is not needed and shows up in the same grep
  command.
2020-12-21 00:09:48 +01:00
Andreas Kling
822dc56ef3 LibGUI: Introduce GML - a simple GUI Markup Language :^)
This patch replaces the UI-from-JSON mechanism with a more
human-friendly DSL.

The current implementation simply converts the GML into a JSON object
that can be consumed by GUI::Widget::load_from_json(). The parser is
not very helpful if you make a mistake.

The language offers a very simple way to instantiate any registered
Core::Object class by simply saying @ClassName

@GUI::Label {
    text: "Hello friends!"
    tooltip: ":^)"
}

Layouts are Core::Objects and can be assigned to the "layout" property:

@GUI::Widget {
    layout: @GUI::VerticalBoxLayout {
        spacing: 2
        margins: [8, 8, 8, 8]
    }
}

And finally, child objects are simply nested within their parent:

@GUI::Widget {
    layout: @GUI::HorizontalBoxLayout {
    }
    @GUI::Button {
        text: "OK"
    }
    @GUI::Button {
        text: "Cancel"
    }
}

This feels a *lot* more pleasant to write than the JSON we had. The fact
that no new code was being written with the JSON mechanism was pretty
telling, so let's approach this with developer convenience in mind. :^)
2020-12-20 11:59:40 +01:00
Andreas Kling
48d74c5356 LibHTTP: Make HTTPRequest::from_raw_request() take a ReadonlyBytes
This allows us to get rid of some ByteBuffer::wrap() usage.
2020-12-19 18:29:13 +01:00
AnotherTest
eb7a943641 Meta: Set a 10 second timeout on Shell tests
Shell-backgrounding seems to be deadlocking at times, until that's
fixed, let's not take 5 hours to fail the build.
2020-12-15 20:58:32 +01:00
Itamar
b1e01066fe Lagom/Fuzzers: Fix creation of ELF::Loader 2020-12-14 23:05:53 +01:00
Itamar
e2d695eabb Build: Build libgcc_s.so
This is the shared version of libgcc
2020-12-14 23:05:53 +01:00
Ben Wiederhake
4b4f23165f Meta+LibHTTP: Fuzz HTTP request parsing 2020-12-02 10:46:40 +01:00
Ben Wiederhake
e85aad6acc Meta: Always check completeness of ALL_THE_DEBUG_MACROS 2020-12-01 11:06:53 +01:00
Ben Wiederhake
59091e1861 Meta: Nicer wording in lint 2020-12-01 11:06:53 +01:00
Ben Wiederhake
a0f6db246e Meta: Document fuzz dump 2020-12-01 11:06:23 +01:00
Luke
030dac8c4d Lagom/Fuzzers: Add URL fuzzer 2020-11-30 11:35:16 +01:00
Luke
5c911ad4b1 Meta: Add GitHub Actions workflow for Lagom with Fuzzers
There are cases where Lagom will build with GCC but not Clang.
This often goes unnoticed for a while as we don't often build with
Clang.

However, this is now important to test in CI because of the
OSS-Fuzz integration.

Note that this only tests the build, it does not run any tests.
Note that it also only builds LagomCore, Lagom and the fuzzers.
It does not build the other programs that use Lagom.
2020-11-29 20:22:56 +01:00
Nico Weber
29e4dc7634 Lagom: Mention OSS-Fuzz in ReadMe
We added OSS-Fuzz integration in #4154, but documentation about it
is spread across several pull requests, IRC, and issues. Let's collect
the important bits in the ReadMe.
2020-11-29 20:22:09 +01:00
AnotherTest
1ecea2f105 Lagom: Add a Shell parser fuzzer 2020-11-29 16:22:44 +01:00
Linus Groh
6a37285d93 Lagom: Add Regex fuzzers 2020-11-29 09:29:26 +01:00
DavidKorczynski
ff8ac60184
Lagom: Various fixes to make Lagom run on OSS-Fuzz (#4176) 2020-11-27 23:57:02 +01:00
Emanuel Sprung
55450055d8 LibRegex: Add a regular expression library
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:

- AK: Add options/flags and Errors for regular expressions

Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.

- AK: Add Lexer for regular expressions

The lexer parses the input and extracts tokens needed to parse a regular
expression.

- AK: Add regex Parser and PosixExtendedParser

This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.

- AK: Add regex matcher

This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.

To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:

```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle

EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");

result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line

EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");

EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```

- AK: Rework regex to work with opcodes objects

This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.

- AK: Add benchmark for regex

- AK: Some optimization in regex for runtime and memory

- LibRegex: Move regex to own Library and fix all the broken stuff

Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).

To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.

pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)

cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
  down the line (especially with shared libs).

Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
2020-11-27 21:32:41 +01:00
Nico Weber
f160df123b Lagom: Rename FuzzBMP to FuzzBMPLoader 2020-11-26 09:59:46 +01:00
Nico Weber
ec4c2f971b Lagom: Make BMP fuzzer look like the other image loader fuzzers 2020-11-26 09:59:46 +01:00
Nico Weber
4f12efe191 Lagom: Add fuzzers for remaining image loaders: ICO, PNG, PBM, PGM 2020-11-26 09:59:46 +01:00
Ben Wiederhake
ec36388f40 Meta: Reduce IRC spam
Fixes #4145.
2020-11-25 20:50:22 +01:00
Linus Groh
4e68f179d6 Lagom: Fix FuzzJs build
This was broken with the JS::Parser::Error position changes, but I don't
actually see a reason to do anything with the parser errors here, so
let's remove it and consider simply not crashing a success. :^)
2020-11-25 20:00:23 +01:00
Nico Weber
f13a38aa4d Lagom: Add a gif loader fuzzer 2020-11-20 21:17:58 +01:00
Nico Weber
812892ef86 Lagom: Add a jpg fuzzer 2020-11-19 21:21:45 +01:00
Nico Weber
3f3a7d128b Lagom: Make fuzzer cmake less repetitive 2020-11-19 21:21:45 +01:00
Nico Weber
f5967c4745 Lagom: Add a PPM fuzzer
It finds the problem fixed in 69518bd178 but nothing else.
2020-11-19 14:04:35 +01:00
Ben Wiederhake
c4933f9bd7 Meta: Fix IRC notification script
What a silly mistake. How did I manage to do that?
2020-11-14 17:14:45 +01:00
Ben Wiederhake
ef9ac8a8a2 Meta: Use SerenityBot for IRC notifications
This avoids "useless" join/part notifications.
2020-11-14 15:30:29 +01:00
Nico Weber
39ce7251f7 Lagom: Use -fsanitize=fuzzer, not -fsanitize=fuzzer-no-link
Fuzzers don't link for me without this change.
2020-11-14 10:09:03 +01:00
Nico Weber
d775dea13c Lagom: Add a gemini fuzzer
Didn't find anything interesting, but might as well check it in.
2020-11-14 10:09:03 +01:00
Nico Weber
8ed83f9a1d Lagom: Augment fuzzing readme a bit
Recommend using asan, don't set the c compiler (c++ compiler is
sufficient), mention how to run on several cores, and how to get less
output.
2020-11-14 10:09:03 +01:00
Ben Wiederhake
cf9419fc4f Meta: Nicer IRC notifications 2020-11-13 08:56:56 +01:00
Nico Weber
5c2e8b6189 Lagom: Add ntpquery to lagom build 2020-11-12 21:21:33 +01:00
Nico Weber
5fcd34b810 Userland: Add an "adjtime" utility
It's a thin userland wrapper around adjtime(2). It can be used
to view current pending time adjustments, and root can use it to
smoothly adjust the system time.

As far as I can tell, other systems don't have a userland utility
for this, but it seems useful. Useful enough that I'm adding it to
the lagom build so I can use it on my linux box too :)
2020-11-10 19:03:08 +01:00
Linus Groh
46cd9157cd Lagom/Fuzzers: Add a Dockerfile for FuzzilliJs
Based on Fedora. This allows building and running FuzzilliJs and Fuzzilli
itself in a Docker/Podman container.
2020-11-08 11:01:07 +01:00
Ben Wiederhake
3cd39097f6 Meta: Add script that runs all lints 2020-11-08 09:58:55 +01:00
Linus Groh
2c9e6585f8 Documentation: Update required GCC version to >= 10
I initially thought as long as Lagom is not built >= 9 would be fine,
but LagomCore is always built for the code generators.
2020-11-07 18:22:18 +01:00
Luke
8928607455 Lagom/Fuzzers: Add Fuzzilli version of FuzzJs
Fuzzilli is a JavaScript engine fuzzer made by googleprojectzero.

https://github.com/googleprojectzero/fuzzilli/
2020-11-07 10:09:55 +01:00
Linus Groh
21912123c4 Travis: Update host GCC to version 10
This allows us to use the latest C++20 features in programs which we
compile and run as part of Lagom.
2020-11-02 13:12:35 +01:00
Brendan Coles
c1bfb8cb0e Meta: lint-shell-scripts: Exit if shellcheck is not installed 2020-11-01 10:29:33 +01:00
AnotherTest
37c089fb7b LibTLS: (Almost) verify certificate chain against root CA certificates
Also adds a very primitive systemwide ca_certs.ini file.
2020-10-30 23:42:03 +01:00
asynts
607931268e CMake: Use CONFIGURE_DEPENDS in existing globs. 2020-10-29 11:52:47 +01:00