This was done with the help of several scripts, I dump them here to
easily find them later:
awk '/#ifdef/ { print "#cmakedefine01 "$2 }' AK/Debug.h.in
for debug_macro in $(awk '/#ifdef/ { print $2 }' AK/Debug.h.in)
do
find . \( -name '*.cpp' -o -name '*.h' -o -name '*.in' \) -not -path './Toolchain/*' -not -path './Build/*' -exec sed -i -E 's/#ifdef '$debug_macro'/#if '$debug_macro'/' {} \;
done
# Remember to remove WRAPPER_GERNERATOR_DEBUG from the list.
awk '/#cmake/ { print "set("$2" ON)" }' AK/Debug.h.in
Else, there's tons of "-- Set runtime path of" spam at build time,
with apparently no way of disabling the build noise other than turning
of rpaths. If the dynamic loader uses them at some point, we probably
want to set them through cflags/ldflags instead of through cmake's
built-in thing anyways, for that reason.
Modify the user mode runtime to insert stack canaries to find stack corruptions.
The `-fstack-protector-strong` variant was chosen because it catches more
issues than vanilla `-fstack-protector`, but doesn't have substantial
performance impact like `-fstack-protector-all`.
Details:
-fstack-protector enables stack protection for vulnerable functions that contain:
* A character array larger than 8 bytes.
* An 8-bit integer array larger than 8 bytes.
* A call to alloca() with either a variable size or a constant size bigger than 8 bytes.
-fstack-protector-strong enables stack protection for vulnerable functions that contain:
* An array of any size and type.
* A call to alloca().
* A local variable that has its address taken.
Example of it catching corrupting in the `stack-smash` test:
```
courage ~ $ ./user/Tests/LibC/stack-smash
[+] Starting the stack smash ...
Error: Stack protector failure, stack smashing detected!
Shell: Job 1 (/usr/Tests/LibC/stack-smash) Aborted
```
RTTI is still disabled for the Kernel, and for the Dynamic Loader. This
allows for much less awkward navigation of class heirarchies in LibCore,
LibGUI, LibWeb, and LibJS (eventually). Measured RootFS size increase
was < 1%, and libgui.so binary size was ~3.3%. The small binary size
increase here seems worth it :^)
* Add SERENITY_ARCH option to CMake for selecting the target toolchain
* Port all build scripts but continue to use i686
* Update GitHub Actions cache to include BuildIt.sh
Previosuly, generation of the SONAME attribute was disabled.
This caused libraries to have relative paths in DT_NEEDED attributes
(e.g "Libraries/libcore.so" instead of just "libcore.so"),
which caused build errors when the working directory during build was
not $SERENITY_ROOT/Build.
This caused the build of ports that use libraries other than libc.so
to fail (e.g the nesalizer port).
Closes#4457
We now configure the gcc spec files to use a different crt files for
static & PIE binaries.
This relieves us from the need to explicitly specify the desired crt0
file in cmake scripts.
Problem:
- These utility functions are only used in `AK`, but are being defined
in the top-level. This clutters the top-level.
Solution:
- Move the utility functions to `Meta/CMake/utils.cmake` and include
where needed.
- Also, move `all_the_debug_macros.cmake` into `Meta/CMake` directory
to consolidate the location of `*.cmake` script files.
Problem:
- Modifying CXXFLAGS directly is an old CMake style.
- The giant and ever-growing list of `*_DEBUG` macros clutters the
top-level CMakeLists.txt.
Solution:
- Use the more current `add_compile_definitions` function.
- Sort all the debug options so that they are easy to view.
- Move the `*_DEBUG` macros to their own file which can be included
directly.
Problem:
- Functions are duplicated in [PBM,PGM,PPM]Loader class
implementations. They are functionally equivalent. This does not
follow the DRY (Don't Repeat Yourself) principle.
Solution:
- Factor out the common functions into a separate file.
- Refactor common code to generic functions.
- Change `PPM_DEBUG` macro to be `PORTABLE_IMAGE_LOADER_DEBUG` to work
with all the supported types. This requires adding the image type to
the debug log messages for easier debugging.
Problem:
- Appending to CMAKE_CXX_FLAGS for everything is cumbersome.
Solution:
- Use the `add_compile_options` built-in function to handle adding
compiler options (and even de-duplicating).
Problem:
- Setting `CMAKE_CXX_FLAGS` directly to effect the version of the C++
standard being used is no longer the recommended best practice.
Solution:
- Set C++20 mode in the compiler by setting `CMAKE_CXX_STANDARD`.
- Force the build system generator not to fallback to the latest
standard supported by the compiler by enabling
`CMAKE_CXX_STANDARD_REQUIRED`. This shouldn't ever be a problem
though since the toolchain is tightly controlled.
- Disable GNU compiler extensions by disabling `CMAKE_CXX_EXTENSIONS`
to preserve the previous flags.
This new subsystem is somewhat replacing the IDE disk code we had with a
new flexible design.
StorageDevice is a generic class that represent a generic storage
device. It is meant that specific storage hardware will override the
interface. StorageController is a generic class that represent
a storage controller that can be found in a machine.
The IDEController class governs two IDEChannels. An IDEChannel is
responsible to manage the master & slave devices of the channel,
therefore an IDEChannel is an IRQHandler.
New serenity_app() targets can be defined which allows application
icons to be emedded directly into the executable. The embedded
icons will then be used when creating an icon for that file in
LibGUI.
This patch replaces the UI-from-JSON mechanism with a more
human-friendly DSL.
The current implementation simply converts the GML into a JSON object
that can be consumed by GUI::Widget::load_from_json(). The parser is
not very helpful if you make a mistake.
The language offers a very simple way to instantiate any registered
Core::Object class by simply saying @ClassName
@GUI::Label {
text: "Hello friends!"
tooltip: ":^)"
}
Layouts are Core::Objects and can be assigned to the "layout" property:
@GUI::Widget {
layout: @GUI::VerticalBoxLayout {
spacing: 2
margins: [8, 8, 8, 8]
}
}
And finally, child objects are simply nested within their parent:
@GUI::Widget {
layout: @GUI::HorizontalBoxLayout {
}
@GUI::Button {
text: "OK"
}
@GUI::Button {
text: "Cancel"
}
}
This feels a *lot* more pleasant to write than the JSON we had. The fact
that no new code was being written with the JSON mechanism was pretty
telling, so let's approach this with developer convenience in mind. :^)
We need to account for how many shared lock instances the current
thread owns, so that we can properly release such references when
yielding execution.
We also need to release the process lock when donating.
The dynamic loader exists as /usr/lib/Loader.so and is loaded by the
kernel when ET_DYN programs are executed.
The dynamic loader is responsible for loading the dependencies of the
main program, allocating TLS storage, preparing all loaded objects for
execution and finally jumping to the entry of the main program.
This prevents zombies created by multi-threaded applications and brings
our model back to closer to what other OSs do.
This also means that SIGSTOP needs to halt all threads, and SIGCONT needs
to resume those threads.
This changes the Thread::wait_on function to not enable interrupts
upon leaving, which caused some problems with page fault handlers
and in other situations. It may now be called from critical
sections, with interrupts enabled or disabled, and returns to the
same state.
This also requires some fixes to Lock. To aid debugging, a new
define LOCK_DEBUG is added that enables checking for Lock leaks
upon finalization of a Thread.
This commit is a mix of several commits, squashed into one because the
commits before 'Move regex to own Library and fix all the broken stuff'
were not fixable in any elegant way.
The commits are listed below for "historical" purposes:
- AK: Add options/flags and Errors for regular expressions
Flags can be provided for any possible flavour by adding a new scoped enum.
Handling of flags is done by templated Options class and the overloaded
'|' and '&' operators.
- AK: Add Lexer for regular expressions
The lexer parses the input and extracts tokens needed to parse a regular
expression.
- AK: Add regex Parser and PosixExtendedParser
This patchset adds a abstract parser class that can be derived to implement
different parsers. A parser produces bytecode to be executed within the
regex matcher.
- AK: Add regex matcher
This patchset adds an regex matcher based on the principles of the T-REX VM.
The bytecode pruduced by the respective Parser is put into the matcher and
the VM will recursively execute the bytecode according to the available OpCodes.
Possible improvement: the recursion could be replaced by multi threading capabilities.
To match a Regular expression, e.g. for the Posix standard regular expression matcher
use the following API:
```
Pattern<PosixExtendedParser> pattern("^.*$");
auto result = pattern.match("Well, hello friends!\nHello World!"); // Match whole needle
EXPECT(result.count == 1);
EXPECT(result.matches.at(0).view.starts_with("Well"));
EXPECT(result.matches.at(0).view.end() == "!");
result = pattern.match("Well, hello friends!\nHello World!", PosixFlags::Multiline); // Match line by line
EXPECT(result.count == 2);
EXPECT(result.matches.at(0).view == "Well, hello friends!");
EXPECT(result.matches.at(1).view == "Hello World!");
EXPECT(pattern.has_match("Well,....")); // Just check if match without a result, which saves some resources.
```
- AK: Rework regex to work with opcodes objects
This patchsets reworks the matcher to work on a more structured base.
For that an abstract OpCode class and derived classes for the specific
OpCodes have been added. The respective opcode logic is contained in
each respective execute() method.
- AK: Add benchmark for regex
- AK: Some optimization in regex for runtime and memory
- LibRegex: Move regex to own Library and fix all the broken stuff
Now regex works again and grep utility is also in place for testing.
This commit also fixes the use of regex.h in C by making `regex_t`
an opaque (-ish) type, which makes its behaviour consistent between
C and C++ compilers.
Previously, <regex.h> would've blown C compilers up, and even if it
didn't, would've caused a leak in C code, and not in C++ code (due to
the existence of `OwnPtr` inside the struct).
To make this whole ordeal easier to deal with (for now), this pulls the
definitions of `reg*()` into LibRegex.
pros:
- The circular dependency between LibC and LibRegex is broken
- Eaiser to test (without accidentally pulling in the host's libc!)
cons:
- Using any of the regex.h functions will require the user to link -lregex
- The symbols will be missing from libc, which will be a big surprise
down the line (especially with shared libs).
Co-Authored-By: Ali Mohammad Pur <ali.mpfard@gmail.com>
Problem:
- CMake is not outputting `compile_commands.json`.
- `compile_commands.json` is used by build integration tooling such as
`clang-tidy`.
Solution:
- Enable `CMAKE_EXPORT_COMPILE_COMMANDS` option so that the file is
output.
