Commit Graph

208 Commits

Author SHA1 Message Date
Ali Mohammad Pur
578d73943a LibC+LibRegex: Move central regex definitions into LibC/bits
This decouples LibRegex from the serenity LibC.
Fixes #15251.
2022-09-20 12:57:21 +01:00
Ben Wiederhake
7c5e30daaa Everywhere: Fix badly-formatted includes 2022-09-17 04:00:54 +00:00
Tim Schumacher
8763dbcccc Everywhere: Remove a bunch of dead write-only variables
LLVM 15 now warns (and thus errors) about this, and there is really no
point in keeping them.
2022-09-16 05:39:28 +00:00
Ali Mohammad Pur
660d2b53b1 LibRegex: Account for eof after \<x> when 'x' leads to legacy behaviour 2022-09-12 16:03:57 +04:30
Ali Mohammad Pur
48442059fc LibRegex: Consume exactly two chars for escaped characters
We were previously consuming an extra char afterwards, which could be
the charclass terminator, leading to possible OOB accesses.
2022-09-12 16:03:57 +04:30
Timothy Flynn
48cb15283a LibRegex: Explicitly check if a character falls into a table-based range
Previously, for a regex such as /[a-sy-z]/i, we would incorrectly think
the character "u" fell into the range "a-s" because neither of the
conditions "u > s && U > s" or "u < a && U < a" would be true, resulting
in the lookup falling back to assuming the character is in the range.

Instead, first explicitly check if the character falls into the range,
rather than checking if it falls outside the range. If the explicit
checks fail, then we know the character is outside the range.
2022-08-29 16:34:47 -04:00
Ali Mohammad Pur
e43b478920 LibRegex: Check code unit count range when accessing by code unit count 2022-07-20 21:25:59 +01:00
Ali Mohammad Pur
598dc74a76 LibRegex: Partially implement the ECMAScript unicodeSets proposal
This skips the new string unicode properties additions, along with \q{}.
2022-07-20 21:25:59 +01:00
Ali Mohammad Pur
7734914909 LibRegex: Refactor parsing 'CharacterEscape' out of 'AtomEscape'
The ECMA262 spec has this as a separate production, and we need it to be
split up for a future commit.
2022-07-20 21:25:59 +01:00
Ali Mohammad Pur
b908f9f6ef LibRegex: Pass parse flags as a struct instead of multiple arguments 2022-07-20 21:25:59 +01:00
sin-ack
5422691f07 LibRegex: Remove RegexStringView(char const*) constructor
This allowed passing in a nullptr for the StringView which will not be
possible once StringView(char const*) is removed.
2022-07-12 23:11:35 +02:00
sin-ack
fbc771efe9 Everywhere: Use default StringView constructor over nullptr
While null StringViews are just as bad, these prevent the removal of
StringView(char const*) as that constructor accepts a nullptr.

No functional changes.
2022-07-12 23:11:35 +02:00
sin-ack
3f3f45580a Everywhere: Add sv suffix to strings relying on StringView(char const*)
Each of these strings would previously rely on StringView's char const*
constructor overload, which would call __builtin_strlen on the string.
Since we now have operator ""sv, we can replace these with much simpler
versions. This opens the door to being able to remove
StringView(char const*).

No functional changes.
2022-07-12 23:11:35 +02:00
sin-ack
c70f45ff44 Everywhere: Explicitly specify the size in StringView constructors
This commit moves the length calculations out to be directly on the
StringView users. This is an important step towards the goal of removing
StringView(char const*), as it moves the responsibility of calculating
the size of the string to the user of the StringView (which will prevent
naive uses causing OOB access).
2022-07-12 23:11:35 +02:00
Ali Mohammad Pur
d348eaf305 LibRegex: Treat inverted Compare entries as disjunctions
[^XYZ] is not(X | Y | Z), we used to translate this to
not(X) | not(Y) | not(Z), this commit makes LibRegex interpret this
pattern as not(X) & not(Y) & not(Z).
2022-07-10 14:26:03 +02:00
Ali Mohammad Pur
fe46b2c141 LibRegex: Correctly track current inversion state in the optimizer
This is currently not important as we do not nest TemporaryInverse.
2022-07-10 14:26:03 +02:00
Ali Mohammad Pur
9c5febe800 LibRegex: Flush compare tables before entering a permanent inverse state 2022-07-10 14:26:03 +02:00
Ali Mohammad Pur
b85666b3d2 LibRegex: Fix lookup table-based range checks in Compare
The lowercase version of a range is not required to be a valid range,
instead of casefolding the range and making it invalid, check twice with
both cases of the input character (which are the same as the input if
not insensitive).
This time includes an actual test :^)
2022-07-09 01:00:44 +00:00
Ali Mohammad Pur
5f012778b8 LibRegex: Use the correct values for comparing LUT entries
Previously we were ignoring the insensitive flag for LUT lookups.
2022-07-05 07:19:13 +02:00
Ali Mohammad Pur
7d01ee63d6 LibRegex: Use proper CharRange constructor instead of bit_casting
Otherwise the range order would be inverted.
2022-07-05 07:19:13 +02:00
Ali Mohammad Pur
6e655b7f89 LibRegex: Fully interpret the Compare Op when looking for overlaps
We had a really naive and simplistic implementation, which lead to
various issues where the optimiser incorrectly rewrote the regex to use
atomic groups; this commit fixes that.
2022-07-04 23:09:53 +02:00
Ali Mohammad Pur
1409a48da6 LibRegex: Check inverse_matched after every op, not just at the end
Fixes #13755.

Co-Authored-By: Damien Firmenich <fir.damien@gmail.com>
2022-04-22 10:02:39 +02:00
Idan Horowitz
086969277e Everywhere: Run clang-format 2022-04-01 21:24:45 +01:00
Ali Mohammad Pur
97a333608e LibRegex: Make codegen+optimisation for alternatives much faster
Just a little thinking outside the box, and we can now parse and
optimise a million copies of "a|" chained together in just a second :^)
2022-02-20 11:53:59 +01:00
Ali Mohammad Pur
4be7239626 LibRegex: Make parse_disjunction() consume all disjunctions in one frame
This helps us not blow up when too many disjunctions are chained togther
in the regex we're parsing.
Fixes #12615.
2022-02-20 11:53:59 +01:00
Ali Mohammad Pur
627bbee055 LibRegex: Allow quantifiers after quantifiable assertions
While quantifying assertions is very much meaningless, the specification
allows them with annex B's extended grammar for browsers, so read and
apply the quantifiers.
Fixes #12373.
2022-02-20 11:53:59 +01:00
Ali Mohammad Pur
3b0943d24c LibRegex: Correct the alternative matching order when one is empty
Previously we were compiling `/a|/` into what effectively would be
`/|a`, which is clearly incorrect.
2022-02-14 11:30:50 +01:00
Ali Mohammad Pur
6a4c8a66ae LibRegex: Only skip full instructions when optimizing alternations
It makes no sense to skip half of an instruction, so make sure to skip
only full instructions!
2022-02-09 21:02:24 +00:00
Timothy Flynn
2212aa2388 LibRegex: Support non-ASCII whitespace characters when matching \s or \S
ECMA-262 defines \s as:

    Return the CharSet containing all characters corresponding to a code
    point on the right-hand side of the WhiteSpace or LineTerminator
    productions.

The LineTerminator production is simply: U+000A, U+000D, U+2028, or
U+2029. Unfortunately there isn't a Unicode property that covers just
those code points.

The WhiteSpace production is: U+0009, U+000B, U+000C, U+FEFF, or any
code point with the Space_Separator general category.

If the Unicode generators are disabled, this will fall back to ASCII
space code points.
2022-02-05 22:30:10 +03:30
Timothy Flynn
3729fd06fa LibRegex: Do not return an Optional from Regex::Matcher::execute
The code path that could return an optional no longer exists as of
commit: a962ee020a
2022-02-05 19:06:50 +03:30
Timothy Flynn
27d3de1f17 LibRegex: Do not continue searching input when the sticky bit is set
This partially reverts commit a962ee020a.

When the sticky bit is set, the global bit should basically be ignored
except by external callers who want their own special behavior. For
example, RegExp.prototype [ @@match ] will use the global flag to
accumulate consecutive matches. But on the first failure, the regex
loop should break.
2022-02-05 19:06:50 +03:30
Ali Mohammad Pur
a962ee020a LibJS+LibRegex: Don't repeat regex match in regexp_exec()
LibRegex already implements this loop in a more performant way, so all
LibJS has to do here is to return things in the right shape, and not
loop over the input string.
Previously this was a quadratic operation on string length, which lead
to crazy execution times on failing regexps - now it's nice and fast :^)

Note that a Regex test has to be updated to remove the stateful flag as
it repeats matching on multiple strings.
2022-02-05 00:09:32 +01:00
Ali Mohammad Pur
2b028f6faa LibRegex+LibJS: Avoid searching for more than one match in JS RegExps
All of JS's regular expression APIs only want a single match, so avoid
trying to produce more (which will be discarded anyway).
2022-02-05 00:09:32 +01:00
Ali Mohammad Pur
5fac41f733 LibRegex: Implement ECMA262 multiline matching without splitting lines
As ECMA262 regex allows `[^]` and literal newlines to match newlines in
the input string, we shouldn't split the input string into lines, rather
simply make boundaries and catchall patterns capable of checking for
these conditions specifically.
2022-01-26 00:53:09 +03:30
Ali Mohammad Pur
aa20210119 LibRegex: Don't return empty vectors from RegexStringView::lines()
Instead, return a vector of one empty string.
2022-01-26 00:53:09 +03:30
Ali Mohammad Pur
cd83325c7c LibRegex: Preserve capture groups and matches across ForkReplace
This makes the (flawed) ForkStay inserted as a loop header unnecessary,
and finally fixes LibRegex rewriting weird loops in weird ways.
2022-01-22 00:35:49 +00:00
Ali Mohammad Pur
bc20e4f71d LibRegex: Add some more information to Compare::Reference debug output 2022-01-22 00:35:49 +00:00
Ali Mohammad Pur
97dde09170 LibRegex: Allow ClearCaptureGroup to create new groups
Instead of leaking all capture groups and selectively clearing some,
simply avoid leaking things and only "define" the ones that need to
exist.
This *actually* implements the capture groups ECMA262 quirk.
Also adds the test removed in the previous commit (to avoid messing up
test runs across bisects).
2022-01-22 00:35:49 +00:00
Ali Mohammad Pur
704e0654b3 Revert "LibRegex: Implement an ECMA262 Regex quirk with negative loo..."
This partially reverts commit c11be92e23.
That commit fixes one thing and breaks many more, a next commit will
implement this quirk in a more sane way.
2022-01-22 00:35:49 +00:00
Ali Mohammad Pur
9eccd4c56e LibRegex: Allow the pattern to match the zero-length end of the string
...only if Multiline is not enabled.
Fixes #11940.
2022-01-21 18:14:08 +03:30
Ali Mohammad Pur
c11be92e23 LibRegex: Implement an ECMA262 Regex quirk with negative lookarounds
This implements the quirk defined by "Note 3" in section "Canonicalize"
(https://tc39.es/ecma262/#sec-runtime-semantics-canonicalize-ch).

Crosses off another quirk from #6042.
2022-01-21 18:14:08 +03:30
Ali Mohammad Pur
bfe8f312f3 LibRegex: Correct jump offset to the start of the loop block
Previously we were jumping to the new end of the previous block (created
by the newly inserted ForkStay), correct the offset to jump to the
correct block as shown in the comments.
Fixes #12033.
2022-01-21 18:14:08 +03:30
Ali Mohammad Pur
9de33629da AK+Everywhere: Make Variant::visit() respect the Variant's constness
...and fix all the instances of visit() taking non-const arguments.
2022-01-14 11:35:40 +03:30
Ali Mohammad Pur
1a35e27490 LibRegex: Make FailForks fail all forks up to the last save point
This makes negative lookarounds with more than one fork behave
correctly.
Fixes #11350.
2021-12-25 18:41:10 +01:00
Hendiadyoin1
27a967371b LibRegex: Remove duplicate declaration of execution_result_name(...) 2021-12-21 18:17:28 -08:00
Hendiadyoin1
303af07df8 LibRegex: Use AK::any_of in Parser::lookahead_any
Equivalent to std::ranges::any_of, which clang-tidy suggests.
2021-12-21 18:17:28 -08:00
Hendiadyoin1
0f4a79a24d LibRegex: Capture this explicitly in RegexStringView::equals lambda
This stops clang-tidy from suggesting that this function can be made
static, although accessing `this->operator==` in the lambda function.
2021-12-21 18:17:28 -08:00
Hendiadyoin1
5885e70df7 LibRegex: Remove some meaningless/useless const-qualifiers
Also replace String creation from `""` with `String::empty()`
2021-12-21 18:17:28 -08:00
Hendiadyoin1
ca69ded9a5 LibRegex: Collapse some if(...) return true; else return false; blocks 2021-12-21 18:17:28 -08:00
Hendiadyoin1
a2563496f5 LibRegex: Remove some else-after-returns 2021-12-21 18:17:28 -08:00