Commit Graph

80 Commits

Author SHA1 Message Date
Liav A
b2fd51f561 AK: Implement string formatting for FixedStringBuffers
To ensure this happens without duplicating code, we allow forcing a
StringBuilder object to only use the inline buffer, so the code in the
AK/Format.cpp file doesn't need to deal with different underlying
storage types (expandable or inline-fixed) at all.
2023-08-12 11:48:48 -06:00
Linus Groh
e76394d96c AK: Remove infallible version of StringBuilder::to_byte_buffer
Also drop the try_ prefix from the fallible function, as it is no longer
needed to distinguish the two.
2023-03-09 15:51:00 +00:00
Karol Baraniecki
b4b283670d AK: Introduce a fallible version of StringBuilder::to_byte_buffer
Name it StringBuilder::try_to_byte_buffer accordingly :^)
2023-03-09 12:59:57 +00:00
Sam Atkins
1453ac79e7 AK: Add StringBuilder::to_fly_string() 2023-02-15 12:48:26 -05:00
Linus Groh
6e7459322d AK: Remove StringBuilder::build() in favor of to_deprecated_string()
Having an alias function that only wraps another one is silly, and
keeping the more obvious name should flush out more uses of deprecated
strings.
No behavior change.
2023-01-27 20:38:49 +00:00
Timothy Flynn
39bda0073e AK: Make StringBuilder::try_append_code_point actually fallible
It currently uses the non-fallible `append` method to append each UTF-8
encoded byte of the code point.
2023-01-08 12:13:15 +01:00
Ben Wiederhake
c2a900b853 Everywhere: Remove unused includes of AK/StdLibExtras.h
These instances were detected by searching for files that include
AK/StdLibExtras.h, but don't match the regex:

\\b(abs|AK_REPLACED_STD_NAMESPACE|array_size|ceil_div|clamp|exchange|for
ward|is_constant_evaluated|is_power_of_two|max|min|mix|move|_RawPtr|RawP
tr|round_up_to_power_of_two|swap|to_underlying)\\b

(Without the linebreaks.)

This regex is pessimistic, so there might be more files that don't
actually use any "extra stdlib" functions.

In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
2023-01-02 20:27:20 -05:00
Ali Mohammad Pur
543890c5c9 AK: Add a fallible StringBuilder::create() factory function
This is nice, and is also used by the Jakt runtime.
2022-12-11 20:44:54 +03:30
Andreas Kling
a3e82eaad3 AK: Introduce the new String, replacement for DeprecatedString
DeprecatedString (formerly String) has been with us since the start,
and it has served us well. However, it has a number of shortcomings
that I'd like to address.

Some of these issues are hard if not impossible to solve incrementally
inside of DeprecatedString, so instead of doing that, let's build a new
String class and then incrementally move over to it instead.

Problems in DeprecatedString:

- It assumes string allocation never fails. This makes it impossible
  to use in allocation-sensitive contexts, and is the reason we had to
  ban DeprecatedString from the kernel entirely.

- The awkward null state. DeprecatedString can be null. It's different
  from the empty state, although null strings are considered empty.
  All code is immediately nicer when using Optional<DeprecatedString>
  but DeprecatedString came before Optional, which is how we ended up
  like this.

- The encoding of the underlying data is ambiguous. For the most part,
  we use it as if it's always UTF-8, but there have been cases where
  we pass around strings in other encodings (e.g ISO8859-1)

- operator[] and length() are used to iterate over DeprecatedString one
  byte at a time. This is done all over the codebase, and will *not*
  give the right results unless the string is all ASCII.

How we solve these issues in the new String:

- Functions that may allocate now return ErrorOr<String> so that ENOMEM
  errors can be passed to the caller.

- String has no null state. Use Optional<String> when needed.

- String is always UTF-8. This is validated when constructing a String.
  We may need to add a bypass for this in the future, for cases where
  you have a known-good string, but for now: validate all the things!

- There is no operator[] or length(). You can get the underlying data
  with bytes(), but for iterating over code points, you should be using
  an UTF-8 iterator.

Furthermore, it has two nifty new features:

- String implements a small string optimization (SSO) for strings that
  can fit entirely within a pointer. This means up to 3 bytes on 32-bit
  platforms, and 7 bytes on 64-bit platforms. Such small strings will
  not be heap-allocated.

- String can create substrings without making a deep copy of the
  substring. Instead, the superstring gets +1 refcount from the
  substring, and it acts like a view into the superstring. To make
  substrings like this, use the substring_with_shared_superstring() API.

One caveat:

- String does not guarantee that the underlying data is null-terminated
  like DeprecatedString does today. While this was nifty in a handful of
  places where we were calling C functions, it did stand in the way of
  shared-superstring substrings.
2022-12-06 15:21:26 +01:00
Linus Groh
57dc179b1f Everywhere: Rename to_{string => deprecated_string}() where applicable
This will make it easier to support both string types at the same time
while we convert code, and tracking down remaining uses.

One big exception is Value::to_string() in LibJS, where the name is
dictated by the ToString AO.
2022-12-06 08:54:33 +01:00
Linus Groh
6e19ab2bbc AK+Everywhere: Rename String to DeprecatedString
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
2022-12-06 08:54:33 +01:00
Lucas CHOLLET
62b8ccaffc StringBuilder: Add try_append_repeated() and append_repeated()
This two methods add the character as many times as specified by the
second parameter.
2022-09-15 14:08:21 +01:00
sin-ack
3f3f45580a Everywhere: Add sv suffix to strings relying on StringView(char const*)
Each of these strings would previously rely on StringView's char const*
constructor overload, which would call __builtin_strlen on the string.
Since we now have operator ""sv, we can replace these with much simpler
versions. This opens the door to being able to remove
StringView(char const*).

No functional changes.
2022-07-12 23:11:35 +02:00
Idan Horowitz
9da8c78133 AK: Add a try variant of StringBuilder::append_escaped_for_json
This will allow us to make a fallible version of the JSON serializers.
2022-02-27 20:37:57 +01:00
Idan Horowitz
8f093e91e0 AK: Exclude StringBuilder String APIs from the Kernel
These APIs are only used by userland, and String is OOM-infallible,
so let's just ifdef it out of the Kernel.
2022-02-16 22:21:37 +01:00
Andreas Kling
c74b6c06a5 AK: Use ByteBuffer::append(u8) in StringBuilder single-char append 2022-02-13 14:44:36 +01:00
Sam Atkins
45cf40653a Everywhere: Convert ByteBuffer factory methods from Optional -> ErrorOr
Apologies for the enormous commit, but I don't see a way to split this
up nicely. In the vast majority of cases it's a simple change. A few
extra places can use TRY instead of manual error checking though. :^)
2022-01-24 22:36:09 +01:00
Daniel Bertalan
1d2f78682b Kernel+AK: Eliminate a couple of temporary String allocations 2021-12-30 14:16:03 +01:00
Andreas Kling
008355c222 AK: Add failable try_* functions to StringBuilder
These will allow us to start using TRY() with StringBuilder operations.
2021-11-17 00:21:13 +01:00
Andreas Kling
8b1108e485 Everywhere: Pass AK::StringView by value 2021-11-11 01:27:46 +01:00
Andreas Kling
a15ed8743d AK: Make ByteBuffer::try_* functions return ErrorOr<void>
Same as Vector, ByteBuffer now also signals allocation failure by
returning an ENOMEM Error instead of a bool, allowing us to use the
TRY() and MUST() patterns.
2021-11-10 21:58:58 +01:00
Daniel Bertalan
fccb06b2cd AK: Use UnicodeUtils::code_point_to_utf8 in StringBuilder 2021-10-15 21:50:19 -07:00
Ali Mohammad Pur
27e3589f61 AK+Kernel: Avoid unescaped control chars in append_escaped_for_json()
Otherwise it could produce invalid JSON.
2021-09-13 14:38:53 +04:30
Ali Mohammad Pur
97e97bccab Everywhere: Make ByteBuffer::{create_*,copy}() OOM-safe 2021-09-06 01:53:26 +02:00
Ali Mohammad Pur
3a9f00c59b Everywhere: Use OOM-safe ByteBuffer APIs where possible
If we can easily communicate failure, let's avoid asserting and report
failure instead.
2021-09-06 01:53:26 +02:00
Timothy Flynn
c16aca7abf AK+Kernel: Add StringBuilder::append overload for UTF-16 views
Currently, to append a UTF-16 view to a StringBuilder, callers must
first convert the view to UTF-8 and then append the copy. Add a UTF-16
overload so callers do not need to hold an entire copy in memory.
2021-08-10 23:07:50 +02:00
Timothy Flynn
5978caf96b AK: Convert StringBuilder to use east-const 2021-08-10 23:07:50 +02:00
Gunnar Beutner
de0aa44bb6 AK: Remove the m_length member for StringBuilder
Instead we can just use ByteBuffer::size() which already keeps track
of the buffer's size.
2021-05-31 14:49:00 +04:30
Gunnar Beutner
4c32a128ef AK: Fix accidentally-quadratic behavior in StringBuilder
Found by OSS Fuzz:

Related commit: 3908a49661

Co-authored-by: Ben Wiederhake <BenWiederhake.GitHub@gmx.de>
2021-05-31 14:49:00 +04:30
Gunnar Beutner
8f755c9d07 AK: Use ByteBuffer::append for the StringBuilder class
Previously the StringBuilder class would use memcpy() to write
directly into the ByteBuffer's buffer. Instead we should use the
append() method which ensures we don't overrun the buffer.
2021-05-31 14:49:00 +04:30
Gunnar Beutner
5f18cf75c5 AK: Replace ByteBuffer::grow with resize()/ensure_capacity()
Previously ByteBuffer::grow() behaved like Vector<T>::resize().
However the function name was somewhat ambiguous - and so this patch
updates ByteBuffer to behave more like Vector<T> by replacing grow()
with resize() and adding an ensure_capacity() method.

This also lets the user change the buffer's capacity without affecting
the size which was not previously possible.

Additionally this patch makes the capacity() method public (again).
2021-05-31 14:49:00 +04:30
Ben Wiederhake
dfd988707c Revert "AK: Fix accidentally-quadratic behavior in StringBuilder"
This reverts commit 2d011961c9.
2021-05-30 21:39:39 +01:00
Ben Wiederhake
2d011961c9 AK: Fix accidentally-quadratic behavior in StringBuilder
Found by OSS Fuzz:
#34451 (old bug)

Related commit: 3908a49661
2021-05-30 14:39:30 +01:00
Gunnar Beutner
598d7f4127 AK: StringBuilder should prefer to use its inline capacity first
Previously StringBuilder would start allocating an external buffer
once the caller has used up more than half of the inline buffer's
capacity. Instead we should prefer to use the inline buffer until
it is full and only then start to allocate an external buffer.
2021-05-18 21:49:10 +02:00
Max Wipfli
f51b0729f5 AK: Implement StringBuilder::append_as_lowercase(char ch)
This patch adds a convenience method to AK::StringBuilder which converts
ASCII uppercase characters to lowercase before appending them.
2021-05-18 21:02:07 +02:00
Gunnar Beutner
3908a49661 AK: Revert removal of StringBuilder::will_append optimization
This was removed as part of the ByteBuffer changes but the allocation
optimization is still necessary at least for non-SerenityOS targets
where malloc_good_size() isn't supported or returns a small value and
causes a whole bunch of unnecessary reallocations.
2021-05-18 08:06:32 +02:00
Gunnar Beutner
fcaf98361f AK: Turn ByteBuffer into a value type
Previously ByteBuffer would internally hold a RefPtr to the byte
buffer and would behave like a reference type, i.e. copying a
ByteBuffer would not create a duplicate byte buffer, but rather
two objects which refer to the same internal buffer.

This also changes ByteBuffer so that it has some internal capacity
much like the Vector<T> type. Unlike Vector<T> however a byte
buffer's data may be uninitialized.

With this commit ByteBuffer makes use of the kmalloc_good_size()
API to pick an optimal allocation size for its internal buffer.
2021-05-16 17:49:42 +02:00
Andreas Kling
834b6508d7 AK: Remove StringBuilder::appendf()
All users have been converted to using AK::Format via appendff().
2021-05-07 21:12:09 +02:00
Brian Gianforcaro
1682f0b760 Everything: Move to SPDX license identifiers in all files.
SPDX License Identifiers are a more compact / standardized
way of representing file license information.

See: https://spdx.dev/resources/use/#identifiers

This was done with the `ambr` search and replace tool.

 ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 11:22:27 +02:00
Andreas Kling
5d180d1f99 Everywhere: Rename ASSERT => VERIFY
(...and ASSERT_NOT_REACHED => VERIFY_NOT_REACHED)

Since all of these checks are done in release builds as well,
let's rename them to VERIFY to prevent confusion, as everyone is
used to assertions being compiled out in release.

We can introduce a new ASSERT macro that is specifically for debug
checks, but I'm doing this wholesale conversion first since we've
accumulated thousands of these already, and it's not immediately
obvious which ones are suitable for ASSERT.
2021-02-23 20:56:54 +01:00
Andreas Kling
54ade31d84 AK: Add some inline capacity to StringBuilder
This patch adds a 128-byte inline buffer that we use before switching
to using a dynamically growing ByteBuffer.

This allows us to avoid heap allocations in many cases, and totally
incidentally also speeds up @nico's favorite test, "disasm /bin/id"
more than 2x. :^)
2020-11-24 22:06:51 +01:00
Andreas Kling
adabcf24ec Everywhere: Add missing <AK/ByteBuffer.h> includes
All of these files were getting ByteBuffer.h from someone else and then
using it. Let's include it explicitly.
2020-11-15 13:11:21 +01:00
Andreas Kling
5e164052f6 AK+Kernel: Escape JSON keys & values
Grab the escaping logic from JSON string value serialization and use
it for serializing all keys and values.

Fixes #3917.
2020-11-02 12:56:36 +01:00
Nico Weber
ce95628b7f Unicode: Try s/codepoint/code_point/g again
This time, without trailing 's'. Ran:

    git grep -l 'codepoint' | xargs sed -ie 's/codepoint/code_point/g
2020-08-05 22:33:42 +02:00
Nico Weber
19ac1f6368 Revert "Unicode: s/codepoint/code_point/g"
This reverts commit ea9ac3155d.
It replaced "codepoint" with "code_points", not "code_point".
2020-08-05 22:33:42 +02:00
Andreas Kling
ea9ac3155d Unicode: s/codepoint/code_point/g
Unicode calls them "code points" so let's follow their style.
2020-08-03 19:06:41 +02:00
Andreas Kling
d4bbc00901 AK: Add StringBuilder::append_codepoint(u32)
Also, implement append(Utf32View) using it.
2020-06-04 21:12:17 +02:00
Andreas Kling
86242f9c18 AK: Add StringBuilder::append(Utf32View)
This encodes the incoming UTF-32 sequence as UTF-8.
2020-05-17 22:35:25 +02:00
Andreas Kling
d8aa2a6997 AK: StringBuilder with 0 initial capacity shouldn't build null String
With 0 initial capacity, we don't allocate an underlying ByteBuffer
for the StringBuilder, which would then lead to a null String() being
returned from to_string().

This patch makes sure we always build a valid String.
2020-05-15 13:50:42 +02:00
Andreas Kling
7d862dd5fc AK: Reduce header dependency graph of String.h
String.h no longer pulls in StringView.h. We do this by moving a bunch
of String functions out-of-line.
2020-03-23 13:48:44 +01:00