Commit Graph

17 Commits

Author SHA1 Message Date
Nico Weber
ce95628b7f Unicode: Try s/codepoint/code_point/g again
This time, without trailing 's'. Ran:

    git grep -l 'codepoint' | xargs sed -ie 's/codepoint/code_point/g
2020-08-05 22:33:42 +02:00
Nico Weber
19ac1f6368 Revert "Unicode: s/codepoint/code_point/g"
This reverts commit ea9ac3155d.
It replaced "codepoint" with "code_points", not "code_point".
2020-08-05 22:33:42 +02:00
Andreas Kling
ea9ac3155d Unicode: s/codepoint/code_point/g
Unicode calls them "code points" so let's follow their style.
2020-08-03 19:06:41 +02:00
AnotherTest
a4e0b585fe AK: Add a way to get the number of valid bytes in a Utf8View 2020-05-18 11:31:43 +02:00
Andreas Kling
6cde7e4d20 AK: Add Utf8View::length_in_codepoints() 2020-05-17 13:05:39 +02:00
Emanuel Sprung
074d935c6e AK, LibGfx, LibGUI: Initialize various variables to zero.
The not initialized variables can lead to compiler warnings that
become errors with the -Werror flag.
2020-02-25 10:18:46 +01:00
Andreas Kling
3bbf4610d2 AK: Add a forward declaration header
You can now #include <AK/Forward.h> to get most of the AK types as
forward declarations.

Header dependency explosion is one of the main contributors to compile
times at the moment, so this is a step towards smaller include graphs.
2020-02-14 23:31:18 +01:00
Sergey Bugaev
c0b32f7b76 Meta: Claim copyright for files created by me
This changes copyright holder to myself for the source code files that I've
created or have (almost) completely rewritten. Not included are the files
that were significantly changed by others even though it was me who originally
created them (think HtmlView), or the many other files I've contributed code to.
2020-01-24 15:15:16 +01:00
Andreas Kling
94ca55cefd Meta: Add license header to source files
As suggested by Joshua, this commit adds the 2-clause BSD license as a
comment block to the top of every source file.

For the first pass, I've just added myself for simplicity. I encourage
everyone to add themselves as copyright holders of any file they've
added or modified in some significant way. If I've added myself in
error somewhere, feel free to replace it with the appropriate copyright
holder instead.

Going forward, all new source files should include a license header.
2020-01-18 09:45:54 +01:00
Andreas Kling
6f4c380d95 AK: Use size_t for the length of strings
Using int was a mistake. This patch changes String, StringImpl,
StringView and StringBuilder to use size_t instead of int for lengths.
Obviously a lot of code needs to change as a result of this.
2019-12-09 17:51:21 +01:00
Andreas Kling
f4e6dae6fe UTF-8: Add Utf8CodepointIterator::codepoint_length_in_bytes()
This allows you to retrieve the length (in bytes) of the codepoint the
iterator is currently pointing at.
2019-10-18 22:49:23 +02:00
Andreas Kling
9f99e285d1 Utf8View: Don't print potentially unterminated string in debug message 2019-09-15 15:46:40 +02:00
Sergey Bugaev
1fb6a7d893 AK: Fix buffer overrun in Utf8CodepointIterator::operator++
The old implementation tried to move forward as long as the current
byte looks like a UTF-8 character continuation byte (has its two
most significant bits set to 10). This is correct as long as we assume
the string is actually valid UTF-8, which we do (we also have a separate
method that can check whether it is the case).

We can't, however, assume that the data after the end of our string
is also valid UTF-8 (in fact, we're not even allowed to look at data
outside out string, but it happens to a valid memory region most of
the time). If the byte after the end of our string also has its most
significant bits set to 10, we would move one byte forward, and then
fail the m_length > 0 assertion.

One way to fix this would be to add a length check inside the loop
condition. The other one, implemented in this commit, is to reimplement
the whole function in terms of decode_first_byte(), which gives us
the length as encoded in the first byte. This also brings it more
in line with the other functions around it that do UTF-8 decoding.
2019-09-08 17:45:10 +02:00
Andreas Kling
fb39e46d3d Utf8View: Try fixing the travis-ci build
There's some overload ambiguity when doing Utf8View("literal")
2019-09-05 19:06:39 +02:00
Sergey Bugaev
55197ed4ef AK: Log UTF-8 validation errors 2019-09-05 16:37:39 +02:00
Sergey Bugaev
c379f43d2a AK: Add some more utility methods to Utf8View 2019-09-05 16:37:39 +02:00
Sergey Bugaev
5d3696174b AK: Add a Utf8View type for iterating over UTF-8 codepoints
Utf8View wraps a StringView and implements begin() and end() that
return a Utf8CodepointIterator, which parses UTF-8-encoded Unicode
codepoints and returns them as 32-bit integers.

This is the first step towards supporting emojis in Serenity ^)
https://github.com/SerenityOS/serenity/issues/490
2019-08-28 13:46:02 +02:00