Commit Graph

16 Commits

Author SHA1 Message Date
Timothy Flynn
d3e809bcd4 LibJS+LibUnicode: Port retrieving the system time zone to ICU 2024-06-26 10:14:02 +02:00
Timothy Flynn
ebdb92eef6 LibUnicode+Everywhere: Merge LibLocale back into LibUnicode
LibLocale was split off from LibUnicode a couple years ago to reduce the
number of applications on SerenityOS that depend on CLDR data. Now that
we use ICU, both LibUnicode and LibLocale are actually linking in this
data. And since vcpkg gives us static libraries, both libraries are over
30MB in size.

This patch reverts the separation and merges LibLocale into LibUnicode
again. We now have just one library that includes the ICU data.

Further, this will let LibUnicode share the locale cache that previously
would only exist in LibLocale.
2024-06-23 19:52:45 +02:00
Timothy Flynn
ab56b8c8dc LibUnicode: Remove the locale-unaware text segmentation implementation 2024-06-20 13:46:54 +02:00
Timothy Flynn
8d7216f4e0 LibUnicode: Replace IDNA ASCII conversion with ICU 2024-06-18 21:07:56 +02:00
Simon Wanner
5bcb019106 LibUnicode: Add IDNA::to_ascii
This implements the ToASCII operation of Unicode Technical Standard 46
2023-12-10 08:04:58 -05:00
Simon Wanner
299d35aadc LibUnicode: Add Punycode::decode 2023-12-10 08:04:58 -05:00
Timothy Flynn
73239fdd82 LibUnicode: Add a unit test for Unicode grapheme and word segmentation
These include tests for previously broken boundary conditions.
2023-02-25 22:23:39 +01:00
Timothy Flynn
1484d3d9f5 LibUnicode: Add a method to check if a code point could start an emoji 2023-02-24 19:48:47 +01:00
matcool
c8d7b0a33a Tests: Add tests for LibUnicode's normalize 2022-10-06 08:24:39 -04:00
Timothy Flynn
9e860d973e LibLocale: Move locale source files to the LibLocale library
Everything is now setup to create the LibLocale library and link it
where needed.
2022-09-05 14:37:16 -04:00
Timothy Flynn
b2d2bb43ce LibLocale: Move locale test files to the LibLocale folder 2022-09-05 14:37:16 -04:00
Timothy Flynn
fc8bf7ac3e LibUnicode+Userland: Migrate generated CLDR data to LibLocaleData
Currently, LibUnicodeData contains the generated UCD and CLDR data. Move
the UCD data to the main LibUnicode library, and rename LibUnicodeData
to LibLocaleData. This is another prepatory change to migrate to
LibLocale.
2022-09-05 14:37:16 -04:00
Timothy Flynn
6d7d9dd324 LibUnicode: Do not assume time zones & meta zones have a 1-to-1 mapping
The generator parses metaZones.json to form a mapping of meta zones to
time zones (AKA "golden zone" in TR-35). This parser errantly assumed
this was a 1-to-1 mapping.
2022-01-06 22:28:01 +01:00
Timothy Flynn
ffb3ba3079 Tests: Link some tests directly against LibUnicodeData
These were missed in 565a880ce5.

This wasn't an issue because these tests don't pledge/unveil anything,
so they could happily dlopen() the library at runtime. But this is now
needed in order to migrate LibUnicode towards weak symbols instead.
2022-01-04 22:49:43 +00:00
Andrew Kaster
58797a1289 Tests: Remove all file(GLOB) from CMakeLists in Tests
Using a file(GLOB) to find all the test files in a directory is an easy
hack to get things started, but has some drawbacks. Namely, if you add
a test, it won't be found again without re-running CMake. `ninja` seems
to do this automatically, but it would be nice to one day stop seeing it
rechecking our globbed directories.
2021-09-02 09:08:23 +02:00
Timothy Flynn
4dda3edc9e LibUnicode: Introduce a Unicode library for interacting with UCD files
The Unicode standard publishes the Unicode Character Database (UCD) with
information about every code point, such as each code point's upper case
mapping. LibUnicode exists to download and parse UCD files at build time
and to provide accessors to that data.

As a start, LibUnicode includes upper- and lower-case code point
converters.
2021-07-26 17:03:55 +01:00