LadybirdBrowser/ladybird
1
1
Fork 0
mirror of https://github.com/LadybirdBrowser/ladybird.git synced 2024-10-05 01:27:21 +03:00
Code Issues Packages Projects Releases Wiki Activity
03c023d7e9
ladybird/Meta/CMake/unicode_data.cmake

166 lines
8.7 KiB
CMake
Raw Normal View History

LibUnicode: Upgrade to Unicode version 14.0.0
2021-09-15 16:35:18 +03:00
set(UCD_VERSION 14.0.0)
LibUnicode: Upgrade to CLDR version 40.0.0 Release notes: https://github.com/unicode-org/cldr-json/releases/tag/40.0.0
2021-11-09 18:50:37 +03:00
set(CLDR_VERSION 40.0.0)
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(UCD_PATH "${CMAKE_BINARY_DIR}/UCD" CACHE PATH "Download location for UCD files")
set(CLDR_PATH "${CMAKE_BINARY_DIR}/CLDR" CACHE PATH "Download location for CLDR files")
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
LibUnicode: Remove cached UCD and CLDR files when their version changes
2021-09-15 16:33:55 +03:00
set(UCD_VERSION_FILE "${UCD_PATH}/version.txt")
set(CLDR_VERSION_FILE "${CLDR_PATH}/version.txt")
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(UNICODE_DATA_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/UnicodeData.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(UNICODE_DATA_PATH "${UCD_PATH}/UnicodeData.txt")
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(SPECIAL_CASING_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/SpecialCasing.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(SPECIAL_CASING_PATH "${UCD_PATH}/SpecialCasing.txt")
LibUnicode: Download and parse the special casing UCD file This adds a SpecialCasing structure to the generated UnicodeData.h/cpp files. This structure contains casing rules for code points which have non-1-to-1 upper-to-lower case code point mappings. Further, these rules may be limited to specific locales or other context.
2021-07-27 17:39:37 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(DERIVED_GENERAL_CATEGORY_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/extracted/DerivedGeneralCategory.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(DERIVED_GENERAL_CATEGORY_PATH "${UCD_PATH}/DerivedGeneralCategory.txt")
LibUnicode: Generate separate tables for General Category properties Previously, each code point's General Category was part of the generated UnicodeData structure. This ultimately presented two problems, one functional and one performance related: * Some General Categories are applied to unassigned code points, for example the Unassigned (Cn) category. Unassigned code points are strictly excluded from UnicodeData.txt, so by relying on that file, the generator is unable to handle these categories. * Lookups for General Categories are slower when searching through the large UnicodeData hash map. Even though lookups are O(1), the hash function turned out to be slower than binary searching through a category-specific table. So, now a table is generated for each General Category. When querying a code point for a category, a binary search is done on each code point range in that category's table to check if code point has that category. Further, General Categories are now parsed from the UCD file DerivedGeneralCategory.txt. This file is a normal "prop list" file and contains the categories for unassigned code points.
2021-08-10 14:17:24 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(PROP_LIST_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/PropList.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(PROP_LIST_PATH "${UCD_PATH}/PropList.txt")
LibUnicode: Download and parse the property list UCD file
2021-07-28 00:37:47 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(DERIVED_CORE_PROP_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/DerivedCoreProperties.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(DERIVED_CORE_PROP_PATH "${UCD_PATH}/DerivedCoreProperties.txt")
LibUnicode: Parse and utilize DerivedCoreProperties DerivedCoreProperties are pseudo-properties that are the union of other categories and properties. For example, the derived property Math is the union of the general category Sm and the property Other_Math. Parsing these is necessary for implementing Unicode property escapes. But it also has the added benefit that LibUnicode now does not need to derive some of these properties at runtime.
2021-07-29 01:39:41 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(DERIVED_BINARY_PROP_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/extracted/DerivedBinaryProperties.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(DERIVED_BINARY_PROP_PATH "${UCD_PATH}/DerivedBinaryProperties.txt")
LibUnicode: Parse UCD DerivedBinaryProperties.txt and generate property
2021-08-04 14:53:24 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(PROP_ALIAS_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/PropertyAliases.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(PROP_ALIAS_PATH "${UCD_PATH}/PropertyAliases.txt")
LibUnicode: Parse and generate PropertyAliases These are all used for Unicode property escapes.
2021-07-29 03:14:42 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(PROP_VALUE_ALIAS_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/PropertyValueAliases.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(PROP_VALUE_ALIAS_PATH "${UCD_PATH}/PropertyValueAliases.txt")
LibUnicode: Generate General Category unions and aliases This downloads the PropertyValueAliases.txt UCD file, which contains a set of General Category aliases. This changes the General Category enumeration to now be generated as a bitmask. This is to easily allow General Category unions. For example, the LC (Cased_Letter) category is the union of the Ll, Lu, and Lt categories.
2021-07-31 20:45:05 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(SCRIPTS_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/Scripts.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(SCRIPTS_PATH "${UCD_PATH}/Scripts.txt")
LibUnicode: Parse UCD Scripts.txt and generate as a Unicode property There are a couple of minor nuances with parsing script values, compared to other properties. In Scripts.txt, the UCD file lists the full name of each script; other properties, like General Category, list the shorter name in their primary files. This means that the aliases listed in PropertyValueAliases.txt are reversed for script values.
2021-08-04 00:11:19 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(SCRIPT_EXTENSIONS_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/ScriptExtensions.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(SCRIPT_EXTENSIONS_PATH "${UCD_PATH}/ScriptExtensions.txt")
LibUnicode: Parse UCD ScriptExtensions.txt and generate property
2021-08-04 14:05:30 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(EMOJI_DATA_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/emoji/emoji-data.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(EMOJI_DATA_PATH "${UCD_PATH}/emoji-data.txt")
LibUnicode: Parse UCD emoji-data.txt and generate Unicode property
2021-08-04 14:46:36 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(NORM_PROPS_URL "https://www.unicode.org/Public/${UCD_VERSION}/ucd/DerivedNormalizationProps.txt")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(NORM_PROPS_PATH "${UCD_PATH}/DerivedNormalizationProps.txt")
LibUnicode: Download and parse DerivedNormalizationProps UCD file This file contains the last properties that LibUnicode is not parsing. Much of the data in this file is not currently used; that is left as a FIXME for when String.prototype.normalize is implemented. Until then, only the code point properties are utilized for regular expression pattern escapes.
2021-08-10 22:29:28 +03:00
LibUnicode: Extract UCD and CLDR versions to a CMake variable This also surrounds expansion of affected URL and path variables with quotes.
2021-09-15 15:46:27 +03:00
set(CLDR_ZIP_URL "https://github.com/unicode-org/cldr-json/releases/download/${CLDR_VERSION}/cldr-${CLDR_VERSION}-json-modern.zip")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(CLDR_ZIP_PATH "${CLDR_PATH}/cldr.zip")
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
LibUnicode: Extract cldr-core dataset from CLDR database
2021-08-30 15:56:28 +03:00
set(CLDR_CORE_SOURCE cldr-core)
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(CLDR_CORE_PATH "${CLDR_PATH}/${CLDR_CORE_SOURCE}")
LibUnicode: Extract cldr-core dataset from CLDR database
2021-08-30 15:56:28 +03:00
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
set(CLDR_LOCALES_SOURCE cldr-localenames-modern)
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(CLDR_LOCALES_PATH "${CLDR_PATH}/${CLDR_LOCALES_SOURCE}")
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
LibUnicode: Extract cldr-misc dataset from CLDR database
2021-09-06 05:46:03 +03:00
set(CLDR_MISC_SOURCE cldr-misc-modern)
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(CLDR_MISC_PATH "${CLDR_PATH}/${CLDR_MISC_SOURCE}")
LibUnicode: Extract cldr-misc dataset from CLDR database
2021-09-06 05:46:03 +03:00
LibUnicode: Extract cldr-numbers dataset from CLDR database This dataset holds the values needed to handle DisplayNames.prototype.of with a type option of "currency".
2021-08-26 15:31:31 +03:00
set(CLDR_NUMBERS_SOURCE cldr-numbers-modern)
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
set(CLDR_NUMBERS_PATH "${CLDR_PATH}/${CLDR_NUMBERS_SOURCE}")
LibUnicode: Extract cldr-numbers dataset from CLDR database This dataset holds the values needed to handle DisplayNames.prototype.of with a type option of "currency".
2021-08-26 15:31:31 +03:00
LibUnicode: Remove cached UCD and CLDR files when their version changes
2021-09-15 16:33:55 +03:00
function(remove_unicode_data_if_version_changed version version_file cache_path)
set(version_differs YES)
if (EXISTS "${version_file}")
file(STRINGS "${version_file}" active_version)
if (version STREQUAL active_version)
set(version_differs NO)
endif()
endif()
if (version_differs)
message(STATUS "Removing outdated ${cache_path} for version ${version}")
file(REMOVE_RECURSE "${cache_path}")
file(WRITE "${version_file}" "${version}")
endif()
endfunction()
LibUnicode: Functionalize downloading of UCD data files
2021-09-15 15:46:56 +03:00
function(download_ucd_file url path)
if (NOT EXISTS "${path}")
get_filename_component(file "${path}" NAME)
message(STATUS "Downloading UCD ${file} from ${url}...")
file(DOWNLOAD "${url}" "${path}" INACTIVITY_TIMEOUT 10)
LibUnicode: Download and parse DerivedNormalizationProps UCD file This file contains the last properties that LibUnicode is not parsing. Much of the data in this file is not currently used; that is left as a FIXME for when String.prototype.normalize is implemented. Until then, only the code point properties are utilized for regular expression pattern escapes.
2021-08-10 22:29:28 +03:00
endif()
LibUnicode: Functionalize downloading of UCD data files
2021-09-15 15:46:56 +03:00
endfunction()
LibUnicode: Functionalize extraction of CLDR data files
2021-09-15 15:50:45 +03:00
function(extract_cldr_file source path)
if(EXISTS "${CLDR_ZIP_PATH}" AND NOT EXISTS "${path}")
message(STATUS "Extracting CLDR ${source} from ${CLDR_ZIP_PATH}...")
LibUnicode: Use consistent variable naming in unicode_data.cmake I kept mixing these up in my head.
2021-09-15 20:27:17 +03:00
execute_process(COMMAND unzip -q "${CLDR_ZIP_PATH}" "${source}/**" -d "${CLDR_PATH}" RESULT_VARIABLE unzip_result)
LibUnicode: Functionalize extraction of CLDR data files
2021-09-15 15:50:45 +03:00
if (NOT unzip_result EQUAL 0)
message(FATAL_ERROR "Failed to unzip ${source} from ${CLDR_ZIP_PATH} with status ${unzip_result}")
endif()
endif()
endfunction()
LibUnicode: Functionalize downloading of UCD data files
2021-09-15 15:46:56 +03:00
if (ENABLE_UNICODE_DATABASE_DOWNLOAD)
LibUnicode: Remove cached UCD and CLDR files when their version changes
2021-09-15 16:33:55 +03:00
remove_unicode_data_if_version_changed("${UCD_VERSION}" "${UCD_VERSION_FILE}" "${UCD_PATH}")
remove_unicode_data_if_version_changed("${CLDR_VERSION}" "${CLDR_VERSION_FILE}" "${CLDR_PATH}")
LibUnicode: Functionalize downloading of UCD data files
2021-09-15 15:46:56 +03:00
download_ucd_file("${UNICODE_DATA_URL}" "${UNICODE_DATA_PATH}")
download_ucd_file("${SPECIAL_CASING_URL}" "${SPECIAL_CASING_PATH}")
download_ucd_file("${DERIVED_GENERAL_CATEGORY_URL}" "${DERIVED_GENERAL_CATEGORY_PATH}")
download_ucd_file("${PROP_LIST_URL}" "${PROP_LIST_PATH}")
download_ucd_file("${DERIVED_CORE_PROP_URL}" "${DERIVED_CORE_PROP_PATH}")
download_ucd_file("${DERIVED_BINARY_PROP_URL}" "${DERIVED_BINARY_PROP_PATH}")
download_ucd_file("${PROP_ALIAS_URL}" "${PROP_ALIAS_PATH}")
download_ucd_file("${PROP_VALUE_ALIAS_URL}" "${PROP_VALUE_ALIAS_PATH}")
download_ucd_file("${SCRIPTS_URL}" "${SCRIPTS_PATH}")
download_ucd_file("${SCRIPT_EXTENSIONS_URL}" "${SCRIPT_EXTENSIONS_PATH}")
download_ucd_file("${EMOJI_DATA_URL}" "${EMOJI_DATA_PATH}")
download_ucd_file("${NORM_PROPS_URL}" "${NORM_PROPS_PATH}")
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
LibUnicode: Functionalize extraction of CLDR data files
2021-09-15 15:50:45 +03:00
if (NOT EXISTS "${CLDR_ZIP_PATH}")
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
message(STATUS "Downloading CLDR database from ${CLDR_ZIP_URL}...")
LibUnicode: Functionalize extraction of CLDR data files
2021-09-15 15:50:45 +03:00
file(DOWNLOAD "${CLDR_ZIP_URL}" "${CLDR_ZIP_PATH}" INACTIVITY_TIMEOUT 10)
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
endif()
LibUnicode: Functionalize extraction of CLDR data files
2021-09-15 15:50:45 +03:00
extract_cldr_file("${CLDR_CORE_SOURCE}" "${CLDR_CORE_PATH}")
extract_cldr_file("${CLDR_LOCALES_SOURCE}" "${CLDR_LOCALES_PATH}")
extract_cldr_file("${CLDR_MISC_SOURCE}" "${CLDR_MISC_PATH}")
extract_cldr_file("${CLDR_NUMBERS_SOURCE}" "${CLDR_NUMBERS_PATH}")
LibUnicode: Make unicode data generation logic more relocatable The previous logic had several checks for Lagom directories and subdirectories. All we really want to do for these header checks is make sure that the files end up in an included folder prefixed with LibUnicode. We also don't need to hard code the path to the generator, the $<TARGET_FILES> generator expression can create the path for us.
2021-07-29 11:18:52 +03:00
set(UNICODE_DATA_HEADER LibUnicode/UnicodeData.h)
set(UNICODE_DATA_IMPLEMENTATION LibUnicode/UnicodeData.cpp)
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
set(UNICODE_LOCALE_HEADER LibUnicode/UnicodeLocale.h)
set(UNICODE_LOCALE_IMPLEMENTATION LibUnicode/UnicodeLocale.cpp)
CMake: Add custom target to build only the generated sources This is needed so all headers and files exist on disk, so that the sonar cloud analyzer can find them when executing the compilation commands contained in compile_commands.json, without actually building. Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2021-08-30 05:27:50 +03:00
set(UNICODE_META_TARGET_PREFIX LibUnicode_)
LibUnicode: Make unicode data generation logic more relocatable The previous logic had several checks for Lagom directories and subdirectories. All we really want to do for these header checks is make sure that the files end up in an included folder prefixed with LibUnicode. We also don't need to hard code the path to the generator, the $<TARGET_FILES> generator expression can create the path for us.
2021-07-29 11:18:52 +03:00
if (CMAKE_CURRENT_BINARY_DIR MATCHES ".*/LibUnicode") # Serenity build.
set(UNICODE_DATA_HEADER UnicodeData.h)
set(UNICODE_DATA_IMPLEMENTATION UnicodeData.cpp)
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
set(UNICODE_LOCALE_HEADER UnicodeLocale.h)
set(UNICODE_LOCALE_IMPLEMENTATION UnicodeLocale.cpp)
CMake: Add custom target to build only the generated sources This is needed so all headers and files exist on disk, so that the sonar cloud analyzer can find them when executing the compilation commands contained in compile_commands.json, without actually building. Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2021-08-30 05:27:50 +03:00
set(UNICODE_META_TARGET_PREFIX "")
Meta: Add LibUnicode (and its tests) to Lagom This is primarily to allow using LibUnicode within LibJS and its REPL. Note: this seems to be the first time that a Lagom dependency requires generated source files. For this to work, some of Lagom's CMakeLists.txt commands needed to be re-organized to include the CMake files that fetch and parse UnicodeData.txt. The paths required to invoke the generator also differ depending on what is currently building (SerenityOS vs. Lagom as part of the Serenity build vs. a standalone Lagom build).
2021-07-25 22:15:47 +03:00
endif()
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
add_custom_command(
LibUnicode: Invoke Unicode data generator a single time It takes a non-neglible amount of time to parse all of the UCD files and generate the Unicode data files. To help compile times, only invoke the generator once.
2021-08-02 17:47:41 +03:00
OUTPUT ${UNICODE_DATA_HEADER} ${UNICODE_DATA_IMPLEMENTATION}
Meta: Allow specifying alternative paths for downloaded Unicode data This lets us possibly share downloaded artifacts between different builds without re-downloading them every time you change toolchains.
2021-09-07 09:49:21 +03:00
COMMAND $<TARGET_FILE:Lagom::GenerateUnicodeData> -h ${UNICODE_DATA_HEADER}.tmp -c ${UNICODE_DATA_IMPLEMENTATION}.tmp -u ${UNICODE_DATA_PATH} -s ${SPECIAL_CASING_PATH} -g ${DERIVED_GENERAL_CATEGORY_PATH} -p ${PROP_LIST_PATH} -d ${DERIVED_CORE_PROP_PATH} -b ${DERIVED_BINARY_PROP_PATH} -a ${PROP_ALIAS_PATH} -v ${PROP_VALUE_ALIAS_PATH} -r ${SCRIPTS_PATH} -x ${SCRIPT_EXTENSIONS_PATH} -e ${EMOJI_DATA_PATH} -n ${NORM_PROPS_PATH}
COMMAND "${CMAKE_COMMAND}" -E copy_if_different ${UNICODE_DATA_HEADER}.tmp ${UNICODE_DATA_HEADER}
COMMAND "${CMAKE_COMMAND}" -E copy_if_different ${UNICODE_DATA_IMPLEMENTATION}.tmp ${UNICODE_DATA_IMPLEMENTATION}
COMMAND "${CMAKE_COMMAND}" -E remove ${UNICODE_DATA_HEADER}.tmp ${UNICODE_DATA_IMPLEMENTATION}.tmp
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
VERBATIM
Meta: Allow specifying alternative paths for downloaded Unicode data This lets us possibly share downloaded artifacts between different builds without re-downloading them every time you change toolchains.
2021-09-07 09:49:21 +03:00
DEPENDS Lagom::GenerateUnicodeData ${UNICODE_DATA_PATH} ${SPECIAL_CASING_PATH} ${DERIVED_GENERAL_CATEGORY_PATH} ${PROP_LIST_PATH} ${DERIVED_CORE_PROP_PATH} ${DERIVED_BINARY_PROP_PATH} ${PROP_ALIAS_PATH} ${PROP_VALUE_ALIAS_PATH} ${SCRIPTS_PATH} ${SCRIPT_EXTENSIONS_PATH} ${EMOJI_DATA_PATH} ${NORM_PROPS_PATH}
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
)
CMake: Add custom target to build only the generated sources This is needed so all headers and files exist on disk, so that the sonar cloud analyzer can find them when executing the compilation commands contained in compile_commands.json, without actually building. Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2021-08-30 05:27:50 +03:00
add_custom_target(generate_${UNICODE_META_TARGET_PREFIX}UnicodeData DEPENDS ${UNICODE_DATA_HEADER} ${UNICODE_DATA_IMPLEMENTATION})
add_dependencies(all_generated generate_${UNICODE_META_TARGET_PREFIX}UnicodeData)
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
add_custom_command(
OUTPUT ${UNICODE_LOCALE_HEADER} ${UNICODE_LOCALE_IMPLEMENTATION}
Meta: Allow specifying alternative paths for downloaded Unicode data This lets us possibly share downloaded artifacts between different builds without re-downloading them every time you change toolchains.
2021-09-07 09:49:21 +03:00
COMMAND $<TARGET_FILE:Lagom::GenerateUnicodeLocale> -h ${UNICODE_LOCALE_HEADER}.tmp -c ${UNICODE_LOCALE_IMPLEMENTATION}.tmp -r ${CLDR_CORE_PATH} -l ${CLDR_LOCALES_PATH} -m ${CLDR_MISC_PATH} -n ${CLDR_NUMBERS_PATH}
COMMAND "${CMAKE_COMMAND}" -E copy_if_different ${UNICODE_LOCALE_HEADER}.tmp ${UNICODE_LOCALE_HEADER}
COMMAND "${CMAKE_COMMAND}" -E copy_if_different ${UNICODE_LOCALE_IMPLEMENTATION}.tmp ${UNICODE_LOCALE_IMPLEMENTATION}
COMMAND "${CMAKE_COMMAND}" -E remove ${UNICODE_LOCALE_HEADER}.tmp ${UNICODE_LOCALE_IMPLEMENTATION}.tmp
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
VERBATIM
Meta: Allow specifying alternative paths for downloaded Unicode data This lets us possibly share downloaded artifacts between different builds without re-downloading them every time you change toolchains.
2021-09-07 09:49:21 +03:00
DEPENDS Lagom::GenerateUnicodeLocale ${CLDR_CORE_PATH} ${CLDR_LOCALES_PATH} ${CLDR_MISC_PATH} ${CLDR_NUMBERS_PATH}
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
)
CMake: Add custom target to build only the generated sources This is needed so all headers and files exist on disk, so that the sonar cloud analyzer can find them when executing the compilation commands contained in compile_commands.json, without actually building. Co-authored-by: Andrew Kaster <akaster@serenityos.org>
2021-08-30 05:27:50 +03:00
add_custom_target(generate_${UNICODE_META_TARGET_PREFIX}UnicodeLocale DEPENDS ${UNICODE_LOCALE_HEADER} ${UNICODE_LOCALE_IMPLEMENTATION})
add_dependencies(all_generated generate_${UNICODE_META_TARGET_PREFIX}UnicodeLocale)
LibUnicode: Download Unicode's CLDR database and generate locale data The Unicode standard publishes a database known as the Common Locale Data Repository (CLDR). This is a massive set of data from which anyone implementing Unicode's Technical Standard #35 may generate their implementation: https://www.unicode.org/reports/tr35/ This commit updates LibUnicode to download the compressed database and extract a small subset. That subset is used to generate a list of available locales and the territories (AKA regions) associated with each locale.
2021-08-25 02:50:18 +03:00
set(UNICODE_DATA_SOURCES ${UNICODE_DATA_HEADER} ${UNICODE_DATA_IMPLEMENTATION} ${UNICODE_LOCALE_HEADER} ${UNICODE_LOCALE_IMPLEMENTATION})
LibUnicode: Introduce a Unicode library for interacting with UCD files The Unicode standard publishes the Unicode Character Database (UCD) with information about every code point, such as each code point's upper case mapping. LibUnicode exists to download and parse UCD files at build time and to provide accessors to that data. As a start, LibUnicode includes upper- and lower-case code point converters.
2021-07-25 22:10:51 +03:00
endif()
Reference in New Issue Copy Permalink