Sarah Hoffmann 51b6d16dc6 overhaul the token analysis interface ... The functional split betweenthe two functions is now that the first one creates the ID that is used in the word table and the second one creates the variants. There no longer is a requirement that the ID is the normalized version. We might later reintroduce the requirement that a normalized version be available but it doesn't necessarily need to be through the ID. The function that creates the ID now gets the full PlaceName. That way it might take into account attributes that were set by the sanitizers. Finally rename both functions to something more sane.		2022-07-29 15:14:11 +02:00
..
cli	move country_info into data submodule	2022-07-06 11:08:36 +02:00
config	add support for external sanitizer modules	2022-07-25 16:10:19 +02:00
data	move country_info into data submodule	2022-07-06 11:08:36 +02:00
db	docs: fix typos	2022-07-20 22:05:31 +08:00
indexer	change indexing order for interpolations	2022-06-02 15:16:46 +02:00
tokenizer	overhaul the token analysis interface	2022-07-29 15:14:11 +02:00
tools	add type annotations to special phrase importer	2022-07-18 09:54:29 +02:00
utils	add class for online centroid computation	2022-06-23 23:42:31 +02:00
conftest.py	switch tests to ICU tokenizer as default	2022-05-10 14:54:50 +02:00
cursor.py	add consistent SPDX copyright headers	2022-01-03 16:23:58 +01:00
dummy_tokenizer.py	move PlaceInfo into data submodule	2022-07-06 10:54:47 +02:00
mock_icu_word_table.py	add tests for new analyzed housenumbers	2022-03-01 09:34:32 +01:00
mock_legacy_word_table.py	add consistent SPDX copyright headers	2022-01-03 16:23:58 +01:00
mocks.py	convert admin --analyse-indexing to new indexing method	2022-07-07 16:20:08 +02:00
pytest.ini	'read_config' is no longer a fixture	2022-04-01 22:52:17 +08:00