Nominatim/nominatim/tokenizer
2021-10-07 09:49:13 +02:00
..
sanitizers add tests for sanitizer tagging language 2021-10-06 12:29:25 +02:00
token_analysis reorganize and complete tests around generic token analysis 2021-10-06 17:03:37 +02:00
__init__.py introduce tokenizer modules 2021-04-30 11:29:57 +02:00
base.py fix argument description for check_database 2021-10-07 09:49:13 +02:00
factory.py unify ICUNameProcessorRules and ICURuleLoader 2021-10-01 12:27:24 +02:00
icu_rule_loader.py add tests for sanitizer tagging language 2021-10-06 12:29:25 +02:00
icu_token_analysis.py use analyser provided in the 'analyzer' property 2021-10-05 14:10:32 +02:00
icu_tokenizer.py use analyser provided in the 'analyzer' property 2021-10-05 14:10:32 +02:00
legacy_tokenizer.py unify ICUNameProcessorRules and ICURuleLoader 2021-10-01 12:27:24 +02:00
place_sanitizer.py introduce sanitizer step before token analysis 2021-10-01 12:27:24 +02:00