osm-search/Nominatim
1
1
Fork 0
mirror of https://github.com/osm-search/Nominatim.git synced 2024-11-23 13:44:36 +03:00
Code Issues Packages Projects Releases Wiki Activity
7cfcbacfc7
Nominatim/nominatim/tokenizer
History
Sarah Hoffmann 7cfcbacfc7 make token analyzers configurable modules 
Adds a mandatory section 'analyzer' to the token-analysis entries
which define, which analyser to use. Currently there is exactly
one, generic, which implements the former ICUNameProcessor.
2021-10-04 17:37:34 +02:00
..
sanitizers add unit tests for new sanatizer functions 2021-10-01 12:27:24 +02:00
token_analysis make token analyzers configurable modules 2021-10-04 17:37:34 +02:00
__init__.py introduce tokenizer modules 2021-04-30 11:29:57 +02:00
base.py unify ICUNameProcessorRules and ICURuleLoader 2021-10-01 12:27:24 +02:00
factory.py unify ICUNameProcessorRules and ICURuleLoader 2021-10-01 12:27:24 +02:00
icu_rule_loader.py make token analyzers configurable modules 2021-10-04 17:37:34 +02:00
icu_tokenizer.py introduce sanitizer step before token analysis 2021-10-01 12:27:24 +02:00
icu_variants.py unify ICUNameProcessorRules and ICURuleLoader 2021-10-01 12:27:24 +02:00
legacy_tokenizer.py unify ICUNameProcessorRules and ICURuleLoader 2021-10-01 12:27:24 +02:00
place_sanitizer.py introduce sanitizer step before token analysis 2021-10-01 12:27:24 +02:00