Sarah Hoffmann 90b40fc3e6 define formal public Python interface for tokenizer ... This introduces an abstract class for the Tokenizer/Analyzer for documentation purposes.		2021-08-16 11:41:54 +02:00
..
__init__.py	introduce tokenizer modules	2021-04-30 11:29:57 +02:00
base.py	define formal public Python interface for tokenizer	2021-08-16 11:41:54 +02:00
factory.py	do not hide errors when importing tokenizer	2021-05-18 16:28:21 +02:00
icu_name_processor.py	split up variant computation for better readability	2021-07-12 14:43:50 +02:00
icu_rule_loader.py	switch to a more flexible variant description format	2021-07-04 10:28:20 +02:00
icu_variants.py	remove default parameter for namedtuple	2021-07-06 22:57:42 +02:00
legacy_icu_tokenizer.py	define formal public Python interface for tokenizer	2021-08-16 11:41:54 +02:00
legacy_tokenizer.py	define formal public Python interface for tokenizer	2021-08-16 11:41:54 +02:00