3 Commits

Author	SHA1	Message	Date
Sarah Hoffmann	fbbdd31399	move word table and normalisation SQL into tokenizer ... Creating and populating the word table is now the responsibility of the tokenizer. The get_maxwordfreq() function has been replaced with a simple template parameter to the SQL during function installation. The number is taken from the parameter list in the database to ensure that it is not changed after installation.	2021-04-30 11:30:51 +02:00
Darkshredder	e5719de657	Added fixture for sql_preprocessor and fixed some issues	2021-03-11 15:39:17 +05:30
Sarah Hoffmann	d2bd6aa78d	introduce jinja2 for preprocessing SQL ... Replaces various hand-crafted replacements of varying format with a single Jinja2 templating mechanism. Allows full access to configuration if necessary.	2021-03-03 17:51:08 +01:00