Sarah Hoffmann
51b6d16dc6
overhaul the token analysis interface
...
The functional split betweenthe two functions is now that the
first one creates the ID that is used in the word table and
the second one creates the variants. There no longer is a
requirement that the ID is the normalized version. We might
later reintroduce the requirement that a normalized version be available
but it doesn't necessarily need to be through the ID.
The function that creates the ID now gets the full PlaceName. That way
it might take into account attributes that were set by the sanitizers.
Finally rename both functions to something more sane.
2022-07-29 15:14:11 +02:00
Sarah Hoffmann
34d27ed45c
move PlaceName into the generic data module
2022-07-29 11:42:20 +02:00
Sarah Hoffmann
094100bbf6
harmonize spelling
...
Stick with the American spelling of Analyze.
2022-07-29 10:52:01 +02:00
Sarah Hoffmann
c8873d34af
harmonize interface of token analysis module
...
The configure() function now receives a Transliterator object instead
of the ICU rules. This harmonizes the parameters with the create
function.
2022-07-29 10:43:07 +02:00
Sarah Hoffmann
f0d640961a
add documentation for custom token analysis
2022-07-29 09:41:28 +02:00
Sarah Hoffmann
3746befd88
add documentation for sanitizer interface
...
Also switches mkdocstrings to 0.18 with the rather unfortunate
consequence that now mkdocstrings-python-legacy is needed as well.
2022-07-28 22:00:29 +02:00
Sarah Hoffmann
a8b037669a
Merge pull request #2780 from lonvia/python-modules-in-project-directory
...
Support for external sanitizer and token analysis modules
2022-07-28 21:58:04 +02:00
Sarah Hoffmann
d819036daa
add support for external token analysis modules
2022-07-25 16:27:22 +02:00
Sarah Hoffmann
6d41046b15
add support for external sanitizer modules
2022-07-25 16:10:19 +02:00
Sarah Hoffmann
7b7203c149
add function for loading plugin modules
...
Loads modules for configurable code like tokenizers, sanitizers, etc.
Supports internal modules, external libraries and code from the
project directory.
2022-07-25 16:10:10 +02:00
Sarah Hoffmann
95d4061b2a
Merge pull request #2775 from lonvia/remove-centos-instructions
...
Remove vagrant scripts for CentOS
2022-07-25 10:29:32 +02:00
Sarah Hoffmann
375b57a96a
vagrant: remove proj dependency and only require php-cli
2022-07-24 10:24:18 +02:00
Sarah Hoffmann
12ace4329d
remove CentOS installation instructions
...
Fixes #2601 .
2022-07-24 10:22:22 +02:00
Sarah Hoffmann
09e0be0e39
Merge pull request #2774 from lonvia/parameter-arrays
...
Ignore URL parameters in array notation
2022-07-23 23:56:32 +02:00
Sarah Hoffmann
cd4bcea894
ignore API parameters in array notation
...
PHP automatically parses parameters in an array notation(foo[]) into
array types. Ignore these parameters as 'unknown'.
Fixes #2763 .
2022-07-23 10:51:44 +02:00
Sarah Hoffmann
1bee151fe3
Merge pull request #2772 from kianmeng/fix-typos
...
docs: fix typos
2022-07-20 17:13:30 +02:00
Kian-Meng Ang
f5e52e748f
docs: fix typos
2022-07-20 22:05:31 +08:00
Sarah Hoffmann
b7f6c7c76a
docs: slightly increase recommended hardware requirements
2022-07-20 10:16:23 +02:00
Sarah Hoffmann
bc7f6209d8
Merge pull request #2770 from lonvia/typed-python
...
Type annotations for Python code
2022-07-19 09:03:30 +02:00
Sarah Hoffmann
372a548c28
CI: remove installation of pip on Ubuntu 20
2022-07-18 12:19:04 +02:00
Sarah Hoffmann
5aad105c73
add explicit cast for fetchone
2022-07-18 10:18:51 +02:00
Sarah Hoffmann
f40c83d025
CIL use psutil type stubs
2022-07-18 09:55:58 +02:00
Sarah Hoffmann
83054af46f
remove typing_extensions requirement
...
The typing_extensions package is only necessary now when running mypy.
It won't be used at runtime anymore.
2022-07-18 09:55:58 +02:00
Sarah Hoffmann
cb81f11422
CI: make type checking strict
2022-07-18 09:55:58 +02:00
Sarah Hoffmann
a849f3c9ec
add type annotations for command line functions
2022-07-18 09:55:54 +02:00
Sarah Hoffmann
25d854dc5c
add type annotations for Tiger import function
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
9963261d8d
add type annotations to special phrase importer
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
459ab3bbdc
add type annotations to database check functions
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
a21d4d3ac4
add type annotations for database import functions
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
4da1f0da6f
add type annotations for migrations
2022-07-18 09:54:29 +02:00
Sarah Hoffmann
17bbe2637a
add type annotations to tool functions
2022-07-18 09:54:27 +02:00
Sarah Hoffmann
6c6bbe5747
add type annotations for ICU tokenizer
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
18b16e06ca
add type annotations for legacy tokenizer
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
e37cfc64d2
add type annotations to ICU tokenizer helper modules
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
77510f4a3b
add typing extensions for Ubuntu22.04
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
d35e3c25b6
add type annotations for token analysis
...
No annotations for ICU types yet.
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
62eedbb8f6
add type hints for sanitizers
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
5617bffe2f
add type annotations for indexer
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
8adab2c6ca
add typing information for postcode formatter
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
d0c44431d0
add typing information for place_info and country_info
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
282a61ce51
add typing information for utils submodule
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
7a1d22ff15
type annotations for non-blocking DB connection
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
0dff71a410
add type annotations for SQL preprocessor
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
26f30bff28
add type annotation to DB utils
...
As a cursor is needed as type, make this a public type.
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
e6775e713c
add typing information to DB properties
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
69f9122bef
add typing annotations for DB status module
...
Requires TypedDict which is only available from Python 3.8. Require
therefore typing_extensions to make the functions available for
earlier Python versions.
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
fc254fc744
adapt use of Connection in bdd tests to name change
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
845c43137a
add type annotations to freeze functions
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
aaf2b6032e
fix uses of config.get_path() to expect None
2022-07-18 09:47:57 +02:00
Sarah Hoffmann
c4928c646d
define type for enivronment dictionaries
2022-07-18 09:47:57 +02:00
