Commit Graph

3008 Commits

Author SHA1 Message Date
Sarah Hoffmann
0d9fe6e49c
Merge pull request #2219 from lonvia/bdd-test-remove-php
BDD tests: run all setup via nominatim Python library
2021-03-17 11:40:34 +01:00
Sarah Hoffmann
ebae3553e0 bdd: run all setup via nominatim Python library
Drops all calls to PHP utility functions. nominatim cli functions
are used where possible, to stay as close to the final code as
possible with the tests.

By removing the PHP calls, the test code now only uses osm2pgsql and
the database module from the build directory.
2021-03-16 22:20:41 +01:00
Sarah Hoffmann
d3ff831b8a
Merge pull request #2216 from lonvia/fix-reverse-interpolation
Reverse: do not prefer interpolations over closer housenumbers
2021-03-15 14:08:54 +01:00
Sarah Hoffmann
4d7c5ec089 reverse: do not prefer interpolations over closer housenumbers
Always look up the closest housenumber before looking up
interpolations. This ensures that closer housenumbers are
preferred over interpolations.

Fixes #2214.
2021-03-15 10:50:04 +01:00
Sarah Hoffmann
81a6b746b8
Merge pull request #2212 from darkshredder/country-name
Ported createCountryNames() to python and Added tests
2021-03-15 09:36:06 +01:00
Darkshredder
f356a75a24 Add setup.php 2021-03-14 15:02:30 +05:30
Sarah Hoffmann
7212fa8630 fix template variable name 2021-03-13 12:05:53 +01:00
Sarah Hoffmann
6cabc44841
Merge pull request #2213 from lonvia/tweak-search-weights
Some more tweaking of the ranking of search interpretations
2021-03-12 15:47:36 +01:00
Darkshredder
b108bd1c1e Linting fix 2021-03-12 18:28:47 +05:30
Darkshredder
077a8c1f95 refactored tests and made changes to code for easy readibility 2021-03-12 18:23:20 +05:30
Darkshredder
7a874d5b97 Ported createCountryNames() to python and added tests 2021-03-12 10:28:41 +05:30
Sarah Hoffmann
9086a794a1
Merge pull request #2204 from darkshredder/tiger-data
Ported tiger-data-import to Python and Added Tarball Support
2021-03-11 22:48:38 +01:00
Sarah Hoffmann
6dd2b9c2ec do not mix partial names with other words
As soon as a housenumber, postcode, etc. appear, the name term
must obviously be closed and no further partial terms can be
appended.
2021-03-11 22:44:49 +01:00
Sarah Hoffmann
3fbe4511f9 make linter happy 2021-03-11 21:14:23 +01:00
Sarah Hoffmann
3933fc3ad3 avoid multi-term partials in names
Names are either full words or single-word partial names.
Searching for multi-word partials yields exactly the same
result as with full words.
2021-03-11 20:42:37 +01:00
Sarah Hoffmann
00b05e2394 higher penalty for special searches
Adds a general higher penalty for special search term and an
additional one if the term is anywhere but the beginning or the
end. Also housenumbers and special searches together are less
likely.
2021-03-11 20:37:51 +01:00
Sarah Hoffmann
d5e8c5e975 do not mix partial and full name terms
If NameNonSearch already contains a partial term, then a
full term must not be added to the Name list anymore.
2021-03-11 20:22:54 +01:00
Sarah Hoffmann
478dfb0639 add one-rank penalty for using partial search
Ensures that full matches are preferred over partial ones even when
the full word consists of only one term.
2021-03-11 17:52:44 +01:00
Sarah Hoffmann
f498e40208 fix result splitting for last search group
When we are in the final iteration of the search groups, it is not
possible to further delay the results. Unconditionally use the
results with the best rank instead.
2021-03-11 17:14:46 +01:00
Sarah Hoffmann
182f5f5d7b give preference to full words in address, too
Full word terms are already preferred for the name part. Adding
only one-word partials to the address, makes it impossible to
give a similar preference for the address part. Each term adds
a rank penalty. The problem here is that we interpret the query
forwards and backwards. Having different penalty systems for
name and address means that the same term ends up with different
penalties and that often leads to interpretations of the wrong
direction being in the way.
2021-03-11 15:03:36 +01:00
Darkshredder
e5719de657 Added fixture for sql_preprocessor and fixed some issues 2021-03-11 15:39:17 +05:30
Darkshredder
8486a83cf5 Added test for tarfile 2021-03-10 18:14:17 +05:30
Darkshredder
ccfad57fca Added test and removed runlegacyscript 2021-03-10 17:18:12 +05:30
Darkshredder
64128b699a fixed linting, refactored threaded sql handling and removed importTigerData() function 2021-03-10 13:28:29 +05:30
Darkshredder
4080fbb95c Test fixes 2021-03-09 01:00:56 +05:30
Darkshredder
14ec83c886 Linting fixes 2021-03-08 23:10:49 +05:30
Darkshredder
122c4618b9 Linting fixes 2021-03-08 22:59:51 +05:30
Darkshredder
2af82975cd Ported tiger-data-import to python and Added Tarball Support 2021-03-08 21:57:56 +05:30
Sarah Hoffmann
35f4695b67
Merge pull request #2200 from lonvia/migrations-for-current-version
Introduce a command for database migration
2021-03-08 10:14:03 +01:00
Sarah Hoffmann
3c9e09545e documentation for new migration command 2021-03-06 16:38:37 +01:00
Sarah Hoffmann
764a41b973 automatic migration from 3.6 release
Adds a 'admin --migrate' command that checks for the current
database version and runs any necessary migrations. Also
has migrations going back to 3.6.
2021-03-06 16:36:57 +01:00
Sarah Hoffmann
9d103503f7
Merge pull request #2197 from lonvia/use-jinja-for-sql-preprocessing
Use jinja2 for SQL preprocessing
2021-03-04 16:36:18 +01:00
Sarah Hoffmann
09f4d767e4 port index creation to python
Also switches to jinja-based preprocessing, which allows to
simplify the SQL files. Use 'if not exists' where possible
so that the step can be rerun to fix missing indexes.
2021-03-04 11:11:47 +01:00
Sarah Hoffmann
dd301cf5ac indexer: ANALYSE must be run outside transactions 2021-03-04 11:06:33 +01:00
Sarah Hoffmann
eacabb0e96 move table creation to jinja-based preprocessing 2021-03-03 22:07:51 +01:00
Sarah Hoffmann
6cda021d9b add new jinja2 requirement 2021-03-03 17:51:08 +01:00
Sarah Hoffmann
d2bd6aa78d introduce jinja2 for preprocessing SQL
Replaces various hand-crafted replacements of varying format with
a single Jinja2 templating mechanism. Allows full access to
configuration if necessary.
2021-03-03 17:51:08 +01:00
Sarah Hoffmann
6b306f30b6
Merge pull request #2194 from grischard/patch-1
Fix typo in .github/actions/build-nominatim/action.yml
2021-03-03 11:34:12 +01:00
Guillaume Rischard
c48fd18344
Update action.yml 2021-03-03 11:20:21 +01:00
Sarah Hoffmann
8ea7e04363
Merge pull request #2192 from lonvia/database-versioning
Introduce database versioning
2021-03-02 15:57:46 +01:00
Sarah Hoffmann
32c2d2b248 document new status fields 2021-03-01 22:21:37 +01:00
Sarah Hoffmann
111cca8c9a return database version with status API 2021-03-01 22:17:16 +01:00
Sarah Hoffmann
7ae9c3a9f0 add database_version setting to tests 2021-03-01 21:49:33 +01:00
Sarah Hoffmann
bf4320a7d6 do not depend on cmdline parameter for creating partition tables
The partition numbers in use only depend on the entries in search_name.
2021-03-01 21:28:39 +01:00
Sarah Hoffmann
3a0a4b9175 save software version in the database
The version represents the software version that was used to
import the data.
2021-03-01 20:35:15 +01:00
Sarah Hoffmann
4faefe156c report software version of status call 2021-03-01 16:47:19 +01:00
Sarah Hoffmann
86273f5e2a introduce database patch level for version
This will be needed later for automatic migrations.
2021-03-01 16:46:19 +01:00
Sarah Hoffmann
b4f64aa770 make sure that calls to PHP legacy scripts are fatal on error 2021-03-01 16:10:45 +01:00
Sarah Hoffmann
976c5e9121 introduce table for in-database properties
Adds a simple table where settings for the database can be
saved. This is useful for state that must not change after
import.
2021-03-01 16:09:17 +01:00
Sarah Hoffmann
db663dd92f remove unused import 2021-03-01 09:26:08 +01:00