osm-search/Nominatim
1
1
Fork 0
mirror of https://github.com/osm-search/Nominatim.git synced 2024-12-01 14:31:42 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,886 Commits 28 Branches 41 Tags 164 MiB
3933fc3ad3
Commit Graph

2886 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sarah Hoffmann
3933fc3ad3 avoid multi-term partials in names 
Names are either full words or single-word partial names.
Searching for multi-word partials yields exactly the same
result as with full words.
 2021-03-11 20:42:37 +01:00
Sarah Hoffmann
00b05e2394 higher penalty for special searches 
Adds a general higher penalty for special search term and an
additional one if the term is anywhere but the beginning or the
end. Also housenumbers and special searches together are less
likely.
 2021-03-11 20:37:51 +01:00
Sarah Hoffmann
d5e8c5e975 do not mix partial and full name terms 
If NameNonSearch already contains a partial term, then a
full term must not be added to the Name list anymore.
 2021-03-11 20:22:54 +01:00
Sarah Hoffmann
478dfb0639 add one-rank penalty for using partial search 
Ensures that full matches are preferred over partial ones even when
the full word consists of only one term.
 2021-03-11 17:52:44 +01:00
Sarah Hoffmann
f498e40208 fix result splitting for last search group 
When we are in the final iteration of the search groups, it is not
possible to further delay the results. Unconditionally use the
results with the best rank instead.
 2021-03-11 17:14:46 +01:00
Sarah Hoffmann
182f5f5d7b give preference to full words in address, too 
Full word terms are already preferred for the name part. Adding
only one-word partials to the address, makes it impossible to
give a similar preference for the address part. Each term adds
a rank penalty. The problem here is that we interpret the query
forwards and backwards. Having different penalty systems for
name and address means that the same term ends up with different
penalties and that often leads to interpretations of the wrong
direction being in the way.
 2021-03-11 15:03:36 +01:00
Sarah Hoffmann
35f4695b67
Merge pull request #2200 from lonvia/migrations-for-current-version 
Introduce a command for database migration
 2021-03-08 10:14:03 +01:00
Sarah Hoffmann
3c9e09545e documentation for new migration command 2021-03-06 16:38:37 +01:00
Sarah Hoffmann
764a41b973 automatic migration from 3.6 release 
Adds a 'admin --migrate' command that checks for the current
database version and runs any necessary migrations. Also
has migrations going back to 3.6.
 2021-03-06 16:36:57 +01:00
Sarah Hoffmann
9d103503f7
Merge pull request #2197 from lonvia/use-jinja-for-sql-preprocessing 
Use jinja2 for SQL preprocessing
 2021-03-04 16:36:18 +01:00
Sarah Hoffmann
09f4d767e4 port index creation to python 
Also switches to jinja-based preprocessing, which allows to
simplify the SQL files. Use 'if not exists' where possible
so that the step can be rerun to fix missing indexes.
 2021-03-04 11:11:47 +01:00
Sarah Hoffmann
dd301cf5ac indexer: ANALYSE must be run outside transactions 2021-03-04 11:06:33 +01:00
Sarah Hoffmann
eacabb0e96 move table creation to jinja-based preprocessing 2021-03-03 22:07:51 +01:00
Sarah Hoffmann
6cda021d9b add new jinja2 requirement 2021-03-03 17:51:08 +01:00
Sarah Hoffmann
d2bd6aa78d introduce jinja2 for preprocessing SQL 
Replaces various hand-crafted replacements of varying format with
a single Jinja2 templating mechanism. Allows full access to
configuration if necessary.
 2021-03-03 17:51:08 +01:00
Sarah Hoffmann
6b306f30b6
Merge pull request #2194 from grischard/patch-1 
Fix typo in .github/actions/build-nominatim/action.yml
 2021-03-03 11:34:12 +01:00
Guillaume Rischard
c48fd18344
Update action.yml 2021-03-03 11:20:21 +01:00
Sarah Hoffmann
8ea7e04363
Merge pull request #2192 from lonvia/database-versioning 
Introduce database versioning
 2021-03-02 15:57:46 +01:00
Sarah Hoffmann
32c2d2b248 document new status fields 2021-03-01 22:21:37 +01:00
Sarah Hoffmann
111cca8c9a return database version with status API 2021-03-01 22:17:16 +01:00
Sarah Hoffmann
7ae9c3a9f0 add database_version setting to tests 2021-03-01 21:49:33 +01:00
Sarah Hoffmann
bf4320a7d6 do not depend on cmdline parameter for creating partition tables 
The partition numbers in use only depend on the entries in search_name.
 2021-03-01 21:28:39 +01:00
Sarah Hoffmann
3a0a4b9175 save software version in the database 
The version represents the software version that was used to
import the data.
 2021-03-01 20:35:15 +01:00
Sarah Hoffmann
4faefe156c report software version of status call 2021-03-01 16:47:19 +01:00
Sarah Hoffmann
86273f5e2a introduce database patch level for version 
This will be needed later for automatic migrations.
 2021-03-01 16:46:19 +01:00
Sarah Hoffmann
b4f64aa770 make sure that calls to PHP legacy scripts are fatal on error 2021-03-01 16:10:45 +01:00
Sarah Hoffmann
976c5e9121 introduce table for in-database properties 
Adds a simple table where settings for the database can be
saved. This is useful for state that must not change after
import.
 2021-03-01 16:09:17 +01:00
Sarah Hoffmann
db663dd92f remove unused import 2021-03-01 09:26:08 +01:00
Sarah Hoffmann
90a5d23016 use tmp_path fixture in config tests 2021-03-01 09:24:04 +01:00
Sarah Hoffmann
99e35d256a
fix typo 2021-03-01 09:07:49 +01:00
Sarah Hoffmann
e14e7c6235
Merge pull request #2186 from lonvia/port-import-to-python 
Move setup procedure to Python
 2021-02-27 12:09:23 +01:00
Sarah Hoffmann
b46adbad22 make sure psql always finishes 
If an execption is raised by other means, we still have to close
the stdin pipe to psql to make sure that it exits and releases its
connection to the database.
 2021-02-27 10:24:40 +01:00
Sarah Hoffmann
afabbeb546 older versions of Postgresql need explicit return type 2021-02-27 09:46:42 +01:00
Sarah Hoffmann
d14a3df10f do not truncate search_name in reverse-only mode 2021-02-27 09:46:42 +01:00
Sarah Hoffmann
9feb84e426 actions: add psutil dependency 2021-02-26 16:50:09 +01:00
Sarah Hoffmann
c7f40e3cee fix verbose flag for PHP wrapper scripts 
The flag must come after the command.
 2021-02-26 16:49:32 +01:00
Sarah Hoffmann
dd03aeb966 bdd: use python library where possible 
Replace calls to PHP scripts with direct calls into the
nominatim Python library where possible. This speed up
tests quite a bit.
 2021-02-26 16:14:29 +01:00
Sarah Hoffmann
15b5906790 move setup function to python 
There are still back-calls to PHP for some of the sub-steps.
These needs some larger refactoring to be moved to Python.
 2021-02-26 15:02:39 +01:00
Sarah Hoffmann
3ee8d9fa75 properly close connections of indexer after use 2021-02-26 12:10:54 +01:00
Sarah Hoffmann
57db5819ef prot load-data function to python 2021-02-25 21:32:40 +01:00
Sarah Hoffmann
3c186f8030 add a function for the intial indexing run 
Also moves postcodes to fully parallel indexing.
 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
db5e78c879 remove unused partitionfunction function 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
c7fd0a7af4 port wikipedia importance functions to python 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
32683f73c7 move import-data option to native python 
This adds a new dependecy to the Python psutil package.
 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
7222235579 introduce custom object for cmdline arguments 
Allows to define special functions over the arguments.

Also splits CLI tests in two files as they have become too many.
 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
f6e894a53a port database setup function to python 
Hide the former PHP functions in a transition command until
they are removed.
 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
b93ec2522e use psql for executing sql files 
This allows to run larger files without needing to keep
them in memory.
 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
af7226393a add function to set up libpq environment 
Instead of parsing the DSN for each external libpq program we
are going to execute, provide a function that feeds them all
necessary parameters through the environment.

osm2pgsql is the first user.
 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
e520613362 convert connect() into a context manager 2021-02-25 18:42:54 +01:00
Sarah Hoffmann
204fe20b4b
Merge pull request #2185 from lonvia/fix-deadlock-handling-for-psycopg27 
Improve deadlock detection for various versions of psycopg2
 2021-02-25 18:39:40 +01:00