osm-search/Nominatim
1
1
Fork 0
mirror of https://github.com/osm-search/Nominatim.git synced 2024-11-13 10:26:11 +03:00
Code Issues Packages Projects Releases Wiki Activity
3,024 Commits 28 Branches 41 Tags 164 MiB
d5fc3b5e99
Commit Graph

3024 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sarah Hoffmann
d5fc3b5e99 subprocess needs string argument 
Compatibility change for Python 3.5.
 2021-04-23 22:50:08 +02:00
Sarah Hoffmann
f8f8c7e534 check for existance of custom .env before opening 2021-04-23 22:50:08 +02:00
Sarah Hoffmann
3a642d50a4 use more generic ImportError to check for module 
ModuleNotFoundError was only introduced in Python 3.6.
 2021-04-23 22:50:08 +02:00
Sarah Hoffmann
9685c68e30 replace usages of fromisoformat() with strptime() 
fromisoformat was only introduced with Python 3.7 while we
still support Python 3.5.

Fixes #2292.
 2021-04-23 22:50:08 +02:00
Sarah Hoffmann
95e6ec091b remove argparse dependency for vagrant scripts 
Users don't need to recreate the manpage.
 2021-04-23 22:50:08 +02:00
Sarah Hoffmann
788baafa26 bdd tests: fix place dependen ranking tests 
The ranks of places may differ for some countries. Force the
place nodes in the test on null island which always uses the
default ranking.
 2021-04-22 17:31:00 +02:00
Sarah Hoffmann
4c31813398
Merge pull request #2288 from RhinoDevel/patch-1 
Replace "nominatim-update" with "nominatim".
 2021-04-22 17:12:25 +02:00
RhinoDevel
b7bae80616
Replace "nominatim-update" with "nominatim". 
If I am not mistaken, the correct command to index imported data via commandline is "nominatim index".
 2021-04-22 15:40:22 +02:00
Sarah Hoffmann
f7e4aa51d3 indexer: reset query counter 
Reset the counter for queries after the asynchronous connections
have been reopened.
 2021-04-21 10:33:45 +02:00
Sarah Hoffmann
696c50459f
Merge pull request #2285 from lonvia/split-indexer-code 
Rework indexer code
 2021-04-20 15:34:14 +02:00
Sarah Hoffmann
50b6d7298c factor out async connection handling into separate class 
Also adds a test for reconnecting regularly while indexing.
 2021-04-20 14:08:37 +02:00
Sarah Hoffmann
26a81654a8 indexer: make self.conn function-local 
Also switches to our internal connect function which gives us
a cursor with a sclar() function.
 2021-04-20 14:08:37 +02:00
Sarah Hoffmann
6430371d7d make index() function private 2021-04-20 14:08:37 +02:00
Sarah Hoffmann
18705b3f18 move analyse function into indexinf function 2021-04-20 14:08:37 +02:00
Sarah Hoffmann
c6bd2bb7fb indexer: move runner into separate file 2021-04-20 14:08:37 +02:00
Sarah Hoffmann
c4fd94bd1a
Merge pull request #2284 from lonvia/cleanup-word-frequency-computation 
Rename and simplify function for word pre-computation
 2021-04-19 18:28:04 +02:00
Sarah Hoffmann
b88b952f56 simplify token precomputation 
Rename function to reflect that it is only used for precomputation.
The token IDs are not really needed, so don't bother to compute
the array of tokens.
 2021-04-19 17:24:19 +02:00
Sarah Hoffmann
d68b02d36a remove unused word recomputation script 
Has been replaced by a script recomputing counts from search_name.
 2021-04-19 16:40:57 +02:00
Sarah Hoffmann
b9b85eb208
Merge pull request #2283 from darkshredder/tiger-data-test-fix 
Fix: tiger-data tarfile test
 2021-04-19 13:56:36 +02:00
Darkshredder
1f898405a6 Fix: tiger-data tarfile test 2021-04-19 16:02:52 +05:30
Sarah Hoffmann
6f6910101e
Merge pull request #2282 from lonvia/add-paths-to-config 
Include software paths in Python config object
 2021-04-19 12:14:25 +02:00
Sarah Hoffmann
79d55357e8 simplify sql and website creation functions 2021-04-19 10:53:30 +02:00
Sarah Hoffmann
4fa6c0ad53 simplify constructor for SQL preprocessor 
Use sql path from config.
 2021-04-19 10:26:25 +02:00
Sarah Hoffmann
8f63f9516b simplify interface for adding tiger data 
Also simplifies tests using existing fixtures.
 2021-04-19 10:26:25 +02:00
Sarah Hoffmann
995ba2c7c2 add library directories to config 
Allows to reduce the number of parameters in functions that take
the config anyway.
 2021-04-19 10:26:25 +02:00
Sarah Hoffmann
830e3be1e6
Merge pull request #2281 from changpingc/changping/fix-tiger-index 
fix index on location_property_tiger (parent_place_id)
 2021-04-19 08:42:59 +02:00
Channgping Chen
29a314a092 fix index on location_property_tiger (parent_place_id) 
Looks like 2af82975cd
accidentally renamed an index. Because of the added "if not
exists" clause, the index doesn't get created. This
significantly slows down reverse queries because they now
require full scans on location_property_tiger.

Without this fix, reverse queries can take 8s on a full
planet install on an r5.8xlarge instance in EC2.
 2021-04-19 00:33:15 +00:00
Sarah Hoffmann
abdba5fdc7
Merge pull request #2280 from AntoJvlt/Fix-special-phrases-import-and-tests-cleaning 
Fix regex and sanity check for the import of special phrases and tests cleaning.
 2021-04-18 11:57:19 +02:00
AntoJvlt
b2ae715699 Only log a warning if a wrong input is detected on the wiki while importing special phrases 2021-04-17 20:19:39 +02:00
AntoJvlt
a95c748363 Fix occurence regex 2021-04-17 19:24:13 +02:00
AntoJvlt
ec859e41c6 Cleaned tests and add database cleaning tests on test_import_from_wiki 2021-04-17 19:23:33 +02:00
Sarah Hoffmann
7aeae9da81
Merge pull request #2279 from lonvia/add-index-for-continued-indexing 
Add index for continued indexing
 2021-04-17 11:51:21 +02:00
Sarah Hoffmann
2ca11ccc6b add tests for continuing import 2021-04-17 11:10:36 +02:00
Sarah Hoffmann
d74ae669e3 add support index when continuing import at index phase 
Indexing scans the placex table sequentially during indexing
on the initial import. That is okay because we know that all
rows need to be processed anywhere. When continuing the import,
however, a large part might already be indexed, so that the
process spends a lot of time going through rows that are no
longer of interest. Create a supporting index for all unindexed
rows to speed up the scan. This is the same index as used later
for updates.
 2021-04-17 11:07:04 +02:00
Sarah Hoffmann
9fabc5572d
Merge pull request #2278 from lonvia/remove-transistion-functions 
Remove transition functions
 2021-04-17 10:13:33 +02:00
Sarah Hoffmann
da98a2102a remove transition functions from Python 2021-04-16 18:41:14 +02:00
Sarah Hoffmann
fb3353b854
Merge pull request #2277 from lonvia/update-osm2pgsql 
Update osm2pgsql to current master
 2021-04-16 17:40:43 +02:00
Sarah Hoffmann
b7e5c54593 remove PHP code for transition functions 2021-04-16 17:28:51 +02:00
Sarah Hoffmann
68beec5590 remove installation of PHP util scripts 2021-04-16 17:09:40 +02:00
Sarah Hoffmann
6ba06d1eb4
Merge pull request #2276 from lonvia/port-country-code-creation-to-python 
Port country code creation to python
 2021-04-16 16:57:04 +02:00
Sarah Hoffmann
0f11e311c4 add test for new postcode import function 2021-04-16 16:11:20 +02:00
Sarah Hoffmann
886a01c796 port function to compute initial postcodes to Python 2021-04-16 16:11:20 +02:00
Sarah Hoffmann
a632b9f86a
Merge pull request #2275 from lonvia/switch-to-absolute-imports 
Use absolute imports in Python code
 2021-04-16 15:04:10 +02:00
Sarah Hoffmann
76b1885595 use absolute imports in Python code 
Relative imports are no longer officially recommended.
 2021-04-16 14:20:09 +02:00
Sarah Hoffmann
c55b409cf6 update osm2pgsql to current master (fixes version output) 2021-04-15 10:24:01 +02:00
Sarah Hoffmann
c64193f839
Merge pull request #2263 from AntoJvlt/special-phrases-autoupdate 
Implemented auto update of special phrases while importing them
 2021-04-15 10:13:25 +02:00
Sarah Hoffmann
28a2a795ba
Merge pull request #2270 from lonvia/simplify-place-boundary-merge 
Simplify matching between place and boundary names
 2021-04-15 10:12:53 +02:00
Sarah Hoffmann
e90adfc7c3 adapt database check to new index layout 2021-04-14 17:52:59 +02:00
Sarah Hoffmann
16267dc021 add migration for new placenode geometry index 2021-04-14 17:52:59 +02:00
Sarah Hoffmann
e7266b52ae simplify name matching between boundary and place node 
Instead of normalising the names simply compare them in lower
case. This removes the dependency on the tokenizer for
linking boundaries and nodes. When looking up the linked places
by place type also allow that one name is simply contained in the
other. This catches the frequent case where one of the names has
an addendum (e.g. Newport vs. City of Newport).

Drops the special index for the name lookup and insted relies
on a slightly extended version of the geometry index used for
reverse lookup. Saves around 100MB on a planet.
 2021-04-14 17:52:59 +02:00