Commit Graph

4080 Commits

Author SHA1 Message Date
Sarah Hoffmann
0a73ed7d64 add secondary importance to API BDD tests
Also fixes a path issue during API test DB creation that could
never possibly have worked.
2022-10-01 11:01:49 +02:00
Sarah Hoffmann
abf349fb0d simplify use of secondary importance
The values in the raster are already normalized between 0 and 2**16,
so a simple conversion to [0, 1] will do.

Check for existance of secondary_importance table statically when
creating the SQL function. For that to work importance tables need
to be created before the functions.
2022-10-01 11:01:49 +02:00
Sarah Hoffmann
3185fad918 load views as a SQL file and rename to 'secondary importance'
The only requirement for secondary importance is that a raster table
comes out of it. The generic name leaves open where the data comes
from.
2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
0ab0f0ea44 Integrated OSM views into importance computation 2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
ac467c7a2d Enhanced the implementation of OSM views GeoTIFF import functionality 2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
c85b74497b Initial implementation of GeoTIFF import functionality 2022-10-01 11:01:49 +02:00
Sarah Hoffmann
3381a92d92
Merge pull request #2832 from lonvia/conditional-analyze-on-indexing
Only run analyze on indexing when work was done
2022-09-28 15:17:40 +02:00
Sarah Hoffmann
a2ee58d8a1 only run analyze on indexing when work was done
This speeds up processing when continuing indexing after it was
interrupted.
2022-09-28 10:22:54 +02:00
Sarah Hoffmann
051f3720ce
Merge pull request #2829 from lonvia/optimize-indexes
Further optimize indexes
2022-09-26 10:02:51 +02:00
Sarah Hoffmann
f017e1e9a1 make sure indexes are used 2022-09-25 14:09:45 +02:00
Sarah Hoffmann
33ba6896a8 further split up the big geometry index
Adds partial indexes for all geometry queries used during import.
A full index is not necessary anymore at that point. Still create
the index afterwards for use in queries.

Also adds documentation for all indexes on where they are used.
2022-09-21 16:21:41 +02:00
Sarah Hoffmann
f4d3ae6f70 consolidate indexes over geometry_sectors
The index over geometry_sectors are mainly used for ordering
the places which need indexing. That means they function effectively
as a TODO list. Consolodate them so that they always only contain
the places which are still to do. Also add the appropriate index
for the boundary indexing phase.
2022-09-21 10:38:58 +02:00
Sarah Hoffmann
860f3559a1 split up large osmid index on placex
This doesn't do anything in terms of lookup speeds but the resulting
indexes are quite a bit smaller.
2022-09-21 09:24:57 +02:00
Sarah Hoffmann
d8be8a7293
fix funding link 2022-09-19 15:39:58 +02:00
Sarah Hoffmann
9750a361c9
add Github Sponsering to funding page 2022-09-19 15:38:56 +02:00
Sarah Hoffmann
ed3dd81d04 run final index creation in parallel 2022-09-19 11:55:25 +02:00
Sarah Hoffmann
bef1aebf1c add function for parallel execution of SQL scripts 2022-09-19 11:52:17 +02:00
Sarah Hoffmann
26688ba35d add link to funding page 2022-09-19 10:30:58 +02:00
Sarah Hoffmann
a1158feeb8
Merge pull request #2818 from lonvia/better-geometry-index
Add index for lookup of addressable areas
2022-09-19 10:18:43 +02:00
Sarah Hoffmann
aef014a47d add indexes for lookup of addressable areas
The generic geometry index has become to slow for that purpose.
2022-09-18 16:57:12 +02:00
Sarah Hoffmann
d6a0947e5a update security policy for 4.1 version 2022-09-13 08:58:31 +02:00
Sarah Hoffmann
bc94318d83 mypy: fix new warnings due to external type updates 2022-09-05 17:39:35 +02:00
Sarah Hoffmann
d4c6e58b57
Merge pull request #2812 from mausch/patch-1
docs: fix links to rank docs
2022-09-05 17:27:09 +02:00
Mauricio Scheffer
66832cf0a5
docs: fix links to rank docs 2022-09-05 11:11:13 +01:00
Sarah Hoffmann
bcfe817212
Merge pull request #2799 from lonvia/fix-inclusions-with-extratags
Ignore irrelevant extra tags on address interpolations
2022-08-13 19:02:27 +02:00
Sarah Hoffmann
07d72f950b
Merge pull request #2739 from tareqpi/collect_os_info.sh
integration of host system information script into Nominatim CLI tool
2022-08-13 19:02:14 +02:00
Sarah Hoffmann
dddfa3a075 ignore irrelevant extra tags on address interpolations
When deciding if an address interpolation has address information, only
look for addr:street and addr:place. If they are not there go looking
for the address on the address nodes. Ignores irrelevant tags like
addr:inclusion.

Fixes #2797.
2022-08-13 14:07:06 +02:00
Tareq Al-Ahdal
74019877a4 Added the feature of collecting host system information to the CI tests 2022-08-13 06:22:13 +08:00
Tareq Al-Ahdal
465d82a92f Integrated 'collect_os_info.py' into Nominatim's CLI tool 2022-08-13 06:18:10 +08:00
Tareq Al-Ahdal
49f889bf09 Enhanced and refactored 'collect_os_info.py'
Changed the script to functional programming paradigm to remove the big number of local attributes to decrease memory usage when running it. Additional OS info are now included.
2022-08-13 06:13:05 +08:00
Tareq Al-Ahdal
5e477e3b5b Merge remote-tracking branch 'upstream/master' into collect_os_info.sh 2022-08-13 05:53:39 +08:00
Sarah Hoffmann
67cfad6a2c
Merge pull request #2798 from lonvia/more-rank-change-fixes
Invalidations when boundaries and places change their rank
2022-08-12 11:42:03 +02:00
Sarah Hoffmann
487e81fe3c more invalidations when boundary changes rank
When a boundary or place changes its address rank, all places where
it participates as address need to be potentially reindexed.
Also use the computed rank when testing place nodes against
boundaries. Boundaries are computed earlier.

Fixes #2794.
2022-08-12 09:48:46 +02:00
Sarah Hoffmann
18f525ac54
Merge pull request #2793 from lonvia/increase-minimum-results
Fix minimum number of results that are searched for
2022-08-09 20:08:45 +02:00
Sarah Hoffmann
e0c184e097 fix base number of returned results
The intent was to always search for at least 10 results.

Improves on #882.
2022-08-09 13:53:20 +02:00
Sarah Hoffmann
78716ab8b9
Merge pull request #2792 from lonvia/new-type-annotations
Adapt to new type annotations from typeshed
2022-08-09 13:52:20 +02:00
Sarah Hoffmann
8d082c13e0 adapt to new type annotations from typeshed
Some more functions frrom psycopg are now properly annotated.
No ignoring necessary anymore.
2022-08-09 11:06:54 +02:00
Sarah Hoffmann
196dc2a659 docs: add types-psutil requirement 2022-08-08 09:46:25 +02:00
Sarah Hoffmann
4fe797d704 remove mypy ignore for psutil.virtual_memory()
Now available in typeshed.
2022-08-08 09:44:45 +02:00
Sarah Hoffmann
3c188164ab
Merge pull request #2789 from lonvia/update-osm2pgsql
Update osm2pgsql (fixes admin_level parsing)
2022-08-08 09:15:58 +02:00
Sarah Hoffmann
5330370076 update osm2pgsql (fix admin_level parsing) 2022-08-07 18:34:47 +02:00
Sarah Hoffmann
eecc73ea1a docs: fix dangling links 2022-08-05 15:29:43 +02:00
Sarah Hoffmann
8c73c0795e docs: update links to vagrant instructions 2022-08-05 15:27:11 +02:00
Sarah Hoffmann
7d68aa8f04 prepare release 4.1.0 2022-08-05 14:33:11 +02:00
Sarah Hoffmann
a0cd96e05e
Merge pull request #2786 from lonvia/export-centroid-for-tokenizer
Export centroid to tokenizer
2022-08-01 11:38:24 +02:00
Sarah Hoffmann
b19c90b9a6 export centroid to tokenizer
May come in handy when developping sanitizers for an area smaller
than country size.
2022-07-31 22:10:58 +02:00
Sarah Hoffmann
e427712cb0
Merge pull request #2784 from lonvia/doscs-customizing-icu-tokenizer
Document the public API of sanitizers and token analysis modules
2022-07-31 19:15:50 +02:00
Sarah Hoffmann
9864b191b1 fix various typos 2022-07-31 17:10:35 +02:00
Sarah Hoffmann
e7574f119e add simple examples of sanitizers and token analysis 2022-07-29 17:15:25 +02:00
Sarah Hoffmann
51b6d16dc6 overhaul the token analysis interface
The functional split betweenthe two functions is now that the
first one creates the ID that is used in the word table and
the second one creates the variants. There no longer is a
requirement that the ID is the normalized version. We might
later reintroduce the requirement that a normalized version be available
but it doesn't necessarily need to be through the ID.

The function that creates the ID now gets the full PlaceName. That way
it might take into account attributes that were set by the sanitizers.

Finally rename both functions to something more sane.
2022-07-29 15:14:11 +02:00