Commit Graph

4117 Commits

Author SHA1 Message Date
Sarah Hoffmann
a1da149211 CI: require lua libraries 2022-11-10 09:38:25 +01:00
Sarah Hoffmann
74405e9684 add migration for place_to_be_deleted table 2022-11-10 09:38:25 +01:00
Sarah Hoffmann
2fac507453 change updates to handle delete/insert workflow
This makes Nominatim compatible with osm2pgsql's default update
modus operandi of deleting and reinserting data. Deletes are diverted
into a TODO table instead of executing them. When data is reinserted,
the corresponding entry in the TODO table is deleted. After updates are
finished, the remaining entries in the TODO table are executed, doing
the same work as the delete trigger did before.

The new behaviour also works against the gazetteer output with its
insert-only mechanism.
2022-11-10 09:38:23 +01:00
Sarah Hoffmann
51ed55cc32 initial flex import scripts
Only implements the extratags style for the moment. Tests pass
for the same behaviour as the gazetteer output. Updates still need
to be done.
2022-11-10 09:37:38 +01:00
Sarah Hoffmann
de2a3bd5f8 bdd tests: make import style configurable
The switch is for development. Tests are not guaranteed to still
work when run with anything but the 'extratags' style.
2022-11-10 09:37:38 +01:00
Sarah Hoffmann
981e9700be add osm2pgsql gazetteer tests
This ports the gazetteer tests from osm2pgsql to BDD tests.
2022-11-10 09:37:38 +01:00
Sarah Hoffmann
b52ce4f9f2
Merge pull request #2869 from mtmail/improve-tiger-install-doc
Tiger install doc: add -refresh website- step
2022-11-09 20:48:39 +01:00
Sarah Hoffmann
64c591da7f fix type issues with calls to pyosmium 2022-11-09 20:46:33 +01:00
Marc Tobias
2387648a85 Tiger install doc: add -refresh website- step 2022-11-09 17:33:31 +01:00
Sarah Hoffmann
846ecff0c5
Merge pull request #2871 from lonvia/fix-timeout-for-updates
Fix timeout for updates
2022-11-09 14:26:39 +01:00
Sarah Hoffmann
26a5b59c28 add types-requests dependency 2022-11-09 09:12:37 +01:00
Sarah Hoffmann
6ddb39fda3 respect socket timeout also in other replication functions 2022-11-09 09:12:37 +01:00
Sarah Hoffmann
1fdcec985a fix timeout use for replication timeout
The timeout parameter is no longer taken into account since
pyosmium switched to the requests library. This adds the parameter
back.
2022-11-09 09:12:37 +01:00
Sarah Hoffmann
30f526c943
Merge pull request #2870 from mtmail/update-github-actions-to-node-16
update those github action packages still using node12
2022-11-08 17:24:53 +01:00
Marc Tobias
253127cb9f update those github action packages still using node12 2022-11-08 15:16:55 +01:00
Sarah Hoffmann
3237ca587f
Merge pull request #2866 from lonvia/reverse-ignore-interpolations-without-parent
Ignore interpolations without parent on reverse search
2022-11-07 09:00:59 +01:00
Sarah Hoffmann
0dbc0ae6d5 ignore interpolations without parent on reverse search
If no parent can be found for an interpolation, there is most
likely a data error involved. So don' t show these interpolations
in reverse search results.
2022-11-05 22:16:09 +01:00
Sarah Hoffmann
7461ff4680
Merge pull request #2865 from Romeo-PHILLIPS/fix/documentation_status_code
Fix: documentation status code
2022-11-05 22:14:44 +01:00
Romeo
afc714e1d3 fix: format 2022-11-04 18:05:40 +01:00
Romeo
3bc0db8d91 fix: markup 2022-11-04 18:04:28 +01:00
Romeo
d573da5b2c fix: 705 Status Code Documentation 2022-11-04 18:03:49 +01:00
Romeo
ecd5a3fdf9 fix: 705 Status Code Documenation 2022-11-04 17:59:36 +01:00
Sarah Hoffmann
543d63e7a9
Merge pull request #2862 from mtmail/remove-version-from-fpm-sock-file
Install scripts: remove version from /var/run/php-fpm filenames
2022-11-04 17:32:50 +01:00
Sarah Hoffmann
7a22ae6bf9
Merge pull request #2863 from lonvia/add-support-for-postgresql-15
Update CI tests to postgresql 15
2022-11-04 17:32:06 +01:00
Sarah Hoffmann
ebe23d6882 update CI tests to postgresql 15 2022-11-04 16:21:15 +01:00
marc tobias
33c805aee0 Install scripts: remove version from /var/run/php-fpm filenames 2022-11-04 14:22:11 +01:00
t-tomek
244b6fcef6
Update DB.php
Use SET instead of UPDATE queries to support read only databases
2022-11-02 14:11:52 +01:00
Sarah Hoffmann
616ff4ae25 actions: pin pyicu to 2.9 2022-10-24 14:21:44 +02:00
Sarah Hoffmann
e221eaa977
Merge pull request #2836 from mtmail/tiger2022
Documentation: remove year from TIGER filename, new 2022 data
2022-10-24 11:21:55 +02:00
Sarah Hoffmann
eed7abb839
Merge pull request #2838 from lonvia/update-osm2pgsql
Update osm2pgsql to latest 1.7.1 release
2022-10-05 18:59:13 +02:00
Sarah Hoffmann
5f6dcd36ed fix flaky API test
The search 'landstr' produces many duplicates so that with
some bad luck 4 or less results may appear. Disable deduplication
to make it more predictable.
2022-10-05 15:16:14 +02:00
Sarah Hoffmann
f395054536 update osm2pgsql to 1.7.1 2022-10-04 21:16:57 +02:00
Sarah Hoffmann
afeafc8aa7
Merge pull request #2835 from lonvia/secondary-importance
Secondary importance
2022-10-04 16:25:47 +02:00
marc tobias
f1ece658f8 Documentation: remove year from TIGER filename 2022-10-04 14:19:36 +02:00
Sarah Hoffmann
b3abb355eb docs: add customization hints for secondary importance
Removing the download links for now as the tile importance
is still too experimental.
2022-10-01 11:01:49 +02:00
Sarah Hoffmann
5877b69d51 do not run unit test when postgis_raster is not available 2022-10-01 11:01:49 +02:00
Sarah Hoffmann
5ec2c1b712 adapt unit tests to changed function names 2022-10-01 11:01:49 +02:00
Sarah Hoffmann
0a73ed7d64 add secondary importance to API BDD tests
Also fixes a path issue during API test DB creation that could
never possibly have worked.
2022-10-01 11:01:49 +02:00
Sarah Hoffmann
abf349fb0d simplify use of secondary importance
The values in the raster are already normalized between 0 and 2**16,
so a simple conversion to [0, 1] will do.

Check for existance of secondary_importance table statically when
creating the SQL function. For that to work importance tables need
to be created before the functions.
2022-10-01 11:01:49 +02:00
Sarah Hoffmann
3185fad918 load views as a SQL file and rename to 'secondary importance'
The only requirement for secondary importance is that a raster table
comes out of it. The generic name leaves open where the data comes
from.
2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
0ab0f0ea44 Integrated OSM views into importance computation 2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
ac467c7a2d Enhanced the implementation of OSM views GeoTIFF import functionality 2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
c85b74497b Initial implementation of GeoTIFF import functionality 2022-10-01 11:01:49 +02:00
Sarah Hoffmann
3381a92d92
Merge pull request #2832 from lonvia/conditional-analyze-on-indexing
Only run analyze on indexing when work was done
2022-09-28 15:17:40 +02:00
Sarah Hoffmann
a2ee58d8a1 only run analyze on indexing when work was done
This speeds up processing when continuing indexing after it was
interrupted.
2022-09-28 10:22:54 +02:00
Sarah Hoffmann
051f3720ce
Merge pull request #2829 from lonvia/optimize-indexes
Further optimize indexes
2022-09-26 10:02:51 +02:00
Sarah Hoffmann
f017e1e9a1 make sure indexes are used 2022-09-25 14:09:45 +02:00
Sarah Hoffmann
33ba6896a8 further split up the big geometry index
Adds partial indexes for all geometry queries used during import.
A full index is not necessary anymore at that point. Still create
the index afterwards for use in queries.

Also adds documentation for all indexes on where they are used.
2022-09-21 16:21:41 +02:00
Sarah Hoffmann
f4d3ae6f70 consolidate indexes over geometry_sectors
The index over geometry_sectors are mainly used for ordering
the places which need indexing. That means they function effectively
as a TODO list. Consolodate them so that they always only contain
the places which are still to do. Also add the appropriate index
for the boundary indexing phase.
2022-09-21 10:38:58 +02:00
Sarah Hoffmann
860f3559a1 split up large osmid index on placex
This doesn't do anything in terms of lookup speeds but the resulting
indexes are quite a bit smaller.
2022-09-21 09:24:57 +02:00