Commit Graph

55 Commits

Author SHA1 Message Date
Sarah Hoffmann
cbaabe7c24 add function to recalculate counts for full-word search term 2017-12-17 16:00:28 +01:00
Sarah Hoffmann
6c1977b448 replace double-quoting with single quotes where applicable 2017-10-26 21:40:33 +02:00
Sarah Hoffmann
8e0ffde3e0 fix CodeSniffer violations 2017-10-08 17:00:59 +02:00
Sarah Hoffmann
bafbf679b6 add script for updating postcodes 2017-10-03 15:58:14 +02:00
Sarah Hoffmann
8c4bcd36ea check that replication URL points to a repo of OSM diffs
Also check that pyosmium does not return None to work around
a bad return code in the current release of pyosmium-get-changes.

Fixes #784.
2017-08-29 21:05:54 +02:00
Sarah Hoffmann
896421fe9f improve update interval computation
Don't do anything if a downloaded diff is empty after all
(may be happening when an empty diff was published upstream).

Correctly compute the waiting interval before checking for new
data. As the interval is now computed based on the date of the
newest object in the database, the configured intervals need
to be adjusted slightly to take into account the time it takes
for the server to publish a diff.
2017-06-11 09:21:24 +02:00
Sarah Hoffmann
4e6c75040e Guard against network failures in getDatabaseDate()
When updating use the date from the diff file instead
as we are guaranteed to get this if the file has been
successfully loaded.
2017-06-09 21:49:31 +02:00
Sarah Hoffmann
1876fe42c9 updates: always remove output file and start with correct sequence id 2017-05-27 17:02:34 +02:00
Sarah Hoffmann
86acaa5264 fix name of constant 2017-05-27 16:42:01 +02:00
Sarah Hoffmann
bd4b1b2d08 switch from osmosis to pyosmium for updates
Pyosmium comes with convenient functions for finding the
right state and does not require external files for
rembering the state. Updates can now conveniently
set up by simply running ./utils/update.php --init-updates
and state is kept directly in the import_status table.

This change requires an update in the database schema.
Run the following to update:

ALTER TABLE import_status ADD COLUMN sequence_id integer;
ALTER TABLE import_status ADD COLUMN indexed boolean;
ALTER TABLE import_osmosis_log ADD COLUMN batchseq integer;
2017-05-25 16:26:09 +02:00
Christian von Roques
c80637b05c deduplicate words: do not look for duplicates in empty partitions 2017-04-15 19:12:25 +02:00
Christian von Roques
3ec67b9193 update: getPostgresVersion requires a database connection 2017-04-15 19:12:25 +02:00
Sarah Hoffmann
bb1c3f23ab reduce maximum line length to 199 for all php files 2016-10-14 00:01:16 +02:00
Marc Tobias Metten
4a321487f3 fix indentation and misc errors according to PSR2 coding style guide 2016-09-14 03:16:46 +02:00
Marc Tobias Metten
2a784fa3d4 fix function declaration errors according to PSR2 coding style guide 2016-09-11 05:22:51 +02:00
marc tobias
3887423381 fix array-related errors according to PSR2 coding style guide 2016-09-10 21:10:52 +02:00
marc tobias
f05ea577f4 bracket spacing for if/else/for/foreach/while/switch according to PSR2 standard 2016-09-08 02:16:22 +01:00
Marc Tobias Metten
832547f192 tabs-to-spaces 2016-09-04 03:19:48 +02:00
Sarah Hoffmann
cf4a44aaf7 introduce chksql function for phpscripts
Checks SQL query results for errors and bails out if it finds one.
Avoids some heavy code duplication all over the place.
2016-06-09 20:37:51 +02:00
Sarah Hoffmann
02bd322c1d remove explicitly set postgres/postgis version
Get the version from the database where necessary or simply
probe for existence of features. Fake hstore_to_json when
necessary.

Bumps the minimum required versions fro postgres to 9.1 and
for postgis to 2.0.
2016-03-30 22:48:18 +02:00
Sarah Hoffmann
a769fc4ff3 remove unused/unmaintained options in import scripts 2016-03-28 22:20:31 +02:00
Sarah Hoffmann
4b46c73088 fix nominatim paths 2016-03-16 19:56:10 +01:00
Sarah Hoffmann
5ccb499a6f install executables and call settings.php explicitly
Executables (including websites) need to be installed in the
build directory, so that they can find the right settings.php.
settings now defines build and source dir.
2016-03-12 09:40:43 +01:00
Sarah Hoffmann
5d7bd37ab2 introduce InstallPath, so that stuff can be executed from build dir 2016-03-12 09:40:43 +01:00
calimeroteknik
fb7d65b8c3 utils/update.php: use fail() whenever possible for errors 2015-12-07 14:08:02 +01:00
calimeroteknik
dbb1f5c0e2 utils/update.php: make the process exit with an non-zero error code if the update fails 2015-12-07 13:54:05 +01:00
Sarah Hoffmann
fe3eb40e3a disable multithreading in osm2pgsql
not really useful for gazetteer plugin and producing too much overhead
2015-08-29 13:14:11 +02:00
Brian Quinion
1d248ac86d Make deduplicate work again. Only currently support 9.3 due to new postgresql functions added. 2014-06-30 17:49:24 +01:00
Kurt Roeckx
45c1e7582f Connect to the correct postgresql port from the settings. 2013-11-13 22:24:10 +01:00
IrlJidel
dbd7f8fdae dont include polling time for batch duration for non-minutely updates, log name of each update step 2013-10-02 10:53:21 +01:00
IrlJidel
1c86275bad output date of steps during import-osmosis 2013-09-30 13:44:50 +01:00
IrlJidel
84d1e521aa show import_osmosis_log insert statements, print completed log after import_status inserts 2013-09-30 13:25:30 +01:00
IrlJidel
7471b09ed3 set start time for indexing step 2013-09-30 13:20:24 +01:00
Sarah Hoffmann
bf182ba2dc make import-osmosis(-all) fail with error code!=0 2013-09-25 23:05:47 +02:00
Sarah Hoffmann
8332fedfd6 make sleep interval dependent on date of last batch 2013-09-19 22:56:55 +02:00
Brian Quinion
65ea0efe68 Remove stray exit from debug 2013-09-19 15:19:31 +01:00
Brian Quinion
120d866e3e Sanity check on download.geofabrik.de update interval 2013-09-19 12:22:36 +01:00
Sarah Hoffmann
77b9edd1e9 allow osm2pgsql cache memory to be set for updates 2013-09-15 23:10:26 +02:00
Sarah Hoffmann
8fd7d319b7 Merge branch 'osmosis_error_return' of https://github.com/woodpeck/Nominatim 2013-09-09 13:46:59 +02:00
Frederik Ramm
b8d4371395 re-run osmosis also on return codes other than 1 2013-08-30 10:42:21 +02:00
Spin0us
85f10f48fd Update update.php 2013-07-31 07:48:29 +02:00
Frederik Ramm
87519de8ab fix indentation 2013-05-16 09:11:19 +02:00
Frederik Ramm
1f2a262ee0 Add osm2pgsql flat node support via settings.php 2013-05-16 09:03:16 +02:00
IrlJidel
3eea62363d fix typo for baseUrl when configuring configuration.txt
use osmosis --read-replication-lag to determine if there are changes before trying to process updates, useful when we are tracking hourly or daily replication updates

set CONST_Replication_Recheck_Interval to 60

skip lag check if CONST_Replication_Update_Interval > 60, for minutelies there's always new diffs to process

use tabs for indent

change sleep for non-minutely updates so that we dont drift tto much or poll excessively

unset $aReplicationLag before each exec attempt

unset $aReplicationLag inside while loop
2013-04-02 21:30:12 +01:00
Sarah Hoffmann
ea2cf36b12 switch to main overpass server 2013-02-26 20:42:57 +01:00
Brian Quinion
661338ff25 Use Etc/UTC timezone for updates for reliability 2012-12-08 21:42:12 +00:00
Sarah Hoffmann
410ce603f7 use Overpass API by default to download OSM objects 2012-05-23 23:32:12 +02:00
Sarah Hoffmann
e54bd028ee make index work and remove more hard-coded DB names 2012-05-23 23:32:12 +02:00
Datendelphin
8be76da629 eliminate php warning, remove setting aPipes (isn't used) 2012-05-23 23:32:12 +02:00
Datendelphin
1461dc603e make .osc files with osmosis 2012-05-23 23:32:11 +02:00