Commit Graph

251 Commits

Author SHA1 Message Date
Sarah Hoffmann
3fda792929 ignore empty flatnode file option
Fixes #902.
2018-02-12 20:47:04 +01:00
marc tobias
e9407cd48d shorten line to please PHP style guide 2018-02-08 17:52:26 +01:00
Marc Tobias Metten
1d6861667b update.php - check file exists before deleting 2018-01-31 00:38:05 +01:00
Sarah Hoffmann
2cf1ff41c0 move nominatim.org links to https
Solves #737.
2018-01-10 23:21:21 +01:00
Sarah Hoffmann
118517b076
Merge pull request #874 from lonvia/check-for-updates
Add function to check if new updates are available
2018-01-10 22:51:12 +01:00
Sarah Hoffmann
45abcbc301 update drop list for new postcode table
Fixes #875.
2018-01-05 22:41:25 +01:00
Sarah Hoffmann
d5df1c8ae3 fix setup when no us_postcode is available 2018-01-05 22:41:05 +01:00
Sarah Hoffmann
9712decefe update URLs in code and documentation
Use https for all openstreetmap addresses, remove defunct
mapquest link and redirect documentation links to
nominatim.org.
2018-01-05 22:38:51 +01:00
Sarah Hoffmann
b06bc799bc add function to check if new updates are available 2018-01-01 22:23:29 +01:00
Sarah Hoffmann
cbaabe7c24 add function to recalculate counts for full-word search term 2017-12-17 16:00:28 +01:00
Marc Tobias Metten
1a1e0ef138 update utils/tiger_county_fips.json data 2017-10-28 00:08:59 +02:00
Sarah Hoffmann
6c1977b448 replace double-quoting with single quotes where applicable 2017-10-26 21:40:33 +02:00
Sarah Hoffmann
9ac401267a tiger import: convert counties to str
For python2 the gdal features come out as str and
cannot be combined with unicode strings.
2017-10-24 22:27:09 +02:00
marc tobias
a71200a57a huge cleanup of tigerAddressImport.py 2017-10-24 22:27:09 +02:00
marc tobias
b062e7e774 huge cleanup of tigerAddressImport.py 2017-10-24 22:27:09 +02:00
marc tobias
8eed1a8bec Tiger 2017 data no longer contains -divroad- field 2017-10-20 15:17:51 +02:00
Sarah Hoffmann
8e0ffde3e0 fix CodeSniffer violations 2017-10-08 17:00:59 +02:00
Sarah Hoffmann
bafbf679b6 add script for updating postcodes 2017-10-03 15:58:14 +02:00
Sarah Hoffmann
8e2ef2842e move psqlRunScript implementation into cmd lib
Function needed for update.php as well.
2017-10-03 14:26:59 +02:00
Sarah Hoffmann
61ed3b8ab3 setup: bail out earl when something is wrong with nominatim.so 2017-09-17 20:07:03 +02:00
Sarah Hoffmann
bb1552be29 setup: error out when web site user does not exist
User is needed to be able to grant rights.
2017-09-17 19:51:00 +02:00
Sarah Hoffmann
5614ece9a1 run psql in quiet mode unless 'verbose' is enabled 2017-09-17 11:34:35 +02:00
Sarah Hoffmann
3546b30473 timestamp info message and repeat warnings at end 2017-09-17 11:06:52 +02:00
Sarah Hoffmann
909b0c7462 Merge pull request #782 from lonvia/rework-postcodes
Rework handling of artificial postcode centroids
2017-09-16 15:54:55 +02:00
Sarah Hoffmann
8c4bcd36ea check that replication URL points to a repo of OSM diffs
Also check that pyosmium does not return None to work around
a bad return code in the current release of pyosmium-get-changes.

Fixes #784.
2017-08-29 21:05:54 +02:00
Sarah Hoffmann
e55ac77c94 add simple tests for postcode import 2017-08-19 19:37:06 +02:00
Sarah Hoffmann
3c9af7f151 move adding postcodes to word table to calculation step 2017-08-19 19:37:06 +02:00
Sarah Hoffmann
5b4bbab9be include GB CodePoint data into location_postcode table 2017-08-19 19:37:06 +02:00
Sarah Hoffmann
53f8459e97 move postcode indexing to end of setup
The search_name tables are needed for finding the parent,
so the rest of the database must be indexed.
2017-08-19 19:37:05 +02:00
Sarah Hoffmann
80ef6cbaab add indexing of artificial postcodes 2017-08-19 19:37:05 +02:00
Sarah Hoffmann
15dbb6383c add new location_postcode table
Artifical postcode centroids are now saved in there.
2017-08-19 19:37:05 +02:00
Sarah Hoffmann
896421fe9f improve update interval computation
Don't do anything if a downloaded diff is empty after all
(may be happening when an empty diff was published upstream).

Correctly compute the waiting interval before checking for new
data. As the interval is now computed based on the date of the
newest object in the database, the configured intervals need
to be adjusted slightly to take into account the time it takes
for the server to publish a diff.
2017-06-11 09:21:24 +02:00
Sarah Hoffmann
4e6c75040e Guard against network failures in getDatabaseDate()
When updating use the date from the diff file instead
as we are guaranteed to get this if the file has been
successfully loaded.
2017-06-09 21:49:31 +02:00
Sarah Hoffmann
0628aa887f adapt postcode centroid import to new address hstore
The postcode now needs to be saved in address->'postcode'
not in the postcode column which is derived only later while
indexing.

Fixes #746.

Thanks to @kkoop.
2017-06-06 20:42:13 +02:00
Sarah Hoffmann
54393addd3 disregard special phrases that do not match fully
Compare the normalized terms imported with the special
terms script with the normalized version of the query string.
Disregard them if they cannot be found. This avoids a significant
number of mismatches due to transliteration issues.

The match will only be done when a normalized word has been set
making this change backwards compatible with older databases.
2017-06-04 23:12:09 +02:00
Sarah Hoffmann
e3fb706c65 add normalized version of special search terms on import
Requires the PHP bindings for libicu, so add that as a requirement.
2017-06-04 23:12:09 +02:00
Sarah Hoffmann
1876fe42c9 updates: always remove output file and start with correct sequence id 2017-05-27 17:02:34 +02:00
Sarah Hoffmann
86acaa5264 fix name of constant 2017-05-27 16:42:01 +02:00
Sarah Hoffmann
bd4b1b2d08 switch from osmosis to pyosmium for updates
Pyosmium comes with convenient functions for finding the
right state and does not require external files for
rembering the state. Updates can now conveniently
set up by simply running ./utils/update.php --init-updates
and state is kept directly in the import_status table.

This change requires an update in the database schema.
Run the following to update:

ALTER TABLE import_status ADD COLUMN sequence_id integer;
ALTER TABLE import_status ADD COLUMN indexed boolean;
ALTER TABLE import_osmosis_log ADD COLUMN batchseq integer;
2017-05-25 16:26:09 +02:00
Sarah Hoffmann
b15b996cc4 remove base country grid from repo
The updated country grid is significantly larger, so expect it to
be downloaded from the nominatim.org website instead.
2017-05-12 23:02:50 +02:00
Sarah Hoffmann
86d5209118 Merge branch 'roques-tetris' 2017-04-22 18:14:05 +02:00
Sarah Hoffmann
291a161cc6 make sure no invalid geometries end up in placex 2017-04-22 10:34:03 +02:00
Sarah Hoffmann
4c7145c293 Merge branch 'tetris' of https://github.com/roques/Nominatim into roques-tetris 2017-04-20 20:56:44 +02:00
Sarah Hoffmann
71e6d5f9a6 fix typo 2017-04-18 22:44:39 +02:00
Sarah Hoffmann
446a5a95bc get pg error on correct connection 2017-04-18 22:29:21 +02:00
Sarah Hoffmann
19edbd3581 send address interpolation import to correct postgres conn 2017-04-18 22:11:58 +02:00
Christian von Roques
b6be33cded explicitly name columns at INSERT 2017-04-18 13:36:02 +02:00
Christian von Roques
c80637b05c deduplicate words: do not look for duplicates in empty partitions 2017-04-15 19:12:25 +02:00
Christian von Roques
3ec67b9193 update: getPostgresVersion requires a database connection 2017-04-15 19:12:25 +02:00
Christian von Roques
40debbc7b5 with PHP7 the result of new() can no longer be assigned by reference 2017-04-15 19:12:25 +02:00