Don't do anything if a downloaded diff is empty after all
(may be happening when an empty diff was published upstream).
Correctly compute the waiting interval before checking for new
data. As the interval is now computed based on the date of the
newest object in the database, the configured intervals need
to be adjusted slightly to take into account the time it takes
for the server to publish a diff.
The postcode now needs to be saved in address->'postcode'
not in the postcode column which is derived only later while
indexing.
Fixes#746.
Thanks to @kkoop.
Compare the normalized terms imported with the special
terms script with the normalized version of the query string.
Disregard them if they cannot be found. This avoids a significant
number of mismatches due to transliteration issues.
The match will only be done when a normalized word has been set
making this change backwards compatible with older databases.
Pyosmium comes with convenient functions for finding the
right state and does not require external files for
rembering the state. Updates can now conveniently
set up by simply running ./utils/update.php --init-updates
and state is kept directly in the import_status table.
This change requires an update in the database schema.
Run the following to update:
ALTER TABLE import_status ADD COLUMN sequence_id integer;
ALTER TABLE import_status ADD COLUMN indexed boolean;
ALTER TABLE import_osmosis_log ADD COLUMN batchseq integer;
The interpolaton computation needs information from the osm2pgsql
slim tables which may not be available when the data is inserted.
Insertion now only adds a line with basic address information to
location_property_osmline. The line is then split during the
indexing, leading to more lines (which are complete in that case)
being inserted.
Fixes#598.
Search results can become odd without the country search
terms, so make their inclusion a mandatory part of the
setup.
Also adds a new configuration variable to restrict the
languages taken into account by Nominatim.
function pgsqlRunPartitionScript was missing global $aCMDResult
setup was failing with
...
PHP Notice: Undefined variable: aCMDResult in /home/roles/nominatim/app/Nominatim/build/utils/setup.php on line 768
NOTICE: type "nearplace" does not exist, skipping
..
ERROR: relation "location_area_large_0" already exists
ERROR: pgsql returned with error code (3)
pgsql returned with error code (3)
Command exited with non-zero status 255
Symlinking does not go well with the separate build directory.
The recommended method is now to serve the website directly from
build/website via an alias. 'website installation' may be
reintroduced later via cmake.
Introduces two new settings CONST_Use_US_Tiger_Data and
CONST_Use_Aux_Location_data, which are disabled by default.
When false the corresponding tables are ignored in queries
and updates.
Aux and tiger tables are no longer created by default. This
has to be done by the corresponding import scripts. The former
aux table creation can be found in sql/aux_tables.sql for
reference.
Get the version from the database where necessary or simply
probe for existence of features. Fake hstore_to_json when
necessary.
Bumps the minimum required versions fro postgres to 9.1 and
for postgis to 2.0.
Executables (including websites) need to be installed in the
build directory, so that they can find the right settings.php.
settings now defines build and source dir.
Apache 2.4 has changed the date format, so that the current regexp
doesn't match anymore, so be more tolerant with the date format.
Also force less fancy output formatting without tables.
This allows address interpolations to work correctly when flatnode storage
is used for node coordinates.
To fix interpolations in an existing database, follow these steps:
* invalidate all interpolations (in psql):
`UPDATE placex SET indexed_status=2 WHERE rank_search = 28`
* disable updates:
./utils/setup.php --create-functions --create-partition-functions
* reindex the whole lot:
./utils/update.php --index --index-instances <number of your cpus>
* enable updates again:
./utils/setup.php --create-functions --enable-diff-updates --create-partition-functions
Also added a sanity check to ensure that accidental removal of admin_level
tags on large areas doesn't cause huge reindexing load. That can be disabled
by setting CONST_Limit_Reindexing to false.
Name of function was changed in postgis 2.1 and now prints ugly
deprecation warnings. For older versions of postgis, function
will be renamed to the new name during the setup of the DB.
To update existing databases with postgis < 2.1 run:
ALTER FUNCTION st_line_interpolate_point(geometry, double precision) RENAME TO ST_LineInterpolatePoint
and then reinstall the SQL functions:
./utils/setup.php --create-functions --enable-diff-updates --create-partition-functions
Use planet_osm_nodes instead of places to get the very last node even if it is
untagged. Then ask for version 1 to avoid being mislead if the node has been
changed in the meantime.
use osmosis --read-replication-lag to determine if there are changes before trying to process updates, useful when we are tracking hourly or daily replication updates
set CONST_Replication_Recheck_Interval to 60
skip lag check if CONST_Replication_Update_Interval > 60, for minutelies there's always new diffs to process
use tabs for indent
change sleep for non-minutely updates so that we dont drift tto much or poll excessively
unset $aReplicationLag before each exec attempt
unset $aReplicationLag inside while loop
Adds word counts from a full planet to the word table. There is a
new configuration option CONST_Max_Word_Frequency which allows to
take into account the word count: the value that was set on import
is used to determine if a word is added to the search_name table.
The value during runtime determines if a single term should be
used for partial search or simply be ignored.
make setup.php fail early if:
- no osm-file argument was supplied for --all or --import-data
- the supplied osm-file is not existing or not readable (file permissions)
Import TIGER data into a temporary table first that later replaces
the current location_property_tiger table. This way index creation
on the table can be delayed until after the import which should
speed up the import and result in significantly smaller indexes.
Also removed index on parent_place_id as it is covered by
idx_location_property_tiger_housenumber_parent_place_id.
function changes:
-----------------
Move to ST_PointOnSurface from ST_Centroid in various places to avoid looking up a point outside the polygon
Move to ST_Covers from ST_Contains to include points on admin boundaries
Re-order preference for get_country_code now our data is better. country_osm_grid is now the preffered source.
Fix code to calculate country code in placex_insert, rank_search test was too early
Add extra field to placex 'calculated_country_code' to improve structure of code
Move split_geometery function out of add_location into its own function
Rewrite split_geometery to be more efficient.
Change place_insert to do more updates and less delete/inserts (delete is slow)
Include wikipedia links in details.php ouput
Cleanup no longer used geometry validation (adding overhead)
Include debug statements in function.sql (--DEBUG: ) and add flag to setup.php to turn them on
setup.php:
----------
add flag --disable-token-precalc to speed up debuging
add flag --index-noanalyse to disable analysising DB at rank 4 and 26 (previously removed, but on my local DB it seems to be required)
add flag --enable-diff-updates (modifier to --create-functions) to turn on the code required for diff updates without having to modify functions.sql
add flag --enable-debug-statements (modifier to --create-functions) to turn on debug warning statements
update.php:
-----------
added flag --no-index to import osmosis changes without indexing them
extend the hack to allow import of JOSM generated osm files
country_grid.sql - reference copy of the sql used to generate the country_osm_grid table, needs cleanup
Allows to restrict the special phrases imported from the wiki.
Blacklist allows to exclude certain class/type combinations.
Whitelist allows to define an allowed subset of types for a class.
Adjust to your liking in settings/phrase_settings.php before running
the specialphrases script.
- new config options for postgresql version and
location of osm2pgsql and osmosis binaries
- new option --no-npi for update.php that supresses writing of NPIs in
osmosis update mode
- add more GRANTs for www-data
