osm-search/Nominatim
1
1
Fork 0
mirror of https://github.com/osm-search/Nominatim.git synced 2024-12-26 06:22:13 +03:00
Code Issues Packages Projects Releases Wiki Activity
2,611 Commits 31 Branches 41 Tags 164 MiB
7889509856
Commit Graph

356 Commits

Author SHA1 Message Date
Sarah Hoffmann
19edbd3581 send address interpolation import to correct postgres conn 2017-04-18 22:11:58 +02:00
Christian von Roques
b6be33cded explicitly name columns at INSERT 2017-04-18 13:36:02 +02:00
Christian von Roques
c80637b05c deduplicate words: do not look for duplicates in empty partitions 2017-04-15 19:12:25 +02:00
Christian von Roques
3ec67b9193 update: getPostgresVersion requires a database connection 2017-04-15 19:12:25 +02:00
Christian von Roques
40debbc7b5 with PHP7 the result of new() can no longer be assigned by reference 2017-04-15 19:12:25 +02:00
Christian von Roques
ccecabfbf4 correct warm.php --search-only help text 2017-04-15 19:12:25 +02:00
Sarah Hoffmann
c8e79397f5 introduce address column for place tables 
The column is a hstore containing the full OSM tag with key and
value.
 2017-04-08 18:58:26 +02:00
Sarah Hoffmann
c48fb88e6b move interpolaton computation into indexing step 
The interpolaton computation needs information from the osm2pgsql
slim tables which may not be available when the data is inserted.

Insertion now only adds a line with basic address information to
location_property_osmline. The line is then split during the
indexing, leading to more lines (which are complete in that case)
being inserted.

Fixes #598.
 2017-02-26 16:46:41 +01:00
Sarah Hoffmann
7273b58bbe make path to wikipedia files configurable 2016-12-28 23:38:48 +01:00
Sarah Hoffmann
a61f852cd2 fix parameter list for renamed postgis functions 
fixes #560
 2016-10-24 20:39:49 +02:00
Sarah Hoffmann
bb1c3f23ab reduce maximum line length to 199 for all php files 2016-10-14 00:01:16 +02:00
Sarah Hoffmann
39580057fc better error message for missing osm2pgsql 2016-10-09 21:04:53 +02:00
Sarah Hoffmann
a855ffe58e replace deprecated postgis functions 
ST_Line_Locate_Point and ST_Distance_Spheroid have changed
spelling in newer version and throw a deprecation warning.
 2016-10-09 20:09:59 +02:00
Sarah Hoffmann
1982978f74 move country search term creation into setup script 
Search results can become odd without the country search
terms, so make their inclusion a mandatory part of the
setup.

Also adds a new configuration variable to restrict the
languages taken into account by Nominatim.
 2016-09-28 23:30:44 +02:00
Marc Tobias Metten
6238ae6032 add namespaces, method visibility according to PSR2 standard 2016-09-16 02:27:36 +02:00
Marc Tobias Metten
93c26a0e65 half-reverting 51155b63, We dont need two of three break statements after all 2016-09-15 19:36:32 +02:00
Marc Tobias Metten
4a321487f3 fix indentation and misc errors according to PSR2 coding style guide 2016-09-14 03:16:46 +02:00
Marc Tobias Metten
51155b639b case statement need : instead of ;. Added more breaks 2016-09-14 03:00:57 +02:00
Marc Tobias Metten
2a784fa3d4 fix function declaration errors according to PSR2 coding style guide 2016-09-11 05:22:51 +02:00
marc tobias
3887423381 fix array-related errors according to PSR2 coding style guide 2016-09-10 21:10:52 +02:00
marc tobias
f05ea577f4 bracket spacing for if/else/for/foreach/while/switch according to PSR2 standard 2016-09-08 02:16:22 +01:00
Marc Tobias Metten
832547f192 tabs-to-spaces 2016-09-04 03:19:48 +02:00
Sarah Hoffmann
5464dec057 factor out parameter parsing into its own class 
Allows to use the parsing functions in website/search.php and
utils/query.php as well.
 2016-08-20 13:43:50 +02:00
Sarah Hoffmann
d594bb238b cleanup interface of PlaceLookup 
Move lookup parameters (place id, type, frac etc.) from members
to parameters for lookup().
 2016-07-26 20:54:14 +02:00
Sarah Hoffmann
1b33999f10 refactor ReverseGeocode 
- remove unused functions and members
- move coordinates from being members to being parameters of lookup()
 2016-07-26 20:54:14 +02:00
IrlJidel
ca92442601 pgsqlRunPartitionScript needs global $aCMDResult 
function pgsqlRunPartitionScript  was missing global $aCMDResult

setup was failing with

...
PHP Notice:  Undefined variable: aCMDResult in /home/roles/nominatim/app/Nominatim/build/utils/setup.php on line 768
NOTICE:  type "nearplace" does not exist, skipping
..
ERROR:  relation "location_area_large_0" already exists
ERROR: pgsql returned with error code (3)
pgsql returned with error code (3)
Command exited with non-zero status 255
 2016-07-14 19:49:58 +01:00
Sarah Hoffmann
cf4a44aaf7 introduce chksql function for phpscripts 
Checks SQL query results for errors and bails out if it finds one.
Avoids some heavy code duplication all over the place.
 2016-06-09 20:37:51 +02:00
Sarah Hoffmann
85c72cdccd remove --create-website function 
Symlinking does not go well with the separate build directory.
The recommended method is now to serve the website directly from
build/website via an alias. 'website installation' may be
reintroduced later via cmake.
 2016-06-08 23:41:36 +02:00
Sarah Hoffmann
2156fb1ad7 remove reference assignment where unnecessary 
Avoids a number of warnings on PHP7.
 2016-06-04 23:46:19 +02:00
Sarah Hoffmann
0fcab79486 fix syntax in load-data function 
Also makes sure that load data runs with at least 2 threads: one
for loading placex and one for loading osmline.
 2016-05-15 09:41:11 +02:00
Sarah Hoffmann
234b1b0293 Merge remote-tracking branch 'markus/master' into master 2016-05-14 21:09:57 +02:00
markigail
f07d620ee8 Change load-data in setup.php. 2016-05-11 10:22:03 +02:00
Marc Tobias Metten
7b4806b2e8 update.php was checking for static file which is no longer part of the project 2016-04-26 20:45:48 +02:00
Markus Gail
89c556d163 Coding style adaptions and correcting small errors from merge. 2016-04-25 13:58:28 +02:00
Markus Gail
b03be15442 Merge remote-tracking branch 'origin/master' into osmline 
Conflicts:
	lib/Geocode.php
	lib/PlaceLookup.php
	sql/functions.sql
	sql/tables.sql
	utils/setup.php
 2016-04-25 11:01:04 +02:00
Markus Gail
7879ad44cd Remove interpolation lines from placex and save them in an extra table. 2016-04-25 09:44:01 +02:00
Sarah Hoffmann
131527bdab Merge pull request #436 from lonvia/remove-location-property-tables 
remove unused location_property_-partion- tables
 2016-04-24 16:05:10 +02:00
Sarah Hoffmann
a244c41ea7 create_sql_functions() needs the commandline parameters 
Fixes one failing test.
 2016-04-23 11:07:12 +02:00
Sarah Hoffmann
a32f6c66b8 remove unused location_property_-partion- tables 2016-04-23 10:07:57 +02:00
Sarah Hoffmann
b4f0b8b728 apply marker filtering to functions.sql also after table creation 
fixes #430
 2016-04-21 00:00:15 +02:00
Sarah Hoffmann
a86563cfb1 make tables for external data (Tiger and aux) configurable 
Introduces two new settings CONST_Use_US_Tiger_Data and
CONST_Use_Aux_Location_data, which are disabled by default.
When false the corresponding tables are ignored in queries
and updates.

Aux and tiger tables are no longer created by default. This
has to be done by the corresponding import scripts. The former
aux table creation can be found in sql/aux_tables.sql for
reference.
 2016-04-07 21:47:51 +02:00
Sarah Hoffmann
4f9f21f661 remove state and county data for US and make postcode import optional 2016-03-30 23:21:13 +02:00
Sarah Hoffmann
02bd322c1d remove explicitly set postgres/postgis version 
Get the version from the database where necessary or simply
probe for existence of features. Fake hstore_to_json when
necessary.

Bumps the minimum required versions fro postgres to 9.1 and
for postgis to 2.0.
 2016-03-30 22:48:18 +02:00
Sarah Hoffmann
a769fc4ff3 remove unused/unmaintained options in import scripts 2016-03-28 22:20:31 +02:00
Markus Gail
bc449e10be change tiger housenumber format to save as lines instead of points and thus new interpolation 2016-03-21 14:07:12 +01:00
Sarah Hoffmann
4ea0074548 fix path for website installations 2016-03-16 20:02:09 +01:00
Sarah Hoffmann
4b46c73088 fix nominatim paths 2016-03-16 19:56:10 +01:00
Sarah Hoffmann
5ccb499a6f install executables and call settings.php explicitly 
Executables (including websites) need to be installed in the
build directory, so that they can find the right settings.php.
settings now defines build and source dir.
 2016-03-12 09:40:43 +01:00
Sarah Hoffmann
5d7bd37ab2 introduce InstallPath, so that stuff can be executed from build dir 2016-03-12 09:40:43 +01:00
Kelvin Wong
e0e812d09b Reverted change on FIPS 35013 to remain ASCII 2016-01-04 23:10:01 +08:00
Kelvin Wong
7e8e952b66 Additional changes to FIPS code 
See http://www2.census.gov/geo/docs/reference/codes/files/national_county.txt
 2016-01-04 16:18:33 +08:00
Kelvin Wong
32c8dabefe Updated FIPS map in utils/tigerAddressImport.py 
See https://www.census.gov/geo/reference/county-changes.html
 2016-01-04 15:34:00 +08:00
calimeroteknik
fb7d65b8c3 utils/update.php: use fail() whenever possible for errors 2015-12-07 14:08:02 +01:00
calimeroteknik
dbb1f5c0e2 utils/update.php: make the process exit with an non-zero error code if the update fails 2015-12-07 13:54:05 +01:00
Brian Quinion
347f8e57c7 Allow for postgis being in the default template - not an error if it already is installed 2015-10-07 14:44:59 +01:00
Sarah Hoffmann
f24b904f9a don't use isset for bool options in setup.php 2015-09-27 11:09:55 +02:00
Sarah Hoffmann
0df8a4e24f add more tables and an experimental warning to new --drop function 2015-09-27 11:00:45 +02:00
Frederik Ramm
b5c9845448 Add --drop option to setup.php 2015-09-26 01:41:19 +02:00
Sarah Hoffmann
fe3eb40e3a disable multithreading in osm2pgsql 
not really useful for gazetteer plugin and producing too much overhead
 2015-08-29 13:14:11 +02:00
Sarah Hoffmann
5581bb67f8 respect --osm2pgsql-cache setting 
fixes #260
 2015-08-08 18:15:14 +02:00
Marc Tobias Metten
7c8c206818 new method /lookup.php - Reverse search by multiple osm ids 2015-06-16 21:11:21 +02:00
Sarah Hoffmann
bf77c72195 amke Tiger import script work with python3 2015-05-08 20:04:22 +02:00
Sarah Hoffmann
3257c04d94 make Tiger data path configurable 2015-05-08 19:48:27 +02:00
Marc Tobias Metten
3b809c40f1 update Tiger import script to use 2014 data 2015-05-04 01:13:20 +02:00
Marc Tobias Metten
3134fb8b40 update Tiger import script to use 2014 data 2015-05-04 01:07:54 +02:00
Sarah Hoffmann
48e0b318f2 remove create-road function, no longer used 
The road tables are filled during indexing. Fixes #21.
 2015-05-03 19:56:08 +02:00
Sarah Hoffmann
0930f6e728 state setup: get latest node from place, planet_osm_nodes may be empty 2015-03-09 21:33:02 +01:00
Sarah Hoffmann
9e3e7bf791 switch remaining GRANTs to configurable web user 
fixes #252
 2015-03-05 21:55:20 +01:00
Sarah Hoffmann
f156da6b63 support tablespaces and www user config in tiger import 2015-01-11 13:44:39 +01:00
Sarah Hoffmann
3db64e7913 support tablespaces for special phrase table 2014-12-28 23:10:10 +01:00
Sarah Hoffmann
c391479c58 reorganise table creation 
- remove unused tables
- make apache user configurable
- introduce configurable tablespaces
 2014-12-02 00:05:21 +01:00
Sarah Hoffmann
04a20a2a0a remove debug output 2014-11-28 22:16:56 +01:00
Sarah Hoffmann
6382b1c111 more tolerant regexp for parsing replication state directories 
Apache 2.4 has changed the date format, so that the current regexp
doesn't match anymore, so be more tolerant with the date format.
Also force less fancy output formatting without tables.
 2014-11-27 22:23:04 +01:00
Sarah Hoffmann
8f09b9378f only check for postgres major and minor version 
fixes #192
 2014-10-29 21:04:28 +01:00
Sarah Hoffmann
2d6f00945a rewrite address interpolation so that planet_osm_node is not needed 
This allows address interpolations to work correctly when flatnode storage
is used for node coordinates.

To fix interpolations in an existing database, follow these steps:

  * invalidate all interpolations (in psql):
    `UPDATE placex SET indexed_status=2 WHERE rank_search = 28`
  * disable updates:
    ./utils/setup.php --create-functions --create-partition-functions
  * reindex the whole lot:
    ./utils/update.php --index --index-instances <number of your cpus>
  * enable updates again:
    ./utils/setup.php --create-functions --enable-diff-updates --create-partition-functions
 2014-10-05 22:16:02 +02:00
Sarah Hoffmann
a1a136778d add finished message to setup.php 2014-09-29 19:31:37 +02:00
Brian Quinion
1d248ac86d Make deduplicate work again. Only currently support 9.3 due to new postgresql functions added. 2014-06-30 17:49:24 +01:00
Sarah Hoffmann
6a5b5b3c36 revert error checking on pgsql restore 
wikipedia dumps create expected errors on restore, fixes #142
 2014-06-05 22:31:59 +02:00
Sarah Hoffmann
71883fc341 add setup option to ignore SQL errors 
needed when restoring partial backups
 2014-05-13 23:23:31 +02:00
Sarah Hoffmann
08b6fffc36 break out of write loop when psql fails 
fixes hanging in --create-functions
 2014-05-13 23:17:05 +02:00
Sarah Hoffmann
090e9e7fb6 make errors from psql in setup fatal 2014-04-13 16:09:27 +02:00
Sarah Hoffmann
e42ff1935f forgot the actual query script 2014-03-03 21:48:17 +01:00
Sarah Hoffmann
cf35f19318 split gb_postcode in table creation and data loading 
Data for gb_postcode now needs to be downloaded first.
Also fixes table creation for postgis 2.0.
 2014-02-21 23:07:41 +01:00
Sarah Hoffmann
98b93df83d recreate place objects when admin_level changes 
Also added a sanity check to ensure that accidental removal of admin_level
tags on large areas doesn't cause huge reindexing load. That can be disabled
by setting CONST_Limit_Reindexing to false.
 2014-02-07 23:37:54 +01:00
Sarah Hoffmann
e0b1e3b66d replace ST_Line_Interpolate_Point with ST_LineInterpolatePoint (DB adaption needed) 
Name of function was changed in postgis 2.1 and now prints ugly
deprecation warnings. For older versions of postgis, function
will be renamed to the new name during the setup of the DB.

To update existing databases with postgis < 2.1 run:

ALTER FUNCTION st_line_interpolate_point(geometry, double precision) RENAME TO ST_LineInterpolatePoint

and then reinstall the SQL functions:

./utils/setup.php --create-functions --enable-diff-updates --create-partition-functions
 2014-02-07 20:10:14 +01:00
Brian Quinion
edc5733715 Script to automatically warm the postgresql query before sending traffic 2014-02-06 16:50:41 +00:00
Kurt Roeckx
45c1e7582f Connect to the correct postgresql port from the settings. 2013-11-13 22:24:10 +01:00
Sarah Hoffmann
8675a2607a use postgis as an extension for versions >= 2.0 2013-10-10 23:13:58 +02:00
IrlJidel
dbd7f8fdae dont include polling time for batch duration for non-minutely updates, log name of each update step 2013-10-02 10:53:21 +01:00
IrlJidel
1c86275bad output date of steps during import-osmosis 2013-09-30 13:44:50 +01:00
IrlJidel
84d1e521aa show import_osmosis_log insert statements, print completed log after import_status inserts 2013-09-30 13:25:30 +01:00
IrlJidel
7471b09ed3 set start time for indexing step 2013-09-30 13:20:24 +01:00
Philippe
1c7afef1f2 added FIPS codes for 60xxx, 66xxx, 69xxx, 72xxx and 78xxx 2013-09-27 17:59:42 -04:00
Sarah Hoffmann
bf182ba2dc make import-osmosis(-all) fail with error code!=0 2013-09-25 23:05:47 +02:00
Sarah Hoffmann
8332fedfd6 make sleep interval dependent on date of last batch 2013-09-19 22:56:55 +02:00
Brian Quinion
65ea0efe68 Remove stray exit from debug 2013-09-19 15:19:31 +01:00
Brian Quinion
120d866e3e Sanity check on download.geofabrik.de update interval 2013-09-19 12:22:36 +01:00
Sarah Hoffmann
77b9edd1e9 allow osm2pgsql cache memory to be set for updates 2013-09-15 23:10:26 +02:00
Sarah Hoffmann
80b4121aca allow setup without partitions 2013-09-15 22:45:47 +02:00
Sarah Hoffmann
8fd7d319b7 Merge branch 'osmosis_error_return' of https://github.com/woodpeck/Nominatim 2013-09-09 13:46:59 +02:00
Sarah Hoffmann
12eb91b2bc Merge branch 'master' of https://github.com/spin0us/Nominatim 2013-09-09 13:45:41 +02:00
Brian Quinion
d8f9b27c93 make missing osmosis non-fatal for 'all' option - just skip it 2013-09-06 10:58:48 +01:00
Brian Quinion
62b5849528 script to compare to nominatim instances 2013-09-06 09:47:46 +01:00
Frederik Ramm
b8d4371395 re-run osmosis also on return codes other than 1 2013-08-30 10:42:21 +02:00
Spin0us
85f10f48fd Update update.php 2013-07-31 07:48:29 +02:00
Brian Quinion
5e6fdc467c Check to see if we can access the website at the configured CONST_Website_BaseURL 2013-06-13 18:07:19 +01:00
Frederik Ramm
87519de8ab fix indentation 2013-05-16 09:11:19 +02:00
Frederik Ramm
1f2a262ee0 Add osm2pgsql flat node support via settings.php 2013-05-16 09:03:16 +02:00
Sarah Hoffmann
ddc46acd26 add support for house numbers without street 2013-04-27 00:57:18 +02:00
Sarah Hoffmann
913341ee86 remove worldboundaries.sql 
fixes #22
 2013-04-20 10:27:56 +02:00
Sarah Hoffmann
01bcb59e44 update Tiger extract scripts for 2012 edition 2013-04-12 20:07:03 +02:00
Sarah Hoffmann
87ad4fb82f make osmosis-init also work for older planets 
Use planet_osm_nodes instead of places to get the very last node even if it is
untagged. Then ask for version 1 to avoid being mislead if the node has been
changed in the meantime.
 2013-04-07 15:34:29 +02:00
Brian Quinion
6ec5a40c82 better osmosis error message 2013-04-03 17:03:39 +01:00
IrlJidel
3eea62363d fix typo for baseUrl when configuring configuration.txt 
use osmosis --read-replication-lag to determine if there are changes before trying to process updates, useful when we are tracking hourly or daily replication updates

set CONST_Replication_Recheck_Interval to 60

skip lag check if CONST_Replication_Update_Interval > 60, for minutelies there's always new diffs to process

use tabs for indent

change sleep for non-minutely updates so that we dont drift tto much or poll excessively

unset $aReplicationLag before each exec attempt

unset $aReplicationLag inside while loop
 2013-04-02 21:30:12 +01:00
IrlJidel
7cb27dc881 make planet replication url configurable so we can use geofabrik's awesome daily country diffs 2013-03-22 12:44:21 +00:00
Brian Quinion
a8608e2b79 calculate the importance value 2013-03-19 19:44:33 +00:00
Brian Quinion
5ae905b20a download dbpedia attempt at geocoords 2013-03-19 19:29:39 +00:00
Brian Quinion
0070817ba5 first draft of wikipedia article/importance import script 2013-03-18 23:57:10 +01:00
Sarah Hoffmann
ea2cf36b12 switch to main overpass server 2013-02-26 20:42:57 +01:00
Brian Quinion
fca2d00617 Basic status page for monitoring nominatim is working 2013-01-24 12:42:42 +00:00
Sarah Hoffmann
87b250e739 split partition.sql into table and function parts 
Makes it easier to patch partition functions on an already existing
database.
 2013-01-19 17:23:21 +01:00
Sarah Hoffmann
20a852b554 support custom psql port in setup.php 
patch by b3nn0 (see http://trac.openstreetmap.org/ticket/4736)
 2013-01-14 22:48:52 +01:00
Brian Quinion
4a3121b52f release in postgis version string is optional 2012-12-18 17:39:01 +00:00
Brian Quinion
96d3cb61c8 add constant to define postgis version. Verify definted version constants are correct during database creation. 2012-12-18 13:09:50 +00:00
Sarah Hoffmann
e9535b3607 check return of DB creation command 2012-12-15 11:31:25 +01:00
Brian Quinion
01ea772255 restrict to one sleep per bucket. A second sleep while already sleeping results in a block. 2012-12-10 01:36:33 +00:00
Brian Quinion
661338ff25 Use Etc/UTC timezone for updates for reliability 2012-12-08 21:42:12 +00:00
Brian Quinion
274f381052 prune list of blocked items if too large, different costs for different page types 2012-12-08 21:39:24 +00:00
Brian Quinion
539d187b5c tiny formating change to utils/blocks.php 2012-12-08 18:56:59 +00:00
Brian Quinion
fa90ece530 add an option for max number of sleeping threads 2012-12-08 18:55:17 +00:00
Brian Quinion
e03ae6d7eb rate limiting using memcache 2012-12-08 17:38:10 +00:00
Brian Quinion
cc24d998e4 move deletable and polygons into default website 2012-12-03 14:26:29 +00:00
Brian Quinion
4f49ef07e6 Merge branch 'master' of github.com:twain47/Nominatim 2012-11-28 14:50:02 +00:00
IrlJidel
74f5929c2a Add css dir to create-website 2012-11-19 16:00:19 +00:00
Brian Quinion
84a5f2779a script to pull language list from openstreetmap wiki 2012-11-14 23:50:52 +00:00
Brian Quinion
678dcbd82a hstore in 9.2 replaces k=>v with hstore(k,v). Update sql to reflect change and introduce compatability function for pre 9.1 postgresql 2012-11-01 01:47:30 +00:00
Brian Quinion
6db3b2328d don't exit during osmosis setup 2012-10-29 14:23:05 +00:00
Brian Quinion
7dbbc43a85 Automatically calculate the state file for replication from the last node in the db 2012-10-22 03:09:38 +01:00
Sarah Hoffmann
c691217901 typo 2012-10-09 22:20:05 +02:00
Sarah Hoffmann
b0242c7901 use calculated_country_code for postcodes 2012-10-09 22:05:37 +02:00
Sarah Hoffmann
8fc2f9551b delete more hardcoded rules for special phrases 
both cases are now sufficiently covered by settings/phrase_settings.php
 2012-09-30 11:46:00 +02:00
Sarah Hoffmann
e73e67001e ignore frequent partial search terms 
Adds word counts from a full planet to the word table. There is a
new configuration option CONST_Max_Word_Frequency which allows to
take into account the word count: the value that was set on import
is used to determine if a word is added to the search_name table.
The value during runtime determines if a single term should be
used for partial search or simply be ignored.
 2012-09-25 00:36:34 +02:00
Sarah Hoffmann
c37a145dc6 patch replication path on --osmosis-init 
planet server layout has changed. Patch in the new path to
minute replications after having osmosis initialise configuration.txt.
 2012-09-23 22:03:28 +02:00
sprengm
7ff1395b96 make setup.php fail early on missing osm-file 
make setup.php fail early if:
  - no osm-file argument was supplied for --all or --import-data
  - the supplied osm-file is not existing or not readable (file permissions)
 2012-08-25 10:00:46 +02:00
Sarah Hoffmann
55b59ea139 import TIGER data into temporary table 
Import TIGER data into a temporary table first that later replaces
the current location_property_tiger table. This way index creation
on the table can be delayed until after the import which should
speed up the import and result in significantly smaller indexes.

Also removed index on parent_place_id as it is covered by
idx_location_property_tiger_housenumber_parent_place_id.
 2012-08-18 10:30:42 +02:00
Sarah Hoffmann
b11718699e fail on error 
- stop indexing if there is any other error than deadlock from the DB
- stop import if osm2pgsql or indexing fails
 2012-07-31 00:31:38 +02:00
Daniele Forsi
38e40cd014 Fix more typos 2012-07-10 15:47:21 +02:00
Sarah Hoffmann
43ce82d08e silently remove leading/trailing spaces 2012-05-30 22:19:49 +02:00
Brian Quinion
c325e1b039 Merge branch 'master' of github.com:twain47/Nominatim 2012-05-24 01:43:23 +02:00
Brian Quinion
a32fe96e16 truncate road partition tables during load 2012-05-24 01:42:59 +02:00
Sarah Hoffmann
35fdb7af0f more verbose error messages 2012-05-24 00:26:16 +02:00
Sarah Hoffmann
410ce603f7 use Overpass API by default to download OSM objects 2012-05-23 23:32:12 +02:00
Sarah Hoffmann
e54bd028ee make index work and remove more hard-coded DB names 2012-05-23 23:32:12 +02:00
Datendelphin
8be76da629 eliminate php warning, remove setting aPipes (isn't used) 2012-05-23 23:32:12 +02:00
Datendelphin
1461dc603e make .osc files with osmosis 2012-05-23 23:32:11 +02:00
Brian Quinion
4be160bd4b revert to plain text dump for 8.4 compatibility (+ permissions fix) 2012-05-23 19:25:43 +02:00
Brian Quinion
f1e381fdd4 fix broken field name / committed debug 2012-05-22 20:21:58 +02:00
Brian Quinion
d74cf03492 use the new country data files 2012-05-22 19:48:35 +02:00
Brian Quinion
8d221e2c82 DB Scheme changes: alter table placex add column calculated_country_code varchar(2); 
function changes:
-----------------
Move to ST_PointOnSurface from ST_Centroid in various places to avoid looking up a point outside the polygon
Move to ST_Covers from ST_Contains to include points on admin boundaries
Re-order preference for get_country_code now our data is better. country_osm_grid is now the preffered source.
Fix code to calculate country code in placex_insert, rank_search test was too early
Add extra field to placex 'calculated_country_code' to improve structure of code
Move split_geometery function out of add_location into its own function
Rewrite split_geometery to be more efficient.
Change place_insert to do more updates and less delete/inserts (delete is slow)
Include wikipedia links in details.php ouput
Cleanup no longer used geometry validation (adding overhead)
Include debug statements in function.sql (--DEBUG: ) and add flag to setup.php to turn them on

setup.php:
----------
add flag --disable-token-precalc to speed up debuging
add flag --index-noanalyse to disable analysising DB at rank 4 and 26 (previously removed, but on my local DB it seems to be required)
add flag --enable-diff-updates (modifier to --create-functions) to turn on the code required for diff updates without having to modify functions.sql
add flag --enable-debug-statements (modifier to --create-functions) to turn on debug warning statements

update.php:
-----------
added flag --no-index to import osmosis changes without indexing them
extend the hack to allow import of JOSM generated osm files

country_grid.sql - reference copy of the sql used to generate the country_osm_grid table, needs cleanup
 2012-05-22 16:27:42 +02:00
Brian Quinion
f17c5345ec Merge branch 'master' of github.com:twain47/Nominatim 2012-05-15 03:06:12 +02:00
Sarah Hoffmann
fd448170c5 remove intermediate ANALYSE while indexing 
Autovacuum does a much better job.
 2012-05-13 10:50:08 +02:00
Sarah Hoffmann
c32fa2172d pre-create word list to avoid duplicates due to parallel indexing 2012-05-13 09:15:56 +02:00
Brian Quinion
0ab9c90708 Can't filter on rank_search before it has been assigned 2012-05-11 19:23:44 +02:00
Brian Quinion
7164fd310d Merge branch 'master' of github.com:twain47/Nominatim 2012-04-30 01:05:13 +01:00
Brian Quinion
1c0fa81d88 calculate importance from wikipedia 2012-04-30 01:05:06 +01:00
mfn
fd3fcea539 Do not swallow PEAR error. 2012-04-26 13:30:49 +02:00
Brian Quinion
9b01ddd641 set osm2pgsql cache memory to a default based on memory in the machine 2012-04-26 01:35:06 +01:00
Sarah Hoffmann
653c05329d Format of replicate-sequence URL has changed 
Additionally update/init import_status in the DB. If this is not
initialized here, subsequent updates of the status won't work.
 2012-04-15 17:45:01 +02:00
Sarah Hoffmann
3f8fcd4055 add switch for osm2pgsql cache size 
also increased the default to 15GB to accaommodate growing planet
 2012-04-11 23:39:07 +02:00
Sarah Hoffmann
7ed4c28405 force UTF-8 database 2012-04-11 22:16:35 +02:00
Brian Quinion
80cf5df1cd Merge branch 'master' of github.com:twain47/Nominatim 2012-03-26 23:57:24 +01:00
Brian Quinion
b99a043c78 Use GB postcode table as definitive source. resort by pressence of search word in output address and other misc very minor changes not commited from MQ servers 2012-03-22 00:34:56 +00:00
Sarah Hoffmann
b6db7c17f9 countries need language list as well 2012-03-21 20:59:35 +01:00
Sarah Hoffmann
248b6ec90f Black/whitelisting for special phrases 
Allows to restrict the special phrases imported from the wiki.
Blacklist allows to exclude certain class/type combinations.
Whitelist allows to define an allowed subset of types for a class.
Adjust to your liking in settings/phrase_settings.php before running
the specialphrases script.
 2012-03-21 00:40:35 +01:00
Sarah Hoffmann
88b07b8fc2 creation of special phrases for country codes (forwardported from gazeteer) 2012-02-26 20:32:40 +00:00
Sarah Hoffmann
01f4723b7b update classtype tables for special phrases along with the places 2012-02-19 10:33:21 +00:00
Sarah Hoffmann
c186a4ead0 extend scripts for Tiger 2011 data 2012-02-17 20:34:28 +00:00
Sarah Hoffmann
5d10d538a5 hack around yes quotes in wiki and add a simple sanity check against wiki accidents 2012-02-13 22:37:03 +00:00
Sarah Hoffmann
8c7bd6beab during initial setup defer creation of non-vital until after nominatim indexing, 
analyse DB during nominatim indexing so the query planer uses indexes correctly
 2012-02-10 21:01:35 +00:00
Sarah Hoffmann
c5e69fadd0 improve import performance for special phrases 
- create a temporary index on placex
- create indices on place_classtype_* tables after insert
- grant www-data permission to tables
 2012-01-28 23:27:25 +00:00
Sarah Hoffmann
acadfcb2c5 extensions where introduced in postgresql in 9.1, not in 9.0 2012-01-21 12:09:43 +00:00
Sarah Hoffmann
cc564e2f31 - remove dependency on intarray extension 
- new config options for postgresql version and 
  location of osm2pgsql and osmosis binaries
- new option --no-npi for update.php that supresses writing of NPIs in
  osmosis update mode
- add more GRANTs for www-data
 2012-01-21 10:21:42 +00:00
Frederik Ramm
89a365787f bugfix nominatim 2011-09-09 16:00:34 +00:00
Brian Quinion
4d16677d2e update place_id to BIGINT 2011-06-14 13:42:46 +00:00
Brian Quinion
5f97080c73 retry loop on osmosis 2011-04-27 12:14:33 +00:00
Brian Quinion
1028c2444c move partiondef config file to settings, add --create-website option to build symlinks automatically 2011-04-06 18:19:40 +00:00
Brian Quinion
981649bf83 diff updates and npi diff updates 2011-03-18 12:36:17 +00:00
Frederik Ramm
b83af9c248 fix potential deadlock when writing a lot of data to a psql pipe and not reading what comes back; 
stdout/stderr weren't used anyway so just connect them through
 2011-02-21 00:02:17 +00:00
Frederik Ramm
a4c63c094f * use osm2pgsql from $PATH if none present locally 
* do not display usage() on successful postcode generation
 2011-02-18 09:18:37 +00:00
Brian Quinion
3ea1b35b32 add logging of broken polygons, improve address export, better diff output in export, bug in address calculation(#3522), reaply #3520, lost in migration 2011-02-07 11:13:18 +00:00
Brian Quinion
31273a42bd aux properties, memory leaks, tweaks to search order 2011-01-21 10:40:44 +00:00
Brian Quinion
2be00f37c1 finally fix adminitrative spelling error, disable fast_update on indexes by default (performance), fixes for incremental updates on new codebase 2011-01-05 14:07:26 +00:00
Brian Quinion
d95a434127 pull in the special phrases from the wiki 2010-12-13 15:55:52 +00:00
Brian Quinion
8193757427 tiger import code 2010-12-07 13:44:48 +00:00
Brian Quinion
631e8d09ab postcode/zipcode improvements, finish work on handling extratags 2010-12-07 13:41:02 +00:00
Brian Quinion
a6453a808d remove the road buffer code - made things slower. Plus some more tiger import code 2010-11-17 15:19:25 +00:00
Brian Quinion
1663dd49a4 more tiger import code 2010-11-16 13:24:52 +00:00
Brian Quinion
d4625e7741 rename to fix spelling mistake 2010-11-16 12:01:03 +00:00
Brian Quinion
1bcf91e336 add in missing county codes for TIGER 2009 2010-11-16 12:00:16 +00:00
Brian Quinion
7df80bbc6f tigger data import 2010-11-15 16:34:27 +00:00