Commit Graph

373 Commits

Author SHA1 Message Date
Sarah Hoffmann
56d24085f9 port multi-region update scripts to nominatim tool
Also updates the documentation. For the simple case of just
importing multiple regions, provide simplified instructions
that use the new multi-file import feature.

Fixes #2365.
2021-08-14 23:55:48 +02:00
Sarah Hoffmann
d81e152804 integrate analyse of indexing into nominatim tool 2021-02-08 22:22:49 +01:00
Sarah Hoffmann
4cb6dc01f3 port replication update function to python 2021-01-30 15:50:34 +01:00
Sarah Hoffmann
8f0885f6cb port check-for-update function to python 2021-01-28 14:50:14 +01:00
Sarah Hoffmann
30cd2f2280 remove API comparison util
This is outdated and unmaintained. There are tools out there
which can do this better. Try, for example
https://github.com/radarlabs/api-diff
2021-01-26 22:46:35 +01:00
Sarah Hoffmann
ff5a237200 move PHP utilities into the lib directory
These are not called directly as programs but used in a library
fashion by the installed utilities. So the library directory
is a better place.
2021-01-13 14:44:45 +01:00
marc tobias
d60f89867b utils/check_import_finished: skip some checks when setup ran with --drop 2020-12-21 20:12:31 +01:00
Sarah Hoffmann
433017b990 move creation of website scripts to setup script
Instead of creating the website wrapper scripts with cmake,
they are now created when --setup-website is called. The
setup of the configuration constants is directly embedded
into the scripts. This means we can get rid of the separate
settings-frontend.php. More importantly however, it means
that it is now possible to set up multiple website directories
from the same build directory.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
d97aed8741 adapt tests to new dotenv environment
DB tests now can simply set the environment to change configuration
variables. API tests still rely on a configuration file.

Also, query.php needs to set up the CONST_* variables to work with
the query scripts. That is a tiny bit messy and duplicates code
but this part will need to be reworked later.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
06d89e1d47 fix various typos 2020-12-19 14:33:04 +01:00
Sarah Hoffmann
992d3faac8 switch all utils to initialising dotenv 2020-12-19 14:33:04 +01:00
Sarah Hoffmann
0947b61808 switch remaining settings to dotenv format
CONST_Search_AreaPolygons and CONST_Search_ReversePlanForAll have
been removed completely.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
15a1666f8a replace database settings with dotenv variant
As we can't refer to the project root dir in the module path, the
module path may now also be a relative directory which is then
taken as being relative to the project root path.

Moves the checkModulePresence() function into the Setup class, so
that it can work on the computed absolute module path.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
25bdd7c6d9 introduce dotenv parsing for setup.php
This adds the notion of a project directory. This is the directory
that holds all necessary files for one specific installation of
Nominatim. Dotenv looks for an .env file in this directory and
adds it to the global environment together with the defaults from
Nominatim's data directory.

Add's symfony's dotenv library as a new dependency.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
ac116980ac make HTTP proxy setup explicit
The setup relies on the project configuration which we want to
explicitly set up in later steps. Therefore proxy setup needs to
be done explicitly as well. There is the added bonus that the
setup is done only for the utils which try to call outside.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
b5480f6e36 reorganise path settings in config
CONST_BasePath is split into separate configuration variables
for binaries, libraries and data. These variables as well as
the installation path are now set in the executable directly and
no longer configurable via project settings.

This is the first step towards an installable software. The
executables should know per installation where to find their
necessary data to execute. Project configuration needs to be
restricted to settings that really concern the specific Nominatim
installation.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
17a8cc5e29 use /usr/bin/env for python script
Makes it easier to use the script with a virtualenv setup.
2020-12-19 14:33:04 +01:00
Sarah Hoffmann
0334915067 update osm2pgsql
Needs now an explicit switch to avoid propagating changes from
nodes to ways to relations.
2020-12-02 18:27:18 +01:00
Sarah Hoffmann
22800d7d59 Search housenumbers with unknown address parts by housenumber term
House numbers need special handling because they may appear after
the street term. That means we canot just use them as the main name
for searches where the address has its own search term entries.
Doing this right now, we are able to find '40, Main St, Town' but not
'Main St 40, Town'.

This switches to using the housenumber token as the name term instead.
House number tokens can get special handling when building the search
query that covers the case where they come after the street.

The main disadvantage is that this once more increases the numbers
of possible search interpretation of which we have already too many.

no penalty for housenumber searches
2020-11-25 11:36:10 +01:00
Sarah Hoffmann
7f55dcef3a use simpler recurse operator for overpass download
Also fixes a typo in the OSM link.
2020-10-31 21:44:28 +01:00
Sarah Hoffmann
ec5743bcc0 add script for detailed explaing of indexing trigger 2020-10-09 17:38:33 +02:00
Sarah Hoffmann
b969392f40 remove removed index from database check 2020-09-30 11:33:15 +02:00
Sarah Hoffmann
a4b30fc649 index admin boundaries before everything else
Avoids irregularities that might happen because the address
rank of a boundary is changed through linking.
2020-08-18 15:17:09 +02:00
Sarah Hoffmann
4b21cc1737 remove more traces of osmosis-init 2020-08-11 10:43:04 +02:00
Sarah Hoffmann
73566a9f15 remove unused functions from setup and update
Removes the defunct --osmosis-init and --no-api switches and the
unsupported (and unnecessary) deduplicate. Also removes
'experimental' from --setup-website as this is a required
function now.
2020-08-06 16:16:35 +02:00
Sarah Hoffmann
b78cd3f4c9 make indexing during updates less quiet
Adjust verbosity behaviour to that of indexing during setup.
2020-07-28 22:35:51 +02:00
Sarah Hoffmann
7837970303 remove connect() in update script
This is now implicit.
2020-07-26 12:27:52 +02:00
Sarah Hoffmann
8cd9550295 implicitly connect to database during setup
Make access to the DB object a function, so that the connection
can be opened implicitly when the object is accessed for the first
time. This way we no longer need to check beforehand if a specific
function of the setup needs DB access or not.

Also move the check for the module to the relevant sub step.
2020-07-26 11:56:00 +02:00
Sarah Hoffmann
354487d7f4
Merge pull request #1829 from krahulreddy/websiteSetup
Added setup-website option
2020-07-01 18:11:50 +02:00
Sarah Hoffmann
214f92c428 make phpcs happy 2020-06-28 18:24:29 +02:00
Sarah Hoffmann
ff1be13d0e disable JIT and parallel processing for osm2pgsql in updates
This is known to cause issues because of bad indexing
statistics.
2020-06-28 18:06:06 +02:00
K Rahul Reddy
37f0b51dff Updated setup.php 2020-06-27 10:45:53 +05:30
K Rahul Reddy
6c406124dd Added setup-website option 2020-06-27 10:45:51 +05:30
marc tobias
f56bac350b FAQ entry for PostgreSQL -invalid page in block- 2020-06-19 21:16:57 +02:00
Sarah Hoffmann
65ee7a8002
Merge pull request #1754 from mtmail/nominatim-db-tests-against-postgres
Nominatim::DB tests against separate postgresql database
2020-04-26 10:20:30 +02:00
Sarah Hoffmann
0b0349f746
Merge pull request #1752 from mtmail/new-oo-shell-class
new PHP Nominatim\Shell class to wrap shell escaping
2020-04-25 16:48:04 +02:00
Sarah Hoffmann
2740974a13
Merge pull request #1758 from krahulreddy/advanced-installations
Advanced installations
2020-04-22 09:59:44 +02:00
K Rahul Reddy
42c80893cb
Fix documentation links (#1760)
Update installation documentation link in VAGRANT.md, update.php
2020-04-19 00:42:24 +02:00
K Rahul Reddy
08e273c0c7 Added scripts for multiple country setup and updates 2020-04-17 21:50:59 +05:30
marc tobias
38c21de0ee Nominatim::DB tests against separate postgresql database 2020-04-13 18:01:37 +02:00
marc tobias
fc40939775 new PHP Nominatim\Shell class to wrap shell escaping 2020-04-12 03:50:40 +02:00
Sarah Hoffmann
cbddfcde5b add structured search to command-line query tool 2020-02-19 11:04:07 +01:00
Sarah Hoffmann
b9171dd10b clean up intermediate tables earlier with --drop
When --drop is given, we can remove all node geometry information
already after the import with osm2pgsql. Also drop all unnecessary
tables before creating the final indices.
2020-02-12 11:03:20 +01:00
marc tobias
76082ac7cb check_import_finished.php - reverse_only mode has less indices 2020-02-06 16:48:06 +01:00
Sarah Hoffmann
a8711ab013 fix verboseness of nominiatim script during updates 2020-01-31 18:18:50 +01:00
Sarah Hoffmann
2979c39628 also adapt indexing command in update script 2020-01-29 11:36:12 +01:00
Sarah Hoffmann
3ff6eccfd7 move trigger creation later in setup 2020-01-23 22:28:43 +01:00
Sarah Hoffmann
586ff0c364
Merge pull request #1638 from mtmail/check-for-invalid-indices
check_import_finished: check for invalid indices
2020-01-14 21:34:35 +01:00
Sarah Hoffmann
f3ba358d50
hint that invalid indices must be manually deleted 2020-01-14 21:33:09 +01:00
marc tobias
6b0afd5d9b check_import_finished: check for invalid indices 2020-01-14 19:36:40 +01:00