Commit Graph

2797 Commits

Author SHA1 Message Date
Sarah Hoffmann
b9517c99ae rename sql directory to lib-sql
Also introduces a separate constant for the sql directory, so that
it can be put separately from the rest of the data if required.
2021-02-09 15:26:56 +01:00
Sarah Hoffmann
db3ced17bb rename lib to lib-php 2021-02-09 11:52:07 +01:00
Sarah Hoffmann
248b4cddab update osm2pgsql (disable install rule) 2021-02-09 09:48:50 +01:00
Sarah Hoffmann
d81e152804 integrate analyse of indexing into nominatim tool 2021-02-08 22:22:49 +01:00
Sarah Hoffmann
0cbf98c020 consolidate warm and db-check into single admin command 2021-02-08 21:05:06 +01:00
Sarah Hoffmann
195f9f5ef3 split cli.py by subcommands
Reduces file size below 1000 lines.
2021-02-08 17:23:05 +01:00
Sarah Hoffmann
a759c5b75b move website into php library directory 2021-02-08 12:00:34 +01:00
Sarah Hoffmann
7dfe645b5f move postcode table setup to sql/
Also moves the call to the setup from the setup-db
step to the calculate-postcodes step. The tables also need
no longer be accessible by the webservice.
2021-02-08 11:53:01 +01:00
Sarah Hoffmann
ca3283cbaa remove unused SQL script 2021-02-08 11:28:24 +01:00
Sarah Hoffmann
861e67dfe8 fix off-by-one error in replication download 2021-02-04 17:04:04 +01:00
Sarah Hoffmann
82ef02cd1a
Merge pull request #2161 from lonvia/timeout-for-replication
Reintroduce timeout for replication file download
2021-02-04 16:52:24 +01:00
Sarah Hoffmann
948217d5e9 reintroduce timeout for replication file download
This ports the --socket-timeout parameter from
pyosmium-get-changes which ensures that the update
process eventually times out on hanging network connections.
2021-02-04 11:47:11 +01:00
Sarah Hoffmann
6cc06828db
Merge pull request #2160 from lonvia/introduce-project-dir
Officially introduce and recommend use of a project directory
2021-02-04 09:52:59 +01:00
Sarah Hoffmann
0b2abfb115 replace make serve with nominatim serve command
With the website directory now tied to the project directory instead
of the build directory, it is no longer possible to use make for
running the web server.
2021-02-03 16:34:31 +01:00
Sarah Hoffmann
b2f8fb6201 add migration info for status table 2021-02-03 14:13:09 +01:00
Sarah Hoffmann
e2329c03fe Revert "increase splitting for large geometries"
This reverts commit 559fe513fa.

Increasing the splitting results in geometries where with rounding
issues at the split points, so that contain operations do not
work as expected anymore.

Fixes #2137.
2021-02-03 10:23:38 +01:00
Sarah Hoffmann
9bca670b4e adapt quick start instructions in README to project dir 2021-02-03 10:17:22 +01:00
Sarah Hoffmann
cb06d1f4ca do not overwrite custom set module paths
Given that the module is now copied to the project directory
when no module path is set, we need the information that the
module path is empty. Therefore hand in the default module path
in a separate variable.
2021-02-02 18:31:25 +01:00
Sarah Hoffmann
36447c488a print project directory before running any command 2021-02-02 11:19:31 +01:00
Sarah Hoffmann
69092030cd make phpcs happy 2021-02-02 11:15:56 +01:00
Sarah Hoffmann
109aa9c428 actions: switch to using separate project dir
Also fixes reverse-only import which not run at all.
2021-02-02 11:03:09 +01:00
Sarah Hoffmann
1d97816c53 docs: add hint about putting the nominatim tool into the PATH 2021-02-02 10:56:40 +01:00
Sarah Hoffmann
7591c4fb42 copy database module on install
When no explicity database module is configured, then the
module is now copied into the project directory and used from
there. This means that Nominatim can be updated to a new
version of the module while existing installation keep their
version of normalisation.
2021-02-02 10:56:40 +01:00
Sarah Hoffmann
60cbeb165e hand in absolute path to nominatim tool to php scripts 2021-02-02 10:56:40 +01:00
Sarah Hoffmann
bddfc109f8 refer to new nominatim tool in configuration comments 2021-02-02 10:56:40 +01:00
Sarah Hoffmann
b05c379b39 change the default location for external data to project dir 2021-02-02 10:56:40 +01:00
Sarah Hoffmann
7ba5283fe8 actions: revert to reletive paths for caching 2021-02-02 10:37:18 +01:00
Sarah Hoffmann
98fe5af07d actions: remove setting custom .env
It only set the pyosmium-get-changes binary which is no longer
needed.
2021-02-02 10:35:30 +01:00
Sarah Hoffmann
59cb1d6c27 remove pyosmium-get-changes detection from cmake
pyosmium-get-changes is not longer used.
2021-02-02 10:33:15 +01:00
Sarah Hoffmann
0ad1b28497
Merge pull request #2155 from lonvia/port-regresh-to-python
Port replication and part of the refrsh function to native Python
2021-02-01 11:50:05 +01:00
Sarah Hoffmann
5f63d4ca1f print nice summary after updates 2021-02-01 10:34:31 +01:00
Sarah Hoffmann
90aaab77fc fix linting issues 2021-01-30 16:42:25 +01:00
Sarah Hoffmann
7158433cd3 disable warning about non-toplevel import
They are needed here so nominatim can be run when osmium
is not installed. Everything except replication will work fine.
2021-01-30 16:29:28 +01:00
Sarah Hoffmann
e629a175ed introduce custom UsageError
This is a exception to be thrown when the error occures because
of bad user data. We don't want to print a full stack trace in
these cases but just tell the user what went wrong.
2021-01-30 16:20:10 +01:00
Sarah Hoffmann
45ea73913f remove setting for PYOSMIUM_BINARY
pyosmium is now called as a library from the python code,
so that pyosmium-get-changes is no longer needed.
2021-01-30 15:55:04 +01:00
Sarah Hoffmann
01e0fd7e13 whitelist pyosmium for pylint 2021-01-30 15:52:49 +01:00
Sarah Hoffmann
4cb6dc01f3 port replication update function to python 2021-01-30 15:50:34 +01:00
Sarah Hoffmann
8f0885f6cb port check-for-update function to python 2021-01-28 14:50:14 +01:00
Sarah Hoffmann
beb0fa0727
Merge pull request #2153 from rizkyarlin/patch-1
fix indentation
2021-01-28 09:06:33 +01:00
Muh. Rizky Eka Arlin
436cb9229b
fix indentation 2021-01-28 14:21:54 +08:00
Sarah Hoffmann
d78f0ba804 port replication initialisation to Python 2021-01-26 22:50:54 +01:00
Sarah Hoffmann
5b46fcad8e convert functon creation to python
The new functions always creates normal and partitioned functions.
Also adds specialised connection and cursor classes for adding
frequently used helper functions.
2021-01-26 22:50:54 +01:00
Sarah Hoffmann
94fa7162be port address level computation to Python
Also adds simple tests for correct table creation.
2021-01-26 22:50:54 +01:00
Sarah Hoffmann
e6c2842b66 move update code for postcode and word count to Python
Adds also tests for the new function to execute a SQL script.
2021-01-26 22:50:54 +01:00
Sarah Hoffmann
e6d9485c4a cli: import python modules for commands on demand
Given that only one command will be executed in the end, it is
not necessary to import what amounts to the whole library. This
becomes in particular important for update functions that have
a dependency on pyosmium. The dependency can remain optional for
people not using updates.
2021-01-26 22:50:54 +01:00
Sarah Hoffmann
30cd2f2280 remove API comparison util
This is outdated and unmaintained. There are tools out there
which can do this better. Try, for example
https://github.com/radarlabs/api-diff
2021-01-26 22:46:35 +01:00
Sarah Hoffmann
2c909c1f0c
Merge pull request #2147 from lonvia/tests-for-python-code
Add basic set of tests for Python code
2021-01-21 10:07:50 +01:00
Sarah Hoffmann
063a4cb403 cli indexer tests need a fake database
The Indexer constructor opens a connection to the given database.
2021-01-20 21:30:27 +01:00
Sarah Hoffmann
42ec67f63c add more tests for CLI parameter parser 2021-01-20 21:30:27 +01:00
Sarah Hoffmann
8c02786820 add tests for indexer 2021-01-20 21:30:27 +01:00