Commit Graph

649 Commits

Author SHA1 Message Date
Sarah Hoffmann
36d068871d disable prepared statements
Prepared statements do not work well with the partial indexes that
Nominatim uses because all Python constants are replaced with
parameters. A query like:

  placex.select().where(placex.c.rank_address.between(4, 25)

gets translated into a prepared query with two parameters:

  SELECT * FROM placex WHERE rank_address BETWEEN %s and %s

And this does not work with a partial index of:

  CREATE INDEX on placex(geometry) WHERE rank_address between 4 and 25
2023-03-28 14:53:45 +02:00
Sarah Hoffmann
6c67a4b500 switch reverse CLI command to Python implementation 2023-03-26 18:09:33 +02:00
Sarah Hoffmann
86b43dc605 make sure PHP and Python reverse code does the same
The only allowable difference is precision of coordinates. Python uses
a precision of 7 digits where possible, which corresponds to the
precision of OSM data.

Also fixes some smaller bugs found by the BDD tests.
2023-03-26 16:21:43 +02:00
Sarah Hoffmann
300921a93e add server glue for reverse API call 2023-03-25 17:04:02 +01:00
Sarah Hoffmann
35b52c4656 add output formatters for ReverseResults
These formatters are written in a way that they can be reused for
search results later.
2023-03-25 15:45:03 +01:00
Sarah Hoffmann
878302a622 ignore NotImplementedErrors when compiling SQL 2023-03-24 11:16:02 +01:00
Sarah Hoffmann
55277738d4 factor out layer checks in reverse function 2023-03-24 10:08:01 +01:00
Sarah Hoffmann
2f54732500 python: implement reverse lookup function
The implementation follows for most part the PHP code but introduces an
additional layer parameter with which the kind of places to be returned
can be restricted. This replaces the hard-coded exclusion lists.
2023-03-23 22:38:37 +01:00
Sarah Hoffmann
41da298b18 add python implementation of reverse
This adds an additional layer parameter and slightly changes the
queries to do more efficient lookups for large area features.
2023-03-23 10:16:50 +01:00
Sarah Hoffmann
ebcf8c2b6b api: make details parameter optional 2023-03-23 10:16:50 +01:00
Sarah Hoffmann
1facfd019b api: generalize error handling
Return a consistent error response which takes into account the chosen
content type. Also adds tests for V1 server glue.
2023-03-23 10:16:50 +01:00
Sarah Hoffmann
00e3a752c9 split SearchResult type
Use adapted types for the different result types. This makes it
easier to have adapted output formatting and means there are only
result fields that are filled.
2023-03-23 10:16:50 +01:00
Sarah Hoffmann
2b7eb4906a bdd: add tests for valid debug output 2023-03-09 20:10:51 +01:00
Sarah Hoffmann
9a5f75dba7
Merge pull request #2993 from biswajit-k/delete-tags
Adds sanitizer for preventing certain tags to enter search index based on parameters
2023-03-09 14:31:45 +01:00
biswajit-k
ca149fb796 Adds sanitizer for preventing certain tags to enter search index based on parameters
fix: pylint error

added docs for delete tags sanitizer

fixed typos in docs and code comments

fix: python typechecking error

fixed rank address type

Revert "fixed typos in docs and code comments"

This reverts commit 6839eea755a87f557895f30524fb5c03dd983d60.

added default parameters and refactored code

added test for all parameters
2023-03-09 14:18:39 +05:30
biswajit-k
36388cafe9 fixed typos in docs and code comments 2023-03-06 17:09:38 +05:30
marc tobias
4be6970bd4 sp_wiki_loader: add more default languages 2023-02-18 13:54:48 +01:00
Sarah Hoffmann
8ed096f938 speed up reverse lookup of place nodes
Add a special index that contains the place nodes buffered by their
respective area according to their search rank. This replaces the
maximum area search for place nodes and reduces drastically the number
of place nodes that need to be retrieved.
2023-02-17 14:10:01 +01:00
Sarah Hoffmann
ee0c5e24bb add a WKB decoder for the Point class
This allows to return point geometries from the database and makes
the SQL a bit simpler.
2023-02-16 17:29:56 +01:00
Sarah Hoffmann
8557105c40 add debug output for unit tests
This uses the debug output facility meant for pretty HTML output
to give us debugging output for the unit tests.
2023-02-14 11:57:37 +01:00
Sarah Hoffmann
24e7ffb289 add HTML-formatted debug output to lookup 2023-02-14 11:57:37 +01:00
Sarah Hoffmann
42c3754dcd add tests for details result formatting and trim results
Values that are None are no longer included in the output to save
a bit of bandwidth.
2023-02-04 21:22:22 +01:00
Sarah Hoffmann
3ac70f7cc2 implement details endpoint in Python servers 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
104722a56a switch details cli command to new Python implementation 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
1924beeb20 add lookup of postcdoe data 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
70f6f9a711 add lookup of tiger data 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
f1ceefe9a6 add lookup of address interpolations 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
e1fc1566f3 fix new linting issues from pylint 2.16 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
189f74a40d add unit tests for lookup function 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
df65c10360 add lookup() call to the library API
Currently only looks places up in placex.
2023-02-04 21:22:22 +01:00
Sarah Hoffmann
16b6484c65 add property cache for API
This caches results from querying nominatim_properties.
2023-01-30 09:36:17 +01:00
Sarah Hoffmann
2156fd4909 adapt annotations for SQLAlchemy 2.x
It is not possible to produce type annotations that work with both
versions 1.4 and 2.0. So keep to the principle of only supporting
newest versions when it comes to mypy. This means that some types
may have to be string quoted to not cause issues when running with
SQLAlchemy 1.4.
2023-01-28 11:51:26 +01:00
Sarah Hoffmann
2e9090d121 adapt status to use SQLAlchemy tables 2023-01-28 11:51:26 +01:00
Sarah Hoffmann
23f2690c54 convert StatusResult to a dataclass
This gives us nice str() functions.
2023-01-28 11:51:26 +01:00
Sarah Hoffmann
5226cd2a0b add SQLAlchemy table schema for Nominatim tables
Also defines an extended connection object that includes access to
the table definitions. Makes it easier to access the tables from
code that has been split off into separate modules.
2023-01-28 11:51:26 +01:00
Sarah Hoffmann
77bec1261e add streaming json writer for JSON output 2023-01-25 15:05:33 +01:00
Sarah Hoffmann
f85b0c6208 allow to add php-compatible endpoints
If the new setting NOMINATIM_SERVE_LEGACY_URLS is set, the servers
expose the endpoints also with the .php suffix to ensure backwards
compatibility.
2023-01-24 21:39:19 +01:00
Sarah Hoffmann
e490a30a4a add support for CORS headers
Adds the additional dependency to sanic-cors for the Sanic server.
2023-01-24 21:39:19 +01:00
Sarah Hoffmann
654b652530 factor out common server implementation code
Most of the server implementation of V1 API now resides in
api.v1.server_glue. The webframeworks only supply some glue code
which is independent to changes in the API code.
2023-01-24 21:39:19 +01:00
Sarah Hoffmann
8f4426fbc8 reorganize code around result formatting
Code is now organized by api version. So formatting has moved to
the api.v1 module. Instead of holding a separate ResultFormatter
object per result format, simply move the functions to the
formater collector and hand in the requested format as a parameter.
Thus reorganized, the api.v1 module can export three simple functions
for result formatting which in turn makes the code that uses
the formatters much simpler.
2023-01-24 17:20:51 +01:00
Sarah Hoffmann
32c1e59622 reorganize api submodule
Use a directory for the submodule where the __init__ file contains
the public API. This makes it easier to separate public interface
from the internal implementation.
2023-01-24 13:28:04 +01:00
Sarah Hoffmann
e56957f047 api: delay setup of initial database connection
Defer database setup until the first call to a function. Needs an
additional lock because the setup still needs to be done sequentially.
2023-01-24 10:56:22 +01:00
Sarah Hoffmann
ce9ed993c8 fix importance recalculation
The signature of the compute_importance() function has changed.
2023-01-22 22:32:16 +01:00
Sarah Hoffmann
02645277c8 fix typo in argument to details CLI command
Fixes #2951.
2023-01-21 10:44:10 +01:00
Sarah Hoffmann
6c61690ef3 do not run osm2pgsql append with mutliple threads
As the updates modify the placex table, there may be deadlocks
when different objects want to forward modifications to the same
place (for example because they are both linked to it).
2023-01-05 11:34:56 +01:00
Sarah Hoffmann
38f467bae3 add event handler for default connection settings 2023-01-03 10:03:00 +01:00
Sarah Hoffmann
0c47558729 convert version to named tuple
Also return the new NominatimVersion rather than a string in the
status result.
2023-01-03 10:03:00 +01:00
Sarah Hoffmann
93b9288c30 fix error message for non-existing database 2023-01-03 10:03:00 +01:00
Sarah Hoffmann
9d31a67116 add unit tests for new Python API 2023-01-03 10:03:00 +01:00
Sarah Hoffmann
7219ee6532 extend BDD API tests to query via Python frameworks
A new config option ENGINE allows to choose between php and any of the
supported Python engines.
2023-01-03 10:03:00 +01:00