osm-search/Nominatim
1
1
Fork 0
mirror of https://github.com/osm-search/Nominatim.git synced 2024-11-29 08:36:24 +03:00
Code Issues Packages Projects Releases Wiki Activity
4,658 Commits 28 Branches 41 Tags 164 MiB
50beac8305
Commit Graph

877 Commits

Author SHA1 Message Date
Sarah Hoffmann
c284df2dc9 restrict range for interpolated housenumbers 
Interpolations are only supported up to 2^32 by the database.
Limit to 8 digits, which is still more than should be needed.
 2023-09-05 11:41:41 +02:00
Sarah Hoffmann
18b2a4c204 avoid interpreting integral numbers as coordinates 
There are addresses of type '123 W 345 N, Reigh', where 'W 345 N' is
the actual name of the street.
 2023-09-05 11:26:47 +02:00
Sarah Hoffmann
26dfb868e9 add test case for bad update 2023-08-25 14:22:49 +02:00
Sarah Hoffmann
a9edd57fe2 send charset again in content-type when returning json 
There are quite a few applications out there that will use some local
encoding when the charset is not explicitly given.
 2023-08-23 20:55:57 +02:00
Sarah Hoffmann
23eed4ff2f fix tag name for housename addresses in layer selection 
Fixes #3156.
 2023-08-19 15:57:33 +02:00
Sarah Hoffmann
fa3ac22a8f adapt tests to changes in search 2023-08-12 16:12:31 +02:00
Sarah Hoffmann
926c4a7d04 fix lookup polygon output 
Fixes #3147.
 2023-08-08 21:31:25 +02:00
Sarah Hoffmann
282c0da941 return an error when q is used together with structured parameters 2023-08-08 15:54:55 +02:00
Sarah Hoffmann
78648f1faf remove lookup by address only 
There are too many lookups where the address is very frequent,
even when many address parts are present.
 2023-08-06 21:00:10 +02:00
Sarah Hoffmann
996026e5ed provide full URL in more field 
This is a regression against the PHP version.

Fixes #3138.
 2023-08-06 17:50:02 +02:00
Sarah Hoffmann
afdbdb02a1 do not lookup by address vector when only few tokens are available 
Names of countries and states are exceedingly rare in the word count
but are very frequent in the address. A short name has the danger
of producing too many results.
 2023-08-02 09:25:47 +02:00
Sarah Hoffmann
252fe42612
Merge pull request #3122 from miku0/sanitizer-final 
Adds sanitizer for Japanese addresses to correspond to block address
 2023-08-01 10:38:58 +02:00
miku0
67e1c7dc72 Moved KANJI_MAP to icu-rules 2023-07-31 11:57:49 +00:00
miku0
4d61cc87cf Add the test of reconbine_place 2023-07-31 02:39:56 +00:00
Sarah Hoffmann
e523da9e12 reintroduce file logging for Python frontend 2023-07-30 19:58:00 +02:00
miku0
67706cec4e add @fail-legacy 2023-07-27 07:33:53 +00:00
Sarah Hoffmann
9448c5e16f add tests for new arm and export Python functions 2023-07-26 11:09:52 +02:00
miku0
0722495434 add japanese sanitizer 2023-07-26 07:54:58 +00:00
Sarah Hoffmann
d545c6d73c mostly remove php-cgi requirement 
This is now only needed for BDD tests against the php API.
 2023-07-26 00:10:11 +02:00
Sarah Hoffmann
f69fea4210 remove now unused run_api_script function 2023-07-25 22:45:29 +02:00
Sarah Hoffmann
4cd0a4ced4 remove now unused run_legacy_script() 2023-07-25 21:39:23 +02:00
Sarah Hoffmann
0804cc0cff port export function to Python 
Some of the parameters have been renoved as they don't make sense
anymore.
 2023-07-25 21:39:23 +02:00
Sarah Hoffmann
faeee7528f move warm script to python code 2023-07-25 21:39:23 +02:00
Sarah Hoffmann
66ecb56cea add tests for new endpoints 2023-07-25 10:57:19 +02:00
Sarah Hoffmann
927d2cc824 do not split names from typed phrases 
When phrases are typed, they should only contain exactly one term.
 2023-07-17 20:09:08 +02:00
Sarah Hoffmann
cc45930ef9 avoid lookup via partials on frequent words 
Drops expensive searches via partials on terms like 'rue de'.

See #2979.
 2023-07-06 12:16:57 +02:00
Sarah Hoffmann
ce17b0eeca
Merge pull request #3101 from lonvia/custom-geometry-type 
Improve use of SQLAlchemy statement cache with search queries
 2023-07-03 11:03:26 +02:00
Sarah Hoffmann
82216ebf8b always run function update on migrations 
This means that we can have migrations which require nothing but
an update of the functions.
 2023-07-01 20:18:59 +02:00
Sarah Hoffmann
9f6f12cfeb move search to bind parameters 2023-07-01 18:03:07 +02:00
Sarah Hoffmann
a873f260cf fix merging of linked names into unnamed boundaries 
The NULL value of the boundaries' name field was erasing all
content when used in SQL operations.
 2023-06-30 22:14:11 +02:00
Sarah Hoffmann
d7a3039c2a also switch legacy tokenizer to new street/place choice behaviour 2023-06-30 17:03:17 +02:00
Sarah Hoffmann
645ea5a057 use information from tokenizer to determine street vs. place address 
So far the SQL logic used the information from the address field
to determine if an address is attached to a street or place.
This changes the logic to use the information provided in the
token_info. This allows sanitizers to enforce a certain parenting
without changing the visible address information.
 2023-06-30 11:08:25 +02:00
Sarah Hoffmann
2755ebe883
Merge pull request #3094 from lonvia/fix-failing-bdd-tests 
Add BDD tests against Python frontend to CI
 2023-06-22 22:28:31 +02:00
Sarah Hoffmann
2d05ff0190 slightly adapt postcode tests 2023-06-22 16:51:59 +02:00
Sarah Hoffmann
0d338fa4c0 bdd: fix faking HTTP headers for python web frameworks 2023-06-22 14:00:33 +02:00
mtmail
15a66e7b7d
Merge branch 'osm-search:master' into check-database-on-frozen-database 2023-06-22 12:14:55 +02:00
Marc Tobias
2337cc653b check-database on frozen db shouldnt recommend indexing 2023-06-21 17:47:57 +02:00
Sarah Hoffmann
9bc5be837b remove useless check 
Found by new mypy version.
 2023-06-21 11:56:39 +02:00
Sarah Hoffmann
b79d5494f9 remove support for sanic framework 
There is no performance gain over falcon or starlette but the special
structure of sanic makes it hard to have exchangable code
 2023-06-21 10:53:57 +02:00
Sarah Hoffmann
36df56b093 fix header name for browser languages 2023-06-20 11:56:43 +02:00
Sarah Hoffmann
d0a1e8e311 tweak postcode search 
Give a preference to left-right reading, i.e <postcode>,<address>
prefers a postcode search while <address>,<postcode> rather does
an address search.

Also exclude non-addressables, countries and state from results when a
postcode is contained in the query.
 2023-06-20 11:56:43 +02:00
Sarah Hoffmann
1f83efa8f2
Merge pull request #3086 from lonvia/close-connection-on-replication 
Close database connections while waiting for the next update cycle
 2023-06-19 15:48:00 +02:00
Sarah Hoffmann
6f3339cc49 close DB connection when waiting for next update cycle 2023-06-19 12:02:51 +02:00
Sarah Hoffmann
771be0e056 do not fail php script generation when curly braces are present 
Fixes #3084.
 2023-06-19 11:23:30 +02:00
Sarah Hoffmann
41bf162306 remove tests for old PHP cli commands 2023-05-26 17:36:05 +02:00
Sarah Hoffmann
8f299838f7 fix various failing BDD tests 2023-05-26 15:08:48 +02:00
Sarah Hoffmann
146a0b29c0 add support for search by houenumber 2023-05-26 14:10:57 +02:00
Sarah Hoffmann
371a780ef4 add server fronting for search endpoint 
This also implements some of the quirks of free-text search of the
V1 API, in particular, search for categories and coordinates.
 2023-05-26 11:40:45 +02:00
Sarah Hoffmann
0608cf1476 switch CLI search command to python implementation 2023-05-24 22:54:54 +02:00
Sarah Hoffmann
f335e78d1e make localisation of results explicit 
Localisation was previously done as part of the formatting but might
also be useful on its own when working with the results directly.
 2023-05-24 18:12:34 +02:00
Sarah Hoffmann
dcfb228c9a add API functions for search functions 
Search is now split into three functions: for free-text search,
for structured search and for search by category. Note that the
free-text search does not have as many hidden features like
coordinate search. Use the search parameters for that.
 2023-05-24 18:05:43 +02:00
Sarah Hoffmann
dc99bbb0af implement actual database searches 2023-05-24 13:52:31 +02:00
Sarah Hoffmann
c42273a4db implement search builder 2023-05-23 11:23:44 +02:00
Sarah Hoffmann
3bf489cd7c implement token assignment 2023-05-22 15:49:03 +02:00
Sarah Hoffmann
d8240f9ee4 add query analyser for legacy tokenizer 2023-05-22 11:07:14 +02:00
Sarah Hoffmann
2448cf2a14 add factory for query analyzer 2023-05-22 09:23:19 +02:00
Sarah Hoffmann
004883bdb1 query analyzer for ICU tokenizer 2023-05-22 08:46:19 +02:00
Sarah Hoffmann
ff66595f7a add data structure for tokenized query 2023-05-21 09:30:57 +02:00
Sarah Hoffmann
d9d8b9c526 add tests for parameter converter 2023-05-18 18:09:07 +02:00
Sarah Hoffmann
bef5cea48e switch API parameters to keyword arguments 
This switches the input parameters for API calls to a generic
keyword argument catch-all which is then loaded into a dataclass
where the parameters are checked and forwarded to internal
function.

The dataclass gives more flexibility with the parameters and makes
it easier to reuse common parameters for the different API calls.
 2023-05-18 17:42:23 +02:00
Marc Tobias
e5f332bd71 when adding Tiger data, check first if database is in frozen state 2023-05-08 14:35:30 +02:00
Sarah Hoffmann
5751686fdc
Merge pull request #3006 from biswajit-k/generalize-filter 
generalize filter function for sanitizers
 2023-04-11 19:20:08 +02:00
Sarah Hoffmann
60c1301fca fix a number of corner cases with interpolation splitting 
Snapping a line to a point before splitting was meant to ensure
that the split point is really on the line. However, ST_Snap() does
not always behave well for this case. It may shorten the interpolation
line in some cases with the result that two points housenumbers
suddenly fall on the same point. It might also shorten the line down
to a single point which then makes ST_Split() crash.

Switch to a combination of ST_LineLocatePoint and ST_LineSubString
instead, which guarantees to keep the original geometry. Explicitly
handle the corner cases, where the split point falls on the beginning
or end of the line.
 2023-04-06 16:54:00 +02:00
Sarah Hoffmann
1dce2b98b4 switch CLI lookup command to Python implementation 2023-04-03 14:40:41 +02:00
Sarah Hoffmann
86c4897c9b add lookup call to server glue 2023-04-03 14:40:41 +02:00
Sarah Hoffmann
2237603677 add tests for new lookup API 2023-04-03 14:40:41 +02:00
Sarah Hoffmann
6e81596609 rename lookup() API to details and add lookup call 
The initial plan to serve /details and /lookup endpoints from
the same API call turned out to be impractical, so the API now
also has deparate functions for both.
 2023-04-03 14:40:41 +02:00
Sarah Hoffmann
ed9cd9f0e5 bdd: disable detail tests searching by place ID 
Place IDs are not stable and cannot be used in tests.
 2023-04-03 10:07:06 +02:00
Sarah Hoffmann
7d30dbebc5 flex style: reinstate postcode boundaries 
Postcode boundaries don't have a name, so need to be imported
unconditionally.
 2023-04-03 09:17:50 +02:00
biswajit-k
8f03c80ce8 generalize filter for sanitizers 2023-04-01 19:24:09 +05:30
Sarah Hoffmann
683a3cb3ec call osm2pgsql postprocessing flush_deleted_places() when adding data 2023-03-31 18:05:07 +02:00
Sarah Hoffmann
1feac2069b add BDD tests for new layers parameter 2023-03-30 09:54:55 +02:00
Sarah Hoffmann
26ee6b6dde python reverse: add support for point geometries in interpolations 2023-03-29 17:21:33 +02:00
Sarah Hoffmann
6c67a4b500 switch reverse CLI command to Python implementation 2023-03-26 18:09:33 +02:00
Sarah Hoffmann
86b43dc605 make sure PHP and Python reverse code does the same 
The only allowable difference is precision of coordinates. Python uses
a precision of 7 digits where possible, which corresponds to the
precision of OSM data.

Also fixes some smaller bugs found by the BDD tests.
 2023-03-26 16:21:43 +02:00
Sarah Hoffmann
35b52c4656 add output formatters for ReverseResults 
These formatters are written in a way that they can be reused for
search results later.
 2023-03-25 15:45:03 +01:00
Sarah Hoffmann
2f54732500 python: implement reverse lookup function 
The implementation follows for most part the PHP code but introduces an
additional layer parameter with which the kind of places to be returned
can be restricted. This replaces the hard-coded exclusion lists.
 2023-03-23 22:38:37 +01:00
Sarah Hoffmann
1facfd019b api: generalize error handling 
Return a consistent error response which takes into account the chosen
content type. Also adds tests for V1 server glue.
 2023-03-23 10:16:50 +01:00
Sarah Hoffmann
00e3a752c9 split SearchResult type 
Use adapted types for the different result types. This makes it
easier to have adapted output formatting and means there are only
result fields that are filled.
 2023-03-23 10:16:50 +01:00
Sarah Hoffmann
81430bd3bd bdd: be more fuzzy with coordinate comparisons 2023-03-09 22:37:45 +01:00
Sarah Hoffmann
93203f355a avoid recent Python dialect 2023-03-09 20:57:43 +01:00
Sarah Hoffmann
3f2296e3ea bdd: extend reverse API tests for format checks 
Reorganise the API reverse tests and extend the checks for the
output format, testing for all expected fields.
 2023-03-09 20:20:50 +01:00
Sarah Hoffmann
2b7eb4906a bdd: add tests for valid debug output 2023-03-09 20:10:51 +01:00
Sarah Hoffmann
db1aa4d02e bdd: replace old formatting strings 2023-03-09 19:49:55 +01:00
Sarah Hoffmann
ad88d7a3e0 bdd: more format checks for reverse XML 2023-03-09 19:40:24 +01:00
Sarah Hoffmann
e42c1c9c7a bdd: new step variant 'result contains in field' 
This replaces the + notation for recursing into result dictionaries.
 2023-03-09 19:31:21 +01:00
Sarah Hoffmann
556bb2386d bdd: factor out computation of result to-check lists 2023-03-09 18:01:45 +01:00
Sarah Hoffmann
1e58cef174 bdd: replace property_list construct with standard check functions 2023-03-09 17:56:28 +01:00
Sarah Hoffmann
01010e443f bdd: remove special case for osm_type field 
The fuzzy field check hide cover formatting errors. Use 'osm' when
only caring about the conent.
 2023-03-09 17:44:34 +01:00
Sarah Hoffmann
da0a7a765e bdd: reorganise field comparisons 
Move comparision on Field values from assert_field() into a
comparator class. Replace BadRowValueAssert with a simpler
check_row() function.
 2023-03-09 17:05:05 +01:00
Sarah Hoffmann
9769a0dcdb bdd: use new check_for_attributes() function also in steps 2023-03-09 16:44:07 +01:00
Sarah Hoffmann
fbff4fa218 bdd: fully check correctness of geojson and geocodejson 
Parse code now checks presence of all required fields and exports
all fields for inspection.
 2023-03-09 16:36:46 +01:00
Sarah Hoffmann
d17ec56e54 bdd: remove OrderedDict 
dicts are guaranteed to keep insertion order by since Python 3.7, making
use of ORderedDict mute.
 2023-03-09 16:08:39 +01:00
biswajit-k
ca149fb796 Adds sanitizer for preventing certain tags to enter search index based on parameters 
fix: pylint error

added docs for delete tags sanitizer

fixed typos in docs and code comments

fix: python typechecking error

fixed rank address type

Revert "fixed typos in docs and code comments"

This reverts commit 6839eea755a87f557895f30524fb5c03dd983d60.

added default parameters and refactored code

added test for all parameters
 2023-03-09 14:18:39 +05:30
Sarah Hoffmann
412ead5f2d adapt PHP tests for debug output 2023-02-20 16:23:28 +01:00
Sarah Hoffmann
d574ceb598 restrict place rank inheritance to address items 
Place tags must have no influence on street- or POI-level
objects.
 2023-02-17 16:25:26 +01:00
Sarah Hoffmann
ee0c5e24bb add a WKB decoder for the Point class 
This allows to return point geometries from the database and makes
the SQL a bit simpler.
 2023-02-16 17:29:56 +01:00
Sarah Hoffmann
8557105c40 add debug output for unit tests 
This uses the debug output facility meant for pretty HTML output
to give us debugging output for the unit tests.
 2023-02-14 11:57:37 +01:00
Sarah Hoffmann
42c3754dcd add tests for details result formatting and trim results 
Values that are None are no longer included in the output to save
a bit of bandwidth.
 2023-02-04 21:22:22 +01:00
Sarah Hoffmann
b742200442 expand details BDD tests 
There are now minor differences in the output between PHP and
Python versions, so introduce specific tests.
 2023-02-04 21:22:22 +01:00