osm-search/Nominatim
1
1
Fork 0
mirror of https://github.com/osm-search/Nominatim.git synced 2024-11-25 19:35:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
4,884 Commits 28 Branches 41 Tags 164 MiB
3acd7df5c4
Commit Graph

918 Commits

Author SHA1 Message Date
Sarah Hoffmann
ca6e65fff1 bdd: be more verbose on HTML parsing error 2024-02-19 18:39:01 +01:00
Sarah Hoffmann
1e0025b095 also switch unit tests for cli 2024-02-19 18:39:01 +01:00
Sarah Hoffmann
05fad607ff make Python frontend default and PHP optional 2024-02-19 18:39:01 +01:00
Sarah Hoffmann
4c19762e33 extratags should become null when empty 
Removing the artifical entries in the extratags may lead to
an empty hstore. Set it to null in that case.

Fixes #3055.
 2024-02-08 10:21:48 +01:00
Sarah Hoffmann
81eed0680c recreate word table when refreshing counts 
The counting touches a large part of the word table, leaving
bloated tables and indexes. Thus recreate the table instead and
swap it in.
 2024-02-04 21:35:10 +01:00
Sarah Hoffmann
fed46240d5 disallow category tokens in the middle of a query string 
This already worked for left-to-right readings and now is also
implemented for right-to-left reading. A qualifier must always be
before or after the name.
 2024-01-28 19:03:11 +01:00
Sarah Hoffmann
9627352ee4 search postcodes for highway areas around the area 
So far the code would only accept postcodes that are inside the area.

Fixes #3304.
 2024-01-26 18:14:11 +01:00
Sarah Hoffmann
e0ca2ce6ec interpret stand-alone special terms always as near term 
Fixes #3298.
 2024-01-16 17:19:21 +01:00
Sarah Hoffmann
dc52d0954e
Merge pull request #3238 from mtmail/check-database-for-version-match 
admin --check-database also checks database vs nominatim version
 2024-01-07 15:24:00 +01:00
Sarah Hoffmann
10a5424a71 do not run near queries on qualifier words 
There is too much potential for confusion (e.g. 'Rio Grande' read
as 'river near Grande') fir too little gain. Use near phrases
instead.
 2024-01-07 11:33:11 +01:00
Marc Tobias
1d7e078a2c check-database also checks database vs nominatim version 2024-01-06 20:56:56 +01:00
Sarah Hoffmann
8e90fa3395 avoid closure variables in lambda statements 
There is a bug in SQLAlchemy that assigns the wrong value to bind
parameters from closure variables when reusing lambda statements
that are later extended with other non-lambda expressions.

Thus either avoid lambda statements with closure variables or extending
them with non-lambda expressions.
 2024-01-05 17:49:28 +01:00
Sarah Hoffmann
ff06b64329 enable all BDD API tests for sqlite 2023-12-07 09:32:02 +01:00
Sarah Hoffmann
6d39563b87 enable all API tests for sqlite and port missing features 2023-12-07 09:32:02 +01:00
Sarah Hoffmann
b06f5fddcb simplify handling of SQL lookup code for search_name 
Use function classes which can be instantiated directly.
 2023-12-07 09:31:00 +01:00
Sarah Hoffmann
8791c6cb69 correctly close API objects during testing 2023-12-07 09:31:00 +01:00
Sarah Hoffmann
1b7c8240ba enable connection pools for sqlite 
Connecting is reasonably expensive because the spatialite extension
needs to be loaded. Disable pooling for tests because there is some
memory leak when quickly opening and closing QueuePools with sqlite
connections.
 2023-12-07 09:31:00 +01:00
Sarah Hoffmann
b2319e52ff correctly exclude streets with housenumber searches 
Street result are not subject to the full filtering in the SQL
query, so recheck.
 2023-11-28 17:53:37 +01:00
Sarah Hoffmann
25279d009a add tests for interaction of category parameter with category terms 2023-11-28 16:56:08 +01:00
Sarah Hoffmann
3f72ca4bca rename use of category as POI search to near_item 
Use the term category only as a short-cut for "tuple of key and value".
 2023-11-28 16:27:05 +01:00
Sarah Hoffmann
70dc4957dc the category parameter in search should result in a qualifier 2023-11-28 12:01:49 +01:00
Sarah Hoffmann
a7f5c6c8f5 drop category tokens when they make up a full phrase 2023-11-26 20:58:50 +01:00
Sarah Hoffmann
d8dca2a3a9 enable BDD tests for sqlite databases 
The database must currently be created by hand and the name handed
in via -DAPI_TEST_DB='sqlite:...'.
 2023-10-23 17:19:12 +02:00
Sarah Hoffmann
0417946153 make reverse API work with sqlite 2023-10-23 17:19:12 +02:00
Sarah Hoffmann
06bbd501fd make status API work with sqlite incl. unit tests 2023-10-23 17:19:12 +02:00
Sarah Hoffmann
07e6c5cf69 make details API work with sqlite incl. unit tests 2023-10-23 17:19:12 +02:00
Sarah Hoffmann
d0c91e4acf make lookup call work with sqlite 
Includes porting unit tests.
 2023-10-23 17:19:12 +02:00
Sarah Hoffmann
6ad397d4a9 Merge branch 'clean-deleted-relations' of https://github.com/lujoh/Nominatim into lujoh-clean-deleted-relations 2023-10-23 10:47:31 +02:00
lujoh
418f381b49 made age a required argument for the -clean-deleted command 2023-10-20 15:31:55 -04:00
Paweł Wroniszewski
2cae37ccde Revert country settings 2023-10-20 12:50:28 +02:00
lujoh
9ec26c60ff adjusted tests for --clean-deleted-relations command 2023-10-17 23:03:37 -04:00
lujoh
06204dfcd8 moved sql function flush_deleted_places() to utils 2023-10-17 18:22:27 -04:00
Paweł Wroniszewski
fbe40e005d Properly validate postcodes with country code 
Include postcode pattern in postcode normalisation regex, instead of
removing it from postcode pattern in config.

It properly handles postcode validation and normalization when country code
is part of the postcode, e.g. for Isle of Man, Jersey, Anguilla, Andorra,
Cayman Islands and more.

Fixes #3227.
 2023-10-17 01:04:07 +02:00
lujoh
e9efef9095 added subcommand to clean deleted relations for issue # 2444 2023-10-16 11:30:58 -04:00
Sarah Hoffmann
12dbfb0777
Merge pull request #3222 from lonvia/fix-river-output 
Make sure the place name always comes first in output
 2023-10-10 11:20:12 +02:00
Sarah Hoffmann
b62dbd1f92 reduce influence of viewbox 
Perfectly matching city names should still get priority.
 2023-10-07 22:00:52 +02:00
Sarah Hoffmann
5011fde176 make sure the place name always comes first in output 
Also deleted some now unused code.
 2023-10-05 16:37:28 +02:00
Sarah Hoffmann
b00b16aa3a more unit tests for search 2023-09-27 15:00:05 +02:00
Sarah Hoffmann
7fcbe13669 move get_addressdata() implementation to Python 
The pgsql function get_addressdata() does a lookup of a lot of data
that is already available in Python.
 2023-09-26 11:21:36 +02:00
Sarah Hoffmann
0e10916b07 adapt tests to improved search 2023-09-20 14:58:54 +02:00
Sarah Hoffmann
5762a5bc80 move localization into add_result_details 
This means that the locale now needs to be handed in into the search
functions already. At least search needs them for reranking.
 2023-09-19 11:17:04 +02:00
Sarah Hoffmann
c284df2dc9 restrict range for interpolated housenumbers 
Interpolations are only supported up to 2^32 by the database.
Limit to 8 digits, which is still more than should be needed.
 2023-09-05 11:41:41 +02:00
Sarah Hoffmann
18b2a4c204 avoid interpreting integral numbers as coordinates 
There are addresses of type '123 W 345 N, Reigh', where 'W 345 N' is
the actual name of the street.
 2023-09-05 11:26:47 +02:00
Sarah Hoffmann
26dfb868e9 add test case for bad update 2023-08-25 14:22:49 +02:00
Sarah Hoffmann
a9edd57fe2 send charset again in content-type when returning json 
There are quite a few applications out there that will use some local
encoding when the charset is not explicitly given.
 2023-08-23 20:55:57 +02:00
Sarah Hoffmann
23eed4ff2f fix tag name for housename addresses in layer selection 
Fixes #3156.
 2023-08-19 15:57:33 +02:00
Sarah Hoffmann
fa3ac22a8f adapt tests to changes in search 2023-08-12 16:12:31 +02:00
Sarah Hoffmann
926c4a7d04 fix lookup polygon output 
Fixes #3147.
 2023-08-08 21:31:25 +02:00
Sarah Hoffmann
282c0da941 return an error when q is used together with structured parameters 2023-08-08 15:54:55 +02:00
Sarah Hoffmann
78648f1faf remove lookup by address only 
There are too many lookups where the address is very frequent,
even when many address parts are present.
 2023-08-06 21:00:10 +02:00
Sarah Hoffmann
996026e5ed provide full URL in more field 
This is a regression against the PHP version.

Fixes #3138.
 2023-08-06 17:50:02 +02:00
Sarah Hoffmann
afdbdb02a1 do not lookup by address vector when only few tokens are available 
Names of countries and states are exceedingly rare in the word count
but are very frequent in the address. A short name has the danger
of producing too many results.
 2023-08-02 09:25:47 +02:00
Sarah Hoffmann
252fe42612
Merge pull request #3122 from miku0/sanitizer-final 
Adds sanitizer for Japanese addresses to correspond to block address
 2023-08-01 10:38:58 +02:00
miku0
67e1c7dc72 Moved KANJI_MAP to icu-rules 2023-07-31 11:57:49 +00:00
miku0
4d61cc87cf Add the test of reconbine_place 2023-07-31 02:39:56 +00:00
Sarah Hoffmann
e523da9e12 reintroduce file logging for Python frontend 2023-07-30 19:58:00 +02:00
miku0
67706cec4e add @fail-legacy 2023-07-27 07:33:53 +00:00
Sarah Hoffmann
9448c5e16f add tests for new arm and export Python functions 2023-07-26 11:09:52 +02:00
miku0
0722495434 add japanese sanitizer 2023-07-26 07:54:58 +00:00
Sarah Hoffmann
d545c6d73c mostly remove php-cgi requirement 
This is now only needed for BDD tests against the php API.
 2023-07-26 00:10:11 +02:00
Sarah Hoffmann
f69fea4210 remove now unused run_api_script function 2023-07-25 22:45:29 +02:00
Sarah Hoffmann
4cd0a4ced4 remove now unused run_legacy_script() 2023-07-25 21:39:23 +02:00
Sarah Hoffmann
0804cc0cff port export function to Python 
Some of the parameters have been renoved as they don't make sense
anymore.
 2023-07-25 21:39:23 +02:00
Sarah Hoffmann
faeee7528f move warm script to python code 2023-07-25 21:39:23 +02:00
Sarah Hoffmann
66ecb56cea add tests for new endpoints 2023-07-25 10:57:19 +02:00
Sarah Hoffmann
927d2cc824 do not split names from typed phrases 
When phrases are typed, they should only contain exactly one term.
 2023-07-17 20:09:08 +02:00
Sarah Hoffmann
cc45930ef9 avoid lookup via partials on frequent words 
Drops expensive searches via partials on terms like 'rue de'.

See #2979.
 2023-07-06 12:16:57 +02:00
Sarah Hoffmann
ce17b0eeca
Merge pull request #3101 from lonvia/custom-geometry-type 
Improve use of SQLAlchemy statement cache with search queries
 2023-07-03 11:03:26 +02:00
Sarah Hoffmann
82216ebf8b always run function update on migrations 
This means that we can have migrations which require nothing but
an update of the functions.
 2023-07-01 20:18:59 +02:00
Sarah Hoffmann
9f6f12cfeb move search to bind parameters 2023-07-01 18:03:07 +02:00
Sarah Hoffmann
a873f260cf fix merging of linked names into unnamed boundaries 
The NULL value of the boundaries' name field was erasing all
content when used in SQL operations.
 2023-06-30 22:14:11 +02:00
Sarah Hoffmann
d7a3039c2a also switch legacy tokenizer to new street/place choice behaviour 2023-06-30 17:03:17 +02:00
Sarah Hoffmann
645ea5a057 use information from tokenizer to determine street vs. place address 
So far the SQL logic used the information from the address field
to determine if an address is attached to a street or place.
This changes the logic to use the information provided in the
token_info. This allows sanitizers to enforce a certain parenting
without changing the visible address information.
 2023-06-30 11:08:25 +02:00
Sarah Hoffmann
2755ebe883
Merge pull request #3094 from lonvia/fix-failing-bdd-tests 
Add BDD tests against Python frontend to CI
 2023-06-22 22:28:31 +02:00
Sarah Hoffmann
2d05ff0190 slightly adapt postcode tests 2023-06-22 16:51:59 +02:00
Sarah Hoffmann
0d338fa4c0 bdd: fix faking HTTP headers for python web frameworks 2023-06-22 14:00:33 +02:00
mtmail
15a66e7b7d
Merge branch 'osm-search:master' into check-database-on-frozen-database 2023-06-22 12:14:55 +02:00
Marc Tobias
2337cc653b check-database on frozen db shouldnt recommend indexing 2023-06-21 17:47:57 +02:00
Sarah Hoffmann
9bc5be837b remove useless check 
Found by new mypy version.
 2023-06-21 11:56:39 +02:00
Sarah Hoffmann
b79d5494f9 remove support for sanic framework 
There is no performance gain over falcon or starlette but the special
structure of sanic makes it hard to have exchangable code
 2023-06-21 10:53:57 +02:00
Sarah Hoffmann
36df56b093 fix header name for browser languages 2023-06-20 11:56:43 +02:00
Sarah Hoffmann
d0a1e8e311 tweak postcode search 
Give a preference to left-right reading, i.e <postcode>,<address>
prefers a postcode search while <address>,<postcode> rather does
an address search.

Also exclude non-addressables, countries and state from results when a
postcode is contained in the query.
 2023-06-20 11:56:43 +02:00
Sarah Hoffmann
1f83efa8f2
Merge pull request #3086 from lonvia/close-connection-on-replication 
Close database connections while waiting for the next update cycle
 2023-06-19 15:48:00 +02:00
Sarah Hoffmann
6f3339cc49 close DB connection when waiting for next update cycle 2023-06-19 12:02:51 +02:00
Sarah Hoffmann
771be0e056 do not fail php script generation when curly braces are present 
Fixes #3084.
 2023-06-19 11:23:30 +02:00
Sarah Hoffmann
41bf162306 remove tests for old PHP cli commands 2023-05-26 17:36:05 +02:00
Sarah Hoffmann
8f299838f7 fix various failing BDD tests 2023-05-26 15:08:48 +02:00
Sarah Hoffmann
146a0b29c0 add support for search by houenumber 2023-05-26 14:10:57 +02:00
Sarah Hoffmann
371a780ef4 add server fronting for search endpoint 
This also implements some of the quirks of free-text search of the
V1 API, in particular, search for categories and coordinates.
 2023-05-26 11:40:45 +02:00
Sarah Hoffmann
0608cf1476 switch CLI search command to python implementation 2023-05-24 22:54:54 +02:00
Sarah Hoffmann
f335e78d1e make localisation of results explicit 
Localisation was previously done as part of the formatting but might
also be useful on its own when working with the results directly.
 2023-05-24 18:12:34 +02:00
Sarah Hoffmann
dcfb228c9a add API functions for search functions 
Search is now split into three functions: for free-text search,
for structured search and for search by category. Note that the
free-text search does not have as many hidden features like
coordinate search. Use the search parameters for that.
 2023-05-24 18:05:43 +02:00
Sarah Hoffmann
dc99bbb0af implement actual database searches 2023-05-24 13:52:31 +02:00
Sarah Hoffmann
c42273a4db implement search builder 2023-05-23 11:23:44 +02:00
Sarah Hoffmann
3bf489cd7c implement token assignment 2023-05-22 15:49:03 +02:00
Sarah Hoffmann
d8240f9ee4 add query analyser for legacy tokenizer 2023-05-22 11:07:14 +02:00
Sarah Hoffmann
2448cf2a14 add factory for query analyzer 2023-05-22 09:23:19 +02:00
Sarah Hoffmann
004883bdb1 query analyzer for ICU tokenizer 2023-05-22 08:46:19 +02:00
Sarah Hoffmann
ff66595f7a add data structure for tokenized query 2023-05-21 09:30:57 +02:00
Sarah Hoffmann
d9d8b9c526 add tests for parameter converter 2023-05-18 18:09:07 +02:00