Commit Graph

102 Commits

Author SHA1 Message Date
Sarah Hoffmann
62b7670e0c for postcodes use rank_search as base rank for finding addresses
The rank_address reflects the position in the address which is
usually lower than what one would expect for a postcode area.
2024-02-28 14:40:36 +01:00
Sarah Hoffmann
1879cf902c
Merge pull request #3346 from lonvia/reduce-artificial-importance
Reduce default importance
2024-02-28 14:21:46 +01:00
Sarah Hoffmann
019a68a4bb
Merge pull request #3345 from lonvia/simplify-large-geometries
Simplify very large polygons that are not used in addresses
2024-02-28 12:06:49 +01:00
Sarah Hoffmann
36b1660121 add support for new middle table format of osm2pgsql
Functions are adapted according to the format detected from the
osm2pgsql property table.
2024-02-27 18:18:19 +01:00
Sarah Hoffmann
56201feb28 simplify very large polygons non used in addresses
Polygons with rank_address = 0 are only used in search and (rarely)
for reverse lookup. Geometries do not need to be precise for that
because topology does not matter. OSM has some very large polygons
of natural features with sizes of more than 10MB. Simplify these
polygons to keep the database and indexes smaller.
2024-02-27 10:16:18 +01:00
Sarah Hoffmann
c6d40d4bf4 reduce importance when computed from search rank 2024-02-27 10:15:54 +01:00
Sarah Hoffmann
4c19762e33 extratags should become null when empty
Removing the artifical entries in the extratags may lead to
an empty hstore. Set it to null in that case.

Fixes #3055.
2024-02-08 10:21:48 +01:00
Sarah Hoffmann
9627352ee4 search postcodes for highway areas around the area
So far the code would only accept postcodes that are inside the area.

Fixes #3304.
2024-01-26 18:14:11 +01:00
Sarah Hoffmann
af85ad390f
Merge pull request #3273 from lonvia/search-with-sqlite
Add forward search capability for SQLite databases
2023-12-12 12:15:22 +01:00
Sarah Hoffmann
c41f2fed21 simplify weigh_search() function
Use JSON arrays which can have mixed types and therefore have
a more logical structure than separate arrays. Avoid JSON dicts
because of their verboseness.
2023-12-07 09:31:00 +01:00
Sarah Hoffmann
8c7140d92b
Merge pull request #3108 from mtmail/remove-legacy-wikipedia-tag-syntax
These days the OSM wikipedia tab no longer contains URLs
2023-12-07 09:24:32 +01:00
Sarah Hoffmann
383e3ccd25 php frontend: fix on-the-fly lookup of postcode areas 2023-11-15 17:45:12 +01:00
lujoh
06204dfcd8 moved sql function flush_deleted_places() to utils 2023-10-17 18:22:27 -04:00
Sarah Hoffmann
7fcbe13669 move get_addressdata() implementation to Python
The pgsql function get_addressdata() does a lookup of a lot of data
that is already available in Python.
2023-09-26 11:21:36 +02:00
Sarah Hoffmann
c5f5ab5363 be more strict about removal from place_to_be_deleted
If the type of a place is changed and then the same insert issued
again, the old data will effectively remain in the table.

Fixed #3168.
2023-08-25 14:22:49 +02:00
Sarah Hoffmann
7c79b07817 fix parameter use for ST_Project
Before postgis 3.4 ST_Project required a geography as input and seemed
to have implicitly converted to geography. Since 3.4 geometry input
is supported but leads to a completely different result.
2023-08-20 13:52:03 +02:00
Marc Tobias
f59a072aa6 These days the OSM wikipedia tab no longer contains URLs 2023-07-07 17:36:11 +02:00
Sarah Hoffmann
ce17b0eeca
Merge pull request #3101 from lonvia/custom-geometry-type
Improve use of SQLAlchemy statement cache with search queries
2023-07-03 11:03:26 +02:00
Sarah Hoffmann
5135041405 replace CASE construct with plpgsql function 2023-07-01 18:15:22 +02:00
Sarah Hoffmann
a873f260cf fix merging of linked names into unnamed boundaries
The NULL value of the boundaries' name field was erasing all
content when used in SQL operations.
2023-06-30 22:14:11 +02:00
Sarah Hoffmann
645ea5a057 use information from tokenizer to determine street vs. place address
So far the SQL logic used the information from the address field
to determine if an address is attached to a street or place.
This changes the logic to use the information provided in the
token_info. This allows sanitizers to enforce a certain parenting
without changing the visible address information.
2023-06-30 11:08:25 +02:00
Sarah Hoffmann
60c1301fca fix a number of corner cases with interpolation splitting
Snapping a line to a point before splitting was meant to ensure
that the split point is really on the line. However, ST_Snap() does
not always behave well for this case. It may shorten the interpolation
line in some cases with the result that two points housenumbers
suddenly fall on the same point. It might also shorten the line down
to a single point which then makes ST_Split() crash.

Switch to a combination of ST_LineLocatePoint and ST_LineSubString
instead, which guarantees to keep the original geometry. Explicitly
handle the corner cases, where the split point falls on the beginning
or end of the line.
2023-04-06 16:54:00 +02:00
Sarah Hoffmann
b8a7319212 use place_to_be_deleted when force deleting objects 2023-04-04 11:09:17 +02:00
Sarah Hoffmann
d574ceb598 restrict place rank inheritance to address items
Place tags must have no influence on street- or POI-level
objects.
2023-02-17 16:25:26 +01:00
Sarah Hoffmann
3f38091421 split query that deletes old objects from placex
placex only has partial indexes over OSM types, so the OSM type
needs to be hardcoded to ensure these indexes are used.
2023-01-02 17:25:38 +01:00
Sarah Hoffmann
922352e215 do not assign postcodes to long linear features
This avoids a postcode in particular for waterway features and
long natural featues like ridges and valleys.

Fixes #2915.
2022-12-10 14:53:08 +01:00
Sarah Hoffmann
c9ff7d2130 drop illegal values for addr:interpolation on update 2022-11-18 17:26:56 +01:00
Sarah Hoffmann
4f05a03d13 handle associatedStreet relations with multiple streets
When a associatedStreet relation has multiple street members
always take the closest one. Avoid geometry operations for
the frequent case that there is only one street.
2022-11-16 17:25:51 +01:00
Sarah Hoffmann
a46348da38 bdd: test placex content when updating with osm2pgsql 2022-11-14 14:48:44 +01:00
Sarah Hoffmann
36cf0eb922 reorganize handling of place type changes
Always replace existing entries in place, never delete them because
a direct delete will cause conflicts.
2022-11-14 13:57:26 +01:00
Sarah Hoffmann
2fac507453 change updates to handle delete/insert workflow
This makes Nominatim compatible with osm2pgsql's default update
modus operandi of deleting and reinserting data. Deletes are diverted
into a TODO table instead of executing them. When data is reinserted,
the corresponding entry in the TODO table is deleted. After updates are
finished, the remaining entries in the TODO table are executed, doing
the same work as the delete trigger did before.

The new behaviour also works against the gazetteer output with its
insert-only mechanism.
2022-11-10 09:38:23 +01:00
Sarah Hoffmann
abf349fb0d simplify use of secondary importance
The values in the raster are already normalized between 0 and 2**16,
so a simple conversion to [0, 1] will do.

Check for existance of secondary_importance table statically when
creating the SQL function. For that to work importance tables need
to be created before the functions.
2022-10-01 11:01:49 +02:00
Tareq Al-Ahdal
0ab0f0ea44 Integrated OSM views into importance computation 2022-10-01 11:01:49 +02:00
Sarah Hoffmann
f017e1e9a1 make sure indexes are used 2022-09-25 14:09:45 +02:00
Sarah Hoffmann
33ba6896a8 further split up the big geometry index
Adds partial indexes for all geometry queries used during import.
A full index is not necessary anymore at that point. Still create
the index afterwards for use in queries.

Also adds documentation for all indexes on where they are used.
2022-09-21 16:21:41 +02:00
Sarah Hoffmann
aef014a47d add indexes for lookup of addressable areas
The generic geometry index has become to slow for that purpose.
2022-09-18 16:57:12 +02:00
Sarah Hoffmann
dddfa3a075 ignore irrelevant extra tags on address interpolations
When deciding if an address interpolation has address information, only
look for addr:street and addr:place. If they are not there go looking
for the address on the address nodes. Ignores irrelevant tags like
addr:inclusion.

Fixes #2797.
2022-08-13 14:07:06 +02:00
Sarah Hoffmann
487e81fe3c more invalidations when boundary changes rank
When a boundary or place changes its address rank, all places where
it participates as address need to be potentially reindexed.
Also use the computed rank when testing place nodes against
boundaries. Boundaries are computed earlier.

Fixes #2794.
2022-08-12 09:48:46 +02:00
Sarah Hoffmann
b19c90b9a6 export centroid to tokenizer
May come in handy when developping sanitizers for an area smaller
than country size.
2022-07-31 22:10:58 +02:00
Kian-Meng Ang
f5e52e748f docs: fix typos 2022-07-20 22:05:31 +08:00
Sarah Hoffmann
0f00f4968c fix up BDD tests for postcode changes
Includes smaller code fixes found by the tests.
2022-06-23 23:42:31 +02:00
Sarah Hoffmann
b7704833e4 icu: switch postcodes to using the pre-formatted one 2022-06-23 23:42:31 +02:00
Sarah Hoffmann
f833cc80df use default ranks when reorganising rank_address
When shifting address ranks, the evaluation is always done against
unshifted address ranks on import because the objects we compare against
have not been indexed yet. This changes for updates when the object have
been touched in the meantime. To ensure consistent behaviour across
imports and updates, always use the  unshifted address ranks.
2022-06-16 11:20:23 +02:00
Sarah Hoffmann
df0142678a improve address ordering with mixes of place and admin areas
Resolves a couple of situations where a mixed use of places areas and
administrative boundaries would result in a hierarchy that did not
properly respect the contains relation.
2022-06-16 10:44:16 +02:00
Sarah Hoffmann
15cf7dd416 add testcase for #2551
This test proves that places that are linked need to be reindexed.
2022-06-05 21:39:17 +02:00
Sarah Hoffmann
2c05fc858a fix rank inheritance from linked places
When taking over the address rank from a linked place, it needs
to be the originally computed rank, not the one that might have
been adjusted in the meantime. The adjustment was made under the
assumption that the node is not linked.
2022-06-05 19:38:14 +02:00
Sarah Hoffmann
bd0e157b91 fix order when searching for addr:* components
When matching addr:* components the preference was given to
matches that do not intersect with the place.
2022-05-31 16:57:37 +02:00
Sarah Hoffmann
1d203fdb3c fix bug with keeping linking on updates
When moving the finding of linked places to the precomputation stage,
it was also moved before the statement where the linked_place_id was
removed from the linkee. The result was that the current linkee was
excluded when looking for a linked place on updates because it was
still linked to the boundary to be updated.

Fixed by allowing to either keep the linkage or change to an unlinked
place.
2022-05-23 10:55:10 +02:00
Sarah Hoffmann
739fe1c2c4 no longer allow fuzzy assignment of country
The fallback country boundaries already contain a sufficiently large
part of the water area, so there is no need to extend the country
assignment even more. Features outside countries should not show a
country in their address.
2022-05-11 11:54:25 +02:00
Sarah Hoffmann
08672cdf0a explicit cast for osm_type parameter in SQL needed
Otherwise PostgreSQL won't correctly pick up the index
condition.
2022-05-02 14:12:17 +02:00