2965 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Sarah Hoffmann	bf4320a7d6	do not depend on cmdline parameter for creating partition tables ... The partition numbers in use only depend on the entries in search_name.	2021-03-01 21:28:39 +01:00
Sarah Hoffmann	3a0a4b9175	save software version in the database ... The version represents the software version that was used to import the data.	2021-03-01 20:35:15 +01:00
Sarah Hoffmann	4faefe156c	report software version of status call	2021-03-01 16:47:19 +01:00
Sarah Hoffmann	86273f5e2a	introduce database patch level for version ... This will be needed later for automatic migrations.	2021-03-01 16:46:19 +01:00
Sarah Hoffmann	b4f64aa770	make sure that calls to PHP legacy scripts are fatal on error	2021-03-01 16:10:45 +01:00
Sarah Hoffmann	976c5e9121	introduce table for in-database properties ... Adds a simple table where settings for the database can be saved. This is useful for state that must not change after import.	2021-03-01 16:09:17 +01:00
Sarah Hoffmann	db663dd92f	remove unused import	2021-03-01 09:26:08 +01:00
Sarah Hoffmann	90a5d23016	use tmp_path fixture in config tests	2021-03-01 09:24:04 +01:00
Sarah Hoffmann	99e35d256a	fix typo	2021-03-01 09:07:49 +01:00
Sarah Hoffmann	e14e7c6235	Merge pull request #2186 from lonvia/port-import-to-python ... Move setup procedure to Python	2021-02-27 12:09:23 +01:00
Sarah Hoffmann	b46adbad22	make sure psql always finishes ... If an execption is raised by other means, we still have to close the stdin pipe to psql to make sure that it exits and releases its connection to the database.	2021-02-27 10:24:40 +01:00
Sarah Hoffmann	afabbeb546	older versions of Postgresql need explicit return type	2021-02-27 09:46:42 +01:00
Sarah Hoffmann	d14a3df10f	do not truncate search_name in reverse-only mode	2021-02-27 09:46:42 +01:00
Sarah Hoffmann	9feb84e426	actions: add psutil dependency	2021-02-26 16:50:09 +01:00
Sarah Hoffmann	c7f40e3cee	fix verbose flag for PHP wrapper scripts ... The flag must come after the command.	2021-02-26 16:49:32 +01:00
Sarah Hoffmann	dd03aeb966	bdd: use python library where possible ... Replace calls to PHP scripts with direct calls into the nominatim Python library where possible. This speed up tests quite a bit.	2021-02-26 16:14:29 +01:00
Sarah Hoffmann	15b5906790	move setup function to python ... There are still back-calls to PHP for some of the sub-steps. These needs some larger refactoring to be moved to Python.	2021-02-26 15:02:39 +01:00
Sarah Hoffmann	3ee8d9fa75	properly close connections of indexer after use	2021-02-26 12:10:54 +01:00
Sarah Hoffmann	57db5819ef	prot load-data function to python	2021-02-25 21:32:40 +01:00
Sarah Hoffmann	3c186f8030	add a function for the intial indexing run ... Also moves postcodes to fully parallel indexing.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	db5e78c879	remove unused partitionfunction function	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	c7fd0a7af4	port wikipedia importance functions to python	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	32683f73c7	move import-data option to native python ... This adds a new dependecy to the Python psutil package.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	7222235579	introduce custom object for cmdline arguments ... Allows to define special functions over the arguments. Also splits CLI tests in two files as they have become too many.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	f6e894a53a	port database setup function to python ... Hide the former PHP functions in a transition command until they are removed.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	b93ec2522e	use psql for executing sql files ... This allows to run larger files without needing to keep them in memory.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	af7226393a	add function to set up libpq environment ... Instead of parsing the DSN for each external libpq program we are going to execute, provide a function that feeds them all necessary parameters through the environment. osm2pgsql is the first user.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	e520613362	convert connect() into a context manager	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	204fe20b4b	Merge pull request #2185 from lonvia/fix-deadlock-handling-for-psycopg27 ... Improve deadlock detection for various versions of psycopg2	2021-02-25 18:39:40 +01:00
Sarah Hoffmann	a1f0fc1a10	improve deadlock detection for various versions of psycopg2 ... Psycopg2 has changed the kind of exception that is emitted on deadlocks between versions 2.7 and 2.8. The code was already trying to catch both kind of errors but because the psycopg2.errors package is unknown in 2.7 and below, the code would throw an exception on anything but a deadlock error. This commit wraps the deadlock handling into a context manager to avoid code duplication and uses module imports to detect if the new error codes are available. Also sets the required psycopg2 version to 2.7 or bigger as versions below are difficult to test.	2021-02-25 18:11:16 +01:00
Sarah Hoffmann	68c3862270	Merge pull request #2182 from lonvia/change-error-for-details ... Return 404 for details when no object is found in database	2021-02-23 09:09:35 +01:00
Sarah Hoffmann	5b7483ada5	return 404 for details when no bject is found in database ... Fixes #2157.	2021-02-22 16:28:29 +01:00
Sarah Hoffmann	72b01148d2	Merge pull request #2181 from lonvia/port-more-tool-functions-to-python ... Port more tool functions to python	2021-02-22 16:11:21 +01:00
Sarah Hoffmann	971df231b0	avoid os.environ as default valie	2021-02-19 19:29:57 +01:00
Sarah Hoffmann	4b32cbe518	fix return code for check database run with 'not applicable'	2021-02-19 18:32:00 +01:00
Sarah Hoffmann	f08078ccca	bdd tests: directly call python code for setup-website	2021-02-19 18:20:55 +01:00
Sarah Hoffmann	389138abfe	port setup-website to python	2021-02-19 17:51:06 +01:00
Sarah Hoffmann	a0ae4945cd	add unit tests for new check_database code	2021-02-18 20:36:11 +01:00
Sarah Hoffmann	b169e4c88c	port check-database function to python ... This change also adapts the hints to use the nominatim tool. Slightly changed checks, so that they are just as effective on a frozen database.	2021-02-18 17:32:30 +01:00
Sarah Hoffmann	a60c34bded	use a frozen DB for API tests ... This way we also test that dropping does the right thing.	2021-02-17 22:35:27 +01:00
Sarah Hoffmann	153dbb71b8	remove unused code	2021-02-17 22:25:23 +01:00
Sarah Hoffmann	101a1f895d	port freeze function to python	2021-02-17 21:43:15 +01:00
Sarah Hoffmann	bd27310c68	Merge pull request #2173 from lonvia/penality-for-housenumberless-places ... Increase penalty for places without housenumber	2021-02-17 17:52:59 +01:00
Sarah Hoffmann	42ecd535b3	Merge pull request #2174 from lonvia/disable-jit-for-osm2pgsql-again ... Disable JIT and parallel execution for osm2pgsql updates again	2021-02-16 21:32:57 +01:00
Sarah Hoffmann	c9838a02ce	disable JIT and parallel execution for osm2pgsql updates again ... The gazetteer output doesn't disable these functions when writing to the place table but the triggers may contain operations that cause misplanning for the query planner.	2021-02-16 18:23:47 +01:00
Sarah Hoffmann	7ebcf602ac	add simple test for result splitting with multiple ranks	2021-02-16 17:59:12 +01:00
Sarah Hoffmann	8eb85f1340	increase penalty for places without housenumber ... Results where the housenumber was dropped are an unlikely result when they refer to something other than a street. Therefore increase their result rank so that other matches are tried first before choosing them as a result. Improves #2167.	2021-02-16 17:47:06 +01:00
Sarah Hoffmann	2a8e3741fa	Merge pull request #2166 from mtmail/tiger-2020 ... documentation: 2020 TIGER data got released	2021-02-16 14:45:14 +01:00
Sarah Hoffmann	684378722c	Merge pull request #2171 from lonvia/update-vagrant-scripts-for-make-install ... Update vagrant scripts for make install	2021-02-16 14:42:38 +01:00
Sarah Hoffmann	286a686f88	switch vagrant scripts to make install	2021-02-16 12:04:34 +01:00