3049 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Sarah Hoffmann	15b5906790	move setup function to python ... There are still back-calls to PHP for some of the sub-steps. These needs some larger refactoring to be moved to Python.	2021-02-26 15:02:39 +01:00
Sarah Hoffmann	3ee8d9fa75	properly close connections of indexer after use	2021-02-26 12:10:54 +01:00
Sarah Hoffmann	57db5819ef	prot load-data function to python	2021-02-25 21:32:40 +01:00
Sarah Hoffmann	3c186f8030	add a function for the intial indexing run ... Also moves postcodes to fully parallel indexing.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	db5e78c879	remove unused partitionfunction function	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	c7fd0a7af4	port wikipedia importance functions to python	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	32683f73c7	move import-data option to native python ... This adds a new dependecy to the Python psutil package.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	7222235579	introduce custom object for cmdline arguments ... Allows to define special functions over the arguments. Also splits CLI tests in two files as they have become too many.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	f6e894a53a	port database setup function to python ... Hide the former PHP functions in a transition command until they are removed.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	b93ec2522e	use psql for executing sql files ... This allows to run larger files without needing to keep them in memory.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	af7226393a	add function to set up libpq environment ... Instead of parsing the DSN for each external libpq program we are going to execute, provide a function that feeds them all necessary parameters through the environment. osm2pgsql is the first user.	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	e520613362	convert connect() into a context manager	2021-02-25 18:42:54 +01:00
Sarah Hoffmann	204fe20b4b	Merge pull request #2185 from lonvia/fix-deadlock-handling-for-psycopg27 ... Improve deadlock detection for various versions of psycopg2	2021-02-25 18:39:40 +01:00
Sarah Hoffmann	a1f0fc1a10	improve deadlock detection for various versions of psycopg2 ... Psycopg2 has changed the kind of exception that is emitted on deadlocks between versions 2.7 and 2.8. The code was already trying to catch both kind of errors but because the psycopg2.errors package is unknown in 2.7 and below, the code would throw an exception on anything but a deadlock error. This commit wraps the deadlock handling into a context manager to avoid code duplication and uses module imports to detect if the new error codes are available. Also sets the required psycopg2 version to 2.7 or bigger as versions below are difficult to test.	2021-02-25 18:11:16 +01:00
Sarah Hoffmann	68c3862270	Merge pull request #2182 from lonvia/change-error-for-details ... Return 404 for details when no object is found in database	2021-02-23 09:09:35 +01:00
Sarah Hoffmann	5b7483ada5	return 404 for details when no bject is found in database ... Fixes #2157.	2021-02-22 16:28:29 +01:00
Sarah Hoffmann	72b01148d2	Merge pull request #2181 from lonvia/port-more-tool-functions-to-python ... Port more tool functions to python	2021-02-22 16:11:21 +01:00
Sarah Hoffmann	971df231b0	avoid os.environ as default valie	2021-02-19 19:29:57 +01:00
Sarah Hoffmann	4b32cbe518	fix return code for check database run with 'not applicable'	2021-02-19 18:32:00 +01:00
Sarah Hoffmann	f08078ccca	bdd tests: directly call python code for setup-website	2021-02-19 18:20:55 +01:00
Sarah Hoffmann	389138abfe	port setup-website to python	2021-02-19 17:51:06 +01:00
Sarah Hoffmann	a0ae4945cd	add unit tests for new check_database code	2021-02-18 20:36:11 +01:00
Sarah Hoffmann	b169e4c88c	port check-database function to python ... This change also adapts the hints to use the nominatim tool. Slightly changed checks, so that they are just as effective on a frozen database.	2021-02-18 17:32:30 +01:00
Sarah Hoffmann	a60c34bded	use a frozen DB for API tests ... This way we also test that dropping does the right thing.	2021-02-17 22:35:27 +01:00
Sarah Hoffmann	153dbb71b8	remove unused code	2021-02-17 22:25:23 +01:00
Sarah Hoffmann	101a1f895d	port freeze function to python	2021-02-17 21:43:15 +01:00
Sarah Hoffmann	bd27310c68	Merge pull request #2173 from lonvia/penality-for-housenumberless-places ... Increase penalty for places without housenumber	2021-02-17 17:52:59 +01:00
Sarah Hoffmann	42ecd535b3	Merge pull request #2174 from lonvia/disable-jit-for-osm2pgsql-again ... Disable JIT and parallel execution for osm2pgsql updates again	2021-02-16 21:32:57 +01:00
Sarah Hoffmann	c9838a02ce	disable JIT and parallel execution for osm2pgsql updates again ... The gazetteer output doesn't disable these functions when writing to the place table but the triggers may contain operations that cause misplanning for the query planner.	2021-02-16 18:23:47 +01:00
Sarah Hoffmann	7ebcf602ac	add simple test for result splitting with multiple ranks	2021-02-16 17:59:12 +01:00
Sarah Hoffmann	8eb85f1340	increase penalty for places without housenumber ... Results where the housenumber was dropped are an unlikely result when they refer to something other than a street. Therefore increase their result rank so that other matches are tried first before choosing them as a result. Improves #2167.	2021-02-16 17:47:06 +01:00
Sarah Hoffmann	2a8e3741fa	Merge pull request #2166 from mtmail/tiger-2020 ... documentation: 2020 TIGER data got released	2021-02-16 14:45:14 +01:00
Sarah Hoffmann	684378722c	Merge pull request #2171 from lonvia/update-vagrant-scripts-for-make-install ... Update vagrant scripts for make install	2021-02-16 14:42:38 +01:00
Sarah Hoffmann	286a686f88	switch vagrant scripts to make install	2021-02-16 12:04:34 +01:00
Sarah Hoffmann	7360e6c5df	use file copy on older cmake to install osm2pgsql ... Fixes #2170.	2021-02-16 11:06:14 +01:00
Sarah Hoffmann	fbe7be760b	ignore failure to get replication date	2021-02-14 12:17:30 +01:00
marc tobias	a3ce89aeff	documentation: 2020 TIGER data got released	2021-02-12 23:57:12 +01:00
Sarah Hoffmann	6a7e0d652b	Merge pull request #2164 from lonvia/add-make-install ... Make Nominatim installable	2021-02-12 10:32:26 +01:00
Sarah Hoffmann	7cc4c53adb	always return 0 for updates unless there is an error ... This is more in line with previous behavioru than returning a status code when no updates are available.	2021-02-11 10:33:49 +01:00
Sarah Hoffmann	24b13a7a87	docs: adapt check-database command	2021-02-10 21:55:04 +01:00
Sarah Hoffmann	b6c2dbf69c	actions: remove install directories before import ... This ensures that any dangling references to the build or source directory are caught by the CI.	2021-02-10 17:59:52 +01:00
Sarah Hoffmann	0e0e9a6809	need test database for analysing cli test	2021-02-10 16:19:51 +01:00
Sarah Hoffmann	ed60154552	actions: test import with installed version of Nominatim	2021-02-10 16:17:52 +01:00
Sarah Hoffmann	85589cf7dc	add 'make install' to installation instructions	2021-02-10 11:15:21 +01:00
Sarah Hoffmann	99dcd10d3f	test for existance of country grid in cmake already ... Given that the file potentially gets installed, it needs to be present during build time already. Checking during the import is too late.	2021-02-10 10:40:36 +01:00
Sarah Hoffmann	745ae02f47	make installation targets conditional to what is built	2021-02-10 10:04:07 +01:00
Sarah Hoffmann	b6bd11f292	add make install target ... Installation includes PHP andPython libraries, settings, the basic country data, the postgresql module and our custom version of osm2pgsql. The latter is installed in our private library directory so that it does not get in the way of a potentially installed osm2pgsql from the distribution.	2021-02-09 21:04:42 +01:00
Sarah Hoffmann	c60a0784ea	adapt unit tests to new directory structure	2021-02-09 20:13:00 +01:00
Sarah Hoffmann	3cb6f3e460	use DataDir constant for data only ... So far the data directory constant has pointed to the source directory to be usable with different subdirectories. Now only the data subdirectory itself is being used with the constant, so point to the directory directly.	2021-02-09 20:04:08 +01:00
Sarah Hoffmann	de37dc9300	forgot to replace one occurence of sql_dir	2021-02-09 19:32:05 +01:00