mirror of
https://github.com/osm-search/Nominatim.git
synced 2024-09-11 19:17:17 +03:00
update minimum required RAM to 64GB
Also adds more background explanation on time and RAM use, as well as a hint that github issues are not good for installation support.
This commit is contained in:
parent
000fe3ddff
commit
88fab44006
@ -15,9 +15,9 @@ Please make sure to add the following information:
|
||||
* the result you are getting
|
||||
* the expected result, preferably a link to the OSM object you want to find,
|
||||
otherwise an address that is as precise as possible
|
||||
|
||||
To get the link to the OSM object, you can try the following:
|
||||
|
||||
|
||||
To get the link to the OSM object, you can try the following:
|
||||
|
||||
* go to https://openstreetmap.org
|
||||
* zoom to the area of the map where you expect the result and
|
||||
zoom in as much as possible
|
||||
@ -26,7 +26,7 @@ Please make sure to add the following information:
|
||||
* find the object of interest in the list that appears on the left side
|
||||
* click on the object and report the URL back that the browser shows
|
||||
|
||||
### When Reporting Problems with your Installation...
|
||||
### When Reporting Bugs...
|
||||
|
||||
Please add the following information to your issue:
|
||||
|
||||
@ -38,6 +38,9 @@ Please add the following information to your issue:
|
||||
if you run from the git repo, the output of `git rev-parse HEAD`)
|
||||
* (if applicable) exact command line of the command that was causing the issue
|
||||
|
||||
Bug reports that do not include extensive information about your system,
|
||||
about the problem and about what you have been trying to debug the problem
|
||||
will be closed.
|
||||
|
||||
## Workflow for Pull Requests
|
||||
|
||||
|
@ -19,6 +19,14 @@ https://nominatim.org/release-docs/develop/ .
|
||||
Installation
|
||||
============
|
||||
|
||||
**Nominatim is a complex piece of software and runs in a complex environment.
|
||||
Installing and running Nominatim is something for experienced system
|
||||
administrators only who can do some trouble-shooting themselves. We are sorry,
|
||||
but we can not provide installation support. We are all doing this in our free
|
||||
time and there is just so much of that time to go around. Do not open issues in
|
||||
our bug tracker if you need help. You can ask questions on the mailing list
|
||||
(see below) or on [help.openstreetmap.org](https://help.openstreetmap.org/).**
|
||||
|
||||
The latest stable release can be downloaded from https://nominatim.org.
|
||||
There you can also find [installation instructions for the release](https://nominatim.org/release-docs/latest/admin/Installation).
|
||||
|
||||
|
@ -29,7 +29,7 @@ Add to your `settings/local.php`:
|
||||
@define('CONST_Osm2pgsql_Flatnode_File', '/path/to/flatnode.file');
|
||||
|
||||
Replace the second part with a suitable path on your system and make sure
|
||||
the directory exists. There should be at least 40GB of free space.
|
||||
the directory exists. There should be at least 64GB of free space.
|
||||
|
||||
## Downloading additional data
|
||||
|
||||
@ -62,7 +62,7 @@ involve a GB or US postcode. This data can be optionally downloaded:
|
||||
|
||||
In its default setup Nominatim is configured to import the full OSM data
|
||||
set for the entire planet. Such a setup requires a powerful machine with
|
||||
at least 32GB of RAM and around 800GB of SSD hard disks. Depending on your
|
||||
at least 64GB of RAM and around 800GB of SSD hard disks. Depending on your
|
||||
use case there are various ways to reduce the amount of data imported. This
|
||||
section discusses these methods. They can also be combined.
|
||||
|
||||
@ -124,7 +124,7 @@ The style can be changed with the configuration `CONST_Import_Style`.
|
||||
To give you an idea of the impact of using the different styles, the table
|
||||
below gives rough estimates of the final database size after import of a
|
||||
2018 planet and after using the `--drop` option. It also shows the time
|
||||
needed for the import on a machine with 32GB RAM, 4 CPUS and SSDs. Note that
|
||||
needed for the import on a machine with 64GB RAM, 4 CPUS and SSDs. Note that
|
||||
the given sizes are just an estimate meant for comparison of style requirements.
|
||||
Your planet import is likely to be larger as the OSM data grows with time.
|
||||
|
||||
@ -135,7 +135,7 @@ street | 42h | 400 GB | 180 GB
|
||||
address | 59h | 500 GB | 260 GB
|
||||
full | 80h | 575 GB | 300 GB
|
||||
|
||||
You can also customize the styles further. For an description of the
|
||||
You can also customize the styles further. For a description of the
|
||||
style format see [the development section](../develop/Import.md).
|
||||
|
||||
## Initial import of the data
|
||||
@ -147,18 +147,44 @@ Download the data to import and load the data with the following command
|
||||
from the build directory:
|
||||
|
||||
```sh
|
||||
./utils/setup.php --osm-file <data file> --all [--osm2pgsql-cache 28000] 2>&1 | tee setup.log
|
||||
./utils/setup.php --osm-file <data file> --all 2>&1 | tee setup.log
|
||||
```
|
||||
|
||||
The `--osm2pgsql-cache` parameter is optional but strongly recommended for
|
||||
planet imports. It sets the node cache size for the osm2pgsql import part
|
||||
(see `-C` parameter in osm2pgsql help). As a rule of thumb, this should be
|
||||
about the same size as the file you are importing but never more than
|
||||
2/3 of RAM available. If your machine starts swapping reduce the size.
|
||||
***Note for full planet imports:*** Even on a perfectly configured machine
|
||||
the import of a full planet takes at least 2 days. Once you see messages
|
||||
with `Rank .. ETA` appear, the indexing process has started. This part takes
|
||||
the most time. There are 30 ranks to process. Rank 26 and 30 are the most complex.
|
||||
They take each about a third of the total import time. If you have not reached
|
||||
rank 26 after two days of import, it is worth revisiting your system
|
||||
configuration as it may not be optimal for the import.
|
||||
|
||||
Computing word frequency for search terms can improve the performance of
|
||||
forward geocoding in particular under high load as it helps PostgreSQL's query
|
||||
planner to make the right decisions. To recompute word counts run:
|
||||
### Notes on memory usage
|
||||
|
||||
In the first step of the import Nominatim uses osm2pgsql to load the OSM data
|
||||
into the PostgreSQL database. This step is very demanding in terms of RAM usage.
|
||||
osm2pgsql and PostgreSQL are running in parallel at this point. PostgreSQL
|
||||
blockes at least the part of RAM that has been configured with the
|
||||
`shared_buffers` parameter during [PostgreSQL tuning](Installation#PostgreSQL_tuning)
|
||||
and needs some memory on top of that. osm2pgsql needs at least 2GB of RAM for
|
||||
its internal data structures, potentially more when it has to process very large
|
||||
relations. In addition it needs to maintain a cache for node locations. The size
|
||||
of this cache can be configured with the parameter `--osm2pgsql-cache`.
|
||||
|
||||
When importing with a flatnode file, it is best to disable the node cache
|
||||
completely and leave the memory for the flatnode file. Nominatim will do this
|
||||
by default, so you do not need to configure anything in this case.
|
||||
|
||||
For imports without a flatnode file, set `--osm2pgsql-cache` approximately to
|
||||
the size of the OSM pbf file (in MB) you are importing. Make sure you leave
|
||||
enough RAM for PostgreSQL and osm2pgsql as mentioned above. If the system starts
|
||||
swapping or you are getting out-of-memory errors, reduce the cache size or
|
||||
even consider using a flatnode file.
|
||||
|
||||
## Tuning the database
|
||||
|
||||
Accurate word frequency information for search terms helps PostgreSQL's query
|
||||
planner to make the right decisions. Recomputing them can improve the performance
|
||||
of forward geocoding in particular under high load. To recompute word counts run:
|
||||
|
||||
```sh
|
||||
./utils/update.php --recompute-word-counts
|
||||
@ -176,7 +202,8 @@ you also need to enable these key phrases like this:
|
||||
./utils/specialphrases.php --wiki-import > specialphrases.sql
|
||||
psql -d nominatim -f specialphrases.sql
|
||||
|
||||
Note that this command downloads the phrases from the wiki link above.
|
||||
Note that this command downloads the phrases from the wiki link above. You
|
||||
need internet access for the step.
|
||||
|
||||
|
||||
## Installing Tiger housenumber data for the US
|
||||
|
@ -55,15 +55,15 @@ For running continuous updates:
|
||||
### Hardware
|
||||
|
||||
A minimum of 2GB of RAM is required or installation will fail. For a full
|
||||
planet import 32GB of RAM or more are strongly recommended.
|
||||
planet import 64GB of RAM or more are strongly recommended. Do not report
|
||||
out of memory problems if you have less than 64GB RAM.
|
||||
|
||||
For a full planet install you will need at least 700GB of hard disk space
|
||||
For a full planet install you will need at least 800GB of hard disk space
|
||||
(take into account that the OSM database is growing fast). SSD disks
|
||||
will help considerably to speed up import and queries.
|
||||
|
||||
On a 6-core machine with 32GB RAM and SSDs the import of a full planet takes
|
||||
a bit more than 2 days. Without SSDs 7-8 days are more realistic.
|
||||
|
||||
Even on a well configured machine the import of a full planet takes
|
||||
at least 2 days. Without SSDs 7-8 days are more realistic.
|
||||
|
||||
## Setup of the server
|
||||
|
||||
@ -73,17 +73,21 @@ You might want to tune your PostgreSQL installation so that the later steps
|
||||
make best use of your hardware. You should tune the following parameters in
|
||||
your `postgresql.conf` file.
|
||||
|
||||
shared_buffers (2GB)
|
||||
maintenance_work_mem (10GB)
|
||||
work_mem (50MB)
|
||||
effective_cache_size (24GB)
|
||||
shared_buffers = 2GB
|
||||
maintenance_work_mem = (10GB)
|
||||
autovacuum_work_mem = 2GB
|
||||
work_mem = (50MB)
|
||||
effective_cache_size = (24GB)
|
||||
synchronous_commit = off
|
||||
checkpoint_segments = 100 # only for postgresql <= 9.4
|
||||
max_wal_size = 1GB # postgresql > 9.4
|
||||
checkpoint_timeout = 10min
|
||||
checkpoint_completion_target = 0.9
|
||||
|
||||
The numbers in brackets behind some parameters seem to work fine for
|
||||
32GB RAM machine. Adjust to your setup.
|
||||
64GB RAM machine. Adjust to your setup. A higher number for `max_wal_size`
|
||||
means that PostgreSQL needs to run checkpoints less often but it does require
|
||||
the additional space on your disk.
|
||||
|
||||
For the initial import, you should also set:
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user