PMTiles is a web-optimized format, allowing the actual file to be read
with HTTP range requests. Supporting this use case instantly allows
Martin to function as a lambda executable accessing PMTiles, but without
any significant investment into devops or hosting large file.
PMTiles config now also allows `http` and `https` protocol.
```
# Publish PMTiles files
pmtiles:
paths:
# specific pmtiles file will be published as mypmtiles source
# (use last portion of the URL without extension)
- http://example.org/path/to/mypmtiles.pmtiles
sources:
# named source matching source name to a single file
pm-src1: https://example.org/path/to/some_pmtiles.pmtiles
```
fixes#884
---------
Co-authored-by: Kyle Slugg-Urbino <35903887+kyleslugg@users.noreply.github.com>
* BREAKING: `martin-cp` will now set `format=pbf` instead of `mvt`. This
is what QGIS and possibly others expect, and this is what tools like
tilelive generates.
* `martin-cp` sets `minzoom` and `maxzoom` metadata values based on the
zoom parameters
* Add `mbtiles meta-update` command to refresh zoom levels based on the
present tiles.
Partially addresses items in #1081
Figure out and fix the bug with `tile-grid` crate or remove the crate
and implement it ourself.
---------
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
When using `martin-cp` or `mbtiles copy` into an existing file, require
the user to set `--on-duplicate (override|ignore|abort)`. This prevents
accidental writing to an existing file.
Make sure all values in the `tiles` table or view are correct:
* zoom_level is between 0 and 30 (max allowed zoom)
* the x,y values are within the limits for the corresponding zoom level
* the column type of z,x,y are all integers
* the `tile_data` is a NULL or a BLOB
Mathematics should be consolidated in the utils crate. Eventually, I
hope we will move it to [tile-grid](https://crates.io/crates/tile-grid)
or [utiles](https://crates.io/crates/utiles) crates and consolidate our
efforts there.
Also, modify earth circumference by a few centimeters (to match various
other implementations): from `.7` to `.685_578_5`. This modified a few
unit tests.
* Now by default mbtiles will not print internal warnings (e.g. that
something is taking too long to execute)
* remove mbtiles --help test - not really needed, but adds complexity
because it has to be updated every time a version changes.
* Use gzip compression for MVT tiles by default
* Allow user to set compression with `--encoding` parameter (same as
browser's header, e.g. use `br,gzip' to encode with brotli as first
choice, unless already encoded as gzip)
* Allow user to pass a query to the Postgres functions with
`--url-query`
* A bit of a cleanup for `mbtiles summary` output
This adds a new utility called `martin-cp` that allows copying any
number of tiles from a single source (or a composite source) to an
mbtiles file.
`martin-cp` is a tool for generating tiles in bulk, and save retrieved
tiles into a new or an existing MBTiles file. It can be used to generate
tiles for a large area or multiple areas. If multiple areas overlap, it
will generate tiles only once. `martin-cp` supports the same
configuration file and CLI arguments as Martin server, so it can support
all sources and even combining sources.
# MBTiles
* New `--agg-hash (update|verify|off)` flag replaces
`--update-agg-tiles-hash` (still supported, but not shown in the help
screen). This allows bypassing aggregate hash validation entirely,
without either updating or validating it.
* Simplify MBTiles SQL generation
* MBTiles now uses faster `1 << zoom` everywhere, and a dedicated TMS
inversion fn
* split up metadata insert and delete into separate fn
* consolidated schema initialization
* ensure db settings (like pragma) are always reset on new files
# Other
* Always sort JSON-serialized keys for consistency
* this affects `/catalog` key ordering, but content is the same
* Minor code cleanup
- Add separators to long literal.
- Use first() instead of get(0)
- Move some SQL-making code to their own functions
- Minor mathematics and rounding cleanup
---------
Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
* remove benchmarks - not used for now, will need to be rewritten
* move some CLI settings to a separate struct
* move config saving to a shared function
* simplify zoom level CLI parsing
* rename main.rs to be the same name as generated binary (we can have
multiple bins later)
Partial fix of #1002
* [x] Move all summary code from `mbtiles/src/mbtiles.rs` to
`mbtiles/src/summary.rs`
* [x] Move Metadata and Validation function to separate files
* [x] Remove `filename` String from the summary stats - not much point
because it is accessible from the mbtiles struct itself
---------
Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
After some thinking, it seems `mbtiles summary` (aliased as `mbtiles
info`) would be a bit better than stats. I renamed and adjusted
documentation, consolidating it in one doc page.
Other changes:
* use file system's file size, reporting 'unknown' if needed
* report page count
* moved bbox computation into a separate function
* inlined a number of things for readability
