When applying raw (no gzip) patches, `mbtiles` was trying to un-gzip
them first. Now handles it properly. Also adds a number of tests to catch these cases.
* `mbtiles diff` now has an additional `--patch-type` param with
`whole`, `bin-diff-raw` and `bin-diff-gz` values:
* `whole` stores different tiles as before - as whole tiles in the
`tiles` table
* `bin-diff-raw` computes binary difference between tiles, and stores
them as brotli-encoded value in a `bsdiffraw` table, together with a
`xxh3_64` hash of the tile as it will be stored after patching
* `bin-diff-gz` same as `bin-diff-raw`, but assumes the tiles are
gzip-compressed, so it uncompresses them before comparing. The `xxh3_64`
stores the hash of the uncompressed tile. The data will be stored in the
`bsdiffrawgz` table (identical structure with above)
* `mbtiles copy --apply-patch` will automatically detect if
`bsdiffrawgz` or `bsdiffraw` tables exist, and will use binary patching.
* `mbtiles apply-patch` does not support binary patching yet
* `mbtiles copy --diff-with-file ... --patch-type ...` is an alias to
`mbtiles diff --patch-type ...`
* Install `nasm` when running CI on Windows (required by rustls new
faster crypto-provider)
* In integration testing, wrap connection in mutex due to `rstest` now
requiring `once` fixtures to be `Sync`
* Upgrade to Brotli v4, and disable FFI due to compilation conflict
* Some small test cleanups
* A bump to dependencies
* Disable sprite generation test - PNGs tests are too flaky
* Add an mbtiles copy test for empty diffs
* `mbtiles update` now allows different types of zoom updates - reset to
content, grow only, or skip (dry run)
* `martin-cp` will now update (grow-only) metadata zooms
Addresses a few concerns in the #1081
PMTiles is a web-optimized format, allowing the actual file to be read
with HTTP range requests. Supporting this use case instantly allows
Martin to function as a lambda executable accessing PMTiles, but without
any significant investment into devops or hosting large file.
PMTiles config now also allows `http` and `https` protocol.
```
# Publish PMTiles files
pmtiles:
paths:
# specific pmtiles file will be published as mypmtiles source
# (use last portion of the URL without extension)
- http://example.org/path/to/mypmtiles.pmtiles
sources:
# named source matching source name to a single file
pm-src1: https://example.org/path/to/some_pmtiles.pmtiles
```
fixes#884
---------
Co-authored-by: Kyle Slugg-Urbino <35903887+kyleslugg@users.noreply.github.com>
* BREAKING: `martin-cp` will now set `format=pbf` instead of `mvt`. This
is what QGIS and possibly others expect, and this is what tools like
tilelive generates.
* `martin-cp` sets `minzoom` and `maxzoom` metadata values based on the
zoom parameters
* Add `mbtiles meta-update` command to refresh zoom levels based on the
present tiles.
Partially addresses items in #1081
