Make sure all values in the `tiles` table or view are correct:
* zoom_level is between 0 and 30 (max allowed zoom)
* the x,y values are within the limits for the corresponding zoom level
* the column type of z,x,y are all integers
* the `tile_data` is a NULL or a BLOB
* Now by default mbtiles will not print internal warnings (e.g. that
something is taking too long to execute)
* remove mbtiles --help test - not really needed, but adds complexity
because it has to be updated every time a version changes.
* Use gzip compression for MVT tiles by default
* Allow user to set compression with `--encoding` parameter (same as
browser's header, e.g. use `br,gzip' to encode with brotli as first
choice, unless already encoded as gzip)
* Allow user to pass a query to the Postgres functions with
`--url-query`
* A bit of a cleanup for `mbtiles summary` output
# MBTiles
* New `--agg-hash (update|verify|off)` flag replaces
`--update-agg-tiles-hash` (still supported, but not shown in the help
screen). This allows bypassing aggregate hash validation entirely,
without either updating or validating it.
* Simplify MBTiles SQL generation
* MBTiles now uses faster `1 << zoom` everywhere, and a dedicated TMS
inversion fn
* split up metadata insert and delete into separate fn
* consolidated schema initialization
* ensure db settings (like pragma) are always reset on new files
# Other
* Always sort JSON-serialized keys for consistency
* this affects `/catalog` key ordering, but content is the same
* Minor code cleanup
Partial fix of #1002
* [x] Move all summary code from `mbtiles/src/mbtiles.rs` to
`mbtiles/src/summary.rs`
* [x] Move Metadata and Validation function to separate files
* [x] Remove `filename` String from the summary stats - not much point
because it is accessible from the mbtiles struct itself
---------
Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
After some thinking, it seems `mbtiles summary` (aliased as `mbtiles
info`) would be a bit better than stats. I renamed and adjusted
documentation, consolidating it in one doc page.
Other changes:
* use file system's file size, reporting 'unknown' if needed
* report page count
* moved bbox computation into a separate function
* inlined a number of things for readability
* `mbtiles apply-diff` is now `apply-patch` (old name is still
supported)
* `mbtiles copy` can now take `--apply-patch <file>` to apply the patch
while copying from source to destination. This way, the source file will
remain unmodified.
* Fix metadata copying
* Introduce a new metadata field `agg_tiles_hash_after_apply` for diff
files
* Added a lot of new info and debug logging
* Simplified Copying interface - not much value in having all the
complex builder pattern here it seems, might as well use a simple
object.
## Testing
* Generate SQLite DBs in memory on the fly to validate just what we need
* Use `insta` for validating DB content
There is now a function `dump(connection) -> Vec<Entry>` to dump the
content of the entire SQLite DB into text with `serde`. At many steps
through the testing, the DB content is validated with the corresponding
.snap file with `insta` crate (which makes this process mega-simple,
including a simple way to "bless" (update) any changes).
## Discovered bugs
* Seems like normalized files do not get copied properly - they contain
extras that should be removed.
* Add `MbtType::FlatWithHash`
* Support copying, diffing and applying diffs to and from any
`MbtTypes`s
* Support validating tile data if hash is contained in `*.mbtiles` file
(i.e it is of `MbtType::FlatWithHash` or `MbtType::Normalized`)
---------
Co-authored-by: rstanciu <rstanciu@rivian.com>
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
* Add ability to generate diff file by specifying `--diff-with-file` to
the `copy` tool
---------
Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
Copy an existing `.mbtiles` file to a new file.
```shell
mbtiles copy <src_file.mbtiles> <dst_file.mbtiles>
```
Optionally filters by zooms. Supports de-duplicated and simple mbtiles
---------
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
* Broke up martin-mbtiles into multiple files
* Made all mbtiles functions take a `SqliteExecutor` -- this way they
can be used with any SQLX connection structs - either a pool connection
or an individual non-pooled connection.
* Simplified mbtiles bin a bit - I realized there is really no need to
pretty print the output for the single value retrieval. Easier to just
dump it to console as is.
* Bump martin-mbtiles to v0.2.0
* Minor fixes in tools docs, cargo.toml, and justfile
* MBTiles tool Integration tests and release publishing
Major thanks to the
[stackoverflow](https://stackoverflow.com/questions/76394665/how-to-pass-sqlx-connection-a-mut-trait-as-a-fn-parameter-in-rust/76395111)
quick reply by @cafce25 on how to use generic sql executor!
