Commit Graph

11 Commits

Author SHA1 Message Date
Yuri Astrakhan
6f08aa9465
Fix mbtiles validation, CI, and logging (#903) 2023-09-29 14:37:18 -04:00
Yuri Astrakhan
1a386b7071 Fix unit tests to use blobs only 2023-09-28 16:57:50 -04:00
Yuri Astrakhan
0fce9641c0
A few cleanups on tile hashing (#861)
* Rename `global_hash` to `agg_tiles_hash`

This is still a big sticking point: what should be the name for the
metadata key for this value? The value represents the hash of all
`z,x,y,tile` over all rows of the `tiles` table (or view). Should it
include `md5` in its name, or should the hash be auto-detected by its
length? (details in #856)

* Generate it based on `tiles` table/view
* validate or generate, but not both (it will always fail otherwise)
* break up logic for per-tile, total, and integrity checks
* delete unused sqlx prep file
2023-09-05 16:25:59 -04:00
rstanciu
875a942fcf
Add integrity and global hash check to validation (#851)
Co-authored-by: rstanciu <rstanciu@rivian.com>
2023-08-31 03:40:40 +00:00
rstanciu
720b682b7e
Add flat hashing table format (#808)
* Add `MbtType::FlatWithHash`
* Support copying, diffing and applying diffs to and from any
`MbtTypes`s
* Support validating tile data if hash is contained in `*.mbtiles` file
(i.e it is of `MbtType::FlatWithHash` or `MbtType::Normalized`)

---------

Co-authored-by: rstanciu <rstanciu@rivian.com>
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
2023-08-16 10:22:01 -07:00
rstanciu
1342b38e75
Add apply-diff command (#747)
Add command `apply-diff` to apply diff file generated from `copy`
command
2023-07-05 21:38:03 +00:00
rstanciu
e004908722
Add diff-with-file option to copy command (#729)
* Add ability to generate diff file by specifying `--diff-with-file` to
the `copy` tool

---------

Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
2023-07-05 16:55:39 +00:00
Yuri Astrakhan
ff29530804
Fix non-deterministic CI tests (#724)
* a race condition due to duplicated webp.mbtiles and webp.pmtiles
* minor cleanup of CI/dockerignore
* dependency bump
2023-06-22 23:01:57 -04:00
Yuri Astrakhan
454d435589 Move invalid test fixture files 2023-06-03 23:06:37 -04:00
Yuri Astrakhan
e92722708f
PG TileJSON changes, add vector_layers (#584)
* make tilejson's `name` be the same as the ID of the source (even if
aliased)
* `/catalog` will always show ID, but now it will hide the `name` if it
is the same as the `id`
* make `description` be the longer version, e.g. `public.table.column`
format - not guaranteed to be stable
* make `vector_layers` have the fields auto-discovered in the PG table
* preserve the order of the serialized json fields

Fixes #583
2023-02-22 16:25:48 +00:00
Yuri Astrakhan
3fcad46500
fix (re-)compression of images (#579)
Compression middleware turned out to be hard to use for image cases - it
simply looks at the content-encoding, and if not set, tries to compress
if accepted by the client.

Instead, now individual routes are configured with either that
middleware, or for tiles, I decompress and optionally recompress if
applicable.

Now encoding is tracked separately from the tile content, making it
cleaner too. Plus lots of tests for mbtiles & pmtiles.

Fixes #577
2023-02-20 10:44:22 -05:00