Commit Graph

10 Commits

Author SHA1 Message Date
Yuri Astrakhan
8b34cd374c
Add metadat copy/apply-diff, new testing framework (#921)
* Fix metadata copying
* Introduce a new metadata field `agg_tiles_hash_after_apply` for diff
files
* Added a lot of new info and debug logging
* Simplified Copying interface - not much value in having all the
complex builder pattern here it seems, might as well use a simple
object.

## Testing
* Generate SQLite DBs in memory on the fly to validate just what we need
* Use `insta` for validating DB content

There is now a function `dump(connection) -> Vec<Entry>` to dump the
content of the entire SQLite DB into text with `serde`. At many steps
through the testing, the DB content is validated with the corresponding
.snap file with `insta` crate (which makes this process mega-simple,
including a simple way to "bless" (update) any changes).

## Discovered bugs
* Seems like normalized files do not get copied properly - they contain
extras that should be removed.
2023-10-10 11:10:17 -04:00
Yuri Astrakhan
0459d3f748
add encoding=UTF8 pragma, refactoring (#919) 2023-10-03 21:20:41 -04:00
Yuri Astrakhan
6b7bcabe49
Cleanup mbtiles, rename TileCopierOptions, testing (#916)
* Rename `TileCopierOptions` -> `TileCopier`
* remove a few un-needed sqlite open to detect mbtiles type
* move `open_and_detect_type` to `MBTiles`
* add `attach_to` to `MBTiles`
* move various table creation fn to mbtiles_queries file
* a few sql format
2023-10-01 23:28:43 -04:00
Yuri Astrakhan
18504ff2cb
Update agg_tiles_hash docs and minor bug (#901)
* Do not open the same mbtiles file more than once at the same time:
reuse the (unsafe) sqlite handle to create a `RusqliteConnection`
instead.
* The copying should set `agg_tiles_hash` in all cases because now it
uses the always available `tiles` table/view.
* a few minor cleanups and renames related to that
2023-09-28 20:12:11 +02:00
Yuri Astrakhan
2679ad2b63 Minor code cleanup 2023-09-05 12:17:44 -04:00
rstanciu
875a942fcf
Add integrity and global hash check to validation (#851)
Co-authored-by: rstanciu <rstanciu@rivian.com>
2023-08-31 03:40:40 +00:00
rstanciu
3e2dceb4bf
Add ability to set mbtiles metadata value (#817)
* Add cli command `meta-set` to set metadata `value`

---------

Co-authored-by: rstanciu <rstanciu@rivian.com>
2023-08-17 11:36:46 -07:00
rstanciu
720b682b7e
Add flat hashing table format (#808)
* Add `MbtType::FlatWithHash`
* Support copying, diffing and applying diffs to and from any
`MbtTypes`s
* Support validating tile data if hash is contained in `*.mbtiles` file
(i.e it is of `MbtType::FlatWithHash` or `MbtType::Normalized`)

---------

Co-authored-by: rstanciu <rstanciu@rivian.com>
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
2023-08-16 10:22:01 -07:00
rstanciu
42c2df46ef
Add ability to copy to existing mbtiles files (#778)
* Allow the `dst` in `mbtiles copy <src> <dst>` command to already
contain data, merging tilesets
* Add `mbtiles copy --on-duplicate (override|ignore|abort)`

---------

Co-authored-by: rstanciu <rstanciu@rivian.com>
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
2023-07-27 18:54:49 +00:00
Yuri Astrakhan
144e10ff20
Upgrade sqlx to v0.7 (#743)
This is mostly a noop, just updating to the new way of storing cached
queries and a few other changes per
https://github.com/launchbadge/sqlx/blob/main/CHANGELOG.md#070---2023-06-30

Note that this uses some of the code from #729 (thanks @upsicleclown !)

A minor unrelated change - a reformat of `martin/release.toml`
2023-07-04 08:05:23 -04:00