chore: rename agg_tiles_hash_in_diff to agg_tiles_hash_after_apply (#1243)

This is just a documentation change - the actual value stored in the
mbtiles was always `agg_tiles_hash_after_apply`, but accidentally the
constant in the code was not renamed, nor were the docs updated.
This commit is contained in:
Yuri Astrakhan 2024-03-11 04:29:50 -04:00 committed by GitHub
parent db094400bb
commit 60ae6cf168
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 31 additions and 14 deletions

View File

@ -2,9 +2,16 @@
## `mbtiles diff` ## `mbtiles diff`
Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can be [applied](#mbtiles-apply-patch) to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire modified dataset. The delta file will contain all tiles that are different between the two files (modifications, insertions, and deletions as `NULL` values), for both the tile and metadata tables. Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can
be [applied](#mbtiles-apply-patch) to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire
modified dataset. The delta file will contain all tiles that are different between the two files (modifications,
insertions, and deletions as `NULL` values), for both the tile and metadata tables.
There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_in_diff`, and a new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff` command will automatically rename the `agg_tiles_hash_in_diff` value back to `agg_tiles_hash` when applying the diff. There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_after_apply`, and a
new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff
file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff`
command will automatically rename the `agg_tiles_hash_after_apply` value back to `agg_tiles_hash` when applying the
diff.
```shell ```shell
# This command will compare `file1.mbtiles` and `file2.mbtiles`, and generate a new diff file `diff.mbtiles`. # This command will compare `file1.mbtiles` and `file2.mbtiles`, and generate a new diff file `diff.mbtiles`.
@ -24,9 +31,15 @@ mbtiles validate file2a.mbtiles
## `mbtiles apply-patch` ## `mbtiles apply-patch`
Apply the diff file generated with the `mbtiles diff` command above to an MBTiles file. The diff file can be applied to the `src_file.mbtiles` that has been previously downloaded to avoid copying/transmitting the entire modified dataset again. The `src_file.mbtiles` will modified in-place. It is also possible to apply the diff file while copying the source file to a new destination file, by using the [`mbtiles copy --apply-patch`](mbtiles-copy.md#mbtiles-copy---apply-patch) command. Apply the diff file generated with the `mbtiles diff` command above to an MBTiles file. The diff file can be applied to
the `src_file.mbtiles` that has been previously downloaded to avoid copying/transmitting the entire modified dataset
again. The `src_file.mbtiles` will modified in-place. It is also possible to apply the diff file while copying the
source file to a new destination file, by using
the [`mbtiles copy --apply-patch`](mbtiles-copy.md#mbtiles-copy---apply-patch) command.
Note that the `agg_tiles_hash_in_diff` metadata value will be renamed to `agg_tiles_hash` when applying the diff. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. Note that the `agg_tiles_hash_after_apply` metadata value will be renamed to `agg_tiles_hash` when applying the diff.
This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be
different after the diff is applied.
```shell ```shell
mbtiles apply-patch src_file.mbtiles diff_file.mbtiles mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
@ -34,7 +47,11 @@ mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
#### Applying diff with SQLite #### Applying diff with SQLite
Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements. Note that this does not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied. Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles
from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles
from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is
passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements. Note that this does
not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied.
```shell ```shell
sqlite3 src_file.mbtiles \ sqlite3 src_file.mbtiles \

View File

@ -17,7 +17,7 @@ use crate::queries::{
use crate::MbtType::{Flat, FlatWithHash, Normalized}; use crate::MbtType::{Flat, FlatWithHash, Normalized};
use crate::{ use crate::{
invert_y_value, reset_db_settings, CopyType, MbtError, MbtType, MbtTypeCli, Mbtiles, invert_y_value, reset_db_settings, CopyType, MbtError, MbtType, MbtTypeCli, Mbtiles,
AGG_TILES_HASH, AGG_TILES_HASH_IN_DIFF, AGG_TILES_HASH, AGG_TILES_HASH_AFTER_APPLY,
}; };
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, EnumDisplay)] #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, EnumDisplay)]
@ -233,13 +233,13 @@ impl MbtileCopierInt {
if dif.is_some() { if dif.is_some() {
// Insert all rows from diffDb.metadata if they do not exist or are different in sourceDb.metadata. // Insert all rows from diffDb.metadata if they do not exist or are different in sourceDb.metadata.
// Also insert all names from sourceDb.metadata that do not exist in diffDb.metadata, with their value set to NULL. // Also insert all names from sourceDb.metadata that do not exist in diffDb.metadata, with their value set to NULL.
// Rename agg_tiles_hash to agg_tiles_hash_in_diff because agg_tiles_hash will be auto-added later // Rename agg_tiles_hash to agg_tiles_hash_after_apply because agg_tiles_hash will be auto-added later
if self.options.diff_with_file.is_some() { if self.options.diff_with_file.is_some() {
// Include agg_tiles_hash value even if it is the same because we will still need it when applying the diff // Include agg_tiles_hash value even if it is the same because we will still need it when applying the diff
sql = format!( sql = format!(
" "
INSERT {on_dupl} INTO metadata (name, value) INSERT {on_dupl} INTO metadata (name, value)
SELECT IIF(name = '{AGG_TILES_HASH}','{AGG_TILES_HASH_IN_DIFF}', name) as name SELECT IIF(name = '{AGG_TILES_HASH}','{AGG_TILES_HASH_AFTER_APPLY}', name) as name
, value , value
FROM ( FROM (
SELECT COALESCE(difMD.name, srcMD.name) as name SELECT COALESCE(difMD.name, srcMD.name) as name
@ -248,14 +248,14 @@ impl MbtileCopierInt {
ON srcMD.name = difMD.name ON srcMD.name = difMD.name
WHERE srcMD.value != difMD.value OR srcMD.value ISNULL OR difMD.value ISNULL OR srcMD.name = '{AGG_TILES_HASH}' WHERE srcMD.value != difMD.value OR srcMD.value ISNULL OR difMD.value ISNULL OR srcMD.name = '{AGG_TILES_HASH}'
) joinedMD ) joinedMD
WHERE name != '{AGG_TILES_HASH_IN_DIFF}'" WHERE name != '{AGG_TILES_HASH_AFTER_APPLY}'"
); );
debug!("Copying metadata, taking into account diff file with {sql}"); debug!("Copying metadata, taking into account diff file with {sql}");
} else { } else {
sql = format!( sql = format!(
" "
INSERT {on_dupl} INTO metadata (name, value) INSERT {on_dupl} INTO metadata (name, value)
SELECT IIF(name = '{AGG_TILES_HASH_IN_DIFF}','{AGG_TILES_HASH}', name) as name SELECT IIF(name = '{AGG_TILES_HASH_AFTER_APPLY}','{AGG_TILES_HASH}', name) as name
, value , value
FROM ( FROM (
SELECT COALESCE(srcMD.name, difMD.name) as name SELECT COALESCE(srcMD.name, difMD.name) as name

View File

@ -32,7 +32,7 @@ pub use update::UpdateZoomType;
mod validation; mod validation;
pub use validation::{ pub use validation::{
calc_agg_tiles_hash, AggHashType, IntegrityCheckType, MbtType, AGG_TILES_HASH, calc_agg_tiles_hash, AggHashType, IntegrityCheckType, MbtType, AGG_TILES_HASH,
AGG_TILES_HASH_IN_DIFF, AGG_TILES_HASH_AFTER_APPLY,
}; };
/// `MBTiles` uses a TMS (Tile Map Service) scheme for its tile coordinates (inverted along the Y axis). /// `MBTiles` uses a TMS (Tile Map Service) scheme for its tile coordinates (inverted along the Y axis).

View File

@ -5,7 +5,7 @@ use sqlx::query;
use crate::queries::detach_db; use crate::queries::detach_db;
use crate::MbtType::{Flat, FlatWithHash, Normalized}; use crate::MbtType::{Flat, FlatWithHash, Normalized};
use crate::{MbtResult, MbtType, Mbtiles, AGG_TILES_HASH, AGG_TILES_HASH_IN_DIFF}; use crate::{MbtResult, MbtType, Mbtiles, AGG_TILES_HASH, AGG_TILES_HASH_AFTER_APPLY};
pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<()> { pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<()> {
let base_mbt = Mbtiles::new(base_file)?; let base_mbt = Mbtiles::new(base_file)?;
@ -53,7 +53,7 @@ pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<(
query(&format!( query(&format!(
" "
INSERT OR REPLACE INTO metadata (name, value) INSERT OR REPLACE INTO metadata (name, value)
SELECT IIF(name = '{AGG_TILES_HASH_IN_DIFF}', '{AGG_TILES_HASH}', name) as name, SELECT IIF(name = '{AGG_TILES_HASH_AFTER_APPLY}', '{AGG_TILES_HASH}', name) as name,
value value
FROM patchDb.metadata FROM patchDb.metadata
WHERE name NOTNULL AND name != '{AGG_TILES_HASH}';" WHERE name NOTNULL AND name != '{AGG_TILES_HASH}';"

View File

@ -26,7 +26,7 @@ pub const AGG_TILES_HASH: &str = "agg_tiles_hash";
/// Metadata key for a diff file, /// Metadata key for a diff file,
/// describing the eventual [`AGG_TILES_HASH`] value once the diff is applied /// describing the eventual [`AGG_TILES_HASH`] value once the diff is applied
pub const AGG_TILES_HASH_IN_DIFF: &str = "agg_tiles_hash_after_apply"; pub const AGG_TILES_HASH_AFTER_APPLY: &str = "agg_tiles_hash_after_apply";
#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, EnumDisplay, Serialize)] #[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, EnumDisplay, Serialize)]
#[enum_display(case = "Kebab")] #[enum_display(case = "Kebab")]