maplibre/martin
1
1
Fork 0
mirror of https://github.com/maplibre/martin.git synced 2024-12-18 20:31:54 +03:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

chore: rename agg_tiles_hash_in_diff to agg_tiles_hash_after_apply (#1243)

Browse Source 
This is just a documentation change - the actual value stored in the
mbtiles was always `agg_tiles_hash_after_apply`, but accidentally the
constant in the code was not renamed, nor were the docs updated.
This commit is contained in:
Yuri Astrakhan 2024-03-11 04:29:50 -04:00 committed by GitHub
parent db094400bb
commit 60ae6cf168
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 31 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
docs/src/mbtiles-diff.md
View File

@ -2,9 +2,16 @@
## `mbtiles diff`
Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can be [applied](#mbtiles-apply-patch) to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire modified dataset. The delta file will contain all tiles that are different between the two files (modifications, insertions, and deletions as `NULL` values), for both the tile and metadata tables.
Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can
be [applied](#mbtiles-apply-patch) to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire
modified dataset. The delta file will contain all tiles that are different between the two files (modifications,
insertions, and deletions as `NULL` values), for both the tile and metadata tables.
There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_in_diff`, and a new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff` command will automatically rename the `agg_tiles_hash_in_diff` value back to `agg_tiles_hash` when applying the diff.
There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_after_apply`, and a
new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff
file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff`
command will automatically rename the `agg_tiles_hash_after_apply` value back to `agg_tiles_hash` when applying the
diff.
```shell
# This command will compare `file1.mbtiles` and `file2.mbtiles`, and generate a new diff file `diff.mbtiles`.
@ -24,9 +31,15 @@ mbtiles validate file2a.mbtiles
## `mbtiles apply-patch`
Apply the diff file generated with the `mbtiles diff` command above to an MBTiles file. The diff file can be applied to the `src_file.mbtiles` that has been previously downloaded to avoid copying/transmitting the entire modified dataset again. The `src_file.mbtiles` will modified in-place. It is also possible to apply the diff file while copying the source file to a new destination file, by using the [`mbtiles copy --apply-patch`](mbtiles-copy.md#mbtiles-copy---apply-patch) command.
Apply the diff file generated with the `mbtiles diff` command above to an MBTiles file. The diff file can be applied to
the `src_file.mbtiles` that has been previously downloaded to avoid copying/transmitting the entire modified dataset
again. The `src_file.mbtiles` will modified in-place. It is also possible to apply the diff file while copying the
source file to a new destination file, by using
the [`mbtiles copy --apply-patch`](mbtiles-copy.md#mbtiles-copy---apply-patch) command.
Note that the `agg_tiles_hash_in_diff` metadata value will be renamed to `agg_tiles_hash` when applying the diff. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied.
Note that the `agg_tiles_hash_after_apply` metadata value will be renamed to `agg_tiles_hash` when applying the diff.
This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be
different after the diff is applied.
```shell
mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
@ -34,7 +47,11 @@ mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
#### Applying diff with SQLite
Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements. Note that this does not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied.
Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles
from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles
from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is
passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements. Note that this does
not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied.
```shell
sqlite3 src_file.mbtiles \

10
mbtiles/src/copier.rs
View File

@ -17,7 +17,7 @@ use crate::queries::{
use crate::MbtType::{Flat, FlatWithHash, Normalized};
use crate::{
invert_y_value, reset_db_settings, CopyType, MbtError, MbtType, MbtTypeCli, Mbtiles,
AGG_TILES_HASH, AGG_TILES_HASH_IN_DIFF,
AGG_TILES_HASH, AGG_TILES_HASH_AFTER_APPLY,
};
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, EnumDisplay)]
@ -233,13 +233,13 @@ impl MbtileCopierInt {
if dif.is_some() {
// Insert all rows from diffDb.metadata if they do not exist or are different in sourceDb.metadata.
// Also insert all names from sourceDb.metadata that do not exist in diffDb.metadata, with their value set to NULL.
// Rename agg_tiles_hash to agg_tiles_hash_in_diff because agg_tiles_hash will be auto-added later
// Rename agg_tiles_hash to agg_tiles_hash_after_apply because agg_tiles_hash will be auto-added later
if self.options.diff_with_file.is_some() {
// Include agg_tiles_hash value even if it is the same because we will still need it when applying the diff
sql = format!(
"
INSERT {on_dupl} INTO metadata (name, value)
SELECT IIF(name = '{AGG_TILES_HASH}','{AGG_TILES_HASH_IN_DIFF}', name) as name
SELECT IIF(name = '{AGG_TILES_HASH}','{AGG_TILES_HASH_AFTER_APPLY}', name) as name
, value
FROM (
SELECT COALESCE(difMD.name, srcMD.name) as name
@ -248,14 +248,14 @@ impl MbtileCopierInt {
ON srcMD.name = difMD.name
WHERE srcMD.value != difMD.value OR srcMD.value ISNULL OR difMD.value ISNULL OR srcMD.name = '{AGG_TILES_HASH}'
) joinedMD
WHERE name != '{AGG_TILES_HASH_IN_DIFF}'"
WHERE name != '{AGG_TILES_HASH_AFTER_APPLY}'"
);
debug!("Copying metadata, taking into account diff file with {sql}");
} else {
sql = format!(
"
INSERT {on_dupl} INTO metadata (name, value)
SELECT IIF(name = '{AGG_TILES_HASH_IN_DIFF}','{AGG_TILES_HASH}', name) as name
SELECT IIF(name = '{AGG_TILES_HASH_AFTER_APPLY}','{AGG_TILES_HASH}', name) as name
, value
FROM (
SELECT COALESCE(srcMD.name, difMD.name) as name

2
mbtiles/src/lib.rs
View File

@ -32,7 +32,7 @@ pub use update::UpdateZoomType;
mod validation;
pub use validation::{
calc_agg_tiles_hash, AggHashType, IntegrityCheckType, MbtType, AGG_TILES_HASH,
AGG_TILES_HASH_IN_DIFF,
AGG_TILES_HASH_AFTER_APPLY,
};
/// `MBTiles` uses a TMS (Tile Map Service) scheme for its tile coordinates (inverted along the Y axis).

4
mbtiles/src/patcher.rs
View File

@ -5,7 +5,7 @@ use sqlx::query;
use crate::queries::detach_db;
use crate::MbtType::{Flat, FlatWithHash, Normalized};
use crate::{MbtResult, MbtType, Mbtiles, AGG_TILES_HASH, AGG_TILES_HASH_IN_DIFF};
use crate::{MbtResult, MbtType, Mbtiles, AGG_TILES_HASH, AGG_TILES_HASH_AFTER_APPLY};
pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<()> {
let base_mbt = Mbtiles::new(base_file)?;
@ -53,7 +53,7 @@ pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<(
query(&format!(
"
INSERT OR REPLACE INTO metadata (name, value)
SELECT IIF(name = '{AGG_TILES_HASH_IN_DIFF}', '{AGG_TILES_HASH}', name) as name,
SELECT IIF(name = '{AGG_TILES_HASH_AFTER_APPLY}', '{AGG_TILES_HASH}', name) as name,
value
FROM patchDb.metadata
WHERE name NOTNULL AND name != '{AGG_TILES_HASH}';"

2
mbtiles/src/validation.rs
View File

@ -26,7 +26,7 @@ pub const AGG_TILES_HASH: &str = "agg_tiles_hash";
/// Metadata key for a diff file,
/// describing the eventual [`AGG_TILES_HASH`] value once the diff is applied
pub const AGG_TILES_HASH_IN_DIFF: &str = "agg_tiles_hash_after_apply";
pub const AGG_TILES_HASH_AFTER_APPLY: &str = "agg_tiles_hash_after_apply";
#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, EnumDisplay, Serialize)]
#[enum_display(case = "Kebab")]