mirror of
https://github.com/maplibre/martin.git
synced 2024-12-24 07:22:52 +03:00
chore: rename agg_tiles_hash_in_diff to agg_tiles_hash_after_apply (#1243)
This is just a documentation change - the actual value stored in the mbtiles was always `agg_tiles_hash_after_apply`, but accidentally the constant in the code was not renamed, nor were the docs updated.
This commit is contained in:
parent
db094400bb
commit
60ae6cf168
@ -2,9 +2,16 @@
|
|||||||
|
|
||||||
## `mbtiles diff`
|
## `mbtiles diff`
|
||||||
|
|
||||||
Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can be [applied](#mbtiles-apply-patch) to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire modified dataset. The delta file will contain all tiles that are different between the two files (modifications, insertions, and deletions as `NULL` values), for both the tile and metadata tables.
|
Copy command can also be used to compare two mbtiles files and generate a delta (diff) file. The diff file can
|
||||||
|
be [applied](#mbtiles-apply-patch) to the `src_file.mbtiles` elsewhere, to avoid copying/transmitting the entire
|
||||||
|
modified dataset. The delta file will contain all tiles that are different between the two files (modifications,
|
||||||
|
insertions, and deletions as `NULL` values), for both the tile and metadata tables.
|
||||||
|
|
||||||
There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_in_diff`, and a new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff` command will automatically rename the `agg_tiles_hash_in_diff` value back to `agg_tiles_hash` when applying the diff.
|
There is one exception: `agg_tiles_hash` metadata value will be renamed to `agg_tiles_hash_after_apply`, and a
|
||||||
|
new `agg_tiles_hash` will be generated for the diff file itself. This is done to avoid confusion when applying the diff
|
||||||
|
file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied. The `apply-diff`
|
||||||
|
command will automatically rename the `agg_tiles_hash_after_apply` value back to `agg_tiles_hash` when applying the
|
||||||
|
diff.
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
# This command will compare `file1.mbtiles` and `file2.mbtiles`, and generate a new diff file `diff.mbtiles`.
|
# This command will compare `file1.mbtiles` and `file2.mbtiles`, and generate a new diff file `diff.mbtiles`.
|
||||||
@ -24,9 +31,15 @@ mbtiles validate file2a.mbtiles
|
|||||||
|
|
||||||
## `mbtiles apply-patch`
|
## `mbtiles apply-patch`
|
||||||
|
|
||||||
Apply the diff file generated with the `mbtiles diff` command above to an MBTiles file. The diff file can be applied to the `src_file.mbtiles` that has been previously downloaded to avoid copying/transmitting the entire modified dataset again. The `src_file.mbtiles` will modified in-place. It is also possible to apply the diff file while copying the source file to a new destination file, by using the [`mbtiles copy --apply-patch`](mbtiles-copy.md#mbtiles-copy---apply-patch) command.
|
Apply the diff file generated with the `mbtiles diff` command above to an MBTiles file. The diff file can be applied to
|
||||||
|
the `src_file.mbtiles` that has been previously downloaded to avoid copying/transmitting the entire modified dataset
|
||||||
|
again. The `src_file.mbtiles` will modified in-place. It is also possible to apply the diff file while copying the
|
||||||
|
source file to a new destination file, by using
|
||||||
|
the [`mbtiles copy --apply-patch`](mbtiles-copy.md#mbtiles-copy---apply-patch) command.
|
||||||
|
|
||||||
Note that the `agg_tiles_hash_in_diff` metadata value will be renamed to `agg_tiles_hash` when applying the diff. This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be different after the diff is applied.
|
Note that the `agg_tiles_hash_after_apply` metadata value will be renamed to `agg_tiles_hash` when applying the diff.
|
||||||
|
This is done to avoid confusion when applying the diff file to the original file, as the `agg_tiles_hash` value will be
|
||||||
|
different after the diff is applied.
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
|
mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
|
||||||
@ -34,7 +47,11 @@ mbtiles apply-patch src_file.mbtiles diff_file.mbtiles
|
|||||||
|
|
||||||
#### Applying diff with SQLite
|
#### Applying diff with SQLite
|
||||||
|
|
||||||
Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements. Note that this does not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied.
|
Another way to apply the diff is to use the `sqlite3` command line tool directly. This SQL will delete all tiles
|
||||||
|
from `src_file.mbtiles` that are set to `NULL` in `diff_file.mbtiles`, and then insert or update all new tiles
|
||||||
|
from `diff_file.mbtiles` into `src_file.mbtiles`, where both files are of `flat` type. The name of the diff file is
|
||||||
|
passed as a query parameter to the sqlite3 command line tool, and then used in the SQL statements. Note that this does
|
||||||
|
not update the `agg_tiles_hash` metadata value, so it will be incorrect after the diff is applied.
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
sqlite3 src_file.mbtiles \
|
sqlite3 src_file.mbtiles \
|
||||||
|
@ -17,7 +17,7 @@ use crate::queries::{
|
|||||||
use crate::MbtType::{Flat, FlatWithHash, Normalized};
|
use crate::MbtType::{Flat, FlatWithHash, Normalized};
|
||||||
use crate::{
|
use crate::{
|
||||||
invert_y_value, reset_db_settings, CopyType, MbtError, MbtType, MbtTypeCli, Mbtiles,
|
invert_y_value, reset_db_settings, CopyType, MbtError, MbtType, MbtTypeCli, Mbtiles,
|
||||||
AGG_TILES_HASH, AGG_TILES_HASH_IN_DIFF,
|
AGG_TILES_HASH, AGG_TILES_HASH_AFTER_APPLY,
|
||||||
};
|
};
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, EnumDisplay)]
|
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, EnumDisplay)]
|
||||||
@ -233,13 +233,13 @@ impl MbtileCopierInt {
|
|||||||
if dif.is_some() {
|
if dif.is_some() {
|
||||||
// Insert all rows from diffDb.metadata if they do not exist or are different in sourceDb.metadata.
|
// Insert all rows from diffDb.metadata if they do not exist or are different in sourceDb.metadata.
|
||||||
// Also insert all names from sourceDb.metadata that do not exist in diffDb.metadata, with their value set to NULL.
|
// Also insert all names from sourceDb.metadata that do not exist in diffDb.metadata, with their value set to NULL.
|
||||||
// Rename agg_tiles_hash to agg_tiles_hash_in_diff because agg_tiles_hash will be auto-added later
|
// Rename agg_tiles_hash to agg_tiles_hash_after_apply because agg_tiles_hash will be auto-added later
|
||||||
if self.options.diff_with_file.is_some() {
|
if self.options.diff_with_file.is_some() {
|
||||||
// Include agg_tiles_hash value even if it is the same because we will still need it when applying the diff
|
// Include agg_tiles_hash value even if it is the same because we will still need it when applying the diff
|
||||||
sql = format!(
|
sql = format!(
|
||||||
"
|
"
|
||||||
INSERT {on_dupl} INTO metadata (name, value)
|
INSERT {on_dupl} INTO metadata (name, value)
|
||||||
SELECT IIF(name = '{AGG_TILES_HASH}','{AGG_TILES_HASH_IN_DIFF}', name) as name
|
SELECT IIF(name = '{AGG_TILES_HASH}','{AGG_TILES_HASH_AFTER_APPLY}', name) as name
|
||||||
, value
|
, value
|
||||||
FROM (
|
FROM (
|
||||||
SELECT COALESCE(difMD.name, srcMD.name) as name
|
SELECT COALESCE(difMD.name, srcMD.name) as name
|
||||||
@ -248,14 +248,14 @@ impl MbtileCopierInt {
|
|||||||
ON srcMD.name = difMD.name
|
ON srcMD.name = difMD.name
|
||||||
WHERE srcMD.value != difMD.value OR srcMD.value ISNULL OR difMD.value ISNULL OR srcMD.name = '{AGG_TILES_HASH}'
|
WHERE srcMD.value != difMD.value OR srcMD.value ISNULL OR difMD.value ISNULL OR srcMD.name = '{AGG_TILES_HASH}'
|
||||||
) joinedMD
|
) joinedMD
|
||||||
WHERE name != '{AGG_TILES_HASH_IN_DIFF}'"
|
WHERE name != '{AGG_TILES_HASH_AFTER_APPLY}'"
|
||||||
);
|
);
|
||||||
debug!("Copying metadata, taking into account diff file with {sql}");
|
debug!("Copying metadata, taking into account diff file with {sql}");
|
||||||
} else {
|
} else {
|
||||||
sql = format!(
|
sql = format!(
|
||||||
"
|
"
|
||||||
INSERT {on_dupl} INTO metadata (name, value)
|
INSERT {on_dupl} INTO metadata (name, value)
|
||||||
SELECT IIF(name = '{AGG_TILES_HASH_IN_DIFF}','{AGG_TILES_HASH}', name) as name
|
SELECT IIF(name = '{AGG_TILES_HASH_AFTER_APPLY}','{AGG_TILES_HASH}', name) as name
|
||||||
, value
|
, value
|
||||||
FROM (
|
FROM (
|
||||||
SELECT COALESCE(srcMD.name, difMD.name) as name
|
SELECT COALESCE(srcMD.name, difMD.name) as name
|
||||||
|
@ -32,7 +32,7 @@ pub use update::UpdateZoomType;
|
|||||||
mod validation;
|
mod validation;
|
||||||
pub use validation::{
|
pub use validation::{
|
||||||
calc_agg_tiles_hash, AggHashType, IntegrityCheckType, MbtType, AGG_TILES_HASH,
|
calc_agg_tiles_hash, AggHashType, IntegrityCheckType, MbtType, AGG_TILES_HASH,
|
||||||
AGG_TILES_HASH_IN_DIFF,
|
AGG_TILES_HASH_AFTER_APPLY,
|
||||||
};
|
};
|
||||||
|
|
||||||
/// `MBTiles` uses a TMS (Tile Map Service) scheme for its tile coordinates (inverted along the Y axis).
|
/// `MBTiles` uses a TMS (Tile Map Service) scheme for its tile coordinates (inverted along the Y axis).
|
||||||
|
@ -5,7 +5,7 @@ use sqlx::query;
|
|||||||
|
|
||||||
use crate::queries::detach_db;
|
use crate::queries::detach_db;
|
||||||
use crate::MbtType::{Flat, FlatWithHash, Normalized};
|
use crate::MbtType::{Flat, FlatWithHash, Normalized};
|
||||||
use crate::{MbtResult, MbtType, Mbtiles, AGG_TILES_HASH, AGG_TILES_HASH_IN_DIFF};
|
use crate::{MbtResult, MbtType, Mbtiles, AGG_TILES_HASH, AGG_TILES_HASH_AFTER_APPLY};
|
||||||
|
|
||||||
pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<()> {
|
pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<()> {
|
||||||
let base_mbt = Mbtiles::new(base_file)?;
|
let base_mbt = Mbtiles::new(base_file)?;
|
||||||
@ -53,7 +53,7 @@ pub async fn apply_patch(base_file: PathBuf, patch_file: PathBuf) -> MbtResult<(
|
|||||||
query(&format!(
|
query(&format!(
|
||||||
"
|
"
|
||||||
INSERT OR REPLACE INTO metadata (name, value)
|
INSERT OR REPLACE INTO metadata (name, value)
|
||||||
SELECT IIF(name = '{AGG_TILES_HASH_IN_DIFF}', '{AGG_TILES_HASH}', name) as name,
|
SELECT IIF(name = '{AGG_TILES_HASH_AFTER_APPLY}', '{AGG_TILES_HASH}', name) as name,
|
||||||
value
|
value
|
||||||
FROM patchDb.metadata
|
FROM patchDb.metadata
|
||||||
WHERE name NOTNULL AND name != '{AGG_TILES_HASH}';"
|
WHERE name NOTNULL AND name != '{AGG_TILES_HASH}';"
|
||||||
|
@ -26,7 +26,7 @@ pub const AGG_TILES_HASH: &str = "agg_tiles_hash";
|
|||||||
|
|
||||||
/// Metadata key for a diff file,
|
/// Metadata key for a diff file,
|
||||||
/// describing the eventual [`AGG_TILES_HASH`] value once the diff is applied
|
/// describing the eventual [`AGG_TILES_HASH`] value once the diff is applied
|
||||||
pub const AGG_TILES_HASH_IN_DIFF: &str = "agg_tiles_hash_after_apply";
|
pub const AGG_TILES_HASH_AFTER_APPLY: &str = "agg_tiles_hash_after_apply";
|
||||||
|
|
||||||
#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, EnumDisplay, Serialize)]
|
#[derive(Debug, Clone, Copy, Hash, PartialEq, Eq, EnumDisplay, Serialize)]
|
||||||
#[enum_display(case = "Kebab")]
|
#[enum_display(case = "Kebab")]
|
||||||
|
Loading…
Reference in New Issue
Block a user