Commit Graph

1410 Commits

Author SHA1 Message Date
Lucas
aeef27d168 Update releases file names 2023-11-20 01:34:21 -05:00
Yuri Astrakhan
f88fd10aa8
Add martin-cp tool to bulk-copy tiles from any Martin-supported sources (#1011)
This adds a new utility called `martin-cp` that allows copying any
number of tiles from a single source (or a composite source) to an
mbtiles file.

`martin-cp` is a tool for generating tiles in bulk, and save retrieved
tiles into a new or an existing MBTiles file. It can be used to generate
tiles for a large area or multiple areas. If multiple areas overlap, it
will generate tiles only once. `martin-cp` supports the same
configuration file and CLI arguments as Martin server, so it can support
all sources and even combining sources.
2023-11-20 06:27:51 +00:00
Yuri Astrakhan
140ed25679
mbtiles improvements, minor changes (#1013)
# MBTiles
* New `--agg-hash (update|verify|off)` flag replaces
`--update-agg-tiles-hash` (still supported, but not shown in the help
screen). This allows bypassing aggregate hash validation entirely,
without either updating or validating it.
* Simplify MBTiles SQL generation
* MBTiles now uses faster `1 << zoom` everywhere, and a dedicated TMS
inversion fn
* split up metadata insert and delete into separate fn
* consolidated schema initialization
* ensure db settings (like pragma) are always reset on new files

# Other
* Always sort JSON-serialized keys for consistency
  * this affects `/catalog` key ordering, but content is the same
* Minor code cleanup
2023-11-19 10:06:37 +00:00
Lucas
a3b2bfd705
Minor cleanup (#1008)
- Add separators to long literal.
- Use first() instead of get(0)
- Move some SQL-making code to their own functions
- Minor mathematics and rounding cleanup

---------

Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
2023-11-17 18:55:05 +00:00
Yuri Astrakhan
cd375ff1b5
Transparent errors, reusable get_composite_tile, minor opt (#1010) 2023-11-17 07:40:20 +00:00
Yuri Astrakhan
53ebd589c8 relax mbtiles pedantic requirements 2023-11-17 00:15:29 -05:00
Yuri Astrakhan
7e25f4c306
Minor refactoring and cleanup (#1007)
* remove benchmarks - not used for now, will need to be rewritten
* move some CLI settings to a separate struct
* move config saving to a shared function
* simplify zoom level CLI parsing
* rename main.rs to be the same name as generated binary (we can have
multiple bins later)
2023-11-16 19:59:43 -05:00
Yuri Astrakhan
d8e386fb0c
Use Rust's 1.74 cargo lint section (#1006)
https://blog.rust-lang.org/2023/11/16/Rust-1.74.0.html
2023-11-17 08:29:25 +08:00
Lucas
e72f53d9ef
Minor mbtiles summary improvements (#1004)
Partial fix of #1002 

* [x] Move all summary code from `mbtiles/src/mbtiles.rs` to
`mbtiles/src/summary.rs`
* [x] Move Metadata and Validation function to separate files
* [x] Remove `filename` String from the summary stats - not much point
because it is accessible from the mbtiles struct itself

---------

Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
2023-11-16 18:36:40 -05:00
Yuri Astrakhan
b56d019f88 bump lock 2023-11-16 17:03:49 -05:00
Yuri Astrakhan
66eb925e47 bump dep, update pmtiles 2023-11-15 18:36:21 -05:00
Yuri Astrakhan
44313e3c6a
Fix mbtiles copy --dst-type arg name, better err msg (#1003)
* Fix `mbtiles copy --dst-type` to use a `-` instead of `_`
* Better error message for `agg_tiles_hash`
2023-11-14 02:29:00 +00:00
Yuri Astrakhan
8fa8f31336 bump cargo.lock, mbtiles ver bump 2023-11-13 16:58:26 -05:00
Yuri Astrakhan
b0e19d3787
Improve bbox formatting (#1001)
Formats bbox using the most appropriate precision for that zoom level
2023-11-13 08:53:15 +00:00
Yuri Astrakhan
021cddcccd
mbtiles summary tool cleanup (#1000)
After some thinking, it seems `mbtiles summary` (aliased as `mbtiles
info`) would be a bit better than stats. I renamed and adjusted
documentation, consolidating it in one doc page.

Other changes:
* use file system's file size, reporting 'unknown' if needed
* report page count
* moved bbox computation into a separate function
* inlined a number of things for readability
2023-11-13 07:50:10 +00:00
Lucas
0398336114
Add mbtiles statistics command (#986)
- [x] Add tile statistics to mbtiles tools
- [x] Add test
- [x] Use 4326 instead of 3857 for tile bounds
- [x] Add document
- [x] Use size-format to prettify output 
- [x]  Statistics  struct Refactor
- [x] Cleanup and reformat

Closes #964 

---------

Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
2023-11-13 06:03:40 +00:00
Yuri Astrakhan
dec09bdabf
Do not serialize missing configs (#999)
Make sure minimum config does not save unused sub-systems.
2023-11-13 04:40:34 +00:00
Yuri Astrakhan
fad89d742f
simplify serde optional handling (#998) 2023-11-12 22:22:04 -05:00
Yuri Astrakhan
490be6b968
a few minor mbtiles cleanups (#997) 2023-11-12 20:23:59 -05:00
Yuri Astrakhan
e3b62f4ce1 update lock 2023-11-12 15:24:31 -05:00
Yuri Astrakhan
1740af7b2a update lock 2023-11-12 12:41:43 -05:00
Yuri Astrakhan
4626a5b3ea use original cuchi/jinja2-action 2023-11-09 23:11:01 -05:00
Yuri Astrakhan
db90780ad4 Update demo dependencies 2023-11-09 22:52:32 -05:00
Yuri Astrakhan
fe05faaa20 bump lock 2023-11-09 22:28:15 -05:00
dependabot[bot]
7cf9779ff4
chore(deps): Bump rustls-pemfile from 1.0.3 to 1.0.4 (#994)
Bumps [rustls-pemfile](https://github.com/rustls/pemfile) from 1.0.3 to
1.0.4.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="dffcbf7fbc"><code>dffcbf7</code></a>
Bump version to 1.0.4</li>
<li><a
href="cc2ac26d5e"><code>cc2ac26</code></a>
docs: update README with 1.0.4 changelog.</li>
<li><a
href="cc3fe92711"><code>cc3fe92</code></a>
Allow additionally parsing \r newlines in PEM files</li>
<li>See full diff in <a
href="https://github.com/rustls/pemfile/compare/v/1.0.3...v/1.0.4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rustls-pemfile&package-manager=cargo&previous-version=1.0.3&new-version=1.0.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-10 02:51:33 +00:00
Yuri Astrakhan
f7c69f8302
Minor cleanup and just reorg (#992)
Only run git push diff on linux to make development on non-linux easier,
and a few preparations for pmtiles over http serving

Fixes #855
2023-11-08 19:46:23 -05:00
Yuri Astrakhan
f2eba9a542 bump lock 2023-11-08 18:41:11 -05:00
Yuri Astrakhan
baf09c52e8 bump lock 2023-11-07 16:01:20 -05:00
dependabot[bot]
79b7a5e279
chore(deps): Bump serde from 1.0.190 to 1.0.192 (#989)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.190 to
1.0.192.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/serde-rs/serde/releases">serde's
releases</a>.</em></p>
<blockquote>
<h2>v1.0.192</h2>
<ul>
<li>Allow internal tag field in untagged variant (<a
href="https://redirect.github.com/serde-rs/serde/issues/2646">#2646</a>,
thanks <a
href="https://github.com/robsdedude"><code>@​robsdedude</code></a>)</li>
</ul>
<h2>v1.0.191</h2>
<ul>
<li>Documentation improvements</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4676abdc9e"><code>4676abd</code></a>
Release 1.0.192</li>
<li><a
href="35700eb23e"><code>35700eb</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/serde/issues/2646">#2646</a>
from robsdedude/fix/2643/allow-tag-field-in-untagged</li>
<li><a
href="59892e7b58"><code>59892e7</code></a>
Release 1.0.191</li>
<li><a
href="97dd07a7d1"><code>97dd07a</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/serde/issues/2647">#2647</a>
from dtolnay/doccfg</li>
<li><a
href="c8bc97c81b"><code>c8bc97c</code></a>
Document &quot;rc&quot; and &quot;unstable&quot; features on
docs.rs</li>
<li><a
href="9dacfbbd69"><code>9dacfbb</code></a>
Fill in more doc(cfg) attributes</li>
<li><a
href="05c2509d07"><code>05c2509</code></a>
Relocate cfg attrs into deref_impl</li>
<li><a
href="64f949b37b"><code>64f949b</code></a>
Relocate cfg attrs into parse_ip_impl and parse_socket_impl</li>
<li><a
href="3f339de36a"><code>3f339de</code></a>
Relocate cfg attrs into seq_impl and map_impl</li>
<li><a
href="215c2b71ef"><code>215c2b7</code></a>
Relocate cfg attrs into forwarded_impl macro</li>
<li>Additional commits viewable in <a
href="https://github.com/serde-rs/serde/compare/v1.0.190...v1.0.192">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=serde&package-manager=cargo&previous-version=1.0.190&new-version=1.0.192)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-07 03:11:29 +00:00
Yuri Astrakhan
bad4efc0d4 link to book from the banner 2023-11-05 21:51:33 -05:00
Yuri Astrakhan
faa255431a Simplify README 2023-11-05 21:50:26 -05:00
Yuri Astrakhan
fe79756097 Bump lock, improve test.sh 2023-11-05 21:23:08 -05:00
Yuri Astrakhan
9ddbbe0dcb
Downgrade nightly CI until rustc is fixed (#985)
See https://github.com/rust-lang/rust/issues/117598
2023-11-05 01:00:50 -05:00
Yuri Astrakhan
ccadb56be4 cargo bump, rm unused font errors 2023-11-04 16:02:55 -04:00
Yuri Astrakhan
73b56e8a90
Include readme as docs for all crates (#981)
This adds documentation in the https://docs.rs site.
2023-10-31 14:04:10 -04:00
Yuri Astrakhan
09dd2bea62 cleanup trailing spaces 2023-10-31 00:56:13 -04:00
Yuri Astrakhan
d9f01d15d4 cleanup docs 2023-10-31 00:53:39 -04:00
Lucas
c6170c5913
Improve MBTiles documentation (#961)
Fixes #918 

---------

Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
2023-10-31 00:41:21 -04:00
Yuri Astrakhan
3a1f7acc95 fix image test extension 2023-10-30 21:52:11 -04:00
Yuri Astrakhan
bfbe52d032 Add crate categories 2023-10-30 18:52:47 -04:00
Yuri Astrakhan
65b1cdcb83 more doc fixes 2023-10-30 18:25:37 -04:00
Yuri Astrakhan
127c989f37 fix font docs 2023-10-30 18:18:03 -04:00
Yuri Astrakhan
163b44982a fix CI 2023-10-30 17:45:31 -04:00
Yuri Astrakhan
c02fa4a668 Fix CI workflow 2023-10-30 17:18:03 -04:00
Yuri Astrakhan
8c750aa3cc bump lock 2023-10-30 14:53:11 -04:00
Yuri Astrakhan
f1241e264c fix CI status badges 2023-10-28 16:29:38 -04:00
Yuri Astrakhan
47b3106ac6
Rename martin-mbtiles crate to mbtiles (#976)
This simplifies usage as both a crate and as a tool
2023-10-28 08:50:49 +00:00
Yuri Astrakhan
76a1b7fb33 fix CI 2023-10-28 04:01:48 -04:00
Yuri Astrakhan
0178ca731d better logging on failure 2023-10-28 02:06:37 -04:00
Yuri Astrakhan
8b274f49f6 save err for all platforms 2023-10-28 01:25:44 -04:00