* Fix metadata copying
* Introduce a new metadata field `agg_tiles_hash_after_apply` for diff
files
* Added a lot of new info and debug logging
* Simplified Copying interface - not much value in having all the
complex builder pattern here it seems, might as well use a simple
object.
## Testing
* Generate SQLite DBs in memory on the fly to validate just what we need
* Use `insta` for validating DB content
There is now a function `dump(connection) -> Vec<Entry>` to dump the
content of the entire SQLite DB into text with `serde`. At many steps
through the testing, the DB content is validated with the corresponding
.snap file with `insta` crate (which makes this process mega-simple,
including a simple way to "bless" (update) any changes).
## Discovered bugs
* Seems like normalized files do not get copied properly - they contain
extras that should be removed.
* Rename `TileCopierOptions` -> `TileCopier`
* remove a few un-needed sqlite open to detect mbtiles type
* move `open_and_detect_type` to `MBTiles`
* add `attach_to` to `MBTiles`
* move various table creation fn to mbtiles_queries file
* a few sql format
Bumps [json-patch](https://github.com/idubrov/json-patch) from 1.0.0 to
1.1.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="50a220ab4d"><code>50a220a</code></a>
chore: Release json-patch version 1.1.0</li>
<li><a
href="d3a5a121bb"><code>d3a5a12</code></a>
fix: tests and suggestion</li>
<li><a
href="227dd997c3"><code>227dd99</code></a>
refactor: do not use recursion but keep a stack</li>
<li><a
href="670981783d"><code>6709817</code></a>
Updating codecov and fixing the test.</li>
<li><a
href="0e1f7aab17"><code>0e1f7aa</code></a>
Use utoipa support for serde_json::Value</li>
<li>See full diff in <a
href="https://github.com/idubrov/json-patch/compare/v1.0.0...v1.1.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
If a PostgreSQL function has an SQL comment, it will try to parse as
JSON and use its values to override the auto-generated TileJSON. It is
recommended to use this form when creating comments to ensure valid JSON
values.
```sql
DO $do$ BEGIN
EXECUTE 'COMMENT ON FUNCTION YOUR_FUNCTION (ARG1_TYPE,ARG2_TYPE,..ARGN_TYPE) IS $tj$' || $$
{
"description": "description override",
...
}
$$::json || '$tj$';
END $do$;
```
Partially implements #822
* Add `MbtType::FlatWithHash`
* Support copying, diffing and applying diffs to and from any
`MbtTypes`s
* Support validating tile data if hash is contained in `*.mbtiles` file
(i.e it is of `MbtType::FlatWithHash` or `MbtType::Normalized`)
---------
Co-authored-by: rstanciu <rstanciu@rivian.com>
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
Add deb package by [cargo-deb](https://crates.io/crates/cargo-deb)
- [ ] add deb package metadata
- [ ] more research on systemd integration
To inspect the package:
```shell
# install cargo-deb dependices
sudo apt install dpkg
sudo apt install dpkg-dev
sudo apt install liblzma-dev
cargo install cargo-deb
cd martin
cargo deb
The Debian package will be created in `target/debian/martin_0.8.7_amd64.deb` .
```
To decompress this package:
```shell
dpkg -x martin_0.8.7_amd64.deb path_of_decompress
```
---------
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
This is mostly a noop, just updating to the new way of storing cached
queries and a few other changes per
https://github.com/launchbadge/sqlx/blob/main/CHANGELOG.md#070---2023-06-30
Note that this uses some of the code from #729 (thanks @upsicleclown !)
A minor unrelated change - a reformat of `martin/release.toml`
I'd like to add optional rustls support for SQLx. This PR implements the
required features, but tests using `--all-features` fail, because
runtime selection features are not additive. I didn't find a solution
apart from avoiding `--all-features`.
---------
Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.29.0 to 1.29.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/tokio/releases">tokio's
releases</a>.</em></p>
<blockquote>
<h2>Tokio v1.29.1</h2>
<h3>Fixed</h3>
<ul>
<li>rt: fix nesting two <code>block_in_place</code> with a
<code>block_on</code> between (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/5837">#5837</a>)</li>
</ul>
<p><a
href="https://redirect.github.com/tokio-rs/tokio/issues/5837">#5837</a>:
<a
href="https://redirect.github.com/tokio-rs/tokio/pull/5837">tokio-rs/tokio#5837</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1b1b9dc7e3"><code>1b1b9dc</code></a>
chore: prepare Tokio v1.29.1 release</li>
<li><a
href="012c848401"><code>012c848</code></a>
rt: fix nesting block_in_place with block_on (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/5837">#5837</a>)</li>
<li>See full diff in <a
href="https://github.com/tokio-rs/tokio/compare/tokio-1.29.0...tokio-1.29.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Dynamically create image sprites for MapLibre rendering, given a
directory with images.
### TODO
* [x] Work with @flother to merge these PRs
* [x] https://github.com/flother/spreet/pull/59 (must have)
* [x] https://github.com/flother/spreet/pull/57
* [x] https://github.com/flother/spreet/pull/56
* [ ] https://github.com/flother/spreet/pull/62 (not required but nice
to have, can upgrade later without any code changes)
* [x] Add docs to the book
* [x] Add CLI param, e.g. `--sprite <dir_path>`
* [x] Don't output `.sprites` in auto-genned config when not in use
### API
Per [MapLibre sprites
API](https://maplibre.org/maplibre-style-spec/sprite/), we need to
support the following:
* `/sprite/<sprite_id>.json` metadata about the sprite file - all coming
from a single directory
* `/sprite/<sprite_id>.png` all images combined into a single PNG
* `/sprite/<sprite_id>@2x.json` same but for high DPI devices
* `/sprite/<sprite_id>@2x.png`
Multiple sprite_id values can be combined into one sprite with the same
pattern as for tile joining:
`/sprite/<sprite_id1>,<sprite_id2>,...,<sprite_idN>[.json|.png|@2x.json|@2x.png]`.
No ID renaming is done, so identical names will override one another.
### Configuration
[Config file](https://maplibre.org/martin/config-file.html) and possibly
CLI should have a simple option to serve sprites. The configuration may
look similar to how mbtiles and pmtiles are configured:
```yaml
# Publish sprite images
sprites:
paths:
# scan this whole dir, matching all image files, and publishing it as "my_images" sprite source
- /path/to/my_images
sources:
# named source matching source name to a directory
my_sprites: /path/to/some_dir
```
Implement #705
* Broke up martin-mbtiles into multiple files
* Made all mbtiles functions take a `SqliteExecutor` -- this way they
can be used with any SQLX connection structs - either a pool connection
or an individual non-pooled connection.
* Simplified mbtiles bin a bit - I realized there is really no need to
pretty print the output for the single value retrieval. Easier to just
dump it to console as is.
* Bump martin-mbtiles to v0.2.0
* Minor fixes in tools docs, cargo.toml, and justfile
* MBTiles tool Integration tests and release publishing
Major thanks to the
[stackoverflow](https://stackoverflow.com/questions/76394665/how-to-pass-sqlx-connection-a-mut-trait-as-a-fn-parameter-in-rust/76395111)
quick reply by @cafce25 on how to use generic sql executor!
* Add functionality to retrieve a metadata value in an mbtiles file by
key; simple implementation for one of the items in #667
* Also, disable TTY in docker-up `just` target
Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.4.0
to 0.5.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md">criterion's
changelog</a>.</em></p>
<blockquote>
<h2>[0.5.0] - 2023-05-23</h2>
<h3>Changed</h3>
<ul>
<li>Replaced lazy_static dependency with once_cell</li>
<li>Improved documentation of the <code>html_reports</code> feature</li>
<li>Replaced atty dependency with is-terminal</li>
<li>MSRV bumped to 1.64</li>
<li>Upgraded clap dependency to v4</li>
<li>Upgraded tempfile dependency to v3.5.0</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Quick mode (<code>--quick</code>) no longer outputs 1ms for measured
times over 5 seconds</li>
<li>Documentation updates</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4a560cb8d8"><code>4a560cb</code></a>
release version 0.5 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/683">#683</a>)</li>
<li><a
href="3e2849ebf8"><code>3e2849e</code></a>
Post v0.4 CHANGELOG updates (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/680">#680</a>)</li>
<li><a
href="637010e50d"><code>637010e</code></a>
Bump clap to version 4 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/679">#679</a>)</li>
<li><a
href="39d7e2f954"><code>39d7e2f</code></a>
Update to tempfile 3.5 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/675">#675</a>)</li>
<li><a
href="8ab2752ccf"><code>8ab2752</code></a>
Fix typo in known_limitations.md (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/674">#674</a>)</li>
<li><a
href="a2112308ef"><code>a211230</code></a>
chore: replace atty with is-terminal (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/628">#628</a>)</li>
<li><a
href="a844eb226b"><code>a844eb2</code></a>
Bump MSRV to 1.60 for csv dependency (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/665">#665</a>)</li>
<li><a
href="2f53607378"><code>2f53607</code></a>
Use OR operator to specify multiple licenses (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/643">#643</a>)</li>
<li><a
href="86dcd7d323"><code>86dcd7d</code></a>
Fix bug where time over the maximum would be reported as 1ms in quick
mode. (...</li>
<li><a
href="2942aee124"><code>2942aee</code></a>
Remove unnecessary <code>pub</code> from example code (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/657">#657</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/bheisler/criterion.rs/compare/0.4.0...0.5.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [ctor](https://github.com/mmastrac/rust-ctor) from 0.1.26 to
0.2.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/mmastrac/rust-ctor/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* [BREAKING] Use source ID (table name) as the default layer ID, instead
of `schema.table.column`
* Add support for the optional `layer_id` table config parameter
Fix#595
* make tilejson's `name` be the same as the ID of the source (even if
aliased)
* `/catalog` will always show ID, but now it will hide the `name` if it
is the same as the `id`
* make `description` be the longer version, e.g. `public.table.column`
format - not guaranteed to be stable
* make `vector_layers` have the fields auto-discovered in the PG table
* preserve the order of the serialized json fields
Fixes#583
Compression middleware turned out to be hard to use for image cases - it
simply looks at the content-encoding, and if not set, tries to compress
if accepted by the client.
Instead, now individual routes are configured with either that
middleware, or for tiles, I decompress and optionally recompress if
applicable.
Now encoding is tracked separately from the tile content, making it
cleaner too. Plus lots of tests for mbtiles & pmtiles.
Fixes#577
DeadPool seem to be better maintained, and most importantly, it properly
reports connection errors - something bb8 does not support yet.
I also removed connection timeout config param - it was introduced while
trying to solve bb8 issue - not needed because connection string
supports it already.
This should resolve some of the issues in #496
This is a partial fix for #496
* BREAKING: Now Martin behaves the same way as `psql` -- by default, if
SSL is available on the server, it will be used, even though it will not
verify that the server has a valid SSL certificate
* Martin now understands `PGSSLCERT`, `PGSSLKEY`, and `PGSSLROOTCERT`
env vars (and corresponding config keys) - same as psql.
* Martin can now process `?sslmode=verify-ca` and `verify-full` (just
like psql). The verify modes require root and/or client cert & key.
* remove `danger_accept_invalid_certs` -- turns out that behavior is
expected by default unless ssl mode is set to verify - which upstream
lib [does not
support](https://github.com/sfackler/rust-postgres/issues/768) - PR
[submitted](https://github.com/sfackler/rust-postgres/pull/988).
* added connection_timeout_ms option for postgres and set it to 5
seconds by default. This way it will fail out earlier.
* added error reporting to bb8 - but it is currently [broken
upstream](https://github.com/djc/bb8/issues/151) - not sure we can fix
it easily, so may need to switch to deadpool later.
* added docker-based TLS test (horray!) - wasn't trivial at all, despite
ending up fairly simple.
Bumps [indoc](https://github.com/dtolnay/indoc) from 1.0.8 to 2.0.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/indoc/releases">indoc's
releases</a>.</em></p>
<blockquote>
<h2>2.0.0</h2>
<ul>
<li>Change handling of final newline at zero levels of indentation (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/55">#55</a>)</li>
<li>Add <a
href="https://docs.rs/indoc/2/indoc/macro.concatdoc.html"><code>concatdoc!</code></a>
macro (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/56">#56</a>)</li>
<li>Raise oldest supported rustc to 1.56</li>
</ul>
<h2>1.0.9</h2>
<ul>
<li>Fix parsing of some edge cases involving angle brackets in
writedoc's expr argument (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/54">#54</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="bbbec14e9d"><code>bbbec14</code></a>
Release 2.0.0</li>
<li><a
href="24a804e0cf"><code>24a804e</code></a>
Add concatdoc docs</li>
<li><a
href="476b937436"><code>476b937</code></a>
Delete paths from .cargo/config</li>
<li><a
href="b5fb2dc83f"><code>b5fb2dc</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/57">#57</a>
from dtolnay/fromstr</li>
<li><a
href="623d666afa"><code>623d666</code></a>
Replace TokenStream::from_str with Literal::from_str</li>
<li><a
href="28d6d25d20"><code>28d6d25</code></a>
Switch to 2021 edition</li>
<li><a
href="46d38c0fcd"><code>46d38c0</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/56">#56</a>
from dtolnay/concat</li>
<li><a
href="4fec8a91e9"><code>4fec8a9</code></a>
Add concatdoc macro</li>
<li><a
href="1f705293a5"><code>1f70529</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/55">#55</a>
from dtolnay/trailing</li>
<li><a
href="31694b5dca"><code>31694b5</code></a>
Preserve last newline even if not indented</li>
<li>Additional commits viewable in <a
href="https://github.com/dtolnay/indoc/compare/1.0.8...2.0.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
