* Make it possible to have configuration specific to the file-based
config sections: pmtiles, mbtiles, and sprites.
* Implement PMTiles directory cache shared between all pmtiles (both
http and local), with configurable max cache size (in MB), or 0 to
disable. Defaults to 32MB (?)
* PMTiles now share web client instance, which optimizes connection
reuse in case multiple pmtiles reside on the same host
* Major refactoring to allow modular reuse, enabling the following build
features:
* **postgres** - enable PostgreSQL/PostGIS tile sources
* **pmtiles** - enable PMTile tile sources
* **mbtiles** - enable MBTile tile sources
* **fonts** - enable font sources
* **sprites** - enable sprite sources
* Use justfile in the CI
Fixes#1093
* `mbtiles update` now allows different types of zoom updates - reset to
content, grow only, or skip (dry run)
* `martin-cp` will now update (grow-only) metadata zooms
Addresses a few concerns in the #1081
PMTiles is a web-optimized format, allowing the actual file to be read
with HTTP range requests. Supporting this use case instantly allows
Martin to function as a lambda executable accessing PMTiles, but without
any significant investment into devops or hosting large file.
PMTiles config now also allows `http` and `https` protocol.
```
# Publish PMTiles files
pmtiles:
paths:
# specific pmtiles file will be published as mypmtiles source
# (use last portion of the URL without extension)
- http://example.org/path/to/mypmtiles.pmtiles
sources:
# named source matching source name to a single file
pm-src1: https://example.org/path/to/some_pmtiles.pmtiles
```
fixes#884
---------
Co-authored-by: Kyle Slugg-Urbino <35903887+kyleslugg@users.noreply.github.com>
This adds a new utility called `martin-cp` that allows copying any
number of tiles from a single source (or a composite source) to an
mbtiles file.
`martin-cp` is a tool for generating tiles in bulk, and save retrieved
tiles into a new or an existing MBTiles file. It can be used to generate
tiles for a large area or multiple areas. If multiple areas overlap, it
will generate tiles only once. `martin-cp` supports the same
configuration file and CLI arguments as Martin server, so it can support
all sources and even combining sources.
# MBTiles
* New `--agg-hash (update|verify|off)` flag replaces
`--update-agg-tiles-hash` (still supported, but not shown in the help
screen). This allows bypassing aggregate hash validation entirely,
without either updating or validating it.
* Simplify MBTiles SQL generation
* MBTiles now uses faster `1 << zoom` everywhere, and a dedicated TMS
inversion fn
* split up metadata insert and delete into separate fn
* consolidated schema initialization
* ensure db settings (like pragma) are always reset on new files
# Other
* Always sort JSON-serialized keys for consistency
* this affects `/catalog` key ordering, but content is the same
* Minor code cleanup
- Add separators to long literal.
- Use first() instead of get(0)
- Move some SQL-making code to their own functions
- Minor mathematics and rounding cleanup
---------
Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
This implements dynamic font protobuf generation, allowing users to
request font ranges on the fly, and combining them in any order, e.g.
`Font1,Font2,Font3`, same as with sprites and tiles
This is a first iteration, without any multithreading support. In
theory, this could be done far faster by generating SDFs with multiple
threads.
### Current process
* during init, figure out all glyphs available in each font, and store
them as a bitset
* during request:
* combine requested bitsets to figure out which glyph should come from
which font file
* load those glyphs from files (using a single instance of the freetype
lib)
* convert them to SDFs and package them into a protobuf
---------
Co-authored-by: Lucas <zhangyijunmetro@hotmail.com>
* Fix metadata copying
* Introduce a new metadata field `agg_tiles_hash_after_apply` for diff
files
* Added a lot of new info and debug logging
* Simplified Copying interface - not much value in having all the
complex builder pattern here it seems, might as well use a simple
object.
## Testing
* Generate SQLite DBs in memory on the fly to validate just what we need
* Use `insta` for validating DB content
There is now a function `dump(connection) -> Vec<Entry>` to dump the
content of the entire SQLite DB into text with `serde`. At many steps
through the testing, the DB content is validated with the corresponding
.snap file with `insta` crate (which makes this process mega-simple,
including a simple way to "bless" (update) any changes).
## Discovered bugs
* Seems like normalized files do not get copied properly - they contain
extras that should be removed.
* Rename `TileCopierOptions` -> `TileCopier`
* remove a few un-needed sqlite open to detect mbtiles type
* move `open_and_detect_type` to `MBTiles`
* add `attach_to` to `MBTiles`
* move various table creation fn to mbtiles_queries file
* a few sql format
Bumps [json-patch](https://github.com/idubrov/json-patch) from 1.0.0 to
1.1.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="50a220ab4d"><code>50a220a</code></a>
chore: Release json-patch version 1.1.0</li>
<li><a
href="d3a5a121bb"><code>d3a5a12</code></a>
fix: tests and suggestion</li>
<li><a
href="227dd997c3"><code>227dd99</code></a>
refactor: do not use recursion but keep a stack</li>
<li><a
href="670981783d"><code>6709817</code></a>
Updating codecov and fixing the test.</li>
<li><a
href="0e1f7aab17"><code>0e1f7aa</code></a>
Use utoipa support for serde_json::Value</li>
<li>See full diff in <a
href="https://github.com/idubrov/json-patch/compare/v1.0.0...v1.1.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
If a PostgreSQL function has an SQL comment, it will try to parse as
JSON and use its values to override the auto-generated TileJSON. It is
recommended to use this form when creating comments to ensure valid JSON
values.
```sql
DO $do$ BEGIN
EXECUTE 'COMMENT ON FUNCTION YOUR_FUNCTION (ARG1_TYPE,ARG2_TYPE,..ARGN_TYPE) IS $tj$' || $$
{
"description": "description override",
...
}
$$::json || '$tj$';
END $do$;
```
Partially implements #822
* Add `MbtType::FlatWithHash`
* Support copying, diffing and applying diffs to and from any
`MbtTypes`s
* Support validating tile data if hash is contained in `*.mbtiles` file
(i.e it is of `MbtType::FlatWithHash` or `MbtType::Normalized`)
---------
Co-authored-by: rstanciu <rstanciu@rivian.com>
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
Add deb package by [cargo-deb](https://crates.io/crates/cargo-deb)
- [ ] add deb package metadata
- [ ] more research on systemd integration
To inspect the package:
```shell
# install cargo-deb dependices
sudo apt install dpkg
sudo apt install dpkg-dev
sudo apt install liblzma-dev
cargo install cargo-deb
cd martin
cargo deb
The Debian package will be created in `target/debian/martin_0.8.7_amd64.deb` .
```
To decompress this package:
```shell
dpkg -x martin_0.8.7_amd64.deb path_of_decompress
```
---------
Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
This is mostly a noop, just updating to the new way of storing cached
queries and a few other changes per
https://github.com/launchbadge/sqlx/blob/main/CHANGELOG.md#070---2023-06-30
Note that this uses some of the code from #729 (thanks @upsicleclown !)
A minor unrelated change - a reformat of `martin/release.toml`
I'd like to add optional rustls support for SQLx. This PR implements the
required features, but tests using `--all-features` fail, because
runtime selection features are not additive. I didn't find a solution
apart from avoiding `--all-features`.
---------
Co-authored-by: Yuri Astrakhan <YuriAstrakhan@gmail.com>
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.29.0 to 1.29.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/tokio/releases">tokio's
releases</a>.</em></p>
<blockquote>
<h2>Tokio v1.29.1</h2>
<h3>Fixed</h3>
<ul>
<li>rt: fix nesting two <code>block_in_place</code> with a
<code>block_on</code> between (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/5837">#5837</a>)</li>
</ul>
<p><a
href="https://redirect.github.com/tokio-rs/tokio/issues/5837">#5837</a>:
<a
href="https://redirect.github.com/tokio-rs/tokio/pull/5837">tokio-rs/tokio#5837</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1b1b9dc7e3"><code>1b1b9dc</code></a>
chore: prepare Tokio v1.29.1 release</li>
<li><a
href="012c848401"><code>012c848</code></a>
rt: fix nesting block_in_place with block_on (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/5837">#5837</a>)</li>
<li>See full diff in <a
href="https://github.com/tokio-rs/tokio/compare/tokio-1.29.0...tokio-1.29.1">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Dynamically create image sprites for MapLibre rendering, given a
directory with images.
### TODO
* [x] Work with @flother to merge these PRs
* [x] https://github.com/flother/spreet/pull/59 (must have)
* [x] https://github.com/flother/spreet/pull/57
* [x] https://github.com/flother/spreet/pull/56
* [ ] https://github.com/flother/spreet/pull/62 (not required but nice
to have, can upgrade later without any code changes)
* [x] Add docs to the book
* [x] Add CLI param, e.g. `--sprite <dir_path>`
* [x] Don't output `.sprites` in auto-genned config when not in use
### API
Per [MapLibre sprites
API](https://maplibre.org/maplibre-style-spec/sprite/), we need to
support the following:
* `/sprite/<sprite_id>.json` metadata about the sprite file - all coming
from a single directory
* `/sprite/<sprite_id>.png` all images combined into a single PNG
* `/sprite/<sprite_id>@2x.json` same but for high DPI devices
* `/sprite/<sprite_id>@2x.png`
Multiple sprite_id values can be combined into one sprite with the same
pattern as for tile joining:
`/sprite/<sprite_id1>,<sprite_id2>,...,<sprite_idN>[.json|.png|@2x.json|@2x.png]`.
No ID renaming is done, so identical names will override one another.
### Configuration
[Config file](https://maplibre.org/martin/config-file.html) and possibly
CLI should have a simple option to serve sprites. The configuration may
look similar to how mbtiles and pmtiles are configured:
```yaml
# Publish sprite images
sprites:
paths:
# scan this whole dir, matching all image files, and publishing it as "my_images" sprite source
- /path/to/my_images
sources:
# named source matching source name to a directory
my_sprites: /path/to/some_dir
```
Implement #705
* Broke up martin-mbtiles into multiple files
* Made all mbtiles functions take a `SqliteExecutor` -- this way they
can be used with any SQLX connection structs - either a pool connection
or an individual non-pooled connection.
* Simplified mbtiles bin a bit - I realized there is really no need to
pretty print the output for the single value retrieval. Easier to just
dump it to console as is.
* Bump martin-mbtiles to v0.2.0
* Minor fixes in tools docs, cargo.toml, and justfile
* MBTiles tool Integration tests and release publishing
Major thanks to the
[stackoverflow](https://stackoverflow.com/questions/76394665/how-to-pass-sqlx-connection-a-mut-trait-as-a-fn-parameter-in-rust/76395111)
quick reply by @cafce25 on how to use generic sql executor!
* Add functionality to retrieve a metadata value in an mbtiles file by
key; simple implementation for one of the items in #667
* Also, disable TTY in docker-up `just` target
Bumps [criterion](https://github.com/bheisler/criterion.rs) from 0.4.0
to 0.5.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/bheisler/criterion.rs/blob/master/CHANGELOG.md">criterion's
changelog</a>.</em></p>
<blockquote>
<h2>[0.5.0] - 2023-05-23</h2>
<h3>Changed</h3>
<ul>
<li>Replaced lazy_static dependency with once_cell</li>
<li>Improved documentation of the <code>html_reports</code> feature</li>
<li>Replaced atty dependency with is-terminal</li>
<li>MSRV bumped to 1.64</li>
<li>Upgraded clap dependency to v4</li>
<li>Upgraded tempfile dependency to v3.5.0</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>Quick mode (<code>--quick</code>) no longer outputs 1ms for measured
times over 5 seconds</li>
<li>Documentation updates</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4a560cb8d8"><code>4a560cb</code></a>
release version 0.5 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/683">#683</a>)</li>
<li><a
href="3e2849ebf8"><code>3e2849e</code></a>
Post v0.4 CHANGELOG updates (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/680">#680</a>)</li>
<li><a
href="637010e50d"><code>637010e</code></a>
Bump clap to version 4 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/679">#679</a>)</li>
<li><a
href="39d7e2f954"><code>39d7e2f</code></a>
Update to tempfile 3.5 (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/675">#675</a>)</li>
<li><a
href="8ab2752ccf"><code>8ab2752</code></a>
Fix typo in known_limitations.md (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/674">#674</a>)</li>
<li><a
href="a2112308ef"><code>a211230</code></a>
chore: replace atty with is-terminal (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/628">#628</a>)</li>
<li><a
href="a844eb226b"><code>a844eb2</code></a>
Bump MSRV to 1.60 for csv dependency (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/665">#665</a>)</li>
<li><a
href="2f53607378"><code>2f53607</code></a>
Use OR operator to specify multiple licenses (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/643">#643</a>)</li>
<li><a
href="86dcd7d323"><code>86dcd7d</code></a>
Fix bug where time over the maximum would be reported as 1ms in quick
mode. (...</li>
<li><a
href="2942aee124"><code>2942aee</code></a>
Remove unnecessary <code>pub</code> from example code (<a
href="https://redirect.github.com/bheisler/criterion.rs/issues/657">#657</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/bheisler/criterion.rs/compare/0.4.0...0.5.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [ctor](https://github.com/mmastrac/rust-ctor) from 0.1.26 to
0.2.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/mmastrac/rust-ctor/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* [BREAKING] Use source ID (table name) as the default layer ID, instead
of `schema.table.column`
* Add support for the optional `layer_id` table config parameter
Fix#595
* make tilejson's `name` be the same as the ID of the source (even if
aliased)
* `/catalog` will always show ID, but now it will hide the `name` if it
is the same as the `id`
* make `description` be the longer version, e.g. `public.table.column`
format - not guaranteed to be stable
* make `vector_layers` have the fields auto-discovered in the PG table
* preserve the order of the serialized json fields
Fixes#583
Compression middleware turned out to be hard to use for image cases - it
simply looks at the content-encoding, and if not set, tries to compress
if accepted by the client.
Instead, now individual routes are configured with either that
middleware, or for tiles, I decompress and optionally recompress if
applicable.
Now encoding is tracked separately from the tile content, making it
cleaner too. Plus lots of tests for mbtiles & pmtiles.
Fixes#577
DeadPool seem to be better maintained, and most importantly, it properly
reports connection errors - something bb8 does not support yet.
I also removed connection timeout config param - it was introduced while
trying to solve bb8 issue - not needed because connection string
supports it already.
This should resolve some of the issues in #496
This is a partial fix for #496
* BREAKING: Now Martin behaves the same way as `psql` -- by default, if
SSL is available on the server, it will be used, even though it will not
verify that the server has a valid SSL certificate
* Martin now understands `PGSSLCERT`, `PGSSLKEY`, and `PGSSLROOTCERT`
env vars (and corresponding config keys) - same as psql.
* Martin can now process `?sslmode=verify-ca` and `verify-full` (just
like psql). The verify modes require root and/or client cert & key.
* remove `danger_accept_invalid_certs` -- turns out that behavior is
expected by default unless ssl mode is set to verify - which upstream
lib [does not
support](https://github.com/sfackler/rust-postgres/issues/768) - PR
[submitted](https://github.com/sfackler/rust-postgres/pull/988).
* added connection_timeout_ms option for postgres and set it to 5
seconds by default. This way it will fail out earlier.
* added error reporting to bb8 - but it is currently [broken
upstream](https://github.com/djc/bb8/issues/151) - not sure we can fix
it easily, so may need to switch to deadpool later.
* added docker-based TLS test (horray!) - wasn't trivial at all, despite
ending up fairly simple.
Bumps [indoc](https://github.com/dtolnay/indoc) from 1.0.8 to 2.0.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/indoc/releases">indoc's
releases</a>.</em></p>
<blockquote>
<h2>2.0.0</h2>
<ul>
<li>Change handling of final newline at zero levels of indentation (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/55">#55</a>)</li>
<li>Add <a
href="https://docs.rs/indoc/2/indoc/macro.concatdoc.html"><code>concatdoc!</code></a>
macro (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/56">#56</a>)</li>
<li>Raise oldest supported rustc to 1.56</li>
</ul>
<h2>1.0.9</h2>
<ul>
<li>Fix parsing of some edge cases involving angle brackets in
writedoc's expr argument (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/54">#54</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="bbbec14e9d"><code>bbbec14</code></a>
Release 2.0.0</li>
<li><a
href="24a804e0cf"><code>24a804e</code></a>
Add concatdoc docs</li>
<li><a
href="476b937436"><code>476b937</code></a>
Delete paths from .cargo/config</li>
<li><a
href="b5fb2dc83f"><code>b5fb2dc</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/57">#57</a>
from dtolnay/fromstr</li>
<li><a
href="623d666afa"><code>623d666</code></a>
Replace TokenStream::from_str with Literal::from_str</li>
<li><a
href="28d6d25d20"><code>28d6d25</code></a>
Switch to 2021 edition</li>
<li><a
href="46d38c0fcd"><code>46d38c0</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/56">#56</a>
from dtolnay/concat</li>
<li><a
href="4fec8a91e9"><code>4fec8a9</code></a>
Add concatdoc macro</li>
<li><a
href="1f705293a5"><code>1f70529</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/55">#55</a>
from dtolnay/trailing</li>
<li><a
href="31694b5dca"><code>31694b5</code></a>
Preserve last newline even if not indented</li>
<li>Additional commits viewable in <a
href="https://github.com/dtolnay/indoc/compare/1.0.8...2.0.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Adds a new [.mbtiles](https://github.com/mapbox/mbtiles-spec/blob/master/1.3/spec.md)
backend, without the grid support. Uses extensive tile content
detection, i.e. if the content is gzipped, png, jpeg, gif, webp.
From CLI, can be as easy as adding a path to a directory that contains a
.mbtiles file (works just like pmtiles support)
```bash
# All *.mbtiles files in this dir will be published.
# The filename will be used as the source ID
martin ./tests/fixtures
```
From configuration file, the path can be specified in a number of ways
(same as pmtiles)
```yaml
mbtiles:
paths:
# scan this whole dir, matching all *.mbtiles files
- /dir-path
# specific mbtiles file will be published as mbtiles2 source
- /path/to/mbtiles2.mbtiles
sources:
# named source matching source name to a single file
pm-src1: /tmp/mbtiles.mbtiles
# named source, where the filename is explicitly set. This way we will be able to add more options later
pm-src2:
path: /tmp/mbtiles.mbtiles
```
Fixes#494
Merge after #548
Adds a new [.pmtiles](https://protomaps.com/docs/pmtiles/) backend.
Supports all formats like png, vector, etc.
From CLI, can be as easy as adding a path to a directory that contains a
.pmtiles file:
```bash
# All *.pmtiles files in this dir will be published.
# The filename will be used as the source ID
martin ./tests/fixtures
```
From configuration file, the path can be specified in a number of ways:
```yaml
pmtiles:
paths:
# scan this whole dir, matching all *.pmtiles files
- /dir-path
# specific pmtiles file will be published as pmtiles2 source
- /path/to/pmtiles2.pmtiles
sources:
# named source matching source name to a single file
pm-src1: /tmp/pmtiles.pmtiles
# named source, where the filename is explicitly set. This way we will be able to add more options later
pm-src2:
path: /tmp/pmtiles.pmtiles
```
Fixes#508
* added manual coverage justfile command
* a lot of small refactorings of config and argument parsing
* feature: support jsonb query param for functions
* cleaned up public/private access
* make all tests populate with a predefined values to avoid issues with
random data
* Support `postgres` config key to be either a list or an object
* Use `PathBuf` instead of `String` where dealing with files
* Merge `SrvConfigBuilder` into `SrvConfig`
* Parse keep_alive as u64
* More config tests
* Since this is a library, all errors should have a strongly typed enum.
* table bounds computing function was not escaping identifiers
* table bounds computation was also silently ignoring all errors
Can now handle several additional Postgres functions to get a tile, plus
tons of small fixes
### Multiple result variants
* `getmvt(z,x,y) -> [bytea,md5]` (single row with two columns)
* `getmvt(z,x,y) -> [bytea]` (single row with a single column)
* `getmvt(z,x,y) -> bytea` (value)
### Multiple input parameter variants
* `getmvt(z, x, y)` or `getmvt(zoom, x, y)` (all 3 vars must be
integers)
* `getmvt(z, x, y, url_query)`, where instead of `url_query` it could be
any other name, but must be of type JSON
### Breaking
* srid is now the same type as PG -- `i32`
* renamed config vals `table_sources` and `function_sources` into
`tables` and `functions`
### Features and fixes
* if postgis is v3.1+, uses margin parameter to extend the search box by
the size of the buffer. I think we should make 3.1 minimal required.
* fixes feature ID issue from #466
* fixes mixed case names for schemas, tables and columns, functions and
parameter names per #389
### Notes
* More dynamic SQL generation in code instead of using external SQL
files. Those should only be used when they are not parametrized.
* The new function/table discovery mechanism: query for all functions in
the database, and match up those functions with the ones configured (if
any), plus adds all the rest of the un-declared ones if discovery mode
is on.
* During table and function discovery, the code generates a map of
`(PgSqlInfo, FunctionInfo)` (or table) tupples containing SQL needed to
get the tile.
* Auto-discovery mode is currently hidden - the discovery is on only
when no tables or functions are configured. TBD - how to configure it in
the future
* The new system allows for an easy way to auto-discover for the
specific schemas only, solving #47
* predictable order of table/function instantiation
* bounding boxes computed in parallel for all tables (when not
configured)
* proper identifier escaping
* test cleanup
fixes#378fixes#466fixes#65fixes#389
[//]: # (dependabot-start)
⚠️ **Dependabot is rebasing this PR** ⚠️
Rebasing might not happen immediately, so don't worry if this takes some
time.
Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.
---
[//]: # (dependabot-end)
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.9.3 to
0.10.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md">env_logger's
changelog</a>.</em></p>
<blockquote>
<h2>[0.10.0] - 2022-11-24</h2>
<p>MSRV changed to 1.60 to hide optional dependencies</p>
<h3>Fixes</h3>
<ul>
<li>Resolved soundness issue by switching from <code>atty</code> to
<code>is-terminal</code></li>
</ul>
<h3>Breaking Changes</h3>
<p>To open room for changing dependencies:</p>
<ul>
<li>Renamed <code>termcolor</code> feature to <code>color</code></li>
<li>Renamed <code>atty</code> feature to <code>auto-color</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ff029fa8b6"><code>ff029fa</code></a>
chore: Release</li>
<li><a
href="389cc52d5a"><code>389cc52</code></a>
docs: Fix changelog links</li>
<li><a
href="2979c4b7b8"><code>2979c4b</code></a>
docs: Update changelog</li>
<li><a
href="4c379170f7"><code>4c37917</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/rust-cli/env_logger/issues/248">#248</a>
from epage/atty</li>
<li><a
href="d55d26980f"><code>d55d269</code></a>
style: Make clippy happy</li>
<li><a
href="066c2192c1"><code>066c219</code></a>
fix: Replace atty with is_terminal</li>
<li><a
href="4db5e87931"><code>4db5e87</code></a>
fix!: Rename termcolor/atty features</li>
<li><a
href="660cf7feb2"><code>660cf7f</code></a>
fix: Bump MSRV to 1.60.0</li>
<li><a
href="e572d04872"><code>e572d04</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/rust-cli/env_logger/issues/244">#244</a>
from epage/update</li>
<li><a
href="f1ff331fa2"><code>f1ff331</code></a>
docs: Fix typos</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-cli/env_logger/compare/v0.9.3...v0.10.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Partial implementation of the #430
* New endpoint structure:
* `GET /` -- a placeholder for the future home page
* `GET /catalog` -- get a list of available sources, as a list of json
blobs.
* `[{id, name, description, attribution, vector_layer}, ...]` (some
fields might be missing)
* `GET /<id>` -- get tilejson for the given source, or a combination of
sources. No `.json` extension
* `GET /<id>/<z>/<x>/<y>` -- get a tile. No format extension.
* `GET /health` -- healthcheck
* Introduce a new tile format support crate (using code from the
maplibre/mbtileserve project)
* Removed the `/rpc/...` routes - all source IDs are accessed in the
same way
* Can print auto-generated configuration or save it to a file
* Refactored to support multiple sources from multiple backends, with a
proper naming conflict resolution
TODO:
* benchmarks need to be rewritten - they were relying on some internal
structures that are no longer there. This might be done as a separate PR
due to a very different internal architecture - might need to rethink
benchmarking approaches.
By default, Martin is now compiled without openssl, simplifying
debugging and simple case usage, whereas the docker build and CI
publishing would still use openssl
Rework CI to run tests locally using the VM-installed Postgres on all
target platforms.
### CI jobs
* Build release versions on Linux/Win/Mac and save build results as
output artifacts
* In a separate VMs (Linux/Win/Mac)
* use
[nyurik/action-setup-postgis](https://github.com/nyurik/action-setup-postgis)
to install postgis and run tests using the built artifacts
* run `cargo test` on Linux only
* copy built artifacts from the build step, and run tests using the
release martin binary
* package and publish if this is a release
### Other changes
* Port some minor changes from the rewrite to porting easier
* minor cleanups
* remove all "expected" data files - too unstable to be usable
A few minor simplifications in the config and appstate:
* default srid seems to be unused.
* simplify Config struct to have non-optional table and function sources. Ok to be empty.
* add a parsing unit test
* rename configs to distinct names for simplicity
I am making this as a separate PR to keep things easier -- the big upcoming PR will use a dynamic dispatch system for all types of sources
Getting ready for multi-backend system, where each backend would have its own configuration.
All 3 config structs (clap args, config, and configbuilder) are now separated into 3 files - the main one, one for service (actix stuff), and one for PostgreSQL.
The files have been moved to `/pg` and `/srv`, but otherwise the logic should be mostly intact.
* Use latest Clap-derive (currently v4rc, but should be public within a few days)
* reorganize configuration to streamline different config sources into one Config (using multiple ConfigBuilders)
* remove all actions and other low-level magic code, making it more straightforward for the most common usage
* replace r2d2 with bb8 to make it all async
* use first significant version in cargo.toml - this makes it easier to maintain
This fixes#349
