* make tilejson's `name` be the same as the ID of the source (even if
aliased)
* `/catalog` will always show ID, but now it will hide the `name` if it
is the same as the `id`
* make `description` be the longer version, e.g. `public.table.column`
format - not guaranteed to be stable
* make `vector_layers` have the fields auto-discovered in the PG table
* preserve the order of the serialized json fields
Fixes#583
Compression middleware turned out to be hard to use for image cases - it
simply looks at the content-encoding, and if not set, tries to compress
if accepted by the client.
Instead, now individual routes are configured with either that
middleware, or for tiles, I decompress and optionally recompress if
applicable.
Now encoding is tracked separately from the tile content, making it
cleaner too. Plus lots of tests for mbtiles & pmtiles.
Fixes#577
DeadPool seem to be better maintained, and most importantly, it properly
reports connection errors - something bb8 does not support yet.
I also removed connection timeout config param - it was introduced while
trying to solve bb8 issue - not needed because connection string
supports it already.
This should resolve some of the issues in #496
This is a partial fix for #496
* BREAKING: Now Martin behaves the same way as `psql` -- by default, if
SSL is available on the server, it will be used, even though it will not
verify that the server has a valid SSL certificate
* Martin now understands `PGSSLCERT`, `PGSSLKEY`, and `PGSSLROOTCERT`
env vars (and corresponding config keys) - same as psql.
* Martin can now process `?sslmode=verify-ca` and `verify-full` (just
like psql). The verify modes require root and/or client cert & key.
* remove `danger_accept_invalid_certs` -- turns out that behavior is
expected by default unless ssl mode is set to verify - which upstream
lib [does not
support](https://github.com/sfackler/rust-postgres/issues/768) - PR
[submitted](https://github.com/sfackler/rust-postgres/pull/988).
* added connection_timeout_ms option for postgres and set it to 5
seconds by default. This way it will fail out earlier.
* added error reporting to bb8 - but it is currently [broken
upstream](https://github.com/djc/bb8/issues/151) - not sure we can fix
it easily, so may need to switch to deadpool later.
* added docker-based TLS test (horray!) - wasn't trivial at all, despite
ending up fairly simple.
Bumps [indoc](https://github.com/dtolnay/indoc) from 1.0.8 to 2.0.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/indoc/releases">indoc's
releases</a>.</em></p>
<blockquote>
<h2>2.0.0</h2>
<ul>
<li>Change handling of final newline at zero levels of indentation (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/55">#55</a>)</li>
<li>Add <a
href="https://docs.rs/indoc/2/indoc/macro.concatdoc.html"><code>concatdoc!</code></a>
macro (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/56">#56</a>)</li>
<li>Raise oldest supported rustc to 1.56</li>
</ul>
<h2>1.0.9</h2>
<ul>
<li>Fix parsing of some edge cases involving angle brackets in
writedoc's expr argument (<a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/54">#54</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="bbbec14e9d"><code>bbbec14</code></a>
Release 2.0.0</li>
<li><a
href="24a804e0cf"><code>24a804e</code></a>
Add concatdoc docs</li>
<li><a
href="476b937436"><code>476b937</code></a>
Delete paths from .cargo/config</li>
<li><a
href="b5fb2dc83f"><code>b5fb2dc</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/57">#57</a>
from dtolnay/fromstr</li>
<li><a
href="623d666afa"><code>623d666</code></a>
Replace TokenStream::from_str with Literal::from_str</li>
<li><a
href="28d6d25d20"><code>28d6d25</code></a>
Switch to 2021 edition</li>
<li><a
href="46d38c0fcd"><code>46d38c0</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/56">#56</a>
from dtolnay/concat</li>
<li><a
href="4fec8a91e9"><code>4fec8a9</code></a>
Add concatdoc macro</li>
<li><a
href="1f705293a5"><code>1f70529</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/dtolnay/indoc/issues/55">#55</a>
from dtolnay/trailing</li>
<li><a
href="31694b5dca"><code>31694b5</code></a>
Preserve last newline even if not indented</li>
<li>Additional commits viewable in <a
href="https://github.com/dtolnay/indoc/compare/1.0.8...2.0.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Adds a new [.mbtiles](https://github.com/mapbox/mbtiles-spec/blob/master/1.3/spec.md)
backend, without the grid support. Uses extensive tile content
detection, i.e. if the content is gzipped, png, jpeg, gif, webp.
From CLI, can be as easy as adding a path to a directory that contains a
.mbtiles file (works just like pmtiles support)
```bash
# All *.mbtiles files in this dir will be published.
# The filename will be used as the source ID
martin ./tests/fixtures
```
From configuration file, the path can be specified in a number of ways
(same as pmtiles)
```yaml
mbtiles:
paths:
# scan this whole dir, matching all *.mbtiles files
- /dir-path
# specific mbtiles file will be published as mbtiles2 source
- /path/to/mbtiles2.mbtiles
sources:
# named source matching source name to a single file
pm-src1: /tmp/mbtiles.mbtiles
# named source, where the filename is explicitly set. This way we will be able to add more options later
pm-src2:
path: /tmp/mbtiles.mbtiles
```
Fixes#494
Merge after #548
Adds a new [.pmtiles](https://protomaps.com/docs/pmtiles/) backend.
Supports all formats like png, vector, etc.
From CLI, can be as easy as adding a path to a directory that contains a
.pmtiles file:
```bash
# All *.pmtiles files in this dir will be published.
# The filename will be used as the source ID
martin ./tests/fixtures
```
From configuration file, the path can be specified in a number of ways:
```yaml
pmtiles:
paths:
# scan this whole dir, matching all *.pmtiles files
- /dir-path
# specific pmtiles file will be published as pmtiles2 source
- /path/to/pmtiles2.pmtiles
sources:
# named source matching source name to a single file
pm-src1: /tmp/pmtiles.pmtiles
# named source, where the filename is explicitly set. This way we will be able to add more options later
pm-src2:
path: /tmp/pmtiles.pmtiles
```
Fixes#508
* added manual coverage justfile command
* a lot of small refactorings of config and argument parsing
* feature: support jsonb query param for functions
* cleaned up public/private access
* make all tests populate with a predefined values to avoid issues with
random data
* Support `postgres` config key to be either a list or an object
* Use `PathBuf` instead of `String` where dealing with files
* Merge `SrvConfigBuilder` into `SrvConfig`
* Parse keep_alive as u64
* More config tests
* Since this is a library, all errors should have a strongly typed enum.
* table bounds computing function was not escaping identifiers
* table bounds computation was also silently ignoring all errors
Can now handle several additional Postgres functions to get a tile, plus
tons of small fixes
### Multiple result variants
* `getmvt(z,x,y) -> [bytea,md5]` (single row with two columns)
* `getmvt(z,x,y) -> [bytea]` (single row with a single column)
* `getmvt(z,x,y) -> bytea` (value)
### Multiple input parameter variants
* `getmvt(z, x, y)` or `getmvt(zoom, x, y)` (all 3 vars must be
integers)
* `getmvt(z, x, y, url_query)`, where instead of `url_query` it could be
any other name, but must be of type JSON
### Breaking
* srid is now the same type as PG -- `i32`
* renamed config vals `table_sources` and `function_sources` into
`tables` and `functions`
### Features and fixes
* if postgis is v3.1+, uses margin parameter to extend the search box by
the size of the buffer. I think we should make 3.1 minimal required.
* fixes feature ID issue from #466
* fixes mixed case names for schemas, tables and columns, functions and
parameter names per #389
### Notes
* More dynamic SQL generation in code instead of using external SQL
files. Those should only be used when they are not parametrized.
* The new function/table discovery mechanism: query for all functions in
the database, and match up those functions with the ones configured (if
any), plus adds all the rest of the un-declared ones if discovery mode
is on.
* During table and function discovery, the code generates a map of
`(PgSqlInfo, FunctionInfo)` (or table) tupples containing SQL needed to
get the tile.
* Auto-discovery mode is currently hidden - the discovery is on only
when no tables or functions are configured. TBD - how to configure it in
the future
* The new system allows for an easy way to auto-discover for the
specific schemas only, solving #47
* predictable order of table/function instantiation
* bounding boxes computed in parallel for all tables (when not
configured)
* proper identifier escaping
* test cleanup
fixes#378fixes#466fixes#65fixes#389
[//]: # (dependabot-start)
⚠️ **Dependabot is rebasing this PR** ⚠️
Rebasing might not happen immediately, so don't worry if this takes some
time.
Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.
---
[//]: # (dependabot-end)
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.9.3 to
0.10.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md">env_logger's
changelog</a>.</em></p>
<blockquote>
<h2>[0.10.0] - 2022-11-24</h2>
<p>MSRV changed to 1.60 to hide optional dependencies</p>
<h3>Fixes</h3>
<ul>
<li>Resolved soundness issue by switching from <code>atty</code> to
<code>is-terminal</code></li>
</ul>
<h3>Breaking Changes</h3>
<p>To open room for changing dependencies:</p>
<ul>
<li>Renamed <code>termcolor</code> feature to <code>color</code></li>
<li>Renamed <code>atty</code> feature to <code>auto-color</code></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ff029fa8b6"><code>ff029fa</code></a>
chore: Release</li>
<li><a
href="389cc52d5a"><code>389cc52</code></a>
docs: Fix changelog links</li>
<li><a
href="2979c4b7b8"><code>2979c4b</code></a>
docs: Update changelog</li>
<li><a
href="4c379170f7"><code>4c37917</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/rust-cli/env_logger/issues/248">#248</a>
from epage/atty</li>
<li><a
href="d55d26980f"><code>d55d269</code></a>
style: Make clippy happy</li>
<li><a
href="066c2192c1"><code>066c219</code></a>
fix: Replace atty with is_terminal</li>
<li><a
href="4db5e87931"><code>4db5e87</code></a>
fix!: Rename termcolor/atty features</li>
<li><a
href="660cf7feb2"><code>660cf7f</code></a>
fix: Bump MSRV to 1.60.0</li>
<li><a
href="e572d04872"><code>e572d04</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/rust-cli/env_logger/issues/244">#244</a>
from epage/update</li>
<li><a
href="f1ff331fa2"><code>f1ff331</code></a>
docs: Fix typos</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-cli/env_logger/compare/v0.9.3...v0.10.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Partial implementation of the #430
* New endpoint structure:
* `GET /` -- a placeholder for the future home page
* `GET /catalog` -- get a list of available sources, as a list of json
blobs.
* `[{id, name, description, attribution, vector_layer}, ...]` (some
fields might be missing)
* `GET /<id>` -- get tilejson for the given source, or a combination of
sources. No `.json` extension
* `GET /<id>/<z>/<x>/<y>` -- get a tile. No format extension.
* `GET /health` -- healthcheck
* Introduce a new tile format support crate (using code from the
maplibre/mbtileserve project)
* Removed the `/rpc/...` routes - all source IDs are accessed in the
same way
* Can print auto-generated configuration or save it to a file
* Refactored to support multiple sources from multiple backends, with a
proper naming conflict resolution
TODO:
* benchmarks need to be rewritten - they were relying on some internal
structures that are no longer there. This might be done as a separate PR
due to a very different internal architecture - might need to rethink
benchmarking approaches.
By default, Martin is now compiled without openssl, simplifying
debugging and simple case usage, whereas the docker build and CI
publishing would still use openssl
Rework CI to run tests locally using the VM-installed Postgres on all
target platforms.
### CI jobs
* Build release versions on Linux/Win/Mac and save build results as
output artifacts
* In a separate VMs (Linux/Win/Mac)
* use
[nyurik/action-setup-postgis](https://github.com/nyurik/action-setup-postgis)
to install postgis and run tests using the built artifacts
* run `cargo test` on Linux only
* copy built artifacts from the build step, and run tests using the
release martin binary
* package and publish if this is a release
### Other changes
* Port some minor changes from the rewrite to porting easier
* minor cleanups
* remove all "expected" data files - too unstable to be usable
A few minor simplifications in the config and appstate:
* default srid seems to be unused.
* simplify Config struct to have non-optional table and function sources. Ok to be empty.
* add a parsing unit test
* rename configs to distinct names for simplicity
I am making this as a separate PR to keep things easier -- the big upcoming PR will use a dynamic dispatch system for all types of sources
Getting ready for multi-backend system, where each backend would have its own configuration.
All 3 config structs (clap args, config, and configbuilder) are now separated into 3 files - the main one, one for service (actix stuff), and one for PostgreSQL.
The files have been moved to `/pg` and `/srv`, but otherwise the logic should be mostly intact.
* Use latest Clap-derive (currently v4rc, but should be public within a few days)
* reorganize configuration to streamline different config sources into one Config (using multiple ConfigBuilders)
* remove all actions and other low-level magic code, making it more straightforward for the most common usage
* replace r2d2 with bb8 to make it all async
* use first significant version in cargo.toml - this makes it easier to maintain
This fixes#349
* Use standard `MIT OR Apache-2.0` license
* Add `MapLibre contributors`
* Add Apache and MIT license files
* The original LICENSE (MIT) file is near-identical but had different line breaks.
* MIT license now also includes copyright maplibre contributors
* Update some deps (the most important is actix-web to 3.x) and some changes in the code run with new versions.
* Some fixes for cargo fmt.
* Fixed "error: this functions return value is unnecessarily wrapped by Result" from github actions (clippy).
