Commit Graph

81 Commits

Author SHA1 Message Date
Yuri Astrakhan
1d7ce7ba06 readme fix 2022-12-12 10:16:23 -05:00
Yuri Astrakhan
2ee517d135
Support z,x,y and record-returning funcs, table rework (#380)
Can now handle several additional Postgres functions to get a tile, plus
tons of small fixes

### Multiple result variants
* `getmvt(z,x,y) -> [bytea,md5]`  (single row with two columns)
* `getmvt(z,x,y) -> [bytea]` (single row with a single column)
* `getmvt(z,x,y) -> bytea` (value)

### Multiple input parameter variants
* `getmvt(z, x, y)` or `getmvt(zoom, x, y)` (all 3 vars must be
integers)
* `getmvt(z, x, y, url_query)`, where instead of `url_query` it could be
any other name, but must be of type JSON

### Breaking
* srid is now the same type as PG -- `i32`
* renamed config vals `table_sources` and `function_sources` into
`tables` and `functions`

### Features and fixes
* if postgis is v3.1+, uses margin parameter to extend the search box by
the size of the buffer. I think we should make 3.1 minimal required.
* fixes feature ID issue from #466
* fixes mixed case names for schemas, tables and columns, functions and
parameter names per #389
 

### Notes
* More dynamic SQL generation in code instead of using external SQL
files. Those should only be used when they are not parametrized.
* The new function/table discovery mechanism: query for all functions in
the database, and match up those functions with the ones configured (if
any), plus adds all the rest of the un-declared ones if discovery mode
is on.
* During table and function discovery, the code generates a map of
`(PgSqlInfo, FunctionInfo)` (or table) tupples containing SQL needed to
get the tile.
* Auto-discovery mode is currently hidden - the discovery is on only
when no tables or functions are configured. TBD - how to configure it in
the future
* The new system allows for an easy way to auto-discover for the
specific schemas only, solving #47
* predictable order of table/function instantiation
* bounding boxes computed in parallel for all tables (when not
configured)
* proper identifier escaping
* test cleanup

fixes #378
fixes #466
fixes #65
fixes #389
2022-12-10 16:20:42 +02:00
Yuri Astrakhan
ff7c31e16f
Improve pre-push hook and debug page (#509)
* write custom git pre-push via justfile
* fix tests/debug.html incorrectly loading test page
* minor cleanup of the obsolete just targets
2022-12-10 09:40:01 +02:00
Yuri Astrakhan
cfc31fae1b
Update minimal legacy db (#500)
* bump docker compose to use latest supported db and postgis
* update justfile to allow simple legacy testing
* update readme

Followup: implement CI testing -- see #502
2022-12-04 14:34:44 +09:00
Yuri Astrakhan
c1b65796f6
Require PostGIS 3 and use ST_TileEnvelope (#497)
* All tests and internal code now uses ST_TileEnvelope function
* Remove `tile_bbox`
* Rename test function sources for clarity - this will be needed in a
subsequent PR to add other function tests
2022-11-30 18:57:27 +02:00
Yuri Astrakhan
66b7fdc4ea
Multi-source support, new path structure (#456)
Partial implementation of the #430

* New endpoint structure:
  * `GET /` -- a placeholder for the future home page
* `GET /catalog` -- get a list of available sources, as a list of json
blobs.
* `[{id, name, description, attribution, vector_layer}, ...]` (some
fields might be missing)
* `GET /<id>` -- get tilejson for the given source, or a combination of
sources. No `.json` extension
  * `GET /<id>/<z>/<x>/<y>` -- get a tile. No format extension.
  * `GET /health` -- healthcheck
* Introduce a new tile format support crate (using code from the
maplibre/mbtileserve project)
* Removed the `/rpc/...` routes - all source IDs are accessed in the
same way
* Can print auto-generated configuration or save it to a file
* Refactored to support multiple sources from multiple backends, with a
proper naming conflict resolution

TODO:
* benchmarks need to be rewritten - they were relying on some internal
structures that are no longer there. This might be done as a separate PR
due to a very different internal architecture - might need to rethink
benchmarking approaches.
2022-11-26 04:46:40 -05:00
Yuri Astrakhan
b8ac719be8
Rework and consolidate CI (#467)
Rework CI to run tests locally using the VM-installed Postgres on all
target platforms.

### CI jobs
* Build release versions on Linux/Win/Mac and save build results as
output artifacts
* In a separate VMs (Linux/Win/Mac)
* use
[nyurik/action-setup-postgis](https://github.com/nyurik/action-setup-postgis)
to install postgis and run tests using the built artifacts
  * run `cargo test` on Linux only
* copy built artifacts from the build step, and run tests using the
release martin binary
  * package and publish if this is a release

### Other changes
* Port some minor changes from the rewrite to porting easier
* minor cleanups
* remove all "expected" data files - too unstable to be usable
2022-10-31 16:28:21 -04:00
Yuri Astrakhan
a0a411f088
Consolidate DB init and tests (#463)
* Add justfile to simplify running all the tests
* Save all PBF outputs to the text files
* Consolidate all tests to reuse the same code
* Consolidate database initialization
* updated readme with the new instructions

Note that while this PR creates "expected" files, the CI cannot validate
the generated results because the output is not stable. Eventually we
may try to output just the non-geometry values to have reasonable tests
comparing against the expected results.
2022-10-28 14:52:39 -04:00
Yuri Astrakhan
5fb3d50b73
v0.6rc1, MapLibre docs, (#407)
* Migrate to MapLibre documentation
* Changed version to v0.6rc1

getting ready for a release
2022-10-07 17:58:30 +00:00
Yuri Astrakhan
47ed143d8f
Clean up configuration system to use Clap (#415)
* Use latest Clap-derive (currently v4rc, but should be public within a few days)
* reorganize configuration to streamline different config sources into one Config (using multiple ConfigBuilders)
2022-09-28 04:19:23 -04:00
Yuri Astrakhan
5ab2cec8f9
Attempt to migrate CI to maplibre/martin (#408)
Note that I manually published [maplibre/martin:latest](https://hub.docker.com/r/maplibre/martin) to docker, so it might be able to pass CI ok.  I have not looked deeply into the existing CI workflow - @stepankuzmin any suggestions on changes to that?  The github actions are now setup

P.S. I am not certain what that whole `brew/tap` thing is - don't know enough about Macs
2022-09-26 21:48:46 -04:00
Yuri Astrakhan
e8ef2597e3
feat: clean up readme and code of conduct (#382) 2022-08-11 07:06:12 -04:00
Yuri Astrakhan
7a14d6a3ed
feat!: remove --watch support (#381)
BREAKING CHANGE: Remove --watch support and ignore the command line parameter
2022-08-10 12:23:44 -04:00
Devin Norgarb
25442cc972
typo in README.md (#361) 2022-07-02 12:44:24 +00:00
Yuri Astrakhan
0126c5f595
Migrate some urbica/martin -> maplibre/martin (#347)
* Migrate some urbica/martin -> maplibre/martin
2022-06-11 10:51:41 +03:00
zhangzhang
5e2b6279b9
add maplibre example (#340) (h/t @sharkAndshark) 2022-05-29 20:00:05 +03:00
Stepan Kuzmin
b3401785b2
feat: add default SRID support (#308)
* feat: add default SRID support
2022-02-13 15:43:52 +03:00
Stepan Kuzmin
396e563fd1
chore: update dependencies (#278)
* chore: update dependencies

* chore: use main instead of master
2021-11-07 17:28:21 +03:00
Stepan Kuzmin
fc9170d164
feat: add multiple geometry columns support in table sources (#269)
* feat: add multiple geometry columns support in table sources

* test: add multiple geometry tests
2021-10-21 12:20:33 +03:00
Stepan Kuzmin
0ecf3551f8
docs: add using with DigitalOcean PostgreSQL 2021-10-18 14:42:11 +03:00
Stepan Kuzmin
7ad7f1ab8b
feat: use openssl for tls connections, add CA_ROOT_FILE support (#268) (h/t @kapcsandi) 2021-10-18 14:35:08 +03:00
Stepan Kuzmin
d4d101c7f9
docs: add using with Heroku Postgres recipe 2021-10-17 17:30:35 +03:00
Stepan Kuzmin
f2d56c2f7d
fix: tiles attribute in tilejson with x-rewrite-url (#266) 2021-10-15 18:19:36 +03:00
Stepan Kuzmin
194a83e63f
feat: add minzoom and maxzoom support (#265) 2021-10-13 14:51:29 +03:00
Stepan Kuzmin
a9cf508c33
refactor: improve debug logging (#264) 2021-10-12 19:01:25 +03:00
Stepan Kuzmin
0071e94b9e
chore: improve logging (#258) 2021-10-08 17:24:16 +03:00
Stepan Kuzmin
9250563d9d
docs: add a note on caching with Nginx (#255) 2021-10-07 17:00:21 +03:00
Stepan Kuzmin
f935db9f1f
chore: add benchmarks (#230) 2021-07-23 11:04:34 +03:00
Stepan Kuzmin
c27864f5b7
Update README.md (#228) 2021-07-22 21:00:54 +03:00
Stepan Kuzmin
477834bb7c
docs: update debugging section 2021-07-16 13:09:18 +03:00
Stepan Kuzmin
3c01125fc5
feat: add composite sources support (#184)
* feat: add composite sources support WIP

* feat: handle empty composite sources

* fix: decompose queries

* docs: add docs on composite sources

* ci: push docker image

* test: add composite source tests
2021-04-24 20:19:37 +03:00
AdriSolid
166dd68d27
Deckgl example (#187) (h/t @AdriSolid)
* docs: add deckgl example

* docs: improve deckgl code example
2021-02-25 16:07:57 +03:00
AdriSolid
7622731ccf
docs: add deckgl example (#186) (h/t @AdriSolid)
* docs: add deckgl example

* Update README.md

Co-authored-by: Stepan Kuzmin <to.stepan.kuzmin@gmail.com>
2021-02-25 13:33:44 +03:00
Stepan Kuzmin
e953aa35a6
docs: add Leaflet example (#185) 2021-02-23 12:01:45 +03:00
Stepan Kuzmin
7cba29b140
docs: remove codecov badge 2021-02-21 21:46:02 +03:00
Mazedur Rahman
c4fcb5e619
docs: add docs for healthz endpoint (#170)
* Added a simple health check to detect service heartbeat

* Test coverage for healthz endpoint

* Docs for healthz endpoint
2021-01-17 10:32:02 +03:00
Stepan Kuzmin
b262058147
doc: hotfix 2020-12-27 16:41:42 +03:00
Stepan Kuzmin
7d5dfa8174
chore: drop mdillon postgis in favor of official image (#157) 2020-12-27 16:35:05 +03:00
Stepan Kuzmin
4bf7668d87
docs: ✏️ update README.md (#154) 2020-12-20 15:45:33 +03:00
Frédéric Rodrigo
308383a29e
Add nginx server port (#147)
Also forward the port and not only the host.
2020-12-15 13:12:16 +03:00
Stepan Kuzmin
c1c59e158d
feat: add ssl/tls support #63 (#90)
* feat: add ssl/tls support #63

* feat: add danger-accept-invalid-certs option

* ci: switch to ekidd/rust-musl-builder:stable

* ci: enable docker/build-push-action
2020-06-02 09:49:21 +03:00
Stepan Kuzmin
1e34676a14
ci: refactor CI (#86)
* style: run fmt

* ci: run release only on tags

* ci: switch to actions/checkout@v2

* ci: add grcov
* ci: update docker job
2020-05-05 14:13:48 +03:00
Stepan Kuzmin
d19a0da3d0
docs: ✏️ add note on X-Forwarded-Proto 2020-03-21 11:43:13 +03:00
Stepan Kuzmin
58f8066b24
Update README.md 2019-10-14 16:57:19 +03:00
Stepan Kuzmin
dec3759a16 docs: ✏️ update README.md 2019-09-30 18:54:27 +03:00
Stepan Kuzmin
72ff83bc90 ci: 🎡 add release notes 2019-09-30 12:42:49 +03:00
Stepan Kuzmin
5b9a864949
ci: 🎡 switch from CircleCI to Github Actions 2019-09-29 15:59:11 +03:00
Stepan Kuzmin
13bec40ac0
fix: 🐛 use dashes in CLI args instead of underscore
BREAKING CHANGE: renamed CLI args: keep_alive -> keep-alive, listen_addresses ->
listen-addresses, pool_size -> pool-size

Closes: #21
2019-09-29 13:24:03 +03:00
Stepan Kuzmin
fb1a668c11 doc: add docker-compose usage example 2019-08-28 14:04:15 +03:00
Stepan Kuzmin
d4fe02264c doc: fix API section 2019-07-03 17:31:01 +03:00