consbio/mbtileserver
1
1
Fork 0
mirror of https://github.com/consbio/mbtileserver.git synced 2024-08-15 18:20:35 +03:00
Code Issues Packages Projects Releases Wiki Activity

Refactor of handlers, middleware, command line options (#100)

Browse Source
This commit is contained in:
Brendan Ward 2020-05-11 11:18:31 -07:00 committed by GitHub
parent f2305b518f
commit ca4a94e0c5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
21 changed files with 1872 additions and 1014 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

114
CHANGELOG.md Normal file
View File

@ -0,0 +1,114 @@
# Changelog
## O.7 (in progress)
This version involved a significant refactor of internal functionality and HTTP
handlers to provide better ability to modify services at runtime, provide
granular control over the endpoints that are exposed, and cleanup handling
of middleware.
Most internal HTTP handlers for `ServiceSet` and `Tileset` in the
`github.com/consbio/mbtileserver/handlers` package are now `http.HandlerFunc`s
instead of custom handlers that returned status codes or errors as in the previous
versions.
The internal routing within these handlers has been modified to enable
tilesets to change at runtime. Previously, we were using an `http.ServeMux`
for all routes, which breaks when the `Tileset` instances pointed to by those
routes have changed at runtime. Now, the top-level `ServiceSet.Handler()`
allows dynamic routing to any `Tileset` instances currently published. Each
`Tileset` is now responsible for routing to its subpaths (e.g., tile endpoint).
The singular public handler endpoint is still an `http.Handler` instance but
no longer takes any parameters. Those parameters are now handled using
configuration options instead.
`ServiceSet` now enables configuration to set the root URL, toggle which endpoints
are exposed and set the internal error logger. These are passed in using a
`ServiceSetConfig` struct when the service is constructed; these configuration
options are not modifiable at runtime.
`Tileset` instances are now created individually from a set of source `mbtiles`
files, instead of generated within `ServiceSet` from a directory. This provides
more granular control over assigning IDs to tilesets as well as creating,
updating, or deleting `Tileset` instances. You must generate unique IDs for
tilesets before adding to the `ServiceSet`; you can use
`handlers.SHA1ID(filename)` to generate a unique SHA1 ID of the service based on
its full filename path, or `handlers.RelativePathID(filename, tilePath)` to
generate the ID from its path and filename within the tile directory `tilePath`.
HMAC authorization has been refactored into middleware external to the Go API.
It now is instantiated as middleware in `main.go`; this provides better
separation of concerns between the server (`main.go`) and the Go API. The API
for interacting with HMAC authorization from the CLI or endpoints remains the
same.
Most of the updates are demonstrated in `main.go`.
### General changes
### Command-line interface
- added support for automatically generating unique tileset IDs using `--generate-ids` option
- added ability to toggle off non-tile endpoints:
- `--disable-preview`: disables the map preview, enabled by default.
- `--disable-svc-list`: disables the list of map services, enabled by default
- `--disable-tilejson`: disables the TileJSON endpoint for each tile service
- `--tiles-only`: shortcut that disables preview, service list, and TileJSON endpoints
- added ability to have multiple tile paths using a comma-delimited list of paths passed to `--dir` option
- moved static assets for map preview that were originally served on `/static`
endpoint to `/services/<tileset_id>/map/static` so that this endpoint is
disabled when preview is disabled via `--disable-preview`.
### Go API
- added `ServiceSetConfig` for configuration options for `ServiceSet` instances
- added `ServiceSet.AddTileset()`, `ServiceSet.UpdateTileset()`,
`ServiceSet.RemoveTileset()`, and `ServiceSet.HasTileset()` functions.
WARNING: these functions are not yet thread-safe.
### Breaking changes
#### Command-line interface:
- ArcGIS endpoints are now opt-in via `--enable-arcgis` option (disabled by default)
- `--path` option has been renamed to `--root-url` for clarity (env var is now `ROOT_URL`)
- `--enable-reload` has been renamed to `--enable-reload-signal`
#### Handlers API
- `ServiceSet.Handler` parameters have been replaced with `ServiceSetConfig`
passed to `handlers.New()` instead.
- removed `handlers.NewFromBaseDir()`, replaced with `handlers.New()` and calling
`ServiceSet.AddTileset()` for each `Tileset` to register.
- removed `ServiceSet.AddDBOnPath()`; this is replaced by calling
`ServiceSet.AddTileset()` for each `Tileset` to register.
## 0.6.1
- upgraded Docker containers to Go 1.14 (solves out of memory issues during builds on small containers)
## 0.6
- fixed bug in map preview when bounds are not defined for a tileset (#84)
- updated Leaflet to 1.6.0 and Mapbox GL to 0.32.0 (larger upgrades contingent on #65)
- fixed issues with `--tls` option (#89)
- added example proxy configuration for Caddy and NGINX (#91)
- fixed issues with map preview page using HTTP basemaps (#90)
- resolved template loading issues (#85)
### Breaking changes - handlers API:
- Removed `TemplatesFromAssets` as it was not used internally, and unlikely used externally
- Removed `secretKey` from `NewFromBaseDir` parameters; this is replaced by calling `SetRequestAuthKey` on a `ServiceSet`.
## 0.5.0
- Added Docker support (#74, #75)
- Fix case-sensitive mbtiles URLs (#77)
- Add support for graceful reloading (#69, #72, #73)
- Add support for environment args (#70)
- All changes prior to 6/1/2019

88
README.md
View File

@ -18,7 +18,7 @@ In addition to tile-level access, it provides:
- TileJSON 2.1.0 endpoint for each tileset, with full metadata
from the mbtiles file.
- a preview map for exploring each tileset.
- a minimal ArcGIS tile map service API (work in progress)
- a minimal ArcGIS tile map service API
We have been able to host a bunch of tilesets on an
[AWS t2.nano](https://aws.amazon.com/about-aws/whats-new/2015/12/introducing-t2-nano-the-smallest-lowest-cost-amazon-ec2-instance/)
@ -66,19 +66,25 @@ Usage:
mbtileserver [flags]
Flags:
-c, --cert string X.509 TLS certificate filename. If present, will be used to enable SSL on the server.
-d, --dir string Directory containing mbtiles files. (default "./tilesets")
--domain string Domain name of this server. NOTE: only used for AutoTLS.
--dsn string Sentry DSN
-h, --help help for mbtileserver
-k, --key string TLS private key
--path string URL root path of this server (if behind a proxy)
-p, --port int Server port. Default is 443 if --cert or --tls options are used, otherwise 8000. (default -1)
-s, --secret-key string Shared secret key used for HMAC request authentication
-t, --tls Auto TLS using Let's Encrypt
-r, --redirect Redirect HTTP to HTTPS
--enable-reload Enable graceful reload
-v, --verbose Verbose logging
-c, --cert string X.509 TLS certificate filename. If present, will be used to enable SSL on the server.
-d, --dir string Directory containing mbtiles files. Directory containing mbtiles files. Can be a comma-delimited list of directories. (default "./tilesets")
--disable-preview Disable map preview for each tileset (enabled by default)
--disable-svc-list Disable services list endpoint (enabled by default)
--disable-tilejson Disable TileJSON endpoint for each tileset (enabled by default)
--domain string Domain name of this server. NOTE: only used for AutoTLS.
--dsn string Sentry DSN
--enable-arcgis Enable ArcGIS Mapserver endpoints
--enable-reload-signal Enable graceful reload using HUP signal to the server process
--generate-ids Automatically generate tileset IDs instead of using relative path
-h, --help help for mbtileserver
-k, --key string TLS private key
-p, --port int Server port. Default is 443 if --cert or --tls options are used, otherwise 8000. (default -1)
-r, --redirect Redirect HTTP to HTTPS
--root-url string Root URL of services endpoint (default "/services")
-s, --secret-key string Shared secret key used for HMAC request authentication
--tiles-only Only enable tile endpoints (shortcut for --disable-svc-list --disable-tilejson --disable-preview)
-t, --tls Auto TLS via Let's Encrypt
-v, --verbose Verbose logging
```
So hosting tiles is as easy as putting your mbtiles files in the `tilesets`
@ -88,6 +94,9 @@ You can have multiple directories in your `tilesets` directory; these will be co
`<tile_dir>/foo/bar/baz.mbtiles` will be available at `/services/foo/bar/baz`.
If `--generate-ids` is provided, tileset IDs are automatically generated using a SHA1 hash of the path to each tileset.
By default, tileset IDs are based on the relative path of each tileset to the base directory provided using `--dir`.
When you want to remove, modify, or add new tilesets, simply restart the server process or use the reloading process below.
If a valid Sentry DSN is provided, warnings, errors, fatal errors, and panics will be reported to Sentry.
@ -98,19 +107,20 @@ If the `--tls` option is provided, the Let's Encrypt Terms of Service are accept
If either `--cert` or `--tls` are provided, the default port is 443.
You can also set up server config using environment variables instead of flags, which may be more helpful when deploying in a docker image. Use the associated flag to determine usage. The following variables are available:
You can also use environment variables instead of flags, which may be more helpful when deploying in a docker image. Use the associated flag to determine usage. The following variables are available:
- `PORT` (`--port`)
- `TILE_DIR` (`--dir`)
- `PATH_PREFIX` (`--path`)
- `GENERATE_IDS` (`--generate-ids`)
- `ROOT_URL_PATH` (`--root-url-path`)
- `DOMAIN` (`--domain`)
- `TLS_CERT` (`--cert`)
- `TLS_PRIVATE_KEY` (`--key`)
- `HMAC_SECRET_KEY` (`--secret-key`)
- `AUTO_TLS` (`--tls`)
- `REDIRECT` (`--redirect`)
- `DSN` (`--dsn`)
- `VERBOSE` (`--verbose`)
- `HMAC_SECRET_KEY` (`--secret-key`)
Example:
@ -136,7 +146,7 @@ mbtileserver:
### Reload
mbtileserver optionally supports graceful reload (without interrupting any in-progress requests). This functionality
must be enabled with the `--enable-reload` flag. When enabled, the server can be reloaded by sending it a `HUP` signal:
must be enabled with the `--enable-reload-signal` flag. When enabled, the server can be reloaded by sending it a `HUP` signal:
```
$ kill -HUP <pid>
@ -275,7 +285,7 @@ file.
`mbtileserver` automatically creates a TileJSON endpoint for each service at `/services/<tileset_id>`.
The TileJSON uses the same scheme and domain name as is used for the incoming request; the `--domain` setting does not
have an affect on auto-generated URLs.
affect auto-generated URLs.
This API provides most elements of the `metadata` table in the mbtiles file as well as others that are
automatically inferred from tile data.
@ -327,10 +337,19 @@ This currently uses `Leaflet` for image tiles and `Mapbox GL JS` for vector tile
## ArcGIS API
This project currently provides a minimal ArcGIS tiled map service API for tiles stored in an mbtiles file.
This is enabled with the `--enable-arcgis` flag.
This should be sufficient for use with online platforms such as [Data Basin](https://databasin.org). Because the ArcGIS API relies on a number of properties that are not commonly available within an mbtiles file, so certain aspects are stubbed out with minimal information.
This API is not intended for use with more full-featured ArcGIS applications such as ArcGIS Desktop.
Available endpoints:
- Service info: `http://localhost:8000/arcgis/rest/services/<tileset_id>/MapServer`
- Layer info: `http://localhost:8000/arcgis/rest/services/<tileset_id>/MapServer/layers`
- Tiles: `http://localhost:8000/arcgis/rest/services/<tileset_id>/MapServer/tile/0/0/0`
## Request authorization
Providing a secret key with `-s/--secret-key` or by setting the `HMAC_SECRET_KEY` environment variable will
@ -347,7 +366,7 @@ A signature is a URL-safe, base64 encoded HMAC hash using the `SHA1` algorithm.
from a randomly generated salt, and the **secret key** string. The hash payload is a combination of the ISO-formatted
date when the hash was created, and the authorized service id.
The following is an example signature, created in Go for the serivce id `test`, the date
The following is an example signature, created in Go for the service id `test`, the date
`2019-03-08T19:31:12.213831+00:00`, the salt `0EvkK316T-sBLA`, and the secret key
`YMIVXikJWAiiR3q-JMz1v2Mfmx3gTXJVNqme5kyaqrY`
@ -429,12 +448,12 @@ $gulp build
Modifying the `.go` files always requires re-running `go build .`.
In case you have modified the templates and static assets, you need to run `go generate ./handlers` to ensure that your modifications
In case you have modified the templates and static assets, you need to run `go generate ./handlers/templates.go` to ensure that your modifications
are embedded into the executable. For this to work, you must have
[github.com/shurcooL/vfsgen)[https://github.com/shurcooL/vfsgen) installed.
```bash
go generate ./handlers/handlers.go
go generate ./handlers/templates.go
```
This will rewrite the `assets_vfsdata.go` which you must commit along with your
@ -447,30 +466,7 @@ But do not forget to perform it in the end.
## Changes
### 0.6.1
- upgraded Docker containers to Go 1.14
### 0.6
- fixed bug in map preview when bounds are not defined for a tileset (#84)
- updated Leaflet to 1.6.0 and Mapbox GL to 0.32.0 (larger upgrades contingent on #65)
- fixed issues with `--tls` option (#89)
- added example proxy configuration for Caddy and NGINX (#91)
- fixed issues with map preview page using HTTP basemaps (#90)
- resolved template loading issues (#85)
- breaking changes:
- `handlers.go`:
- Removed `TemplatesFromAssets` as it was not used internally, and unlikely used externally
- Removed `secretKey` from `NewFromBaseDir` parameters; this is replaced by calling `SetRequestAuthKey` on a `ServiceSet`.
### 0.5.0
- Added Docker support (#74, #75)
- Fix case-sensitive mbtiles URLs (#77)
- Add support for graceful reloading (#69, #72, #73)
- Add support for environment args (#70)
- All changes prior to 6/1/2019
See [CHANGELOG](CHANGELOG.md).
## Contributors ✨

519
handlers/arcgis.go
View File

@ -2,12 +2,9 @@ package handlers
import (
"encoding/json"
"fmt"
"math"
"net/http"
"strings"
"github.com/consbio/mbtileserver/mbtiles"
)
type arcGISLOD struct {
@ -63,265 +60,307 @@ type arcGISLayer struct {
CurrentVersion float32 `json:"currentVersion"`
}
const (
earthRadius = 6378137.0
earthCircumference = math.Pi * earthRadius
initialResolution = 2 * earthCircumference / 256
dpi uint8 = 96
)
const ArcGISRoot = "/arcgis/rest/services/"
var webMercatorSR = arcGISSpatialReference{Wkid: 3857}
var geographicSR = arcGISSpatialReference{Wkid: 4326}
// wrapJSONP writes b (JSON marshalled to bytes) as a JSONP response to
// w if the callback query parameter is present, and writes b as a JSON
// response otherwise. Any error that occurs during writing is returned.
func wrapJSONP(w http.ResponseWriter, r *http.Request, b []byte) (err error) {
callback := r.URL.Query().Get("callback")
// arcGISServiceJSON returns ArcGIS standard JSON describing the ArcGIS
// tile service.
func (ts *Tileset) arcgisServiceJSON() ([]byte, error) {
db := ts.db
imgFormat := db.TileFormatString()
metadata, err := db.ReadMetadata()
if err != nil {
return nil, err
}
name, _ := metadata["name"].(string)
description, _ := metadata["description"].(string)
attribution, _ := metadata["attribution"].(string)
tags, _ := metadata["tags"].(string)
credits, _ := metadata["credits"].(string)
if callback != "" {
w.Header().Set("Content-Type", "application/javascript")
_, err = w.Write([]byte(fmt.Sprintf("%s(%s);", callback, b)))
// TODO: make sure that min and max zoom always populated
minZoom, _ := metadata["minzoom"].(uint8)
maxZoom, _ := metadata["maxzoom"].(uint8)
// TODO: extract dpi from the image instead
var lods []arcGISLOD
for i := minZoom; i <= maxZoom; i++ {
scale, resolution := calcScaleResolution(i, dpi)
lods = append(lods, arcGISLOD{
Level: i,
Resolution: resolution,
Scale: scale,
})
}
minScale := lods[0].Scale
maxScale := lods[len(lods)-1].Scale
bounds, ok := metadata["bounds"].([]float32)
if !ok {
bounds = []float32{-180, -85, 180, 85} // default to world bounds
}
extent := geoBoundsToWMExtent(bounds)
tileInfo := map[string]interface{}{
"rows": 256,
"cols": 256,
"dpi": dpi,
"origin": map[string]float32{
"x": -20037508.342787,
"y": 20037508.342787,
},
"spatialReference": webMercatorSR,
"lods": lods,
}
documentInfo := map[string]string{
"Title": name,
"Author": attribution,
"Comments": "",
"Subject": "",
"Category": "",
"Keywords": tags,
"Credits": credits,
}
out := map[string]interface{}{
"currentVersion": "10.4",
"id": ts.id,
"name": name,
"mapName": name,
"capabilities": "Map,TilesOnly",
"description": description,
"serviceDescription": description,
"copyrightText": attribution,
"singleFusedMapCache": true,
"supportedImageFormatTypes": strings.ToUpper(imgFormat),
"units": "esriMeters",
"layers": []arcGISLayerStub{
{
ID: 0,
Name: name,
ParentLayerID: -1,
DefaultVisibility: true,
SubLayerIDs: nil,
MinScale: minScale,
MaxScale: maxScale,
},
},
"tables": []string{},
"spatialReference": webMercatorSR,
"minScale": minScale,
"maxScale": maxScale,
"tileInfo": tileInfo,
"documentInfo": documentInfo,
"initialExtent": extent,
"fullExtent": extent,
"exportTilesAllowed": false,
"maxExportTilesCount": 0,
"resampling": false,
}
bytes, err := json.Marshal(out)
if err != nil {
return nil, err
}
return bytes, nil
}
// arcgisServiceHandler is an http.HandlerFunc that returns standard ArcGIS
// JSON for a given ArcGIS tile service
func (ts *Tileset) arcgisServiceHandler(w http.ResponseWriter, r *http.Request) {
svcJSON, err := ts.arcgisServiceJSON()
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("Could not render ArcGIS Service JSON for %v: %v", r.URL.Path, err)
return
}
w.Header().Set("Content-Type", "application/json")
_, err = w.Write(b)
return
}
err = wrapJSONP(w, r, svcJSON)
func (s *ServiceSet) arcgisService(id string, db *mbtiles.DB) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
imgFormat := db.TileFormatString()
metadata, err := db.ReadMetadata()
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("Could not read metadata for tileset %v", id)
}
name, _ := metadata["name"].(string)
description, _ := metadata["description"].(string)
attribution, _ := metadata["attribution"].(string)
tags, _ := metadata["tags"].(string)
credits, _ := metadata["credits"].(string)
// TODO: make sure that min and max zoom always populated
minZoom, _ := metadata["minzoom"].(uint8)
maxZoom, _ := metadata["maxzoom"].(uint8)
// TODO: extract dpi from the image instead
var lods []arcGISLOD
for i := minZoom; i <= maxZoom; i++ {
scale, resolution := calcScaleResolution(i, dpi)
lods = append(lods, arcGISLOD{
Level: i,
Resolution: resolution,
Scale: scale,
})
}
minScale := lods[0].Scale
maxScale := lods[len(lods)-1].Scale
bounds, ok := metadata["bounds"].([]float32)
if !ok {
bounds = []float32{-180, -85, 180, 85} // default to world bounds
}
extent := geoBoundsToWMExtent(bounds)
tileInfo := map[string]interface{}{
"rows": 256,
"cols": 256,
"dpi": dpi,
"origin": map[string]float32{
"x": -20037508.342787,
"y": 20037508.342787,
},
"spatialReference": webMercatorSR,
"lods": lods,
}
documentInfo := map[string]string{
"Title": name,
"Author": attribution,
"Comments": "",
"Subject": "",
"Category": "",
"Keywords": tags,
"Credits": credits,
}
out := map[string]interface{}{
"currentVersion": "10.4",
"id": id,
"name": name,
"mapName": name,
"capabilities": "Map,TilesOnly",
"description": description,
"serviceDescription": description,
"copyrightText": attribution,
"singleFusedMapCache": true,
"supportedImageFormatTypes": strings.ToUpper(imgFormat),
"units": "esriMeters",
"layers": []arcGISLayerStub{
{
ID: 0,
Name: name,
ParentLayerID: -1,
DefaultVisibility: true,
SubLayerIDs: nil,
MinScale: minScale,
MaxScale: maxScale,
},
},
"tables": []string{},
"spatialReference": webMercatorSR,
"minScale": minScale,
"maxScale": maxScale,
"tileInfo": tileInfo,
"documentInfo": documentInfo,
"initialExtent": extent,
"fullExtent": extent,
"exportTilesAllowed": false,
"maxExportTilesCount": 0,
"resampling": false,
}
bytes, err := json.Marshal(out)
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("cannot marshal ArcGIS service info JSON: %v", err)
}
return http.StatusOK, wrapJSONP(w, r, bytes)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("Could not render ArcGIS Service JSON to JSONP for %v: %v", r.URL.Path, err)
}
}
func (s *ServiceSet) arcgisLayers(id string, db *mbtiles.DB) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
metadata, err := db.ReadMetadata()
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("Could not read metadata for tileset %v", id)
}
// arcGISLayersJSON returns JSON for the layers in a given ArcGIS tile service
func (ts *Tileset) arcgisLayersJSON() ([]byte, error) {
metadata, err := ts.db.ReadMetadata()
if err != nil {
return nil, err
}
name, _ := metadata["name"].(string)
description, _ := metadata["description"].(string)
attribution, _ := metadata["attribution"].(string)
name, _ := metadata["name"].(string)
description, _ := metadata["description"].(string)
attribution, _ := metadata["attribution"].(string)
bounds, ok := metadata["bounds"].([]float32)
if !ok {
bounds = []float32{-180, -85, 180, 85} // default to world bounds
}
extent := geoBoundsToWMExtent(bounds)
bounds, ok := metadata["bounds"].([]float32)
if !ok {
bounds = []float32{-180, -85, 180, 85} // default to world bounds
}
extent := geoBoundsToWMExtent(bounds)
minZoom, _ := metadata["minzoom"].(uint8)
maxZoom, _ := metadata["maxzoom"].(uint8)
minScale, _ := calcScaleResolution(minZoom, dpi)
maxScale, _ := calcScaleResolution(maxZoom, dpi)
minZoom, _ := metadata["minzoom"].(uint8)
maxZoom, _ := metadata["maxzoom"].(uint8)
minScale, _ := calcScaleResolution(minZoom, dpi)
maxScale, _ := calcScaleResolution(maxZoom, dpi)
// for now, just create a placeholder root layer
emptyArray := []interface{}{}
emptyLayerArray := []arcGISLayerStub{}
// for now, just create a placeholder root layer
emptyArray := []interface{}{}
emptyLayerArray := []arcGISLayerStub{}
var layers [1]arcGISLayer
layers[0] = arcGISLayer{
ID: 0,
DefaultVisibility: true,
ParentLayer: nil,
Name: name,
Description: description,
Extent: extent,
MinScale: minScale,
MaxScale: maxScale,
CopyrightText: attribution,
HTMLPopupType: "esriServerHTMLPopupTypeAsHTMLText",
Fields: emptyArray,
Relationships: emptyArray,
SubLayers: emptyLayerArray,
CurrentVersion: 10.4,
Capabilities: "Map",
}
var layers [1]arcGISLayer
layers[0] = arcGISLayer{
ID: 0,
DefaultVisibility: true,
ParentLayer: nil,
Name: name,
Description: description,
Extent: extent,
MinScale: minScale,
MaxScale: maxScale,
CopyrightText: attribution,
HTMLPopupType: "esriServerHTMLPopupTypeAsHTMLText",
Fields: emptyArray,
Relationships: emptyArray,
SubLayers: emptyLayerArray,
CurrentVersion: 10.4,
Capabilities: "Map",
}
out := map[string]interface{}{
"layers": layers,
}
out := map[string]interface{}{
"layers": layers,
}
bytes, err := json.Marshal(out)
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("cannot marshal ArcGIS layer info JSON: %v", err)
}
bytes, err := json.Marshal(out)
if err != nil {
return nil, err
}
return bytes, nil
}
return http.StatusOK, wrapJSONP(w, r, bytes)
// arcgisLayersHandler is an http.HandlerFunc that returns standard ArcGIS
// Layers JSON for a given ArcGIS tile service
func (ts *Tileset) arcgisLayersHandler(w http.ResponseWriter, r *http.Request) {
layersJSON, err := ts.arcgisLayersJSON()
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("Could not render ArcGIS layer JSON for %v: %v", r.URL.Path, err)
return
}
err = wrapJSONP(w, r, layersJSON)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("Could not render ArcGIS layers JSON to JSONP for %v: %v", r.URL.Path, err)
}
}
func (s *ServiceSet) arcgisLegend(id string, db *mbtiles.DB) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
// arcgisLegendJSON returns minimal ArcGIS legend JSON for a given ArcGIS
// tile service. Legend elements are not yet supported.
func (ts *Tileset) arcgisLegendJSON() ([]byte, error) {
metadata, err := ts.db.ReadMetadata()
if err != nil {
return nil, err
}
metadata, err := db.ReadMetadata()
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("Could not read metadata for tileset %v", id)
}
name, _ := metadata["name"].(string)
name, _ := metadata["name"].(string)
// TODO: pull the legend from ArcGIS specific metadata tables
var elements [0]interface{}
var layers [1]map[string]interface{}
// TODO: pull the legend from ArcGIS specific metadata tables
var elements [0]interface{}
var layers [1]map[string]interface{}
layers[0] = map[string]interface{}{
"layerId": 0,
"layerName": name,
"layerType": "",
"minScale": 0,
"maxScale": 0,
"legend": elements,
}
layers[0] = map[string]interface{}{
"layerId": 0,
"layerName": name,
"layerType": "",
"minScale": 0,
"maxScale": 0,
"legend": elements,
}
out := map[string]interface{}{
"layers": layers,
}
out := map[string]interface{}{
"layers": layers,
}
bytes, err := json.Marshal(out)
if err != nil {
return nil, err
}
return bytes, nil
}
bytes, err := json.Marshal(out)
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("cannot marshal ArcGIS legend info JSON: %v", err)
}
return http.StatusOK, wrapJSONP(w, r, bytes)
// arcgisLegendHandler is an http.HandlerFunc that returns minimal ArcGIS
// legend JSON for a given ArcGIS tile service
func (ts *Tileset) arcgisLegendHandler(w http.ResponseWriter, r *http.Request) {
legendJSON, err := ts.arcgisLegendJSON()
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("Could not render ArcGIS legend JSON for %v: %v", r.URL.Path, err)
return
}
err = wrapJSONP(w, r, legendJSON)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("Could not render ArcGIS legend JSON to JSONP for %v: %v", r.URL.Path, err)
}
}
func (s *ServiceSet) arcgisTiles(db *mbtiles.DB) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
// split path components to extract tile coordinates x, y and z
pcs := strings.Split(r.URL.Path[1:], "/")
// strip off /arcgis/rest/services/ and then
// we should have at least <id> , "MapServer", "tiles", <z>, <y>, <x>
l := len(pcs)
if l < 6 || pcs[5] == "" {
return http.StatusBadRequest, fmt.Errorf("requested path is too short")
}
z, y, x := pcs[l-3], pcs[l-2], pcs[l-1]
tc, _, err := tileCoordFromString(z, x, y)
if err != nil {
return http.StatusBadRequest, err
}
// arcgisTileHandler returns an image tile or blank image for a given
// tile request within a given ArcGIS tile service
func (ts *Tileset) arcgisTileHandler(w http.ResponseWriter, r *http.Request) {
db := ts.db
var data []byte
err = db.ReadTile(tc.z, tc.x, tc.y, &data)
// split path components to extract tile coordinates x, y and z
pcs := strings.Split(r.URL.Path[1:], "/")
// strip off /arcgis/rest/services/ and then
// we should have at least <id> , "MapServer", "tiles", <z>, <y>, <x>
l := len(pcs)
if l < 6 || pcs[5] == "" {
http.Error(w, "requested path is too short", http.StatusBadRequest)
return
}
z, y, x := pcs[l-3], pcs[l-2], pcs[l-1]
tc, _, err := tileCoordFromString(z, x, y)
if err != nil {
http.Error(w, "invalid tile coordinates", http.StatusBadRequest)
return
}
var data []byte
err = db.ReadTile(tc.z, tc.x, tc.y, &data)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("cannot fetch tile from DB for z=%d, x=%d, y=%d for %v: %v", tc.z, tc.x, tc.y, r.URL.Path, err)
return
}
if data == nil || len(data) <= 1 {
// Return blank PNG for all image types
w.Header().Set("Content-Type", "image/png")
_, err = w.Write(BlankPNG())
if err != nil {
// augment error info
t := "tile"
err = fmt.Errorf("cannot fetch %s from DB for z=%d, x=%d, y=%d: %v", t, tc.z, tc.x, tc.y, err)
return http.StatusInternalServerError, err
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("could not return blank image for %v: %v", r.URL.Path, err)
}
} else {
w.Header().Set("Content-Type", db.ContentType())
_, err = w.Write(data)
if data == nil || len(data) <= 1 {
// Return blank PNG for all image types
w.Header().Set("Content-Type", "image/png")
_, err = w.Write(BlankPNG())
} else {
w.Header().Set("Content-Type", db.ContentType())
_, err = w.Write(data)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("could not write tile data to response for %v: %v", r.URL.Path, err)
}
return http.StatusOK, err
}
}
@ -337,13 +376,6 @@ func geoBoundsToWMExtent(bounds []float32) arcGISExtent {
}
}
func calcScaleResolution(zoomLevel uint8, dpi uint8) (float64, float64) {
var denom = 1 << zoomLevel
resolution := initialResolution / float64(denom)
scale := float64(dpi) * 39.37 * resolution // 39.37 in/m
return scale, resolution
}
// Cast interface to a string if not nil, otherwise empty string
func toString(s interface{}) string {
if s != nil {
@ -370,17 +402,18 @@ func geoToMercator(longitude, latitude float64) (float64, float64) {
// ArcGISHandler returns a http.Handler that serves the ArcGIS endpoints of the ServiceSet.
// The function ef is called with any occurring error if it is non-nil, so it
// can be used for e.g. logging with logging facilities of the caller.
func (s *ServiceSet) ArcGISHandler(ef func(error)) http.Handler {
m := http.NewServeMux()
rootPath := "/arcgis/rest/services/"
// func (s *ServiceSet) ArcGISHandler(ef func(error)) http.Handler {
// m := http.NewServeMux()
for id, db := range s.tilesets {
p := rootPath + id + "/MapServer"
m.Handle(p, wrapGetWithErrors(ef, hmacAuth(s.arcgisService(id, db), s.secretKey, id)))
m.Handle(p+"/layers", wrapGetWithErrors(ef, hmacAuth(s.arcgisLayers(id, db), s.secretKey, id)))
m.Handle(p+"/legend", wrapGetWithErrors(ef, hmacAuth(s.arcgisLegend(id, db), s.secretKey, id)))
// root := "/arcgis/rest/services/"
m.Handle(p+"/tile/", wrapGetWithErrors(ef, hmacAuth(s.arcgisTiles(db), s.secretKey, id)))
}
return m
}
// for id, ts := range s.tilesets {
// prefix := root + id + "/MapServer"
// m.Handle(prefix, wrapGetWithErrors(ef, hmacAuth(ts.arcgisServiceHandler(), s.secretKey, id)))
// m.Handle(prefix+"/layers", wrapGetWithErrors(ef, hmacAuth(ts.arcgisLayersHandler(), s.secretKey, id)))
// m.Handle(prefix+"/legend", wrapGetWithErrors(ef, hmacAuth(ts.arcgisLegendHandler(), s.secretKey, id)))
// m.Handle(prefix+"/tile/", wrapGetWithErrors(ef, hmacAuth(ts.arcgisTileHandler(), s.secretKey, id)))
// }
// return m
// }

12
handlers/assets_vfsdata.go
View File

File diff suppressed because one or more lines are too long

502
handlers/handlers.go
View File

@ -1,502 +0,0 @@
package handlers
//go:generate go run -tags=dev assets_generate.go
import (
"bytes"
"crypto/hmac"
"crypto/sha1"
"crypto/subtle"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"html/template"
"io"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/shurcooL/httpfs/html/vfstemplate"
log "github.com/sirupsen/logrus"
"github.com/consbio/mbtileserver/mbtiles"
)
const maxSignatureAge = time.Duration(15) * time.Minute
// scheme returns the underlying URL scheme of the original request.
func scheme(r *http.Request) string {
if r.TLS != nil {
return "https"
}
if scheme := r.Header.Get("X-Forwarded-Proto"); scheme != "" {
return scheme
}
if scheme := r.Header.Get("X-Forwarded-Protocol"); scheme != "" {
return scheme
}
if ssl := r.Header.Get("X-Forwarded-Ssl"); ssl == "on" {
return "https"
}
if scheme := r.Header.Get("X-Url-Scheme"); scheme != "" {
return scheme
}
return "http"
}
type handlerFunc func(http.ResponseWriter, *http.Request) (int, error)
func wrapGetWithErrors(ef func(error), hf handlerFunc) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" {
status := http.StatusMethodNotAllowed
http.Error(w, http.StatusText(status), status)
return
}
status, err := hf(w, r) // run the handlerFunc and obtain the return codes
if err != nil && ef != nil {
ef(err) // handle the error with the supplied function
}
// in case it's an error, write the status code for the requester
if status >= 400 {
http.Error(w, http.StatusText(status), status)
}
})
}
// hmacAuth wraps handler functions to provide request authentication. If
// -s/--secret-key is provided at startup, this function will enforce proper
// request signing. Otherwise, it will simply pass requests through to the
// handler.
func hmacAuth(hf handlerFunc, secretKey string, serviceId string) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
// If secret key isn't set, allow all requests
if secretKey == "" {
return hf(w, r)
}
query := r.URL.Query()
rawSignature := query.Get("signature")
if rawSignature == "" {
rawSignature = r.Header.Get("X-Signature")
}
if rawSignature == "" {
return 400, errors.New("No signature provided")
}
rawSignDate := query.Get("date")
if rawSignDate == "" {
rawSignDate = r.Header.Get("X-Signature-Date")
}
if rawSignDate == "" {
return 400, errors.New("No signature date provided")
}
signDate, err := time.Parse(time.RFC3339Nano, rawSignDate)
if err != nil {
return 400, errors.New("Signature date is not valid RFC3339")
}
if time.Now().Sub(signDate) > maxSignatureAge {
return 400, errors.New("Signature is expired")
}
signatureParts := strings.SplitN(rawSignature, ":", 2)
if len(signatureParts) != 2 {
return 400, errors.New("Signature does not contain salt")
}
salt, signature := signatureParts[0], signatureParts[1]
key := sha1.New()
key.Write([]byte(salt + secretKey))
hash := hmac.New(sha1.New, key.Sum(nil))
message := fmt.Sprintf("%s:%s", rawSignDate, serviceId)
hash.Write([]byte(message))
checkSignature := base64.RawURLEncoding.EncodeToString(hash.Sum(nil))
if subtle.ConstantTimeCompare([]byte(signature), []byte(checkSignature)) == 1 {
return hf(w, r)
}
return 400, errors.New("Invalid signature")
}
}
// ServiceInfo consists of two strings that contain the image type and a URL.
type ServiceInfo struct {
ImageType string `json:"imageType"`
URL string `json:"url"`
}
// ServiceSet is the base type for the HTTP handlers which combines multiple
// mbtiles.DB tilesets.
type ServiceSet struct {
tilesets map[string]*mbtiles.DB
templates *template.Template
Domain string
Path string
secretKey string
}
// New returns a new ServiceSet. Use AddDBOnPath to add a mbtiles file.
func New() *ServiceSet {
s := &ServiceSet{
tilesets: make(map[string]*mbtiles.DB),
templates: template.New("_base_"),
}
// load templates
vfstemplate.ParseFiles(Assets, s.templates, "map.html", "map_gl.html")
return s
}
// AddDBOnPath interprets filename as mbtiles file which is opened and which will be
// served under "/services/<urlPath>" by Handler(). The parameter urlPath may not be
// nil, otherwise an error is returned. In case the DB cannot be opened the returned
// error is non-nil.
func (s *ServiceSet) AddDBOnPath(filename string, urlPath string) error {
var err error
if urlPath == "" {
return fmt.Errorf("path parameter may not be empty")
}
ts, err := mbtiles.NewDB(filename)
if err != nil {
return fmt.Errorf("Invalid mbtiles file %q: %v", filename, err)
}
s.tilesets[urlPath] = ts
return nil
}
// NewFromBaseDir returns a ServiceSet that combines all .mbtiles files under
// the directory at baseDir. The DBs will all be served under their relative paths
// to baseDir. If baseDir does not exist, is not a valid path, or does not contain
// any valid .mbtiles files, an empty ServiceSet will be returned along with the error.
func NewFromBaseDir(baseDir string) (*ServiceSet, error) {
s := New()
var filenames []string
err := filepath.Walk(baseDir, func(p string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if _, err := os.Stat(p + "-journal"); err == nil {
// Don't try to load .mbtiles files that are being written
return nil
}
if ext := filepath.Ext(p); ext == ".mbtiles" {
filenames = append(filenames, p)
}
return nil
})
if err != nil {
return s, err
}
if len(filenames) == 0 {
return s, fmt.Errorf("no tilesets found in %s", baseDir)
}
for _, filename := range filenames {
subpath, err := filepath.Rel(baseDir, filename)
if err != nil {
return nil, fmt.Errorf("unable to extract URL path for %q: %v", filename, err)
}
e := filepath.Ext(filename)
p := filepath.ToSlash(subpath)
id := p[:len(p)-len(e)]
err = s.AddDBOnPath(filename, id)
if err != nil {
log.Warnf("%s\nThis tileset will not be available from the API", err.Error())
}
}
return s, nil
}
// SetRequestAuthKey sets the secret key used to verify that incoming requests
// are authorized. If blank, no authorization is performed.
func (s *ServiceSet) SetRequestAuthKey(key string) {
s.secretKey = key
}
// Size returns the number of tilesets in this ServiceSet
func (s *ServiceSet) Size() int {
return len(s.tilesets)
}
// rootURL returns the root URL of the service. If s.Domain is non-empty, it
// will be used as the hostname. If s.Path is non-empty, it will be used as a
// prefix.
func (s *ServiceSet) rootURL(r *http.Request) string {
host := r.Host
if len(s.Domain) > 0 {
host = s.Domain
}
root := fmt.Sprintf("%s://%s", scheme(r), host)
if len(s.Path) > 0 {
root = fmt.Sprintf("%s/%s", root, s.Path)
}
return root
}
func (s *ServiceSet) listServices(w http.ResponseWriter, r *http.Request) (int, error) {
rootURL := fmt.Sprintf("%s%s", s.rootURL(r), r.URL)
services := []ServiceInfo{}
for id, tileset := range s.tilesets {
services = append(services, ServiceInfo{
ImageType: tileset.TileFormatString(),
URL: fmt.Sprintf("%s/%s", rootURL, id),
})
}
bytes, err := json.Marshal(services)
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("cannot marshal services JSON: %v", err)
}
w.Header().Set("Content-Type", "application/json")
_, err = w.Write(bytes)
return http.StatusOK, err
}
func (s *ServiceSet) tileJSON(id string, db *mbtiles.DB, mapURL bool) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
query := ""
if r.URL.RawQuery != "" {
query = "?" + r.URL.RawQuery
}
svcURL := fmt.Sprintf("%s%s", s.rootURL(r), r.URL.Path)
imgFormat := db.TileFormatString()
out := map[string]interface{}{
"tilejson": "2.1.0",
"id": id,
"scheme": "xyz",
"format": imgFormat,
"tiles": []string{fmt.Sprintf("%s/tiles/{z}/{x}/{y}.%s%s", svcURL, imgFormat, query)},
}
if mapURL {
out["map"] = fmt.Sprintf("%s/map", svcURL)
}
metadata, err := db.ReadMetadata()
if err != nil {
return http.StatusInternalServerError, err
}
for k, v := range metadata {
switch k {
// strip out values above
case "tilejson", "id", "scheme", "format", "tiles", "map":
continue
// strip out values that are not supported or are overridden below
case "grids", "interactivity", "modTime":
continue
// strip out values that come from TileMill but aren't useful here
case "metatile", "scale", "autoscale", "_updated", "Layer", "Stylesheet":
continue
default:
out[k] = v
}
}
if db.HasUTFGrid() {
out["grids"] = []string{fmt.Sprintf("%s/tiles/{z}/{x}/{y}.json%s", svcURL, query)}
}
bytes, err := json.Marshal(out)
if err != nil {
return http.StatusInternalServerError, fmt.Errorf("cannot marshal service info JSON: %v", err)
}
w.Header().Set("Content-Type", "application/json")
_, err = w.Write(bytes)
return http.StatusOK, err
}
}
// executeTemplates first tries to find the template with the given name for
// the ServiceSet. If that fails because it is not available, an HTTP status
// Internal Server Error is returned.
func (s *ServiceSet) executeTemplate(w http.ResponseWriter, name string, data interface{}) (int, error) {
t := s.templates.Lookup(name)
if t == nil {
return http.StatusInternalServerError, fmt.Errorf("template not found %q", name)
}
buf := &bytes.Buffer{}
err := t.Execute(buf, data)
if err != nil {
return http.StatusInternalServerError, err
}
_, err = io.Copy(w, buf)
return http.StatusOK, err
}
func (s *ServiceSet) serviceHTML(id string, db *mbtiles.DB) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
p := struct {
URL string
ID string
}{
fmt.Sprintf("%s%s", s.rootURL(r), strings.TrimSuffix(r.URL.Path, "/map")),
id,
}
switch db.TileFormat() {
default:
return s.executeTemplate(w, "map", p)
case mbtiles.PBF:
return s.executeTemplate(w, "map_gl", p)
}
}
}
type tileCoord struct {
z uint8
x, y uint64
}
// tileCoordFromString parses and returns tileCoord coordinates and an optional
// extension from the three parameters. The parameter z is interpreted as the
// web mercator zoom level, it is supposed to be an unsigned integer that will
// fit into 8 bit. The parameters x and y are interpreted as longitude and
// latitude tile indices for that zoom level, both are supposed be integers in
// the integer interval [0,2^z). Additionally, y may also have an optional
// filename extension (e.g. "42.png") which is removed before parsing the
// number, and returned, too. In case an error occurred during parsing or if the
// values are not in the expected interval, the returned error is non-nil.
func tileCoordFromString(z, x, y string) (tc tileCoord, ext string, err error) {
var z64 uint64
if z64, err = strconv.ParseUint(z, 10, 8); err != nil {
err = fmt.Errorf("cannot parse zoom level: %v", err)
return
}
tc.z = uint8(z64)
const (
errMsgParse = "cannot parse %s coordinate axis: %v"
errMsgOOB = "%s coordinate (%d) is out of bounds for zoom level %d"
)
if tc.x, err = strconv.ParseUint(x, 10, 64); err != nil {
err = fmt.Errorf(errMsgParse, "first", err)
return
}
if tc.x >= (1 << z64) {
err = fmt.Errorf(errMsgOOB, "x", tc.x, tc.z)
return
}
s := y
if l := strings.LastIndex(s, "."); l >= 0 {
s, ext = s[:l], s[l:]
}
if tc.y, err = strconv.ParseUint(s, 10, 64); err != nil {
err = fmt.Errorf(errMsgParse, "y", err)
return
}
if tc.y >= (1 << z64) {
err = fmt.Errorf(errMsgOOB, "y", tc.y, tc.z)
return
}
return
}
// tileNotFoundHandler writes the default response for a non-existing tile of type f to w
func tileNotFoundHandler(w http.ResponseWriter, f mbtiles.TileFormat) (int, error) {
var err error
switch f {
case mbtiles.PNG, mbtiles.JPG, mbtiles.WEBP:
// Return blank PNG for all image types
w.Header().Set("Content-Type", "image/png")
w.WriteHeader(http.StatusOK)
_, err = w.Write(BlankPNG())
case mbtiles.PBF:
// Return 204
w.WriteHeader(http.StatusNoContent)
default:
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"message": "Tile does not exist"}`)
}
return http.StatusOK, err // http.StatusOK doesn't matter, code was written by w.WriteHeader already
}
func (s *ServiceSet) tiles(db *mbtiles.DB) handlerFunc {
return func(w http.ResponseWriter, r *http.Request) (int, error) {
// split path components to extract tile coordinates x, y and z
pcs := strings.Split(r.URL.Path[1:], "/")
// we are expecting at least "services", <id> , "tiles", <z>, <x>, <y plus .ext>
l := len(pcs)
if l < 6 || pcs[5] == "" {
return http.StatusBadRequest, fmt.Errorf("requested path is too short")
}
z, x, y := pcs[l-3], pcs[l-2], pcs[l-1]
tc, ext, err := tileCoordFromString(z, x, y)
if err != nil {
return http.StatusBadRequest, err
}
var data []byte
// flip y to match the spec
tc.y = (1 << uint64(tc.z)) - 1 - tc.y
isGrid := ext == ".json"
switch {
case !isGrid:
err = db.ReadTile(tc.z, tc.x, tc.y, &data)
case isGrid && db.HasUTFGrid():
err = db.ReadGrid(tc.z, tc.x, tc.y, &data)
default:
err = fmt.Errorf("no grid supplied by tile database")
}
if err != nil {
// augment error info
t := "tile"
if isGrid {
t = "grid"
}
err = fmt.Errorf("cannot fetch %s from DB for z=%d, x=%d, y=%d: %v", t, tc.z, tc.x, tc.y, err)
return http.StatusInternalServerError, err
}
if data == nil || len(data) <= 1 {
return tileNotFoundHandler(w, db.TileFormat())
}
if isGrid {
w.Header().Set("Content-Type", "application/json")
if db.UTFGridCompression() == mbtiles.ZLIB {
w.Header().Set("Content-Encoding", "deflate")
} else {
w.Header().Set("Content-Encoding", "gzip")
}
} else {
w.Header().Set("Content-Type", db.ContentType())
if db.TileFormat() == mbtiles.PBF {
w.Header().Set("Content-Encoding", "gzip")
}
}
_, err = w.Write(data)
return http.StatusOK, err
}
}
// Handler returns a http.Handler that serves the endpoints of the ServiceSet.
// The function ef is called with any occurring error if it is non-nil, so it
// can be used for e.g. logging with logging facilities of the caller.
// When the publish parameter is true, a listing of all available services and
// an endpoint with a HTML slippy map for each service are served by the Handler.
func (s *ServiceSet) Handler(ef func(error), publish bool) http.Handler {
m := http.NewServeMux()
if publish {
m.Handle("/services", wrapGetWithErrors(ef, hmacAuth(s.listServices, s.secretKey, "")))
}
for id, db := range s.tilesets {
p := "/services/" + id
m.Handle(p, wrapGetWithErrors(ef, hmacAuth(s.tileJSON(id, db, publish), s.secretKey, id)))
m.Handle(p+"/tiles/", wrapGetWithErrors(ef, hmacAuth(s.tiles(db), s.secretKey, id)))
if publish {
m.Handle(p+"/map", wrapGetWithErrors(ef, hmacAuth(s.serviceHTML(id, db), s.secretKey, id)))
}
}
return m
}

9
handlers/handlers_test.go
View File

@ -1,9 +0,0 @@
package handlers_test
import (
"testing"
)
func testFoo(t *testing.T) {
}

24
handlers/id.go Normal file
View File

@ -0,0 +1,24 @@
package handlers
import (
"crypto/sha1"
"encoding/base64"
"path/filepath"
)
// SHA1ID generates a URL safe base64 encoded SHA1 hash of the filename.
func SHA1ID(filename string) string {
// generate IDs from hash of full file path
filenameHash := sha1.Sum([]byte(filename))
return base64.RawURLEncoding.EncodeToString(filenameHash[:])
}
// RelativePathID returns a relative path from the basedir to the filename.
func RelativePathID(filename, baseDir string) (string, error) {
subpath, err := filepath.Rel(baseDir, filename)
if err != nil {
return "", err
}
subpath = filepath.ToSlash(subpath)
return subpath[:len(subpath)-len(filepath.Ext(filename))], nil
}

85
handlers/middleware.go Normal file
View File

@ -0,0 +1,85 @@
package handlers
import (
"crypto/hmac"
"crypto/sha1"
"crypto/subtle"
"encoding/base64"
"fmt"
"net/http"
"strings"
"time"
)
// HandlerWrapper is a type definition for a function that takes an http.Handler
// and returns an http.Handler
type HandlerWrapper func(http.Handler) http.Handler
// maxSignatureAge defines the maximum amount of time, in seconds
// that an HMAC signature can remain valid
const maxSignatureAge = time.Duration(15) * time.Minute
// HMACAuthMiddleware wraps incoming requests to enforce HMAC signature authorization.
// All requests are expected to have either "signature" and "date" query parameters
// or "X-Signature" and "X-Signature-Date" headers.
func HMACAuthMiddleware(secretKey string, serviceSet *ServiceSet) HandlerWrapper {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
query := r.URL.Query()
rawSignature := query.Get("signature")
if rawSignature == "" {
rawSignature = r.Header.Get("X-Signature")
}
if rawSignature == "" {
http.Error(w, "No signature provided", http.StatusUnauthorized)
return
}
rawSignDate := query.Get("date")
if rawSignDate == "" {
rawSignDate = r.Header.Get("X-Signature-Date")
}
if rawSignDate == "" {
http.Error(w, "No signature date provided", http.StatusUnauthorized)
return
}
signDate, err := time.Parse(time.RFC3339Nano, rawSignDate)
if err != nil {
http.Error(w, "Signature date is not valid RFC3339", http.StatusBadRequest)
return
}
if time.Now().Sub(signDate) > maxSignatureAge {
http.Error(w, "Signature is expired", http.StatusUnauthorized)
return
}
signatureParts := strings.SplitN(rawSignature, ":", 2)
if len(signatureParts) != 2 {
http.Error(w, "Signature does not contain salt", http.StatusBadRequest)
return
}
salt, signature := signatureParts[0], signatureParts[1]
tilesetID := serviceSet.IDFromURLPath(r.URL.Path)
key := sha1.New()
key.Write([]byte(salt + secretKey))
hash := hmac.New(sha1.New, key.Sum(nil))
message := fmt.Sprintf("%s:%s", rawSignDate, tilesetID)
hash.Write([]byte(message))
checkSignature := base64.RawURLEncoding.EncodeToString(hash.Sum(nil))
if subtle.ConstantTimeCompare([]byte(signature), []byte(checkSignature)) != 1 {
// Signature is not valid for the requested resource
// either tilesetID does not match in the signature, or date
http.Error(w, "Signature not authorized for resource", http.StatusUnauthorized)
return
}
next.ServeHTTP(w, r)
})
}
}

45
handlers/request.go Normal file
View File

@ -0,0 +1,45 @@
package handlers
import (
"fmt"
"net/http"
)
// scheme returns the underlying URL scheme of the original request.
func scheme(r *http.Request) string {
if r.TLS != nil {
return "https"
}
if scheme := r.Header.Get("X-Forwarded-Proto"); scheme != "" {
return scheme
}
if scheme := r.Header.Get("X-Forwarded-Protocol"); scheme != "" {
return scheme
}
if ssl := r.Header.Get("X-Forwarded-Ssl"); ssl == "on" {
return "https"
}
if scheme := r.Header.Get("X-Url-Scheme"); scheme != "" {
return scheme
}
return "http"
}
type handlerFunc func(http.ResponseWriter, *http.Request) (int, error)
// wrapJSONP writes b (JSON marshalled to bytes) as a JSONP response to
// w if the callback query parameter is present, and writes b as a JSON
// response otherwise. Any error that occurs during writing is returned.
func wrapJSONP(w http.ResponseWriter, r *http.Request, b []byte) (err error) {
callback := r.URL.Query().Get("callback")
if callback != "" {
w.Header().Set("Content-Type", "application/javascript")
_, err = w.Write([]byte(fmt.Sprintf("%s(%s);", callback, b)))
return
}
w.Header().Set("Content-Type", "application/json")
_, err = w.Write(b)
return
}

258
handlers/serviceset.go Normal file
View File

@ -0,0 +1,258 @@
package handlers
import (
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"net/url"
"sort"
"strings"
)
// ServiceSetConfig provides configuration options for a ServiceSet
type ServiceSetConfig struct {
EnableServiceList bool
EnableTileJSON bool
EnablePreview bool
EnableArcGIS bool
RootURL *url.URL
ErrorWriter io.Writer
}
// ServiceSet is a group of tilesets plus configuration options.
// It provides access to all tilesets from a root URL.
type ServiceSet struct {
tilesets map[string]*Tileset
enableServiceList bool
enableTileJSON bool
enablePreview bool
enableArcGIS bool
domain string
rootURL *url.URL
errorWriter io.Writer
}
// New returns a new ServiceSet.
// If no ServiceSetConfig is provided, the service is initialized with default
// values of ServiceSetConfig.
func New(cfg *ServiceSetConfig) (*ServiceSet, error) {
if cfg == nil {
cfg = &ServiceSetConfig{}
}
s := &ServiceSet{
tilesets: make(map[string]*Tileset),
enableServiceList: cfg.EnableServiceList,
enableTileJSON: cfg.EnableTileJSON,
enablePreview: cfg.EnablePreview,
enableArcGIS: cfg.EnableArcGIS,
rootURL: cfg.RootURL,
errorWriter: cfg.ErrorWriter,
}
return s, nil
}
// AddTileset adds a single tileset identified by idGenerator using the filename.
// If a service already exists with that ID, an error is returned.
func (s *ServiceSet) AddTileset(filename, id string) error {
if _, ok := s.tilesets[id]; ok {
return fmt.Errorf("Tileset already exists for ID: %q", id)
}
path := s.rootURL.Path + "/" + id
ts, err := newTileset(s, filename, id, path)
if err != nil {
return err
}
s.tilesets[id] = ts
return nil
}
// UpdateTileset reloads the Tileset identified by id, if it already exists.
// Otherwise, this returns an error.
// Any errors encountered updating the Tileset are returned.
func (s *ServiceSet) UpdateTileset(id string) error {
ts, ok := s.tilesets[id]
if !ok {
return fmt.Errorf("Tileset does not exist with ID: %q", id)
}
err := ts.reload()
if err != nil {
return err
}
return nil
}
// RemoveTileset removes the Tileset and closes the associated mbtiles file
// identified by id, if it already exists.
// If it does not exist, this returns without error.
// Any errors encountered removing the Tileset are returned.
func (s *ServiceSet) RemoveTileset(id string) error {
ts, ok := s.tilesets[id]
if !ok {
return nil
}
err := ts.delete()
if err != nil {
return err
}
// remove from tilesets and router
delete(s.tilesets, id)
return nil
}
// HasTileset returns true if the tileset identified by id exists within this
// ServiceSet.
func (s *ServiceSet) HasTileset(id string) bool {
if _, ok := s.tilesets[id]; ok {
return true
}
return false
}
// Size returns the number of tilesets in this ServiceSet
func (s *ServiceSet) Size() int {
return len(s.tilesets)
}
// ServiceInfo provides basic information about the service.
type ServiceInfo struct {
ImageType string `json:"imageType"`
URL string `json:"url"`
Name string `json:"name"`
}
// logError writes to the configured ServiceSet.errorWriter if available
// or the standard logger otherwise.
func (s *ServiceSet) logError(format string, args ...interface{}) {
if s.errorWriter != nil {
s.errorWriter.Write([]byte(fmt.Sprintf(format, args...)))
} else {
log.Printf(format, args...)
}
}
// serviceListHandler is an http.HandlerFunc that provides a listing of all
// published services in this ServiceSet
func (s *ServiceSet) serviceListHandler(w http.ResponseWriter, r *http.Request) {
rootURL := fmt.Sprintf("%s://%s%s", scheme(r), r.Host, r.URL)
services := []ServiceInfo{}
// sort ids alpabetically
var ids []string
for id := range s.tilesets {
ids = append(ids, id)
}
sort.Strings(ids)
for _, id := range ids {
ts := s.tilesets[id]
services = append(services, ServiceInfo{
ImageType: ts.tileFormatString(),
URL: fmt.Sprintf("%s/%s", rootURL, id),
Name: ts.name,
})
}
bytes, err := json.Marshal(services)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
s.logError("Error marshalling service list JSON for %v: %v", r.URL.Path, err)
return
}
w.Header().Set("Content-Type", "application/json")
_, err = w.Write(bytes)
if err != nil {
s.logError("Error writing service list content: %v", err)
}
}
// tilesetHandler is an http.HandlerFunc that handles a given tileset
// and associated subpaths
func (s *ServiceSet) tilesetHandler(w http.ResponseWriter, r *http.Request) {
id := s.IDFromURLPath((r.URL.Path))
if id == "" {
http.Error(w, "404 page not found", http.StatusNotFound)
return
}
s.tilesets[id].router.ServeHTTP(w, r)
}
// IDFromURLPath extracts a tileset ID from a URL Path.
// If no valid ID is found, a blank string is returned.
func (s *ServiceSet) IDFromURLPath(id string) string {
root := s.rootURL.Path + "/"
if strings.HasPrefix(id, root) {
id = strings.TrimPrefix(id, root)
// test exact match first
if _, ok := s.tilesets[id]; ok {
return id
}
// Split on /tiles/ and /map/ and trim /map
i := strings.LastIndex(id, "/tiles/")
if i != -1 {
id = id[:i]
} else if s.enablePreview {
id = strings.TrimSuffix(id, "/map")
i = strings.LastIndex(id, "/map/")
if i != -1 {
id = id[:i]
}
}
} else if s.enableArcGIS && strings.HasPrefix(id, ArcGISRoot) {
id = strings.TrimPrefix(id, ArcGISRoot)
// MapServer should be a reserved word, so should be OK to split on it
id = strings.Split(id, "/MapServer")[0]
} else {
// not on a subpath of service roots, so no id
return ""
}
// make sure tileset exists
if _, ok := s.tilesets[id]; ok {
return id
}
return ""
}
// Handler returns a http.Handler that serves the endpoints of the ServiceSet.
// The function ef is called with any occurring error if it is non-nil, so it
// can be used for e.g. logging with logging facilities of the caller.
func (s *ServiceSet) Handler() http.Handler {
m := http.NewServeMux()
root := s.rootURL.Path + "/"
// Route requests at the tileset or subpath to the corresponding tileset
m.HandleFunc(root, s.tilesetHandler)
if s.enableServiceList {
m.HandleFunc(s.rootURL.Path, s.serviceListHandler)
}
if s.enableArcGIS {
m.HandleFunc(ArcGISRoot, s.tilesetHandler)
}
return m
}

14
handlers/static.go
View File

@ -3,16 +3,20 @@ package handlers
import (
"net/http"
"path"
"strings"
"time"
)
// Static returns an http.Handler that will serve the contents of
// the subdirectory "/static/dist" of the Assets.
func Static() http.Handler {
// staticHandler returns a handler that retrieves static files from the virtual
// assets filesystem based on a path. The URL prefix of the resource where
// these are accessed is first trimmed before requesting from the filesystem.
func staticHandler(prefix string) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
f, err := Assets.Open(path.Join("/static/dist", r.URL.Path))
filePath := strings.TrimPrefix(r.URL.Path, prefix)
f, err := Assets.Open(path.Join("/static/dist", filePath))
if err != nil {
http.Error(w, http.StatusText(http.StatusNotFound), http.StatusNotFound)
// not an error, file was not found
http.NotFound(w, r)
return
}
defer f.Close()

38
handlers/templates.go Normal file
View File

@ -0,0 +1,38 @@
package handlers
//go:generate go run -tags=dev assets_generate.go
import (
"bytes"
"fmt"
"html/template"
"io"
"net/http"
"github.com/shurcooL/httpfs/html/vfstemplate"
)
var templates *template.Template
func init() {
// load templates
templates = template.New("_base_")
vfstemplate.ParseFiles(Assets, templates, "map.html", "map_gl.html")
}
// executeTemplates first tries to find the template with the given name for
// the ServiceSet. If that fails because it is not available, an HTTP status
// Internal Server Error is returned.
func executeTemplate(w http.ResponseWriter, name string, data interface{}) (int, error) {
t := templates.Lookup(name)
if t == nil {
return http.StatusInternalServerError, fmt.Errorf("template not found %q", name)
}
buf := &bytes.Buffer{}
err := t.Execute(buf, data)
if err != nil {
return http.StatusInternalServerError, err
}
_, err = io.Copy(w, buf)
return http.StatusOK, err
}

142
handlers/templates/map.html
View File

@ -4,8 +4,8 @@
<head lang="en">
<meta charset="UTF-8" />
<title>{{.ID}} Preview</title>
<script src="/static/core.min.js"></script>
<link href="/static/core.min.css" rel="stylesheet" />
<script src="{{.URL}}/map/static/core.min.js"></script>
<link href="{{.URL}}/map/static/core.min.css" rel="stylesheet" />
<style>
html {
height: 100%;
@ -49,6 +49,9 @@
<body>
<div id="Map"></div>
<script>
// Load raw JSON directly from template
var tileJSON = {{.TileJSON}};
var basemaps = [
L.tileLayer(
"//{s}.arcgisonline.com/ArcGIS/rest/services/World_Topo_Map/MapServer/tile/{z}/{y}/{x}",
@ -90,72 +93,79 @@
];
var map = L.map("Map", {});
map.addControl(
L.control.basemaps({
position: "bottomright",
basemaps: basemaps,
tileX: 0,
tileY: 0,
tileZ: 1
})
);
var layer = null;
d3.json("./").then(function(tileJSON) {
if (tileJSON.bounds) {
var b = tileJSON.bounds;
// TODO: optimize and prevent jitter
map.fitBounds([
[b[1], b[0]],
[b[3], b[2]]
]);
} else {
map.fitWorld();
}
if (tileJSON.bounds) {
var b = tileJSON.bounds;
if (tileJSON.maxzoom && tileJSON.maxzoom < map.getZoom()) {
map.setZoom(tileJSON.maxzoom);
}
// TODO: optimize and prevent jitter
map.fitBounds([
[b[1], b[0]],
[b[3], b[2]]
]);
} else {
map.fitWorld();
}
layer = L.tileLayer(tileJSON.tiles[0], {
minZoom: tileJSON.minzoom || 0,
maxZoom: tileJSON.maxzoom || 23
});
if (tileJSON.maxzoom && tileJSON.maxzoom < map.getZoom()) {
map.setZoom(tileJSON.maxzoom);
}
map.addLayer(layer);
var legendJSON = tileJSON.legend;
if (legendJSON && legendJSON.search(/\{/) === 0) {
legendJSON = JSON.parse(legendJSON);
// Make sure this is the legend JSON structure we expect
if (legendJSON.length && legendJSON[0].elements) {
map.addControl(
L.control.base64legend({
position: "topright",
legends: legendJSON,
collapseSimple: true,
detectStretched: true
})
);
}
}
if (tileJSON.grids && tileJSON.grids.length > 0) {
var gridURL = tileJSON.grids[0];
var utfgrid = L.utfGrid(gridURL, {
resolution: 4,
pointerCursor: true,
mouseInterval: 66 // Delay for mousemove events
});
utfgrid.addTo(map);
var infoContainer = L.DomUtil.create(
"div",
"info",
L.DomUtil.get("Map")
);
var textNode = L.DomUtil.create("h3", "", infoContainer);
textNode.innerHTML =
"This service has UTF-8 Grids; see console for grid data.";
utfgrid.on("mouseover", function(e) {
console.log("UTF grid data:", e.data);
});
}
var layer = L.tileLayer(tileJSON.tiles[0], {
minZoom: tileJSON.minzoom || 0,
maxZoom: tileJSON.maxzoom || 23
});
map.addLayer(layer);
var legendJSON = tileJSON.legend;
if (legendJSON && legendJSON.search(/\{/) === 0) {
legendJSON = JSON.parse(legendJSON);
// Make sure this is the legend JSON structure we expect
if (legendJSON.length && legendJSON[0].elements) {
map.addControl(
L.control.base64legend({
position: "topright",
legends: legendJSON,
collapseSimple: true,
detectStretched: true
})
);
}
}
if (tileJSON.grids && tileJSON.grids.length > 0) {
var gridURL = tileJSON.grids[0];
var utfgrid = L.utfGrid(gridURL, {
resolution: 4,
pointerCursor: true,
mouseInterval: 66 // Delay for mousemove events
});
utfgrid.addTo(map);
var infoContainer = L.DomUtil.create(
"div",
"info",
L.DomUtil.get("Map")
);
var textNode = L.DomUtil.create("h3", "", infoContainer);
textNode.innerHTML =
"This service has UTF-8 Grids; see console for grid data.";
utfgrid.on("mouseover", function(e) {
console.log("UTF grid data:", e.data);
});
}
map.zoomControl.setPosition("topleft");
map.addControl(
L.control.zoomBox({ modal: true, position: "topleft" })
@ -176,16 +186,6 @@
});
map.addControl(slider);
map.addControl(
L.control.basemaps({
position: "bottomright",
basemaps: basemaps,
tileX: 0,
tileY: 0,
tileZ: 1
})
);
</script>
</body>
</html>

139
handlers/templates/map_gl.html
View File

@ -8,9 +8,9 @@
name="viewport"
content="initial-scale=1,maximum-scale=1,user-scalable=no"
/>
<script src="https://api.tiles.mapbox.com/mapbox-gl-js/v0.32.0/mapbox-gl.js"></script>
<script src="https://api.tiles.mapbox.com/mapbox-gl-js/v1.8.0/mapbox-gl.js"></script>
<link
href="https://api.tiles.mapbox.com/mapbox-gl-js/v0.32.0/mapbox-gl.css"
href="https://api.tiles.mapbox.com/mapbox-gl-js/v1.8.0/mapbox-gl.css"
rel="stylesheet"
/>
<style>
@ -49,79 +49,80 @@
};
var map;
mapboxgl.util.getJSON("{{.URL}}", function(e, tilejson) {
var layers = [];
tilejson.vector_layers.forEach(function(srcLyr, i) {
layers.push({
id: "overlay-poly-" + i,
source: "overlay",
"source-layer": srcLyr.id,
filter: ["==", "$type", "Polygon"],
type: "fill",
paint: {
"fill-color": "orange",
"fill-opacity": 0.5,
"fill-outline-color": "red"
}
});
// Load raw JSON directly from template
var tileJSON = {{.TileJSON}};
layers.push({
id: "overlay-line-" + i,
source: "overlay",
"source-layer": srcLyr.id,
filter: ["==", "$type", "LineString"],
type: "line",
paint: {
"line-color": "red",
"line-opacity": 0.75,
"line-width": 2
}
});
layers.push({
id: "overlay-point-" + i,
source: "overlay",
"source-layer": srcLyr.id,
filter: ["==", "$type", "Point"],
type: "circle",
paint: {
"circle-radius": 6,
"circle-color": "#F00",
"circle-opacity": 1
}
});
var layers = [];
tileJSON.vector_layers.forEach(function(srcLyr, i) {
layers.push({
id: "overlay-poly-" + i,
source: "overlay",
"source-layer": srcLyr.id,
filter: ["==", "$type", "Polygon"],
type: "fill",
paint: {
"fill-color": "orange",
"fill-opacity": 0.5,
"fill-outline-color": "red"
}
});
map = new mapboxgl.Map({
container: "map",
style: {
version: 8,
sources: {
basemap: basemapSource,
overlay: {
type: "vector",
tiles: tilejson.tiles,
minzoom: tilejson.minzoom,
maxzoom: tilejson.maxzoom
}
},
layers: [basemapStyle].concat(layers)
},
maxZoom: basemapStyle.maxzoom,
zoom: tilejson.bounds ? 0 : tilejson.center[2],
center: tilejson.center.slice(0, 2)
layers.push({
id: "overlay-line-" + i,
source: "overlay",
"source-layer": srcLyr.id,
filter: ["==", "$type", "LineString"],
type: "line",
paint: {
"line-color": "red",
"line-opacity": 0.75,
"line-width": 2
}
});
if (tilejson.bounds) {
var bounds = [
tilejson.bounds.slice(0, 2),
tilejson.bounds.slice(2, tilejson.bounds.length)
];
map.fitBounds(bounds);
}
map.addControl(new mapboxgl.Navigation());
layers.push({
id: "overlay-point-" + i,
source: "overlay",
"source-layer": srcLyr.id,
filter: ["==", "$type", "Point"],
type: "circle",
paint: {
"circle-radius": 6,
"circle-color": "#F00",
"circle-opacity": 1
}
});
});
map = new mapboxgl.Map({
container: "map",
style: {
version: 8,
sources: {
basemap: basemapSource,
overlay: {
type: "vector",
tiles: tileJSON.tiles,
minzoom: tileJSON.minzoom,
maxzoom: tileJSON.maxzoom
}
},
layers: [basemapStyle].concat(layers)
},
maxZoom: basemapStyle.maxzoom,
zoom: tileJSON.bounds ? 0 : tileJSON.center[2],
center: tileJSON.center.slice(0, 2)
});
if (tileJSON.bounds) {
var bounds = [
tileJSON.bounds.slice(0, 2),
tileJSON.bounds.slice(2, tileJSON.bounds.length)
];
map.fitBounds(bounds);
}
map.addControl(new mapboxgl.NavigationControl());
</script>
</body>
</html>

71
handlers/tile.go Normal file
View File

@ -0,0 +1,71 @@
package handlers
import (
"fmt"
"math"
"strconv"
"strings"
)
const (
earthRadius = 6378137.0
earthCircumference = math.Pi * earthRadius
initialResolution = 2 * earthCircumference / 256
dpi uint8 = 96
)
type tileCoord struct {
z uint8
x, y uint64
}
// tileCoordFromString parses and returns tileCoord coordinates and an optional
// extension from the three parameters. The parameter z is interpreted as the
// web mercator zoom level, it is supposed to be an unsigned integer that will
// fit into 8 bit. The parameters x and y are interpreted as longitude and
// latitude tile indices for that zoom level, both are supposed be integers in
// the integer interval [0,2^z). Additionally, y may also have an optional
// filename extension (e.g. "42.png") which is removed before parsing the
// number, and returned, too. In case an error occurred during parsing or if the
// values are not in the expected interval, the returned error is non-nil.
func tileCoordFromString(z, x, y string) (tc tileCoord, ext string, err error) {
var z64 uint64
if z64, err = strconv.ParseUint(z, 10, 8); err != nil {
err = fmt.Errorf("cannot parse zoom level: %v", err)
return
}
tc.z = uint8(z64)
const (
errMsgParse = "cannot parse %s coordinate axis: %v"
errMsgOOB = "%s coordinate (%d) is out of bounds for zoom level %d"
)
if tc.x, err = strconv.ParseUint(x, 10, 64); err != nil {
err = fmt.Errorf(errMsgParse, "first", err)
return
}
if tc.x >= (1 << z64) {
err = fmt.Errorf(errMsgOOB, "x", tc.x, tc.z)
return
}
s := y
if l := strings.LastIndex(s, "."); l >= 0 {
s, ext = s[:l], s[l:]
}
if tc.y, err = strconv.ParseUint(s, 10, 64); err != nil {
err = fmt.Errorf(errMsgParse, "y", err)
return
}
if tc.y >= (1 << z64) {
err = fmt.Errorf(errMsgOOB, "y", tc.y, tc.z)
return
}
return
}
func calcScaleResolution(zoomLevel uint8, dpi uint8) (float64, float64) {
var denom = 1 << zoomLevel
resolution := initialResolution / float64(denom)
scale := float64(dpi) * 39.37 * resolution // 39.37 in/m
return scale, resolution
}

336
handlers/tileset.go Normal file
View File

@ -0,0 +1,336 @@
package handlers
import (
"encoding/json"
"fmt"
"html/template"
"net/http"
"path/filepath"
"strings"
"github.com/consbio/mbtileserver/mbtiles"
)
// Tileset provides a tileset constructed from an mbtiles file
type Tileset struct {
svc *ServiceSet
db *mbtiles.DB
id string
name string
tileformat mbtiles.TileFormat
published bool
router *http.ServeMux
}
// newTileset constructs a new Tileset from an mbtiles filename.
// Tileset is registered at the passed in path.
// Any errors encountered opening the tileset are returned.
func newTileset(svc *ServiceSet, filename, id, path string) (*Tileset, error) {
db, err := mbtiles.NewDB(filename)
if err != nil {
return nil, fmt.Errorf("Invalid mbtiles file %q: %v", filename, err)
}
metadata, err := db.ReadMetadata()
if err != nil {
return nil, fmt.Errorf("Invalid mbtiles file %q: %v", filename, err)
}
name, ok := metadata["name"].(string)
if !ok {
name = strings.TrimSuffix(filepath.Base(filename), filepath.Ext(filename))
}
ts := &Tileset{
svc: svc,
db: db,
id: id,
name: name,
tileformat: db.TileFormat(),
published: true,
}
// setup routes for tileset
m := http.NewServeMux()
m.HandleFunc(path+"/tiles/", ts.tileHandler)
if svc.enableTileJSON {
m.HandleFunc(path, ts.tileJSONHandler)
}
if svc.enablePreview {
m.HandleFunc(path+"/map", ts.previewHandler)
staticPrefix := path + "/map/static/"
m.Handle(staticPrefix, staticHandler(staticPrefix))
}
if svc.enableArcGIS {
arcgisRoot := ArcGISRoot + id + "/MapServer"
m.HandleFunc(arcgisRoot, ts.arcgisServiceHandler)
m.HandleFunc(arcgisRoot+"/layers", ts.arcgisLayersHandler)
m.HandleFunc(arcgisRoot+"/legend", ts.arcgisLegendHandler)
m.HandleFunc(arcgisRoot+"/tile/", ts.arcgisTileHandler)
}
ts.router = m
return ts, nil
}
// Reload reloads the mbtiles file from disk using the same filename as
// used when this was first constructed
func (ts *Tileset) reload() error {
if ts.db == nil {
return nil
}
filename := ts.db.Filename()
err := ts.db.Close()
if err != nil {
return err
}
db, err := mbtiles.NewDB(filename)
if err != nil {
return fmt.Errorf("Invalid mbtiles file %q: %v", filename, err)
}
ts.db = db
return nil
}
// Delete closes and deletes the mbtiles file for this tileset
func (ts *Tileset) delete() error {
if ts.db != nil {
err := ts.db.Close()
if err != nil {
return err
}
}
ts.db = nil
ts.published = false
return nil
}
// tileFormatString returns the tile format string of the underlying mbtiles file
func (ts *Tileset) tileFormatString() string {
return ts.tileformat.String()
}
// TileJSON returns the TileJSON (as a map of strings to interface{} values)
// for the tileset. This can be rendered into templates or returned via a
// handler.
func (ts *Tileset) TileJSON(svcURL string, query string) (map[string]interface{}, error) {
if ts == nil || !ts.published {
return nil, fmt.Errorf("Tileset does not exist")
}
db := ts.db
imgFormat := db.TileFormatString()
out := map[string]interface{}{
"tilejson": "2.1.0",
"scheme": "xyz",
"format": imgFormat,
"tiles": []string{fmt.Sprintf("%s/tiles/{z}/{x}/{y}.%s%s", svcURL, imgFormat, query)},
"name": ts.name,
}
metadata, err := db.ReadMetadata()
if err != nil {
return nil, err
}
for k, v := range metadata {
switch k {
// strip out values above
case "tilejson", "id", "scheme", "format", "tiles", "map":
continue
// strip out values that are not supported or are overridden below
case "grids", "interactivity", "modTime":
continue
// strip out values that come from TileMill but aren't useful here
case "metatile", "scale", "autoscale", "_updated", "Layer", "Stylesheet":
continue
default:
out[k] = v
}
}
if db.HasUTFGrid() {
out["grids"] = []string{fmt.Sprintf("%s/tiles/{z}/{x}/{y}.json%s", svcURL, query)}
}
return out, nil
}
// tilesJSONHandler is an http.HandlerFunc for the TileJSON endpoint of the tileset
func (ts *Tileset) tileJSONHandler(w http.ResponseWriter, r *http.Request) {
if ts == nil || !ts.published {
http.NotFound(w, r)
return
}
query := ""
if r.URL.RawQuery != "" {
query = "?" + r.URL.RawQuery
}
tilesetURL := fmt.Sprintf("%s://%s%s", scheme(r), r.Host, r.URL.Path)
tileJSON, err := ts.TileJSON(tilesetURL, query)
if ts.svc.enablePreview {
tileJSON["map"] = fmt.Sprintf("%s/map", tilesetURL)
}
bytes, err := json.Marshal(tileJSON)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("could not render TileJSON for %v: %v", r.URL.Path, err)
return
}
w.Header().Set("Content-Type", "application/json")
_, err = w.Write(bytes)
if err != nil {
ts.svc.logError("could not write tileJSON content for %v: %v", r.URL.Path, err)
}
}
// tileHandler is an http.HandlerFunc for the tile endpoint of the tileset.
// If a tile is not found, the handler returns a blank image if the tileset
// has images, and an empty response if the tileset has vector tiles.
func (ts *Tileset) tileHandler(w http.ResponseWriter, r *http.Request) {
if ts == nil || !ts.published {
// In order to not break any requests from when this tileset was published
// return the appropriate not found handler for the original tile format.
tileNotFoundHandler(w, r, ts.tileformat)
return
}
db := ts.db
// split path components to extract tile coordinates x, y and z
pcs := strings.Split(r.URL.Path[1:], "/")
// we are expecting at least "services", <id> , "tiles", <z>, <x>, <y plus .ext>
l := len(pcs)
if l < 6 || pcs[5] == "" {
http.Error(w, "requested path is too short", http.StatusBadRequest)
return
}
z, x, y := pcs[l-3], pcs[l-2], pcs[l-1]
tc, ext, err := tileCoordFromString(z, x, y)
if err != nil {
http.Error(w, "invalid tile coordinates", http.StatusBadRequest)
return
}
var data []byte
// flip y to match the spec
tc.y = (1 << uint64(tc.z)) - 1 - tc.y
isGrid := ext == ".json"
switch {
case !isGrid:
err = db.ReadTile(tc.z, tc.x, tc.y, &data)
case isGrid && db.HasUTFGrid():
err = db.ReadGrid(tc.z, tc.x, tc.y, &data)
default:
err = fmt.Errorf("no grid supplied by tile database")
}
if err != nil {
// augment error info
t := "tile"
if isGrid {
t = "grid"
}
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("cannot fetch %s from DB for z=%d, x=%d, y=%d at path %v: %v", t, tc.z, tc.x, tc.y, r.URL.Path, err)
return
}
if data == nil || len(data) <= 1 {
tileNotFoundHandler(w, r, ts.tileformat)
return
}
if isGrid {
w.Header().Set("Content-Type", "application/json")
if db.UTFGridCompression() == mbtiles.ZLIB {
w.Header().Set("Content-Encoding", "deflate")
} else {
w.Header().Set("Content-Encoding", "gzip")
}
} else {
w.Header().Set("Content-Type", db.ContentType())
if db.TileFormat() == mbtiles.PBF {
w.Header().Set("Content-Encoding", "gzip")
}
}
_, err = w.Write(data)
if err != nil {
ts.svc.logError("Could not write tile data for %v: %v", r.URL.Path, err)
}
}
// previewHandler is an http.HandlerFunc that renders the map preview template
// appropriate for the type of tileset. Image tilesets use Leaflet, whereas
// vector tilesets use Mapbox GL.
func (ts *Tileset) previewHandler(w http.ResponseWriter, r *http.Request) {
if ts == nil || !ts.published {
http.NotFound(w, r)
return
}
query := ""
if r.URL.RawQuery != "" {
query = "?" + r.URL.RawQuery
}
tilesetURL := fmt.Sprintf("%s://%s%s", scheme(r), r.Host, strings.TrimSuffix(r.URL.Path, "/map"))
tileJSON, err := ts.TileJSON(tilesetURL, query)
bytes, err := json.Marshal(tileJSON)
if err != nil {
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
ts.svc.logError("could not render tileJSON for preview for %v: %v", r.URL.Path, err)
return
}
p := struct {
URL string
ID string
TileJSON template.JS
}{
tilesetURL,
ts.id,
template.JS(string(bytes)),
}
switch ts.db.TileFormat() {
default:
executeTemplate(w, "map", p)
case mbtiles.PBF:
executeTemplate(w, "map_gl", p)
}
}
// tileNotFoundHandler is an http.HandlerFunc that writes the default response
// for a non-existing tile of type f to w
func tileNotFoundHandler(w http.ResponseWriter, r *http.Request, f mbtiles.TileFormat) {
switch f {
case mbtiles.PNG, mbtiles.JPG, mbtiles.WEBP:
// Return blank PNG for all image types
w.Header().Set("Content-Type", "image/png")
w.WriteHeader(http.StatusOK)
w.Write(BlankPNG())
case mbtiles.PBF:
// Return 204
w.WriteHeader(http.StatusNoContent)
default:
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusNotFound)
fmt.Fprint(w, `{"message": "Tile does not exist"}`)
}
}

211
main.go
View File

@ -7,6 +7,7 @@ import (
"os/exec"
"os/signal"
"strconv"
"strings"
"syscall"
"golang.org/x/crypto/acme"
@ -14,6 +15,7 @@ import (
"net"
"net/http"
"net/url"
"os"
"time"
@ -46,7 +48,7 @@ var rootCmd = &cobra.Command{
}
}
if reload {
if enableReloadSignal {
if isChild := os.Getenv("MBTS_IS_CHILD"); isChild != "" {
serve()
} else {
@ -59,34 +61,49 @@ var rootCmd = &cobra.Command{
}
var (
port int
tilePath string
certificate string
privateKey string
pathPrefix string
domain string
secretKey string
sentryDSN string
verbose bool
autotls bool
redirect bool
reload bool
port int
tilePath string
certificate string
privateKey string
rootURLStr string
domain string
secretKey string
sentryDSN string
verbose bool
autotls bool
redirect bool
enableReloadSignal bool
generateIDs bool
enableArcGIS bool
disablePreview bool
disableTileJSON bool
disableServiceList bool
tilesOnly bool
)
func init() {
flags := rootCmd.Flags()
flags.IntVarP(&port, "port", "p", -1, "Server port. Default is 443 if --cert or --tls options are used, otherwise 8000.")
flags.StringVarP(&tilePath, "dir", "d", "./tilesets", "Directory containing mbtiles files.")
flags.StringVarP(&tilePath, "dir", "d", "./tilesets", "Directory containing mbtiles files. Can be a comma-delimited list of directories.")
flags.BoolVarP(&generateIDs, "generate-ids", "", false, "Automatically generate tileset IDs instead of using relative path")
flags.StringVarP(&certificate, "cert", "c", "", "X.509 TLS certificate filename. If present, will be used to enable SSL on the server.")
flags.StringVarP(&privateKey, "key", "k", "", "TLS private key")
flags.StringVar(&pathPrefix, "path", "", "URL root path of this server (if behind a proxy)")
flags.StringVar(&rootURLStr, "root-url", "/services", "Root URL of services endpoint")
flags.StringVar(&domain, "domain", "", "Domain name of this server. NOTE: only used for AutoTLS.")
flags.StringVarP(&secretKey, "secret-key", "s", "", "Shared secret key used for HMAC request authentication")
flags.StringVar(&sentryDSN, "dsn", "", "Sentry DSN")
flags.BoolVarP(&verbose, "verbose", "v", false, "Verbose logging")
flags.BoolVarP(&autotls, "tls", "t", false, "Auto TLS via Let's Encrypt")
flags.BoolVarP(&redirect, "redirect", "r", false, "Redirect HTTP to HTTPS")
flags.BoolVarP(&reload, "enable-reload", "", false, "Enable graceful reload")
flags.BoolVarP(&enableArcGIS, "enable-arcgis", "", false, "Enable ArcGIS Mapserver endpoints")
flags.BoolVarP(&enableReloadSignal, "enable-reload-signal", "", false, "Enable graceful reload using HUP signal to the server process")
flags.BoolVarP(&disablePreview, "disable-preview", "", false, "Disable map preview for each tileset (enabled by default)")
flags.BoolVarP(&disableTileJSON, "disable-tilejson", "", false, "Disable TileJSON endpoint for each tileset (enabled by default)")
flags.BoolVarP(&disableServiceList, "disable-svc-list", "", false, "Disable services list endpoint (enabled by default)")
flags.BoolVarP(&tilesOnly, "tiles-only", "", false, "Only enable tile endpoints (shortcut for --disable-svc-list --disable-tilejson --disable-preview)")
flags.StringVar(&sentryDSN, "dsn", "", "Sentry DSN")
flags.BoolVarP(&verbose, "verbose", "v", false, "Verbose logging")
if env := os.Getenv("PORT"); env != "" {
p, err := strconv.Atoi(env)
@ -100,6 +117,14 @@ func init() {
tilePath = env
}
if env := os.Getenv("GENERATE_IDS"); env != "" {
p, err := strconv.ParseBool(env)
if err != nil {
log.Fatalln("GENERATE_IDS must be a bool(true/false)")
}
generateIDs = p
}
if env := os.Getenv("TLS_CERT"); env != "" {
certificate = env
}
@ -108,24 +133,15 @@ func init() {
privateKey = env
}
if env := os.Getenv("PATH_PREFIX"); env != "" {
pathPrefix = env
if env := os.Getenv("ROOT_URL"); env != "" {
rootURLStr = env
}
if env := os.Getenv("DOMAIN"); env != "" {
domain = env
}
if env := os.Getenv("DSN"); env != "" {
sentryDSN = env
}
if env := os.Getenv("VERBOSE"); env != "" {
p, err := strconv.ParseBool(env)
if err != nil {
log.Fatalln("VERBOSE must be a bool(true/false)")
}
verbose = p
if secretKey == "" {
secretKey = os.Getenv("HMAC_SECRET_KEY")
}
if env := os.Getenv("AUTO_TLS"); env != "" {
@ -144,8 +160,24 @@ func init() {
redirect = p
}
if secretKey == "" {
secretKey = os.Getenv("HMAC_SECRET_KEY")
if env := os.Getenv("DSN"); env != "" {
sentryDSN = env
}
if env := os.Getenv("ENABLE_ARCGIS"); env != "" {
p, err := strconv.ParseBool(env)
if err != nil {
log.Fatalln("ENABLE_ARCGIS must be a bool(true/false)")
}
enableArcGIS = p
}
if env := os.Getenv("VERBOSE"); env != "" {
p, err := strconv.ParseBool(env)
if err != nil {
log.Fatalln("VERBOSE must be a bool(true/false)")
}
verbose = p
}
}
@ -175,6 +207,24 @@ func serve() {
log.Debugln("Added logging hook for Sentry")
}
if tilesOnly {
disableServiceList = true
disableTileJSON = true
disablePreview = true
}
if !strings.HasPrefix(rootURLStr, "/") {
log.Fatalln("Value for --root-url must start with \"/\"")
}
if strings.HasSuffix(rootURLStr, "/") {
log.Fatalln("Value for --root-url must not end with \"/\"")
}
rootURL, err := url.Parse(rootURLStr)
if err != nil {
log.Fatalf("Could not parse --root-url value %q\n", rootURLStr)
}
certExists := len(certificate) > 0
keyExists := len(privateKey) > 0
domainExists := len(domain) > 0
@ -183,10 +233,6 @@ func serve() {
log.Fatalln("Both certificate and private key are required to use SSL")
}
if len(pathPrefix) > 0 && !domainExists {
log.Fatalln("Domain is required if path is provided")
}
if autotls && !domainExists {
log.Fatalln("Domain is required to use auto TLS")
}
@ -199,14 +245,52 @@ func serve() {
log.Fatalln("Certificate or tls options are required to use redirect")
}
svcSet, err := handlers.NewFromBaseDir(tilePath)
if err != nil {
log.Errorf("Unable to create services for mbtiles in '%v': %v\n", tilePath, err)
}
if len(secretKey) > 0 {
log.Infoln("An HMAC request authorization key was set. All incoming must be signed.")
svcSet.SetRequestAuthKey(secretKey)
}
svcSet, err := handlers.New(&handlers.ServiceSetConfig{
RootURL: rootURL,
ErrorWriter: &errorLogger{log: log.New()},
EnableServiceList: !disableServiceList,
EnableTileJSON: !disableTileJSON,
EnablePreview: !disablePreview,
EnableArcGIS: enableArcGIS,
})
if err != nil {
log.Fatalln("Could not construct ServiceSet")
}
for _, path := range strings.Split(tilePath, ",") {
// Discover all tilesets
log.Infof("Searching for tilesets in %v\n", path)
filenames, err := mbtiles.ListDBs(path)
if err != nil {
log.Errorf("Unable to list mbtiles in '%v': %v\n", path, err)
}
if len(filenames) == 0 {
log.Errorf("No tilesets found in %s", path)
}
// Register all tilesets
for _, filename := range filenames {
var id string
var err error
if generateIDs {
id = handlers.SHA1ID(filename)
} else {
id, err = handlers.RelativePathID(filename, path)
if err != nil {
log.Errorf("Could not generate ID for tileset: %q", filename)
continue
}
}
err = svcSet.AddTileset(filename, id)
if err != nil {
log.Errorf("Could not add tileset for %q with ID %q\n%v", filename, id, err)
}
}
}
// print number of services
@ -215,28 +299,22 @@ func serve() {
e := echo.New()
e.HideBanner = true
e.Pre(middleware.RemoveTrailingSlash())
if verbose {
e.Use(middleware.Logger())
}
e.Use(middleware.Recover())
e.Use(middleware.CORS())
gzip := middleware.Gzip()
staticPrefix := "/static"
if pathPrefix != "" {
staticPrefix = "/" + pathPrefix + staticPrefix
// log all requests if verbose mode
if verbose {
e.Use(middleware.Logger())
}
staticHandler := http.StripPrefix(staticPrefix, handlers.Static())
e.GET(staticPrefix+"*", echo.WrapHandler(staticHandler), gzip)
ef := func(err error) {
log.Errorf("%v", err)
// setup auth middleware if secret key is set
if secretKey != "" {
hmacAuth := handlers.HMACAuthMiddleware(secretKey, svcSet)
e.Use(echo.WrapMiddleware(hmacAuth))
}
h := echo.WrapHandler(svcSet.Handler(ef, true))
e.GET("/*", h)
a := echo.WrapHandler(svcSet.ArcGISHandler(ef))
e.GET("/arcgis/rest/services/*", a)
// Get HTTP.Handler for the service set, and wrap for use in echo
e.GET("/*", echo.WrapHandler(svcSet.Handler()))
// Start the server
fmt.Println("\n--------------------------------------")
@ -256,7 +334,7 @@ func serve() {
var listener net.Listener
if reload {
if enableReloadSignal {
f := os.NewFile(3, "")
listener, err = net.FileListener(f)
} else {
@ -271,7 +349,7 @@ func serve() {
// Listen for SIGHUP (graceful shutdown)
go func(e *echo.Echo) {
if !reload {
if !enableReloadSignal {
return
}
@ -430,3 +508,14 @@ func supervise() {
fmt.Println("")
}
}
// errorLogger wraps logrus logger so that we can pass it into the handlers
type errorLogger struct {
log *log.Logger
}
// It implements the required io.Writer interface
func (el *errorLogger) Write(p []byte) (n int, err error) {
el.log.Errorln(string(p))
return len(p), nil
}

4
mbtiles/export_test.go Normal file
View File

@ -0,0 +1,4 @@
package mbtiles
// Export for testing.
var StringToFloats = stringToFloats

36
mbtiles/mbtiles.go
View File

@ -10,6 +10,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"time"
@ -76,6 +77,7 @@ func (t TileFormat) ContentType() string {
type DB struct {
filename string // name of tile mbtiles file
db *sql.DB // database connection for mbtiles file
id string // unique ID of mbtiles file
tileformat TileFormat // tile format: PNG, JPG, PBF, WEBP
timestamp time.Time // timestamp of file, for cache control headers
hasUTFGrid bool // true if mbtiles file contains additional tables with UTFGrid data
@ -89,7 +91,7 @@ func NewDB(filename string) (*DB, error) {
//Saves last modified mbtiles time for setting Last-Modified header
fileStat, err := os.Stat(filename)
if err != nil {
return nil, fmt.Errorf("could not read file stats for mbtiles file: %s\n", filename)
return nil, fmt.Errorf("could not read file stats for mbtiles file: %s", filename)
}
db, err := sql.Open("sqlite3", filename)
@ -122,8 +124,10 @@ func NewDB(filename string) (*DB, error) {
if tileformat == GZIP {
tileformat = PBF // GZIP masks PBF, which is only expected type for tiles in GZIP format
}
out := DB{
db: db,
filename: filename,
tileformat: tileformat,
timestamp: fileStat.ModTime().Round(time.Second), // round to nearest second
}
@ -178,6 +182,11 @@ func (tileset *DB) ReadTile(z uint8, x uint64, y uint64, data *[]byte) error {
return nil
}
// Filename returns the Filename of the DB.
func (tileset *DB) Filename() string {
return tileset.filename
}
// ReadGrid reads a UTFGrid with identifiers z, x, y into provided *[]byte. data
// will be nil if the grid does not exist in the database, and an error will be
// raised. This merges in grid key data. The data is returned in
@ -391,3 +400,28 @@ func stringToFloats(str string) ([]float64, error) {
}
return out, nil
}
// ListDBs finds all mbtiles files within a base directory and all subdirectories.
// Any mbtiles files that are currently being written (identified by -journal extension)
// are skipped.
// Any errors encountered while walking the directory tree are returned.
func ListDBs(baseDir string) ([]string, error) {
var filenames []string
err := filepath.Walk(baseDir, func(p string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if _, err := os.Stat(p + "-journal"); err == nil {
// Don't try to load .mbtiles files that are being written
return nil
}
if ext := filepath.Ext(p); ext == ".mbtiles" {
filenames = append(filenames, p)
}
return nil
})
if err != nil {
return nil, err
}
return filenames, err
}

239
mbtiles/mbtiles_test.go
View File

@ -6,6 +6,85 @@ import (
"github.com/consbio/mbtileserver/mbtiles"
)
func Test_ListDBs(t *testing.T) {
var expected = []string{
"testdata/geography-class-jpg.mbtiles",
"testdata/geography-class-png.mbtiles",
"testdata/world_cities.mbtiles",
"testdata/openstreetmap/open-streets-dc.mbtiles",
}
filenames, err := mbtiles.ListDBs("./testdata")
if err != nil {
t.Error("Could not list mbtiles files in testdata directory")
}
found := 0
for _, expectedFilename := range expected {
for _, filename := range filenames {
if filename == expectedFilename {
found += 1
}
}
}
if found != len(expected) {
t.Error("Did not list all expected mbtiles files in testdata directory")
}
// invalid directory should raise an error
_, err = mbtiles.ListDBs("./invalid")
if err == nil {
t.Error("Did not fail to list mbtiles in invalid directory")
}
// valid directory with no mbtiles should be empty
filenames, err = mbtiles.ListDBs("../handlers")
if err != nil {
t.Error("Failed when listing valid directory")
}
if len(filenames) != 0 {
t.Error("Directory with no mbtiles files did not return 0 mbtiles files")
}
}
func Test_stringToFloats(t *testing.T) {
var conditions = []struct {
in string
out []float64
}{
{"0", []float64{0}},
{"0,1.5", []float64{0, 1.5}},
{"0, 1.5, 123.456 ", []float64{0, 1.5, 123.456}},
}
for _, condition := range conditions {
result, err := mbtiles.StringToFloats(condition.in)
if err != nil {
t.Errorf("Unexpected error in stringToFloats: %q", err)
}
if len(result) != len(condition.out) {
t.Errorf("Failed stringToFloats: %q => %v, expected %v", condition.in, result, condition.out)
}
for i, expected := range condition.out {
if expected != condition.out[i] {
t.Errorf("Failed stringToFloats: %q => %v, expected %v", condition.in, result, condition.out)
}
}
}
var invalids = []string{
"", "a", "1,a", "1, ",
}
for _, invalid := range invalids {
_, err := mbtiles.StringToFloats(invalid)
if err == nil {
t.Errorf("stringToFloats did not fail as expected: %q", invalid)
}
}
}
func Test_TileFormat_String(t *testing.T) {
var conditions = []struct {
in mbtiles.TileFormat
@ -48,11 +127,24 @@ func Test_TileFormat_ContentType(t *testing.T) {
func Test_NewDB(t *testing.T) {
// valid tileset should not raise error
_, err := mbtiles.NewDB("./testdata/geography-class-png.mbtiles")
filename := "./testdata/geography-class-png.mbtiles"
db, err := mbtiles.NewDB(filename)
if err != nil {
t.Errorf("Valid tileset could not be opened: %q", err)
}
// closing mbtiles file should not raise error
err = db.Close()
if err != nil {
t.Error("Closing mbtiles file raised error")
}
// nonexistent tileset should raise error
_, err = mbtiles.NewDB("does_not_exist.mbtiles")
if err == nil {
t.Error("Nonexistent tileset did not raise validation error")
}
// invalid tileset should raise error
_, err = mbtiles.NewDB("./testdata/invalid.mbtiles")
if err == nil {
@ -65,3 +157,148 @@ func Test_NewDB(t *testing.T) {
t.Error("Invalid tileset did not raise validation error")
}
}
func Test_Metadata(t *testing.T) {
filename := "./testdata/geography-class-png.mbtiles"
db, err := mbtiles.NewDB(filename)
expectedMetadata := map[string]interface{}{
"name": "Geography Class",
"description": "One of the example maps that comes with TileMill - a bright & colorful world map that blends retro and high-tech with its folded paper texture and interactive flag tooltips. ",
"minzoom": 0,
"maxzoom": 1,
}
metadata, err := db.ReadMetadata()
if err != nil {
t.Error("Error raised when reading metadata")
}
for key, expectedValue := range expectedMetadata {
value, ok := metadata[key]
if !ok {
t.Errorf("Metadata missing expected key: %q", key)
}
if value != expectedValue {
t.Errorf("Metadata value '%v' does not match expected value '%v'", value, expectedValue)
}
}
var expectedBounds = []float64{-180, -85.0511, 180, 85.0511}
bounds, ok := metadata["bounds"]
if !ok {
t.Error("Metadata missing expected key: bounds")
}
boundsValues := bounds.([]float64)
if len(boundsValues) != 4 {
t.Error("Metadata bounds not expected length")
}
for i, expectedValue := range expectedBounds {
if boundsValues[i] != expectedValue {
t.Errorf("Metadata bounds does not have expected values. Found: %v expected: %v", boundsValues[i], expectedValue)
}
}
}
func Test_ReadTile(t *testing.T) {
filename := "./testdata/geography-class-png.mbtiles"
db, err := mbtiles.NewDB(filename)
// valid tile should return data
var data []byte
err = db.ReadTile(0, 0, 0, &data)
if err != nil {
t.Error("Error raised when reading valid tile")
}
if data == nil {
t.Error("Did not read tile data")
}
// missing tile should return nil
err = db.ReadTile(10, 0, 0, &data)
if err != nil {
t.Error("Error raised when reading missing tile")
}
if data != nil {
t.Error("Tile data should have been empty for missing tile")
}
}
func Test_ReadGrid(t *testing.T) {
filename := "./testdata/geography-class-png.mbtiles"
db, err := mbtiles.NewDB(filename)
// valid UTF grid should return data
var data []byte
err = db.ReadGrid(0, 0, 0, &data)
if err != nil {
t.Error("Error raised when reading valid UTF grid")
}
if data == nil {
t.Error("Did not read UTF grid data")
}
// missing UTF grid should return nil
err = db.ReadGrid(10, 0, 0, &data)
if err != nil {
t.Error("Error raised when reading missing UTF grid")
}
if data != nil {
t.Error("Tile data should have been empty for missing UTF grid")
}
}
func Test_Property_Methods(t *testing.T) {
filename := "./testdata/geography-class-png.mbtiles"
db, err := mbtiles.NewDB(filename)
if err != nil {
t.Errorf("Valid tileset could not be opened: %q", err)
}
if db.Filename() != filename {
t.Errorf("Unexpected filename: %q => %q", filename, db.Filename())
}
if db.TileFormat() != mbtiles.PNG {
t.Errorf("TileFormat %v is not expected value", db.TileFormat())
}
if db.TileFormatString() != "png" {
t.Errorf("TileFormatString %q is not expected value 'png'", db.TileFormatString())
}
if db.ContentType() != "image/png" {
t.Errorf("ContentType %q is not expected value 'image/png'", db.ContentType())
}
if !db.HasUTFGrid() {
t.Error("Tileset with UTF grids claims to not have UTF grids")
}
if db.UTFGridCompression() != mbtiles.ZLIB {
t.Errorf("UTF grid compression %v is not expected value", db.UTFGridCompression())
}
filename = "./testdata/world_cities.mbtiles"
db, err = mbtiles.NewDB(filename)
if err != nil {
t.Errorf("Valid tileset could not be opened: %q", err)
}
if db.TileFormat() != mbtiles.PBF {
t.Errorf("TileFormat %v is not expected value", db.TileFormat())
}
if db.TileFormatString() != "pbf" {
t.Errorf("TileFormatString %q is not expected value 'pbf'", db.TileFormatString())
}
if db.ContentType() != "application/x-protobuf" {
t.Errorf("ContentType %q is not expected value 'application/x-protobuf'", db.ContentType())
}
if db.HasUTFGrid() {
t.Error("Tileset with no UTF grids claims to have UTF grids")
}
}
// filename := "./testdata/geography-class-png.mbtiles"
// db, err := mbtiles.NewDB(filename)

BIN
mbtiles/testdata/incomple.mbtiles-journal vendored Normal file
View File

Binary file not shown.