A fresh attempt at overviewing the map model docs

book/src/SUMMARY.md

@@ -16,12 +16,13 @@
   - [API](dev/api.md)
   - [Testing](dev/testing.md)
 - [Map model](map/README.md)
+  - [Details](map/details.md)
   - [Importing](map/importing/README.md)
     - [convert_osm](map/importing/convert_osm.md)
     - [Road/intersection geometry](map/importing/geometry.md)
     - [The rest](map/importing/rest.md)
+    - [Misc](map/importing/misc.md)
   - [Live edits](map/edits.md)
-  - [Misc](map/misc.md)
   - [Exporting](map/platform.md)
 - [Traffic simulation](trafficsim/README.md)
   - [Discrete event simulation](trafficsim/discrete_event.md)

book/src/map/README.md

@@ -1,105 +1,130 @@
 # Map model
 
-A/B Street builds a rich representation of a city map using OpenStreetMap (OSM)
+A/B Street transforms OpenStreetMap (OSM) data into a detailed geometric and
-and other sources. This chapter describes how.
+semantic representation of the world for traffic simulation. This chapter
+describes that map model, with the hopes that it'll be useful for purposes
+beyond this project.
 
-TODO: Integrate pictures from
+## Overview
-[these slides](https://docs.google.com/presentation/d/1cF7qFtjAzkXL_r62CjxBvgQnLvuQ9I2WTE2iX_5tMCY/edit?usp=sharing).
 
-[This recorded presentation](https://youtu.be/chYd5I-5oyc?t=439) covers some of
+A `Map` covers everything inside some hand-drawn boundary, usually scoped to a
-this.
+city or a few of a city's districts. Unlike OSM, it doesn't cover the entire
+world; it only has areas specifically extracted for some purpose.
 
-## The map
+A map consists of many objects. Mainly, there are roads, broken down into
+individual lanes, and intersections. A road is a single segment connecting
+exactly two intersections (as opposed to OSM, where a single "way" may span many
+intersections). Lanes within a road have a specific type, which dictates their
+direction of travel (or lack of travel, like on-street parking) and uses.
+Sidewalks are represented as bidirectional lanes. Roads connect at
+intersections, which contain an explicit set of turns, each linking a source
+lane to a destination lane.
 
-A single city is broken down into different pieces...
+Maps also contain parking lots and buildings, which connect to the nearest
+driveable lane and a sidewalk. Maps have water and park areas, only used for
+drawing. They also represent public transit stops and routes.
 
-A/B Street comes with a few maps, each defined by a bounding/clipping polygon
+## How is a map used?
-for some portion of Seattle. Each map has these objects:
 
-- **Roads**: A single road connects two intersections, carrying OSM metadata and
+Unlike some GIS systems, maps don't use any kind of database -- they're just a
-  containing some child lanes.
+file, anywhere from 1 to ~500MB (depending on the size of their boundary). Once
-- **Lanes**: An individual lane of traffic. Driving (any vehicle), bus-only, and
+loaded into memory, different objects from the map can be accessed directly,
-  bike-only lanes have a direction. On-street parking lanes don't allow any
+along with a large API to perform various queries.
-  movement, and they have some number of parking spots. Sidewalks are
-  bidirectional.
-- **Intersections**: An intersection has references to all of the incoming and
-  outgoing lanes. Most intersections have a stop sign or traffic signal policy
-  controlling movement through it.
-  - **Border** intersections on the edge of the map are special places where
-    agents may appear or disappear.
-- **Turns**: A turn connects one lane to another, via some intersection.
-  (Sidewalks are bidirectional, so specifying the intersection is necessary to
-  distinguish crosswalks at each end of a sidewalk.)
-- **Buildings**: A building has a position, OSM metadata, and a **front path**
-  connecting the edge of the building to the nearest sidewalk. Most trips in A/B
-  Street begin and end at buildings. Some buildings also contain a number of
-  off-street parking spots.
-- **Area**: An area has geometry and OSM metadata and represents a body of
-  water, forest, park, etc. They're just used for drawing.
-- **Bus stop**: A bus stop is placed some distance along a sidewalk, with a
-  pointer to the position on the adjacent driving or bus lane where a bus stops
-  for pick-up.
-- **Bus route**: A bus route has a name and a list of stops that buses will
-  cycle between. In the future, they'll include information about the
-  frequency/schedule of the route.
-- **Parking lot**: A parking lot is connected to a road, has a shape, and has
-  some internal driving "aisles." The number and position of individual parking
-  spots is auto-generated.
 
-## Coordinate system
+Most of the map's API is read-only; once built, a map doesn't change until
+user-created edits are applied.
 
-A/B Street converts (longitude, latitude) coordinates into a simpler form.
+The pipeline to import a map from OSM data (and also optional supplementary,
+city-specific data) is complex and may take a few minutes to run, but it happens
+once offline. Applications using maps just read the final file.
 
-- An (x, y) point starts with the top-left of the bounding polygon as the
+## Features
-  origin. Note this is screen drawing order, not a Cartesian plane (with Y
-  increasing upwards) -- so angle calculations account for this.
-- The (x, y) values are f64's trimmed to a few decimal places, with way more
-  precision than is really needed. These might become actual fixed-point
-  integers later, but for now, a `Pt2D` skirts around Rust's limits on f64's by
-  guaranteeing no NaN's or infinities and thus providing the full `Eq` trait.
-- A few places in map conversion compare points using different thresholds,
-  usually below 1 meter. Ideally these epsilon comparisons could be eliminated
-  in favor of a fixed-point integer representation, but for now, explicit
-  thresholds are useful.
 
-## Invariants
+Why use A/B Street's map model instead of processing OSM directly?
 
-Ideally, the finalized maps would satisfy a list of invariants, simplifying the
+TODO: Order these better. For each one, show before/after pictures
-traffic simulation and drawing code built on top. But the input data is quite
-messy and for now, most of these aren't quite guaranteed to be true.
 
-- Some minimum length for lanes and turns. Very small lanes can't be drawn, tend
+### Area clipping
-  to break intersection polygons, and may lead to gridlocked traffic.
-- Some guarantees that positions along adjacent lanes actually match up, even
-  though different lanes on the same road may have different lengths. Examples
-  include the position of a bus stop on the sidewalk and bus lane matching up.
-  - Additionally, parking lanes without an adjacent driving lane or bus stops
-    without any driving or bus lanes make no sense and should never occur.
-- Connectivity -- any sidewalk should be reachable from any other, and most
-  driving lanes should be accessible from any others. There are exceptions due
-  to border intersections -- if a car spawns on a highway along the border of
-  the map, it may be forced to disappear on the opposite border of the map, if
-  the highway happens to not have any exits within the map boundary.
 
-## Connectivity
+Bodies of water, forests, parks, and other areas are represented in OSM as
+relations, requiring the user to stitch together multiple polylines in undefined
+orders and handle inner holes. A/B Street maps handle all of that, and also clip
+the area's polygon to the boundary of the entire map -- including coastlines.
 
-For a single mode, each lane is connected to two intersections. Turns connect
+### Road and intersection geometry
-two lanes. There are no turns between sidewalks and driving/bike/bus lanes.
 
-All buildings and parking lots have driveways. This must connect to a sidewalk,
+OSM represents roads as a polyline of the physical center of the road. A/B
-allowing pedestrians to enter/exit that object. The driveway OPTIONALLY connects
+Street infers the number and type of lanes from OSM metadata, then creates
-to the nearest driveable lane. This allows cars to enter/exit that object for
+individual lanes of appropriate width, each with a center-line and polygon for
-parking.
+geometry. At intersections, the roads and lanes are "trimmed back" to avoid
+overlapping, and the "common area" becomes the intersection's polygon. This
+heuristic process is reasonably robust to complex shapes, with special treatment
+of highway on/off-ramps, although it does still have some bugs.
 
-Public transit stops are located somewhere on a sidewalk. They're associated
+### Turns
-with a driveable position where the bus or train stops. In the future, this will
-need to account for dedicated surface-level platforms and for underground
-transit stations, likely associated with a building.
 
-There's a concept of "parking blackholes." If you treat every road as
+At each intersection, A/B Street infers all legal movements between vehicle
-bidirectional without access restrictions, then the graph is connected. But the
+lanes and sidewalks. This process makes use of OSM metadata about turn lanes,
-more detailed view has to factor in one-way roads and things near the map
+inferring reasonable defaults for multi-lane roads. OSM turn restriction
-border. These blackholes influence where cars will try to look for parking
+relations, which may span a sequence of several roads to describe U-turns around
-(since we don't want them entering a blackhole and getting stuck) and also, for
+complex intersections, are also used.
-temporary/unintentional reasons, where pedestrian<->bicycle transitions will
+
-happen.
+### Parking lots
+
+OSM models parking lots as areas along with the driveable aisles. Usually the
+capacity of a lot isn't tagged. A/B Street automatically fills paring lots with
+individual stalls along the aisles, estimating the capacity just from this
+geometry.
+
+### Stop signs
+
+At unsignalized intersections, A/B Street infers which roads have to stop, and
+which have right-of-way.
+
+### Traffic signals
+
+OSM has no way to describe how traffic signals are configured. A/B Street models
+fixed-timer signals, automatically inferring the number of phases, their
+duration, and the movements that are prioritized and permitted during each
+phase.
+
+### Pathfinding
+
+A/B Street can determine routes along lanes and turns for vehicles and
+pedestrians. These routes obey OSM's turn restriction relations that span
+multiple road segments. They also avoid roads that're tagged as not allowing
+through-traffic, depending on the route's origin and destination and vehicle
+type. The pathfinding optionally makes use of contraction hierarchies to greatly
+speed up query performance, at the cost of a slower offline importing process.
+
+### Bridge z-ordering
+
+OSM tags bridges and tunnels, but the roads that happen to pass underneath
+bridges aren't mapped. A/B Street detects these and represents the z-order for
+drawing.
+
+### Buildings
+
+Similar to areas, A/B Street consolidates the geometry of OSM buildings, which
+may be split into multiple polygons. Each building is also associated with the
+nearest driveable lane and sidewalk, and metadata is used to infer a land-use
+(like residential and commercial) and commercial amenities available.
+
+### Experimental: public transit
+
+A/B Street uses bus stops and route relations from OSM to build a model of
+public transit routes. OSM makes few guarantees about how the specifics of the
+route are specified, but A/B Street produces specific paths, handling clipping
+to the map boundary.
+
+... All of this isn't the case yet, but it's a WIP!
+
+### Experimental: separated cyclepaths, tramways, and walking paths
+
+Some cyclepaths, tram lines, and footpaths in OSM are tagged as separate ways,
+with no association to a "main" road. Sometimes this is true -- they're
+independent trails that only occasionally cross roads. But often they run
+alongside a road. A/B Street attempts to detect these and "snap" them to the
+main road as extra lanes.
+
+... But this doesn't work yet at all.

book/src/map/details.md

@@ -0,0 +1,105 @@
+# Map model details
+
+A/B Street builds a rich representation of a city map using OpenStreetMap (OSM)
+and other sources. This chapter describes how.
+
+TODO: Integrate pictures from
+[these slides](https://docs.google.com/presentation/d/1cF7qFtjAzkXL_r62CjxBvgQnLvuQ9I2WTE2iX_5tMCY/edit?usp=sharing).
+
+[This recorded presentation](https://youtu.be/chYd5I-5oyc?t=439) covers some of
+this.
+
+## The map
+
+A single city is broken down into different pieces...
+
+A/B Street comes with a few maps, each defined by a bounding/clipping polygon
+for some portion of Seattle. Each map has these objects:
+
+- **Roads**: A single road connects two intersections, carrying OSM metadata and
+  containing some child lanes.
+- **Lanes**: An individual lane of traffic. Driving (any vehicle), bus-only, and
+  bike-only lanes have a direction. On-street parking lanes don't allow any
+  movement, and they have some number of parking spots. Sidewalks are
+  bidirectional.
+- **Intersections**: An intersection has references to all of the incoming and
+  outgoing lanes. Most intersections have a stop sign or traffic signal policy
+  controlling movement through it.
+  - **Border** intersections on the edge of the map are special places where
+    agents may appear or disappear.
+- **Turns**: A turn connects one lane to another, via some intersection.
+  (Sidewalks are bidirectional, so specifying the intersection is necessary to
+  distinguish crosswalks at each end of a sidewalk.)
+- **Buildings**: A building has a position, OSM metadata, and a **front path**
+  connecting the edge of the building to the nearest sidewalk. Most trips in A/B
+  Street begin and end at buildings. Some buildings also contain a number of
+  off-street parking spots.
+- **Area**: An area has geometry and OSM metadata and represents a body of
+  water, forest, park, etc. They're just used for drawing.
+- **Bus stop**: A bus stop is placed some distance along a sidewalk, with a
+  pointer to the position on the adjacent driving or bus lane where a bus stops
+  for pick-up.
+- **Bus route**: A bus route has a name and a list of stops that buses will
+  cycle between. In the future, they'll include information about the
+  frequency/schedule of the route.
+- **Parking lot**: A parking lot is connected to a road, has a shape, and has
+  some internal driving "aisles." The number and position of individual parking
+  spots is auto-generated.
+
+## Coordinate system
+
+A/B Street converts (longitude, latitude) coordinates into a simpler form.
+
+- An (x, y) point starts with the top-left of the bounding polygon as the
+  origin. Note this is screen drawing order, not a Cartesian plane (with Y
+  increasing upwards) -- so angle calculations account for this.
+- The (x, y) values are f64's trimmed to a few decimal places, with way more
+  precision than is really needed. These might become actual fixed-point
+  integers later, but for now, a `Pt2D` skirts around Rust's limits on f64's by
+  guaranteeing no NaN's or infinities and thus providing the full `Eq` trait.
+- A few places in map conversion compare points using different thresholds,
+  usually below 1 meter. Ideally these epsilon comparisons could be eliminated
+  in favor of a fixed-point integer representation, but for now, explicit
+  thresholds are useful.
+
+## Invariants
+
+Ideally, the finalized maps would satisfy a list of invariants, simplifying the
+traffic simulation and drawing code built on top. But the input data is quite
+messy and for now, most of these aren't quite guaranteed to be true.
+
+- Some minimum length for lanes and turns. Very small lanes can't be drawn, tend
+  to break intersection polygons, and may lead to gridlocked traffic.
+- Some guarantees that positions along adjacent lanes actually match up, even
+  though different lanes on the same road may have different lengths. Examples
+  include the position of a bus stop on the sidewalk and bus lane matching up.
+  - Additionally, parking lanes without an adjacent driving lane or bus stops
+    without any driving or bus lanes make no sense and should never occur.
+- Connectivity -- any sidewalk should be reachable from any other, and most
+  driving lanes should be accessible from any others. There are exceptions due
+  to border intersections -- if a car spawns on a highway along the border of
+  the map, it may be forced to disappear on the opposite border of the map, if
+  the highway happens to not have any exits within the map boundary.
+
+## Connectivity
+
+For a single mode, each lane is connected to two intersections. Turns connect
+two lanes. There are no turns between sidewalks and driving/bike/bus lanes.
+
+All buildings and parking lots have driveways. This must connect to a sidewalk,
+allowing pedestrians to enter/exit that object. The driveway OPTIONALLY connects
+to the nearest driveable lane. This allows cars to enter/exit that object for
+parking.
+
+Public transit stops are located somewhere on a sidewalk. They're associated
+with a driveable position where the bus or train stops. In the future, this will
+need to account for dedicated surface-level platforms and for underground
+transit stations, likely associated with a building.
+
+There's a concept of "parking blackholes." If you treat every road as
+bidirectional without access restrictions, then the graph is connected. But the
+more detailed view has to factor in one-way roads and things near the map
+border. These blackholes influence where cars will try to look for parking
+(since we don't want them entering a blackhole and getting stuck) and also, for
+temporary/unintentional reasons, where pedestrian<->bicycle transitions will
+happen.

book/src/map/importing/README.md

@@ -1,6 +1,23 @@
 # Importing
 
-Overview of the process. The importer tool.
+This chapter describes the process of transforming OSM extracts into A/B
+Street's map model. The steps are:
 
-Don't be afraid of how complicated this seems. It started simple -- just bring
+1.  A large .osm file is clipped to a hand-drawn boundary region, using
-in OSM roads, chop into pieces, generate turns.
+    `osmconvert`
+2.  The `convert_osm` crate reads the clipped `.osm`, and a bunch of optional
+    supplementary files, and produces a `RawMap`
+3.  Part of the `map_model` crate transforms the `RawMap` into the final `Map`
+4.  Other applications read and use the `Map` file
+
+The `importer` crate orchestrates these steps, along with automatically
+downloading any missing input data.
+
+The rest of these sections describe each step in a bit more detail. Keeping the
+docs up-to-date is hard; the best reference is the code, which is hopefully
+organized clearly.
+
+Don't be afraid of how complicated this pipeline seems -- each step is
+relatively simple. If it helps, imagine how this started -- just chop up OSM
+ways into road segments, infer lanes for each road, and infer turns between the
+lanes.

book/src/map/importing/convert_osm.md

@@ -1,46 +1,43 @@
 # From OSM to RawMap (`convert_osm` crate)
 
 The first phase of map building reads in data from OSM files and a few others,
-producing a serialized `RawMap`. Importing all maps (one for each pre-defined
+producing a serialized `RawMap`.
-bounding polygon) takes a few minutes. Players don't see this cost; it only
-takes a few seconds to load a serialized map.
 
-- `osm.rs`: Read .osm, extracting the points for road-like ways, buildings, and
+Only major steps are described; see the code for the rest.
-  areas
+
-  - Areas usually come from a relation of multiple ways, with the points out of
+## extract.rs
-    order. Gluing all the points together fails when the .osm has some ways
+
-    clipped out. In that case, try to trace along the map boundary if the
+Read .osm, extracting the points for road-like ways, buildings, and areas
-    partial area intersects the boundary in a clear way. Otherwise, just use a
+
-    straight line to try to close off the polygon.
+- Areas usually come from a relation of multiple ways, with the points out of
-  - Also read traffic signal locations and turn restrictions between OSM ways
+  order. Gluing all the points together fails when the .osm has some ways
-- `split_ways.rs`: Split OSM ways into road segments
+  clipped out. In that case, try to trace along the map boundary if the partial
-  - OSM ways cross many intersections, so treat points with multiple ways and
+  area intersects the boundary in a clear way. Otherwise, just use a straight
-    the points at the beginning and end of a way as intersections, then split
+  line to try to close off the polygon.
-    the way into road segments between two intersections.
+- Also read traffic signal locations and turn restrictions between OSM ways
-  - This phase remembers which road segment is the beginning and end of the OSM
+
-    way, for per-lane turn restrictions later
+## split_ways.rs
-  - Apply turn restrictions between roads here. Since OSM ways cross many
+
-    intersections, the turn restrictions only apply to one particular road
+Split OSM ways into road segments
-    segment that gets created from the way. Make sure the destination of the
+
-    restriction is actually incident to a particular source road.
+- OSM ways cross many intersections, so treat points with multiple ways and the
-- `clip.rs`: Clip the map to the boundary polygon
+  points at the beginning and end of a way as intersections, then split the way
-  - Osmosis options in `import.sh` preserve ways that cross the boundary
+  into road segments between two intersections.
-  - Trim roads that cross the boundary. There may be cases where a road dips out
+- This phase remembers which road segment is the beginning and end of the OSM
-    of bounds, then immediately comes back in. Disconnecting it isn't ideal, but
+  way, for per-lane turn restrictions later
-    it's better to manually tune the boundary polygon when this happens than try
+- Apply turn restrictions between roads here. Since OSM ways cross many
-    to preserve lots of out-of-bounds geometry.
+  intersections, the turn restrictions only apply to one particular road segment
-  - Area polygons are intersected with the boundary polygon using the `clipping`
+  that gets created from the way. Make sure the destination of the restriction
-    crate
+  is actually incident to a particular source road.
-- `lib.rs`: Remove cul-de-sacs (roads that begin and end at the same
+
-  intersection), because they mess up parking hints and pathfinding.
+## clip
-- `lib.rs`: Apply parking hints from a King County GIS blockface dataset
+
-  - Match each blockface to the nearest edge of a road
+Clip the map to the boundary polygon
-  - Interpret the metadata to assign on-street parking there or not
+
-- `lib.rs`: Apply offstreet parking hints from a King County GIS dataset
+- `osmconvert` options preserve ways that cross the boundary
-  - Match each point to the building containing it, plumbing through the number
+- Trim roads that cross the boundary. There may be cases where a road dips out
-    of spots
+  of bounds, then immediately comes back in. Disconnecting it isn't ideal, but
-- `lib.rs`: **Disabled**: Apply sidewalk presence hints from a King County GIS
+  it's better to manually tune the boundary polygon when this happens than try
-  dataset
+  to preserve lots of out-of-bounds geometry.
-  - Match each sidewalk line to the nearest edge of a road
+- Area polygons are intersected with the boundary polygon using the `clipping`
-  - Update the road to have a sidewalk on none, one, or both sides
+  crate
-- `lib.rs` using the `srtm` module: Load (extremely poor quality) elevation data