A/B Street

All documentation lives here. Some chapters are only intended for a technical audience.

A/B Street Instructions

This is an alpha-quality demo. Please email dabreegster@gmail.com or file a Github issue if you hit problems.

Installing the game

Grab a pre-built binary release -- updated every Sunday, announced at r/abstreet:

  • Windows
    • Unzip the folder, then run play_abstreet.bat. You'll probably getting a warning about running software from an unknown publisher.
  • Mac
    • Unzip the directory, then run play_abstreet.sh.
    • If you get an error about the developer unverified, follow this. Help needed to start signing the release!
    • If that just opens a text file instead of running the game, then instead open terminal, cd to the directory you just unzipped. Then do: cd game; RUST_BACKTRACE=1 ./game 1> ../output.txt 2>&1
    • Help needed to package this as a Mac .app, to make this process simpler
  • Linux
    • Unzip the directory, then run play_abstreet.sh.
  • FreeBSD, thanks to Yuri

Or you can try playing directly in your web browser -- still experimental for now.

Or you can compile from source.

Playing the game

  • Use the tutorial to learn the controls.
  • Play the challenges for directed gameplay.
  • Try out any ideas in the sandbox.

Data source licensing

A/B Street binary releases contain pre-built maps that combine data from:

Other binary data bundled in:

Help map out on-street parking

parking_mapper

This guide assumes you've edited OSM before. Contact dabreegster@gmail.com if you have any trouble. Also give me a heads up when you make some edits, so I can regenerate the maps!

  1. Install A/B Street
  2. Choose Contribute parking data on the main screen
  3. Change the map if you'd like to focus somewhere in particular
  4. Click a road with unknown parking
  5. Select what kind of on-street parking the road has
  6. Repeat
  7. Click Generate OsmChange file
  8. Upload the diff.osc file by adding a layer in JOSM (or send it to me)

Like all edits to OSM, to figure out ground-truth, you can survey in-person or use Bing Streetside. Do not use data from Google Maps to edit OSM.

FAQ

Why?

I'm trying to build a realistic traffic simulation of Seattle using OSM data, then use it to strengthen proposals for pedestrianized streets, improving the bike network, and mitigating the West Seattle bridge closure. A/B Street is only as good as its data, and parking is one of the biggest gaps. Missing data means unrealistic traffic as vehicles contend for few parking spots, and roads that look much wider than they are in reality.

Why put this data in OSM?

Why can't I just grab parking data from SDOT's map, using the blockface dataset? Well, I'm trying -- when you see a parking lane in the tool, it's coming from blockface, unless that road in OSM is tagged. But the blockface dataset is comically wrong in many places -- for example, the Montlake bridge apparently has unrestricted parking?! King County GIS has confirmed the dataset isn't meant to be used for this level of detail.

Plus, if the data is in OSM, anybody else can make use of it.

How does the tool work?

A/B Street attempts to render individual lanes and intersections from OSM data. This makes it useful to audit the lane tags in OSM, including parking:lane. The tool tracks your edits and when you generate the OsmChange file, it grabs modified ways from the OSM API to generate a diff. You can inspect the diff, load it in JOSM, and upload.

Your changes won't immediately be reflected in A/B Street. Let me know when you've done some amount of mapping, and I'll regenerate the maps from fresh data.

Why use this tool?

You don't have to; this tool or ID or JOSM all work. But the UI is clunky for this specific purpose. (Also, if you find this tool clunky in any way, let me know and I'll fix it.) There's also a proposed StreetComplete quest.

What about parking restrictions?

There are many parking:lane tags to indicate restricted parking zones, time restrictions, etc. Feel free to map that in ID or JOSM, but I'm just looking to make a first pass over a wide area.

What about off-street parking?

Ideally I'd also like to know how many private parking spots are available to residents of each building. But I don't know of an OSM schema for mapping this, or a practical way to collect this data. Let me know if you have ideas.

What about long roads where parking appears and disappears?

The tool won't help. Use your favorite editor to split the way when the lane configuration changes. Also feel free to just skip these areas.

How to coordinate with other mappers?

If somebody wants to set up HOT tasking, that'd be great, but I don't expect so many people to jump on this.

I noticed weird roads in the tool

Welcome to my world. ;) If the number of lanes seems wrong, select the road and check the OSM tags. I'm inferring lanes from that. Feel free to make manual OSM edits to fix any problems you see. (I'd like to extend this tool to make that easier; let me know if you have ideas how to do this.)

I want to map an area, but there's no option for it

To keep the release size small, I'm not including all maps yet. Let me know what you'd like to see included.

Or if you have a .osm file, try the quick start guide.

Importing a new city into A/B Street

This process isn't easy yet. Please email dabreegster@gmail.com or file a Github issue if you hit problems. I'd really appreciate help and PRs to improve this.

Quick start

Use this if you want to import a city on your computer without making it available to other users yet.

  • If you're using the binary release and have a .osm file, just do: ./importer --oneshot=map.osm.

  • If you're building from source, do: ./import.sh --oneshot=map.osm. If you can't run import.sh, make sure you have all dependencies. If you're using Windows and the console logs appear in a new window, try running the command from import.sh directly, changing the $@ at the end to --oneshot=map.osm or whatever arguments you're passing in.

The oneshot importer will will generate a new file in data/system/maps that you can then load in the game. If you have an Osmosis polygon filter (see below), you can also pass --oneshot_clip=clip.poly to improve the result. You should first make sure your .osm has been clipped: osmconvert large_map.osm -B=clipping.poly --complete-ways -o=smaller_map.osm.

By default, driving on the right is assumed. Use --oneshot_drive_on_left to invert.

How to get .osm files

If the area is small enough, try the "export" tool on https://www.openstreetmap.org. You can download larger areas from https://download.bbbike.org/ or http://download.geofabrik.de/index.html, then clip them to a smaller area. Use geojson.io or geoman.io to draw a boundary around the region you want to simulate and save the geojson locally. Use cargo run --bin geojson_to_osmosis < boundary.geojson > clipping.poly to convert that geojson to the Osmosis format required by osmconvert.

Note that you may hit problems if you use JOSM to download additional data to a .osm file. Unless it updates the <bounds/> element, A/B Street will clip out anything extra. The best approach is to explicitly specify the boundary with --oneshot_clip.

Including the city to A/B street more permanently

Follow this guide to add a new city to A/B street by default so other users can use it as well.

  1. Make sure you can run import.sh -- see the instructions. You'll need Rust, osmconvert, gdal, etc.

  2. Create a new directory: mkdir -p data/input/your_city/polygons

  3. Use geojson.io or geoman.io to draw a boundary around the region you want to simulate and save the geojson locally.

  4. Use cargo run --bin geojson_to_osmosis < boundary.geojson > clipping.poly to convert that geojson to the Osmosis format required by osmconvert.

  5. Create a new module in importer/src/ for your city, copying importer/src/krakow.rs as a guide. Edit that file in the obvious way. The main thing you'll need is a .osm or .osm.pbf file to download that contains your city. The clipping polygon will be applied to that.

  6. Update importer/src/main.rs to reference your new module, following krakow as an example.

  7. Update map_belongs_to_city in updater/src/main.rs

  8. Run it: ./import.sh --city=your_city --raw --map

  9. Update .gitignore, following krakow as an example.

Send a PR with your changes! I'll generate everything and make it work with updater, so most people don't have to build everything from scratch.

Next steps

OpenStreetMap isn't the only data source we need. If you look at the import pipeline for Seattle, you'll see many more sources for parking, GTFS bus schedules, person/trip demand data for scenarios, etc. Most of these aren't standard between cities. If you want to make your city more realistic, we'll have to import more data. Get in touch.

You may notice issues with OSM data while using A/B Street. Some of these are bugs in A/B Street itself, but others are incorrectly tagged lanes. Some resources for fixing OSM:

How A/B Street works

The overview:

  1. A detailed map of Seattle is built from OpenStreetMap (OSM)
  2. A realistic set of daily trips by car, bike, foot, and bus are simulated
  3. You make small changes to roads and intersections
  4. You explore how these changes affect the trips

Details below. Many limitations are mentioned; improvements are ongoing. I'll add pictures to explain better when I get time.

Driving

  • Movement: no acceleration, go the full speed limit of the road unless there's a slower vehicle in front
  • Lanes
    • No over-taking or lane-changing in the middle of a road, only at intersections
    • Strange choice of lanes -- the least full at the time of arrival
    • Narrow two-way neighborhood roads where, in practice, only one car at a time can go are currently full two-way roads
  • Routing is based on fastest time assuming no traffic
    • No rerouting if the driver encounters a traffic jam

Parking

  • Types
    • On-street: parallel parking lanes from GeoData blockface dataset and manually mapped
    • Off-street: most buildings have at least a few parking spots in a driveway or carport
      • Currently experimenting in the downtown map: set the number of available spots based on number of cars seeded at midnight
    • Parking lots: the number of spots is inferred
  • Restrictions
    • All spots are public except for the few spots associated with each building
    • No time restrictions or modeling of payment
  • How cars park
    • Drivers won't look for parking until they first reach their destination building. Then they'll drive to the nearest open parking spot (magically knowing what spots are open, even if they're a few blocks away). If somebody else has taken the spot when they arrive, they'll try again.
    • Once a driver finds an open spot, they'll take 10-15 seconds to park. They block the road behind them in the meantime. There are no conflicts between pedestrians and cars when using a driveway. Cars won't make left turns into or out of driveways.
  • Some parking along the boundary of the map is "blackholed", meaning it's impossible to actually reach it. Nobody will use these spots.

Biking

  • Choice of lane
    • Multi-use trails like the Burke Gilman and separated cycle-tracks like the one along Broadway are currently missing
    • Cyclists won't use an empty parking lane
    • On roads without a bike lane, cyclists currently won't stick to the rightmost lane
    • No over-taking yet, so cars can get stuck behind a bike even if there's a passing lane
  • Elevation change isn't factored into route choice or speed yet; pretend everybody has an e-bike
  • Beginning or ending a cycling trip takes 30-45 seconds. Locking up at bike racks with limited capacity isn't modeled; in practice, it's always easy in Seattle to find a place to lock up.

Walking

  • Not using sidewalk and crosswalk data from OSM yet
  • No jay-walking, even on empty residential streets
  • Pedestrians can't use roads without sidewalks at all
    • When a road only has a sidewalk on one side, driveways will cross the road
  • Pedestrians can "ghost" through each other; crowds of people can grow to any size

Transit

  • The modeling of buses is extremely simple and buggy; I'll work on this soon
  • No light rail yet

Intersections

  • Conflicting movements are coarse: a second vehicle won't start a conflicting turn, even if the first vehicle is physically out of the way but still partially in the intersection
  • Most of the time, vehicles won't "block the box" -- if there's no room in the target lane, a vehicle won't start turning and risk getting stuck in the intersection
  • Traffic signals
    • Only fixed timers; no actuated signals or centralized control yet
    • The timing and stages are automatically guessed, except some intersections are manually mapped
    • No pedestrian beg buttons; walk signals always come on
    • The signal doesn't change for rush hour or weekday/weekend traffic; there's one pattern all day
  • Turn restrictions from OSM are applied
    • Per lane (left turn only from leftmost lane), entire roads, multiple intersections

People and trips

  • A "synthetic population" of ~700,000 people come from PSRC's Soundcast model
    • Soundcast uses census, land-use, vehicle counts, and commuter surveys. The current data is from 2014.
    • All driving trips are currently single-occupancy; no car-pooling or ridesharing
    • Parked cars are initially placed at midnight based on the number of trips between buildings
  • Each person's schedule never changes
    • Your changes to the map won't yet convince somebody to take a bus or walk instead of drive

Map edits

  • Types of edits
    • Change types of lanes. Sometimes this is unrealistic based on actual road width, but data for this is unavailable.
    • Reversing direction of lanes
    • Changing stop signs
    • Changing traffic signal timing
    • Closing roads and intersections for construction, forcing rerouting
  • Disconnecting the map
    • Generally you can't close sidewalks or make changes to make buildings unreachable
    • You shouldn't be able to make bus stops unreachable, but currently this is buggy

Case studies

Note: Most of these still aren't started, because the baseline simulation in the relevant area isn't working. Unknown traffic signal timing, bad guesses at the amount of off-street parking, lanes tagged incorrectly in OpenStreetMap, and simulation bugs cause unrealistic gridlock. It's hard to evaluate a change without a realistic baseline.

In progress:

TODO:

  • Close Broadway and Pine to through-traffic
  • Traffic signal timing at Montlake/520 and Montlake/Pacific
    • Walking around here is frustrating, and pre-COVID, vehicle traffic got fairly stuck
  • Pedestrianizing the Ave (u-district)
    • I can't find the proposal anymore; maybe this?
  • Eastlake bike lanes / RapidRide J
    • See here and here
    • Need to audit lanes in OSM along Eastlake
    • Especially with the Fairview Ave bridge out, detouring to the Cheshiahud loop isn't as useful
  • Madison / RapidRide G
  • Bus lanes on Denny
  • Bike Master Plan
  • Downtown one-way snake
    • An old crazy idea I've always wanted to try
  • Unsorted ideas

Lake Washington Blvd Stay Healthy Street

Draft, updated May 7, 2020 by Dustin Carlino (dabreegster@gmail.com)

In April 2020, Seattle Department of Transportation started rolling out Stay Healthy Streets, restricting roads to through-traffic to give people walking and biking more space for social distancing. Seattle Neighborhood Greenways soon proposed extending this to a 130-mile network.

Selecting the streets requires some planning:

These streets were selected to amplify outdoor exercise opportunities for areas with limited open space options, low car ownership and routes connecting people to essential services and food take out. We also ensured street closures did not impact newly opened food pick up loading zones, parking around hospitals for service for health care professionals, and bus routes.

I've spent the last two years building A/B Street, software to explore the impacts of changes like this on different modes of transportation. So, let's try implementing part of the proposed network and see what happens!

NOTE: You might want to read how A/B Street works first.

Lake Washington Blvd

Let's start with one part of the proposal, closing Lake Washington Blvd to cars through the Arboretum. There's already a multi-use trail alongside this stretch, but its width makes it difficult to maintain 6 feet from people. There are some parking lots that become inaccessible with this proposal, but they're currently closed anyway.

edits

First attempt

Let's get started! If you want to follow along, install A/B Street, open sandbox mode, and switch the map to Lake Washington corridor. Zoom in on the southern tip of the Arboretum and hop into edit mode. We can see Lake Washington Blvd just has one travel lane in each direction here. Click each lane, convert it to a bike lane, and repeat north until Foster Island Road.

When we leave edit mode, the traffic simulation resets to midnight. Nothing really interesting happens until 5 or 6am, so we'll speed up time. Watching the section of road we edited, we'll only see pedestrians and bikes use this stretch of road. If we want, we can click an individual person and follow along their journey.

Something's weird though. There's lots of traffic cutting northbound through the neighborhood, along 29th, Ward, and 28th. We can open up the throughput layer to find which roads have the most traffic. More usefully, we can select "compare before edits" to see what roads are getting more or less traffic because of the road we modified. As expected, there's much less traffic along Lake Wash Blvd, but it's also clear that lots of cars are now cutting through 26th Ave E.

Traffic calming

Let's say you want to nudge traffic to use 23rd Ave, the nearest north/south arterial, instead. (A/B Street is an unopinionated tool; if you have a different goal in mind, try using it for that instead.) In this simulation, drivers pick the fastest route, so we could try lowering speed limits or make some of the residential streets connecting to Madison one-way, discouraging through-traffic. In reality, the speed limit changes could be implemented through traffic calming or cheap, temporary alternatives.

Next steps

I'm working to model "local access only" roads in A/B Street, and I'll describe how to measure the impact on travel times. Stay tuned to see more of the proposed network simulated, and get in touch if you'd like to help out!

West Seattle mitigations

Draft, updated June 23, 2020 by Dustin Carlino (dabreegster@gmail.com)

In March 2020, the West Seattle bridge was closed due to cracks forming. As of May, COVID-19's impact on commuting means the area still hasn't seen how the area will handle losing the main route to the rest of Seattle. A local group, HPAC, published a list of requests to SDOT to prepare the area for these changes.

This page will try to explore some of the problems and solutions from HPAC's document using A/B Street, a traffic simulator designed to explore the impacts of changes like this on different modes of transportation.

NOTE: You might want to read how A/B Street works first.

16th Ave SW and SW Holden St

HPAC has been asking for a protected left-turn stage at this intersection. I'm unfamiliar with this intersection and currently unable to scout in-person, so I'm blindly guessing the traffic signal currently has just two stages:

existing_diagram

From watching the traffic, it seems like the east/west direction is busier, with lots of eastbound traffic headed towards WA-509. Holden St has no turn lanes, so a protected left turn stage makes sense. Let's make the change and see what happens:

Unfortuately, we can't evaluate the change yet, because the simulation gets stuck with unrealistic traffic jams in other parts of the map. This is mostly due to data quality issues in OpenStreetMap and incorrectly guessed traffic signal timings. These problems can be fixed with the help of somebody familiar with the area.

Re-evaluate arterials

The 9th item from HPAC's list asks for measuring the amount of east-west traffic to figure out what streets people are using as arterials. That's an easy analysis, using the throughput layer.

By 6am, the busiest streets include Admiral Way, S Charlestown, SW Genesee, SW Alaska, SW Holden, and SW Roxbury St. Again, it's necessary to first fix data quality problems and run a full day before doing more analysis.

Once the simulation is running smoothly, A/B Street can be used to make changes -- like lowering speed limits, adding a protected left turn stage, or converting part of the road into a bus lane -- and evaluate the effects on individual trips and aggregate groups.

Repair the bridge

Community proposals now includes a "repair the bridge" option, which should restore things to how they were before March 2020. This is useful as a baseline, to explore what traffic patterns were like before the closure.

Developer guide

Getting started

You will first need:

One-time setup:

  1. Download the repository: git clone https://github.com/dabreegster/abstreet.git

  2. Grab the minimal amount of data to get started: cargo run --bin updater

  3. Run the game: RUST_BACKTRACE=1 cargo run --bin game --release. On Windows, set environment variables like this: set RUST_BACKTRACE=1 && cargo run --bin game --release

Development tips

  • Generated API documentation
  • Compile faster by just doing cargo run. The executable will have debug stack traces and run more slowly. You can do cargo run --release to build in optimized release mode; compilation will be slower, but the executable much faster.
  • Some in-game features are turned off by default or don't have a normal menu to access them. The list:
    • To toggle developer mode: press Control+S in game, or cargo run -- --dev
    • To warp to an object by numeric ID: press Control+j
    • To enter debug mode with all sorts of goodies: press Control+D
  • You can start the game in different modes using flags:
    • cargo run --bin game -- --dev data/system/maps/downtown.bin starts on a particular map
    • cargo run --bin game -- data/system/scenarios/downtown/weekday.bin starts with a scenario (which is tied to a certain map)
    • cargo run --bin game -- --challenge=trafficsig/tut2 starts on a particular challenge. See the list of aliases by passing in a bad value here.
    • cargo run --bin game -- data/player/saves/montlake/no_edits_unnamed/00h00m20.3s.bin restores an exact simulation state. Savestates are found in debug mode (Control+D) -- they're probably confusing for the normal player experience, so they're hidden for now.
    • cargo run --bin game -- --tutorial=12 starts somewhere in the tutorial
    • Adding --edits='name of edits' starts with edits applied to the map.

Downloading more cities

As data formats change over time, things in the data/ directory not under version control will get out of date. At any time, you can run cargo run --bin updater from the main repository directory to update only the files that have changed.

You can also opt into downloading updates for more cities by editing data/config. Opting into everything looks like this:

runtime: seattle,huge_seattle,krakow,berlin,xian,tel_aviv,london
input: seattle,huge_seattle,krakow,berlin,xian,tel_aviv,london

runtime downloads new maps and scenarios in data/system/. input is used for building those maps -- see below.

Building map data

You can skip this section if you're just touching code in game, widgetry, and sim.

To run all pieces of the importer, you'll need some extra dependencies:

The first stage of the importer, --raw, will download input files from OSM, King County GIS, and so on. If the mirrors are slow or the files vanish, you could fill out data/config and use the updater described above to grab the latest input.

You can rerun specific stages of the importer:

  • If you're modifying the initial OSM data -> RawMap conversion in convert_osm, you need ./import.sh --raw --map.
  • If you're modifying map_model but not the OSM -> RawMap conversion, then you just need ./import.sh --map.
  • If you're modifying the demand model for Seattle, you can add --scenario to regenerate.
  • By default, all maps are regenerated. You can also specify a single map: ./import.sh --map downtown.
  • By default, Seattle is assumed as the city. You have to specify otherwise: ./import.sh --city=los_angeles --map downtown_la.

You can also make the importer import a new city.

Understanding stuff

The docs listed at https://github.com/dabreegster/abstreet#documentation explain things like map importing and how the traffic simulation works.

Code organization

If you're going to dig into the code, it helps to know what all the crates are. The most interesting crates are map_model, sim, and game.

Constructing the map:

  • convert_osm: extract useful data from OpenStreetMap and other data sources, emit intermediate map format
  • kml: extract shapes from KML shapefiles
  • map_model: the final representation of the map, also conversion from the intermediate map format into the final format
  • map_editor: GUI for modifying geometry of maps and creating maps from scratch. pretty abandoned as of June 2020
  • importer: tool to run the entire import pipeline
  • updater: tool to download/upload large files used in the import pipeline
  • map_tests: tests for the map import pipeline

Traffic simulation:

  • sim: all of the agent-based simulation logic
  • headless: tool to run a simulation without any visualization

Graphics:

  • game: the GUI and main gameplay
  • widgetry: a GUI and 2D OpenGL rendering library, using glium + winit + glutin

Common utilities:

  • abstutil: a grab-bag of IO helpers, timing and logging utilities, etc
  • geom: types for GPS and map-space points, lines, angles, polylines, polygons, circles, durations, speeds

Other:

  • traffic_seitan: a bug-finding tool that randomly generates live map edits

Code conventions

All code is automatically formatted using https://github.com/rust-lang/rustfmt; please run cargo +nightly fmt before sending a PR. (You have to install the nightly toolchain just for fmt)

cargo fmt can't yet organize imports, but we follow a convention to minimize conflict with what some IDEs do. Follow existing code to group imports: std, external crates, other crates in the project, the current crate, then finally any module declarations.

The error handling is unfortunately inconsistent. The goal is to gracefully degrade instead of crashing the game. If a crash does happen, make sure the logs will have enough context to reproduce and debug. For example, giving up when some geometry problem happens isn't ideal, but at least make sure to print the road / agent IDs or whatever will help find the problem. It's fine to crash during map importing, since the player won't deal with this, and loudly stopping problems is useful. It's also fine to crash when initially constructing all of the renderable map objects, because this crash will consistently happen at startup-time and be noticed by somebody developing before a player gets to it.

Prefer using info!, warn!, error!, etc from the log crate. Or if a Timer is available and you want to collect all notes together, timer.note. There are still many places calling println!, but we're trying to clean these up.

See the testing strategy page.

Profiling

Use https://github.com/flamegraph-rs/flamegraph, just running it on the binaries you build normally.

Development notes

Find packages to upgrade: cargo outdated -R

Deal with compile tile: cargo bloat --time

Find why two binary crates aren't sharing dependencies: https://old.reddit.com/r/rust/comments/cqceu4/common_crates_in_cargo_workspace_recompiled/

Where's a dependency coming from? cargo tree -i -p syn

Diff screencaps: http://www.imagemagick.org/Usage/compare/#methods

Debug OpenGL calls:

apitrace trace --api gl ../target/debug/game
qapitrace game.trace
apitrace dump game.trace

Understand XML: just use firefox

Building releases

Cross-compilation notes: https://github.com/rust-embedded/cross Or use https://github.com/japaric/trust

Initially have to:

cargo install cross
sudo apt-get install docker.io
sudo usermod -aG docker ${USER}

Then:

sudo systemctl start docker
cross build --release --target x86_64-pc-windows-gnu --bin game
wine target/x86_64-pc-windows-gnu/release/game.exe data/system/maps/montlake.bin

Markdown

For formatting:

sudo apt-get install npm
cd ~; mkdir npm; cd npm
npm init --yes
npm install prettier --save-dev --save-exact

Videos

# Fullscreen
ffmpeg -f x11grab -r 25 -s 1920x960 -i :0.0+0,55 -vcodec huffyuv raw.avi

ffmpeg -ss 10.0 -t 5.0 -i raw.avi -f gif -filter_complex "[0:v] fps=12,scale=1024:-1,split [a][b];[a] palettegen [p];[b][p] paletteuse" screencast.gif

Faster linking

sudo apt-get install lld

Stick this in ~/.cargo/config:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    "-C", "link-arg=-fuse-ld=lld",
]

git

Keep a fork up to date:

# Once
git remote add upstream https://github.com/rust-windowing/glutin/

git fetch upstream
git merge upstream/master
git diff upstream/master

Refactoring

perl -pi -e 's/WrappedComposite::text_button\(ctx, (.+?), (.+?)\)/Btn::text_fg(\1).build_def\(ctx, \2\)/' `find|grep rs|xargs`

Stack overflow

rust-gdb --args ../target/release/game --dev

Drawing diagrams

draw.io

Mapping

xodo on Android for annotating maps in the field

OSM tools

osmcha.org for recent changes

To upload diffs:

java -jar ~/Downloads/josm-tested.jar ~/abstreet/map_editor/diff.osc

JOSM: Press (and release T), then click to pan. Download a relevant layer, select the .osc, merge, then upload.

Fonts

fontdrop.info

Release checklist

What things are sensitive to changes in map data and simulation rules?

  • tutorial
  • optimize commute challenges

What things do I always forget to test?

  • DPI issues, use --scale_factor

API

Suppose you're tired of manually fiddling with traffic signals, and you want to use machine learning to do it. You can run A/B Street without graphics and automatically control it through an API.

Examples

This Python example has everything you need to get started.

See all example code -- there are different experiments in Go and Python that automate running a simulation, measuring some metric, and making a change to improve the metric.

Control flow

The headless API server that you run contains a single map and simulation at a time. Even though you can theoretically have multiple clients make requests to it simultaneously, the server will only execute one at a time. If you're trying to do something other than use one script to make API calls in sequence, please get in touch, so we can figure out something better suited to your use case.

When you start the headless server, it always loads the montlake map with the weekday scenario. The only way you can change this is by calling /sim/load. For example:

curl http://localhost:1234/sim/load -d '{ "scenario": "data/system/scenarios/downtown/monday.bin", "modifiers": [], "edits": null }' -X POST`

You can also pass flags like --infinite_parking to the server to control SimOptions. These settings will apply for the entire lifetime of the server; you can't change them later.

API details

Under construction: The API will keep changing. There are no backwards compatibility guarantees yet. Please make sure I know about your project, so I don't break your client code.

For now, the API is JSON over HTTP. The exact format is unspecified, error codes are missing, etc. A summary of the commands available so far:

  • /sim
    • GET /sim/reset: Reset all temporary map edits and the simulation state. The trips that will run don't change; they're determined by the scenario specified by the last call to /sim/load. If you made live map edits using things like /traffic-signals/set, they'll be reset to the edits from /sim/load.
    • POST /sim/load: Switch the scenario being simulated, and also optionally sets the map edits.
    • GET /sim/get-time: Returns the current simulation time.
    • GET /sim/goto-time?t=06:30:00: Simulate until 6:30 AM. If the time you specify is before the current time, you have to call /sim/reset first.
    • POST /sim/new-person: The POST body must be an ExternalPerson in JSON format.
  • /traffic-signals
    • GET /traffic-signals/get?id=42: Returns the traffic signal of intersection #42 in JSON.
    • POST /traffic-signals/set: The POST body must be a ControlTrafficSignal in JSON format.
    • GET /traffic-signals/get-delays?id=42&t1=03:00:00&t2=03:30:00: Returns the delay experienced by every agent passing through intersection #42 from 3am to 3:30, grouped by direction of travel.
    • GET /traffic-signals/get-cumulative-thruput?id=42: Returns the number of agents passing through intersection #42 since midnight, grouped by direction of travel.
    • GET /traffic-signals/get-all-current-state: Returns the current state of all traffic signals, including the stage timing, waiting, and accepted agents.
  • /data
    • GET /data/get-finished-trips: Returns a JSON list of all finished trips. Each tuple is (time the trip finished in seconds after midnight, trip ID, mode, duration of trip in seconds). The mode is either a string like "Walk" or "Drive", or null if the trip was cancelled for any reason.
    • GET /data/get-agent-positions: Returns a JSON list of all active agents. Vehicle type (or pedestrian), person ID, and position is included.
    • GET /data/get-road-thruput: Returns a JSON list of (road, agent type, hour since midnight, throughput for that one hour period).
  • /map
    • GET /map/get-edits: Returns the current map edits in JSON. You can save this to a file in data/player/edits/map_name/ and later use it in-game normally. You can also later run the headless server with --edits=name_of_edits.
    • GET /map/get-edit-road-command?id=123: Returns an object that can be modified and then added to map edits.

Working with the map model

If you need to deeply inspect the map, you can dump it to JSON:

cargo run --bin dump_map data/system/maps/montlake.bin > montlake.json

See some example code that reads this JSON and finds buildings.

The format of the map isn't well-documented yet. See the generated API docs and the map model docs in the meantime.

Working with individual trips

You can use the /sim/new-person API in the middle of a simulation, if needed. If possible, it's simpler to create a Scenario as input.

Working with Scenarios

You can import trips from your own data.

You can also generate different variations of one of the demand models by specifying an RNG seed:

cargo run --bin random_scenario -- --rng=123 --map=data/system/maps/montlake.bin > data/system/scenarios/montlake/home_to_work.json

You can also dump Scenarios (the file that defines all of the people and trips) to JSON:

cargo run --bin dump_scenario data/system/scenarios/montlake/weekday.bin > montlake_weekday.json

You can modify the JSON, then put the file back in the appropriate directory and use it in-game:

cargo run --bin game data/system/scenarios/montlake/modified_scenario.json

The Scenario format is also undocumented, but see the generated API docs anyway.

Testing strategy

Unit tests

As you've probably noticed, there aren't many. Lots of the interesting behavior in A/B Street - UI interactions, details of the simulation, map importing -- would take lots of infrastructure to specify a setup and expected outcomes. If you have ideas for new tests, contributions always welcome! In the meantime, one useful test covers how OSM tags translate into individual lanes.

Screenshot diffs

Downloading fresh OSM data or modifying any part of the map importing pipeline could easily break things. Expressing invariants about the map output is hard, because importing is far from perfect, and OSM data is often quite buggy. So the approach to preventing regressions here is to look for visual changes to the final rendered map.

  1. When a new map is opted into this type of test, somebody manually squints carefully at it and sanity checks that it works to some degree.
  2. They use the screen capture tool in debug mode to tile the map into 1920x960 chunks and screengrab everything.
  3. Later, somebody regenerates the map with some possible changes.
  4. They grab screenshots again, then use compare_screenshots.sh to quickly look at the visual diff. Changes to intersection geometry, number of lanes, rendering, etc are all easy to spot.
  5. If this manual inspection of the diff is good, they commit the new screenshots as the new goldenfiles.

data/regen.sh

This tool regenerates all maps and scenarios from scratch. cargo run --bin updater -- --dry then reveals what files have changed.

Additionally, this script does a few more tests:

  • --prebake runs the full weekday scenario on two maps that've previously been coerced into being gridlock-free
  • --smoketest simulates one hour on all maps, flushing out bugs with bus spawning, agents hitting odd parts of the map, etc
  • --check_proposals makes sure the edits shipped with the game still load properly

map_tests

The map_tests crate runs the full importer against really simple .osm files. To iterate rapidly on interpreting turn restrictions, for example, it produces goldenfiles describing all turns in the tiny map.

Old tests

Once upon a time, I made a little test harness that would run the simulation headlessly (without graphics), set up certain situations forcing a car to park in a certain spot, and asserted that different sim/src/events.rs were produced in the right order. The map_editor tool was used to manually draw really simple maps for these situations. I deleted everything, because the effort to specify the input and expected output were too tedious to maintain, and this never really helped catch bugs. There was a way to label roads and buildings in the synthetic maps, so the test code could assert person 2 made it to the "house" building, but even with all of this, it was pretty hard.

This approach is maybe worth reviving, though.

Mass importing many maps

For https://github.com/dabreegster/abstreet/issues/326, I'm starting to figure out how to import hundreds of maps into A/B Street. There are many issues with scaling up the number of supported maps. This document just focuses on importing.

The current approach

https://download.bbbike.org/ conveniently has 200 OSM extracts for major cities world-wide. The data/bbike.sh script downloads these. Then data/mass_import.sh attempts to import them into A/B Street.

The bbike extracts, however, cover huge areas surrounding major cities. Importing such large areas is slow, and the result is too large to work well in A/B Street or the OSM viewer. Ideally, we want just the area concentrated around the "core" of each city.

https://github.com/dabreegster/abstreet/blob/master/convert_osm/src/bin/extract_cities.rs transforms a huge .osm file into smaller pieces, each focusing on one city core. This tool looks for administrative boundary relations tagged as cities, produces a clipping polygon covering the city, and uses osmconvert to produce a smaller .osm file. The tool has two strategies for generating clipping polygons. One is to locate the admin_centre or label node for the region, then generate a circle of fixed radius around that point. Usually this node is located in the city core, so it works reasonably, except for "narrow" cities along a coast. The other strategy glues together the relation's multipolygon boundary, then simplifies the shape (usually with thousands of points) using a convex hull. This strategy tends to produce results that're too large, because city limits are often really huge.

Problems

  • Outside the US, administrative boundaries don't always have a "city" defined. In Tokyo in particular, this name isn't used. I'm not sure which boundary level to use yet.
  • The tool assumes driving on the right everywhere. OSM has https://wiki.openstreetmap.org/wiki/Key:driving_side, but this is usually tagged at the country level, which isn't included in the bbike extracts.
  • The resulting maps are all "flattened" in A/B Street's list, so you can't see any hierarchy of areas. Two cities with the same name from different areas will arbitrarily collide.

Map model

A/B Street transforms OpenStreetMap (OSM) data into a detailed geometric and semantic representation of the world for traffic simulation. This chapter describes that map model, with the hopes that it'll be useful for purposes beyond this project.

Overview

A Map covers everything inside some hand-drawn boundary, usually scoped to a city or a few of a city's districts. Unlike OSM, it doesn't cover the entire world; it only has areas specifically extracted for some purpose.

A map consists of many objects. Mainly, there are roads, broken down into individual lanes, and intersections. A road is a single segment connecting exactly two intersections (as opposed to OSM, where a single "way" may span many intersections). Lanes within a road have a specific type, which dictates their direction of travel (or lack of travel, like on-street parking) and uses. Sidewalks are represented as bidirectional lanes. Roads connect at intersections, which contain an explicit set of turns, each linking a source lane to a destination lane.

Maps also contain parking lots and buildings, which connect to the nearest driveable lane and a sidewalk. Maps have water and park areas, only used for drawing. They also represent public transit stops and routes.

How is a map used?

Unlike some GIS systems, maps don't use any kind of database -- they're just a file, anywhere from 1 to ~500MB (depending on the size of their boundary). Once loaded into memory, different objects from the map can be accessed directly, along with a large API to perform various queries.

Most of the map's API is read-only; once built, a map doesn't change until user-created edits are applied.

The pipeline to import a map from OSM data (and also optional supplementary, city-specific data) is complex and may take a few minutes to run, but it happens once offline. Applications using maps just read the final file.

Features

Why use A/B Street's map model instead of processing OSM directly?

TODO: Order these better. For each one, show before/after pictures

Area clipping

Bodies of water, forests, parks, and other areas are represented in OSM as relations, requiring the user to stitch together multiple polylines in undefined orders and handle inner holes. A/B Street maps handle all of that, and also clip the area's polygon to the boundary of the entire map -- including coastlines.

Road and intersection geometry

OSM represents roads as a polyline of the physical center of the road. A/B Street infers the number and type of lanes from OSM metadata, then creates individual lanes of appropriate width, each with a center-line and polygon for geometry. At intersections, the roads and lanes are "trimmed back" to avoid overlapping, and the "common area" becomes the intersection's polygon. This heuristic process is reasonably robust to complex shapes, with special treatment of highway on/off-ramps, although it does still have some bugs.

Turns

At each intersection, A/B Street infers all legal movements between vehicle lanes and sidewalks. This process makes use of OSM metadata about turn lanes, inferring reasonable defaults for multi-lane roads. OSM turn restriction relations, which may span a sequence of several roads to describe U-turns around complex intersections, are also used.

Parking lots

OSM models parking lots as areas along with the driveable aisles. Usually the capacity of a lot isn't tagged. A/B Street automatically fills paring lots with individual stalls along the aisles, estimating the capacity just from this geometry.

Stop signs

At unsignalized intersections, A/B Street infers which roads have to stop, and which have right-of-way.

Traffic signals

OSM has no way to describe how traffic signals are configured. A/B Street models fixed-timer signals, automatically inferring the number of phases, their duration, and the movements that are prioritized and permitted during each phase.

Pathfinding

A/B Street can determine routes along lanes and turns for vehicles and pedestrians. These routes obey OSM's turn restriction relations that span multiple road segments. They also avoid roads that're tagged as not allowing through-traffic, depending on the route's origin and destination and vehicle type. The pathfinding optionally makes use of contraction hierarchies to greatly speed up query performance, at the cost of a slower offline importing process.

Bridge z-ordering

OSM tags bridges and tunnels, but the roads that happen to pass underneath bridges aren't mapped. A/B Street detects these and represents the z-order for drawing.

Buildings

Similar to areas, A/B Street consolidates the geometry of OSM buildings, which may be split into multiple polygons. Each building is also associated with the nearest driveable lane and sidewalk, and metadata is used to infer a land-use (like residential and commercial) and commercial amenities available.

Experimental: public transit

A/B Street uses bus stops and route relations from OSM to build a model of public transit routes. OSM makes few guarantees about how the specifics of the route are specified, but A/B Street produces specific paths, handling clipping to the map boundary.

... All of this isn't the case yet, but it's a WIP!

Experimental: separated cyclepaths, tramways, and walking paths

Some cyclepaths, tram lines, and footpaths in OSM are tagged as separate ways, with no association to a "main" road. Sometimes this is true -- they're independent trails that only occasionally cross roads. But often they run alongside a road. A/B Street attempts to detect these and "snap" them to the main road as extra lanes.

... But this doesn't work yet at all.

Map model details

A/B Street builds a rich representation of a city map using OpenStreetMap (OSM) and other sources. This chapter describes how.

TODO: Integrate pictures from these slides.

This recorded presentation covers some of this.

The map

A single city is broken down into different pieces...

A/B Street comes with a few maps, each defined by a bounding/clipping polygon for some portion of Seattle. Each map has these objects:

  • Roads: A single road connects two intersections, carrying OSM metadata and containing some child lanes.
  • Lanes: An individual lane of traffic. Driving (any vehicle), bus-only, and bike-only lanes have a direction. On-street parking lanes don't allow any movement, and they have some number of parking spots. Sidewalks are bidirectional.
  • Intersections: An intersection has references to all of the incoming and outgoing lanes. Most intersections have a stop sign or traffic signal policy controlling movement through it.
    • Border intersections on the edge of the map are special places where agents may appear or disappear.
  • Turns: A turn connects one lane to another, via some intersection. (Sidewalks are bidirectional, so specifying the intersection is necessary to distinguish crosswalks at each end of a sidewalk.)
  • Buildings: A building has a position, OSM metadata, and a front path connecting the edge of the building to the nearest sidewalk. Most trips in A/B Street begin and end at buildings. Some buildings also contain a number of off-street parking spots.
  • Area: An area has geometry and OSM metadata and represents a body of water, forest, park, etc. They're just used for drawing.
  • Bus stop: A bus stop is placed some distance along a sidewalk, with a pointer to the position on the adjacent driving or bus lane where a bus stops for pick-up.
  • Bus route: A bus route has a name and a list of stops that buses will cycle between. In the future, they'll include information about the frequency/schedule of the route.
  • Parking lot: A parking lot is connected to a road, has a shape, and has some internal driving "aisles." The number and position of individual parking spots is auto-generated.

Coordinate system

A/B Street converts (longitude, latitude) coordinates into a simpler form.

  • An (x, y) point starts with the top-left of the bounding polygon as the origin. Note this is screen drawing order, not a Cartesian plane (with Y increasing upwards) -- so angle calculations account for this.
  • The (x, y) values are f64's trimmed to a few decimal places, with way more precision than is really needed. These might become actual fixed-point integers later, but for now, a Pt2D skirts around Rust's limits on f64's by guaranteeing no NaN's or infinities and thus providing the full Eq trait.
  • A few places in map conversion compare points using different thresholds, usually below 1 meter. Ideally these epsilon comparisons could be eliminated in favor of a fixed-point integer representation, but for now, explicit thresholds are useful.

Invariants

Ideally, the finalized maps would satisfy a list of invariants, simplifying the traffic simulation and drawing code built on top. But the input data is quite messy and for now, most of these aren't quite guaranteed to be true.

  • Some minimum length for lanes and turns. Very small lanes can't be drawn, tend to break intersection polygons, and may lead to gridlocked traffic.
  • Some guarantees that positions along adjacent lanes actually match up, even though different lanes on the same road may have different lengths. Examples include the position of a bus stop on the sidewalk and bus lane matching up.
    • Additionally, parking lanes without an adjacent driving lane or bus stops without any driving or bus lanes make no sense and should never occur.
  • Connectivity -- any sidewalk should be reachable from any other, and most driving lanes should be accessible from any others. There are exceptions due to border intersections -- if a car spawns on a highway along the border of the map, it may be forced to disappear on the opposite border of the map, if the highway happens to not have any exits within the map boundary.

Connectivity

For a single mode, each lane is connected to two intersections. Turns connect two lanes. There are no turns between sidewalks and driving/bike/bus lanes.

All buildings and parking lots have driveways. This must connect to a sidewalk, allowing pedestrians to enter/exit that object. The driveway OPTIONALLY connects to the nearest driveable lane. This allows cars to enter/exit that object for parking.

Public transit stops are located somewhere on a sidewalk. They're associated with a driveable position where the bus or train stops. In the future, this will need to account for dedicated surface-level platforms and for underground transit stations, likely associated with a building.

There's a concept of "parking blackholes." If you treat every road as bidirectional without access restrictions, then the graph is connected. But the more detailed view has to factor in one-way roads and things near the map border. These blackholes influence where cars will try to look for parking (since we don't want them entering a blackhole and getting stuck) and also, for temporary/unintentional reasons, where pedestrian<->bicycle transitions will happen.

Importing

This chapter describes the process of transforming OSM extracts into A/B Street's map model. The steps are:

  1. A large .osm file is clipped to a hand-drawn boundary region, using osmconvert
  2. The convert_osm crate reads the clipped .osm, and a bunch of optional supplementary files, and produces a RawMap
  3. Part of the map_model crate transforms the RawMap into the final Map
  4. Other applications read and use the Map file

The importer crate orchestrates these steps, along with automatically downloading any missing input data.

The rest of these sections describe each step in a bit more detail. Keeping the docs up-to-date is hard; the best reference is the code, which is hopefully organized clearly.

Don't be afraid of how complicated this pipeline seems -- each step is relatively simple. If it helps, imagine how this started -- just chop up OSM ways into road segments, infer lanes for each road, and infer turns between the lanes.

From OSM to RawMap (convert_osm crate)

The first phase of map building reads in data from OSM files and a few others, producing a serialized RawMap.

Only major steps are described; see the code for the rest.

extract.rs

Read .osm, extracting the points for road-like ways, buildings, and areas

  • Areas usually come from a relation of multiple ways, with the points out of order. Gluing all the points together fails when the .osm has some ways clipped out. In that case, try to trace along the map boundary if the partial area intersects the boundary in a clear way. Otherwise, just use a straight line to try to close off the polygon.
  • Also read traffic signal locations and turn restrictions between OSM ways

split_ways.rs

Split OSM ways into road segments

  • OSM ways cross many intersections, so treat points with multiple ways and the points at the beginning and end of a way as intersections, then split the way into road segments between two intersections.
  • This phase remembers which road segment is the beginning and end of the OSM way, for per-lane turn restrictions later
  • Apply turn restrictions between roads here. Since OSM ways cross many intersections, the turn restrictions only apply to one particular road segment that gets created from the way. Make sure the destination of the restriction is actually incident to a particular source road.

clip

Clip the map to the boundary polygon

  • osmconvert options preserve ways that cross the boundary
  • Trim roads that cross the boundary. There may be cases where a road dips out of bounds, then immediately comes back in. Disconnecting it isn't ideal, but it's better to manually tune the boundary polygon when this happens than try to preserve lots of out-of-bounds geometry.
  • Area polygons are intersected with the boundary polygon using the clipping crate

Road/intersection geometry: RawMap to InitialMap

The remainder of map construction is done in the map_model crate. There's one intermediate structure between RawMap and Map, called InitialMap.

  • make/remove_disconnected.rs: Remove disconnected roads
    • Just floodfill from some road, assuming all roads are bidirectional, to get different partitions.
    • Remove roads from all but the largest partition
  • make/initial/mod.rs and make/initial/lane_specs.rs: Interpret OSM tags to figure out what lanes are on each side of each road, also figuring out the total width of the road.
  • make/initial/geometry.rs: Figure out the polygon for each intersection, and trim back road center-lines to end at a face of the polygon.
    • For every road touching the intersection, get the polyline of each side, based on the road's width
      • See appendix for how to shift polylines
    • Sort all the polylines by the angle to the intersection's shared point
    • Intersect every polyline with every other polyline
      • More specifically -- the second half of each polyline, to get the correct collision point
      • Look at the perpendicular infinite line to the collision point on the shifted polyline, then find where it hits the original center line. Trim back the center line by the max distance from these collisions.
    • Compute the intersection's polygon by considering collisions between adjacent roads' polylines
    • Deal with short roads and floating point issues by deduping any adjacent points closer than 0.1m

InitialMap to Map

Still in the map_model crate.

  • map.rs's make_half_map: Expand roads to lanes, using the list of lane types from before
  • make/turns.rs: Generate turns for every intersection.
    • Vehicle turns (for cars, bikes, buses)
      • Consider every pair of roads in the intersection. Try to match up lane types -- if there's a bike lane on both roads, don't add a turn from driving->bike or bike->driving. If there's not, then fallback to transitions between different lane types.
      • Classify the turn based on the difference between the angle of the incoming lane's last line and the outgoing lane's first line
        • For straight turns, use the Cartesian product to link every incoming with every outgoing lane. If the indices dont match up, the turn becomes a LaneChangeLeft or LaneChangeRight turn. This is used later for intersection policies to prioritize turns appropriately.
        • Right and left turns only originate from the one lane on the appropriate side
    • Walking turns for pedestrians
      • Consider pairs of adjacent roads around the intersection
        • Make a crosswalk to the other side of the road, assuming there's a sidewalk on both sides
        • Make a shared sidewalk corner over to the adjacent road
        • If the adjacent road doesn't have a sidewalk on the close side, then consider skipping that road and making a crosswalk over to the next road. An example of this is a crosswalk over a highway on/off ramp.
    • Verify all the turns so far are unique
    • Filter by the OSM turn restrictions ("only straight" between road1 and road2)
    • Try to apply the OSM per-lane restrictions ("straight or left" from lane 3)
      • The number of lanes in the OSM metadata might not match up with how many lanes created
      • Some of these OSM tags are just completely wrong sometimes. If the filter makes an incoming lane lose all of its turns, then ignore that tag.
  • make/parking_blackholes.rs: Find well-connected roads near "blackhole" lanes.
    • Starting from most driving/biking lanes, most other lanes are reachable. Some aren't -- such as one-way highways inevitably leading from or to a border. These are "blackholes" -- pathfinding to or from here may fail.
    • Find the largest strongly-connected component (SCC) in the driving graph. From every other lane (a blackhole), floodfill both forwards and backwards to find the nearest driving lane part of the main SCC.
    • Later, if a car needs to park by a building on a blackhole road, it'll instead start searching for parking at the redirect. This prevents it from being forced to instead exit the map through a border.
  • make/buildings.rs: Match buildings up with sidewalks
    • Find the closest sidewalk polyline to each building's center. Then draw a straight line for the front path between the edge of the building and the sidewalk point.
    • Filter out buildings too far away from any sidewalk
    • The front path might cross through other buildings; this is probably not worth fixing.
  • make/buildings.rs: Same for parking lots
    • Similar process to match parking lots to nearest sidewalk and driving lane
    • Try to place parking spots along both sides of parking aisles
    • Filter out overlapping spots
  • make/bridges.rs: Find what roads lie beneath bridges, and update their Z-order accordingly for later drawing.
  • stop_signs.rs: Instantiate default stop sign policies
    • Rank incoming roads by OSM priority (arterial beats residential)
    • If there's only one rank, then make an all-way stop
    • Otherwise, the highest rank gets priority and others stop
      • Check if there are any conflicts based on this. If so, then fall-back to an all way stop.
  • traffic_signals.rs: Instantiate default traffic signal policies
    • Apply the first predefined policy that works.
      • 4-way 4 stage, 4-way 2 stage, 3-way 3-stage, degenerate policy for 2 roads, 2-stage for 4 one-ways
      • Fallback to a greedy assignment that just randomly starts a new stage, adds all compatible turns, and repeats until all turns are present priority in some stage.
  • pathfind/mod.rs: Prepare pathfinding
    • A/B Street uses contraction hierarchies (CH) for fast routing, using the fast_paths crate.
    • pathfind/vehicle.rs: For cars, bikes, buses
      • There's a separate CH for cars, buses, and bikes, since they can use slightly different sets of lanes.
      • Building the CH for buses and bikes is much faster than the one for cars, because the algorithm can re-use the node ordering from the first CH.
      • Every lane is a node in the graph, even if it's not an appropriate lane type -- it might change later, and reusing orderings is vital for speed.
      • If two lanes are connected by a turn, then there's an edge in the graph.
        • The edge weight is the length of the lane and turn. Later this could take into account speed limit, penalize lane-changing and left turns, etc.
    • pathfind/walking.rs: For pedestrians
      • Only sidewalk lanes are nodes in the graph -- sidewalks can't ever be changed in A/B Street, so there's no concern about reusing node orderings.
      • All turns between two sidewalks become edges, again using length
      • When actually pathfinding, we get back a list of sidewalks. The actual paths used in the traffic simulation specify forwards or backwards on a sidewalk. Looking at adjacent pairs of sidewalks lets us easily stitch together exact directions.
  • make/bus_stops.rs: Match bus stops with a sidewalk
    • Also precompute the position where the bus stops on the adjacent driving or bus lane.
    • This "equivalent position on another lane" process has a few weird cases, since two lanes on the same road might have different lengths. Right now, the same distance from the start of the lane is used, with clamping for shorter lanes. Ideally, the position would be found by projecting a perpendicular line out from one lane to the other.
  • make/bus_stops.rs: Finalize the list of bus routes
    • Between each pair of adjacent bus stops, run pathfinding to verify there's actually a path for the bus to follow. If any are disconnected, remove the bus route
    • Remove bus stops that have no routes serving them.
  • pathfind/walking.rs: Precompute the CH for pedestrians who will use buses
    • Nodes in the graph are sidewalks and every bus stop
    • There's an edge with weight 0 between a bus stop and its sidewalk
    • There's also an edge with weight 0 between bus stops that're adjacent via some route. Ideally this weight would account for the time until the next bus and the time spent on the bus, etc.
    • Later when figuring out which bus to use for a pedestrian, the resulting list of nodes is scanned for the first and last bus stop along the same route.

Development tricks

  • Separate phases for fast incremental development
    • Don't reimport all data from OSM every time there's a change to part of the map construction code!
    • For slow steps that don't change often, make them separate binaries -- hence convert_osm being separate from the rest.
  • Don't be afraid of manual intervention
    • The data isn't perfect. It's easy to spend lots of time fiddling with code to automatically handle all problems
    • Instead of automatically resolving problems, prefer good tooling for finding and specifying fixes
    • Be careful of derivative structures that could get out of sync with OSM. Prefer contributing real fixes to OSM.
  • Screenshot diff testing
    • When working on the code for intersection geometry, it's easy to check a few example cases get fixed by some change. But what if another part of the map regresses somehow?
    • Take screenshots of the entire map, keep the checksums under version control, look at the diffs visually, and manually verify any changes.
    • Implementation details: One huge gif or png is too slow to read and write, so take a bunch of tiled screenshots covering everything. Amusingly, rendering to a file with glium is slow unless compiling in release mode (which isn't an option for quick incremental development). So instead, pan to each section of the map, render it, call an external screenshot utility, and move on -- just don't wiggle the mouse during this process!
  • Different IDs for objects make sense during different phases
    • For the final product, lanes and such are just a contiguous array, indexed by numeric IDs.
    • But sometimes, we need IDs that're the same between different boundary polygons of maps, so that player edits can be applied anywhere. Using (longitude, latitude) pairs hits floating-point serialization and comparison issues, so referring to roads as (OSM way ID, OSM node ID 1, OSM node ID 2) works instead.

Appendix: PolyLines

Add some pictures here to demonstrate how polyline shifting works, the explode-to-infinity problem, and the bevel/miter fix.

Live edits

A key feature of A/B Street is the player editing the map and seeing how traffic responds. The possible edits include:

  • Change lane types (driving, bus, bike, parking -- sidewalks are fixed)
  • Change speed limits
  • Reverse a lane
  • Change a stop sign policy (which roads have a stop sign and which have priority)
  • Change a traffic signal policy

The map conversion process outlined above takes a few minutes, so reusing this process directly to compute a map with edits wouldn't work at all for real gameplay. Instead, the process for applying edits is incremental:

  • Figure out the actual diff between edits and the current map
    • This is necessary for correctness, but also speeds up a sequence of edits made in the UI -- only one or two lanes or intersections actually changes each time. Of course when loading some saved edits, lots of things might change.
  • For any changed roads, make sure any bus stop on it have a good pointer to their equivalent driving position for the bus.
  • For any modified intersections, recompute turns and the default intersection policies
  • Recompute all the CHs for cars, buses, and bikes -- note sidewalks and bus stops never change
    • This is the slowest step. Critically, the fast_paths crate lets a previous node ordering be reused. If just a few edge weights change, then recomputing is much faster than starting from scratch.
    • While making edits in the UI, we don't actually need to recompute the CH after every little tweak. When the player exits edit mode, only then do we recompute everything.

A list of lanes and intersections actually modified is then returned to the drawing layer, which uploads new geometry to the GPU accordingly.

A/B Street's map model as a platform

A/B Street's representation of a city, built mostly from OSM and lots of heuristics, is likely useful to other projects. This doc brainstorms what it would look like to properly expose it to other users.

To sum up what the map model provides: geometry + semantics.

Use cases

  • Different UIs (particularly 3D / VR) for exploring cities as they are or as they could be, like Streetmix 3D and Complete Street Rule
  • Importing slices of a city as assets into a game engine like Godot
  • A new OSM viewer/editor, particularly focused on POIs
  • Something focusing on 15-minute neighborhoods, with isochrones and nearby amenities

TODO: Give a quick Python example of what interacting with the end goal could look like.

Just data is not enough

At first glance, the existing Map structure could be written to some format with a nicely documented schema. This would certainly be useful, but it's not nearly enough. Interpreting the data sometimes requires lots of code, which already exists -- so why not expose it to users as well?

Examples in OSM where I wish "standard libraries" existed to interpret the data:

A/B Street solves these problems (or at least it tries to), but by itself, the resulting data isn't always useful. So some examples of where a library would be needed too:

  • Pathfinding. ABST does lots of work especially to handle "live" map edits and cheaply regenerate contraction hierarchies. Also, pathfinding requires obeying OSM turn restrictions that span multiple roads -- this prevents even plain old Dijkstra's from working correctly.
  • Getting geometry in different forms. Lanes are stored as a PolyLine, but what if a consumer wants the thickened Polygon, either as points, or maybe even pre-triangulated vertices and indices?

How would an API/library work?

The traditional approach is to link against part of A/B Street as a library and call it through language-specific bindings. The more language-agnostic option is defining an API (maybe JSON or protobuf) and having clients run a local A/B Street server, making HTTP requests to it. This is like the "sidecar" pattern in microservice-land.

Compatibility

Really have to think through this carefully. Some examples of big changes on the horizon:

  • Additive: separate cycleways and tramways. Likely no schema change.
  • Modify: traffic signals will get more complex
  • Modify: we'll likely try again to merge tiny intersections together, which would get rid of the current guarantees that a road/intersection is associated to one particular OSM object

Layering

Clients should be able to opt into different data layers. For example, A/B Street strips out OSM building tags right now to keep filesizes small. But an OSM viewer would want to keep this (and likely discard the large contraction hierarchies). So some pieces of the map model need to be teased apart into optional pieces, and probably loaded in as separate files.

The bigger vision

Depending what other open source projects are on board, the general idea is to start assembling an ecosystem of libraries/tooling to make it easier to build new things off of open GIS data.

The end state might look like this. A few separate applications would exist, all running both natively and in the browser:

  • A/B Street the game, more or less in its current form
  • A new OpenStreetMap viewer, likely focused on visualizing roads and points-of-interest in detail
  • The street parking OSM editor, and other OSM editors specialized for mapping certain things
  • A new app focusing on 15-minute neighborhoods, using isochrones to show amenities available nearby
    • Ideally, allow editing current land use / zoning, to let people explore how new policies might get closer to a 15-minute neighborhood.
    • Possibly GOAT does all of this already, and this new thing shouldn't be built
  • A new app for creating story maps, showing events that occur over time, with lots of detail about the surrounding environment

All of these would make use of some common libraries, which should be extracted out cleanly from A/B Street today:

  • the map model and OSM importer
  • the widgetry UI library
  • some common code for specifically interacting with maps in widgetry
  • a tool to generate a traffic demand model from OSM data, optional census data, etc
  • the discrete-event traffic simulation that A/B Street uses today
  • core geometry/utility libraries

But note only the first application would use things like the simulation library. The point of more cleanly modularizing these pieces is to make it easier for new people to build different pieces, without having to understand and be coupled to everything else. Also, as appropriate, these pieces should use common data formats (like shared-row) to be interoperable with Streetmix, Complete Streets, etc.

A/B Street's Traffic Simulation

This article describes how cars, bikes, buses, and pedestrians are modeled in A/B Street. All code lives in the sim crate.

This recorded presentation covers some of this.

Discrete-event simulation

The traffic simulation models different agents (cars, bikes, buses, pedestrians, and intersections) over time. Agents don't constantly sense and react to the world every second; instead, they remain in some state until something interesting happens. This is a discrete-event architecture -- events are scheduled for some time in the future, and handling them changes the state of some agents. The core simulation loop simply processes events in order -- see scheduler.rs and the step method in sim.rs.

Cars

(Note: Cars, bikes, and buses are all modeled the same way -- bikes just have a max speed, and buses/bikes can use restricted lanes.)

Cars move through a sequence of lanes and turns (movements through an intersection). They queue and can't over-take a slow lead vehicle. The main simplifying assumption in A/B Street is that cars can instantly accelerate and decelerate. This wouldn't model highway driving at all, where things like jam waves are important, but it's reasonable for in-city driving. The essence of scarcity is the capacity on lanes and the contention at intersections. What happens in between isn't vital to get exactly right.

A car has a few states (mechanics/car.rs):

  • Crossing some distance of a lane/turn over some time interval
  • Queued behind another car on a lane/turn
  • WaitingToAdvance at the end of a lane, blocked on an intersection
  • A few states where the car stays in one place: Parking, Unparking, and Idling (for buses at a stop)

State transitions happen in mechanics/driving.rs. This is best explained by an example sequence:

  • A car enters the Unparking state, taking a fixed 30s to exit a parking spot and enter the adjacent driving lane. The driving lane is blocked during this time, to mimic somebody pulling out from a parallel parking spot.
  • The car is now fully somewhere on the driving lane. It enters the Crossing state, covering the remaining distance to the end of the road. The time interval is calculated assuming the car travels at the max speed limit of the road.
  • After that time, the car checks if there's anybody in the queue before it. Nope? Then it attempts to initiate a turn through the intersection, but the stop sign says no, so the car enters the WaitingToAdvance state.
  • Some time later, the stop sign wakes up the car. The car starts the turn, entering the Crossing state again.
  • After finishing the turn, the car starts Crossing the next lane. When it's finished, it turns out there are a few cars ahead of it, so it enters the Queued state.
  • When the lead vehicle directly in front of the car exits the lane, it wakes up the car, putting it in the Crossing state, starting at the appropriate following distance behind the lead vehicle. This prevents the car from immediately warping to the end of the lane when the lead vehicle is out of the way.
  • And so on...

Exact positions

For a discrete-event simulation, we don't usually care exactly where on a lane a car is at some time. But we do need to know for drawing and for a few cases during simulation, such as determining when a bus is lined up with a bus stop in the middle of a lane. mechanics/queue.rs handles this, computing the distance of every car in a lane. For cars in the Crossing state, we linearly interpolate distance based on the current time. Of course, cars have to remain in order, so Queued cars are limited by the lead vehicle's position + the lead vehicle's length + a fixed following distance of 1m.

Another case where we need to know exact positions of cars is to prevent the first vehicle on a lane from hitting the back of a car who just left the lane. All vehicles have length, and position is tracked by the front of the car. When a car's front leaves a lane, its back is still partly in the lane. Logically, the new lead car in the lane still needs to act like it's Queued. So each lane keeps a "laggy head", pointing to the car with its back partly in the lane. After the laggy head has made it sufficient distance along its new turn or lane, the laggy head on the old lane can be erased, unblocking the lead vehicle. This requires calculating exact distances and some occasionally expensive cases where we have to schedule frequent events to check when a laggy head is clear.

Lane-changing

Lane-changing (LCing) deserves special mention. A/B Street cheats by not allowing it on lanes themselves. Instead, at intersections, cars can perform turns that shift them over any number of lanes. These LCing turns conflict with other turns appropriately, so the contention is still modeled. Why do it this way? In a previous project, I tried opportunistic LCing. If a car had room to warp to the equivalent distance on the adjacent lane without causing a crash, it would start LCing, then take a fixed time to slide over, blocking both lanes throughout. This meant cars often failed to LC when they needed to, forcing them to reroute, botching their trip times. In many cases the cars would be permanently stuck, because pathfinding would return paths requiring LCing that couldn't be pulled off in practice due to really short roads. Why not try making the car slow down if needed? Eventually it might have to stop, which could lead to unrealistic gridlock. This LCing model was using a detailed discrete-time model with cars accelerating properly; maybe it's easier with A/B Street's simplified movement model.

Currently in A/B Street, cars will pick the least backed-up lane when there's a choice. They make this decision once when they reach the front of a queue; look for opportunistically_lanechange in router.rs. The decision could be improved.

Pedestrians

Pedestrian modeling -- in mechanics/walking.rs is way simpler. Pedestrians don't need to queue on sidewalks; they can "ghost" through each other. In Seattle, there aren't huge crowds of people walking and slowing down, except for niche cases like Pike Place Market. So in A/B Street, the only scarce resource modeled is the time spent waiting to cross intersections.

Intersections

I need to flesh this section out. See mechanics/intersections.rs for how stop signs and traffic signals work. Two things I need to fix before making this section interesting:

  • Only wake up relevant agents when a previous agent finishes a turn.
  • Don't let an agent start a low-priority turn (like an unprotected left) if it'll force a high-priority vehicle approaching to wait. The approaching vehicle is still in the Crossing state, so we need to notify intersections ahead of time of intended turns and an ETA.

One contributor to permanent gridlock is cars and bikes being stuck in an intersection, preventing conflicting turns from being performed. To help avoid this, one of the last checks that stop signs and traffic signals perform before accepting a new turn request is that the target lane has enough space for the new vehicle. This is "reserved" space, not necessarily currently occupied by vehicles in that lane. This accounts for other vehicles performing a turn bound for that lane. See try_to_reserve_entry in mechanics/queue.rs. When a car completely leaves a lane (determined by the "laggy head" described above), this space is freed, and blocked cars are woken up.

Appendix: discrete-time simulation

A/B Street's first traffic model was discrete-time, meaning that every agent reacted to the world every 0.1s. Cars had a more realistic kinematics model, accelerating to change speed and gradually come to a halt. Cars did a worst-case estimation of how far ahead they need to lookahead in order to satisfy different constraints:

  • Don't exceed any speed limits
  • Don't hit the lead vehicle (which might suddenly slam on its brakes)
  • Stop at the end of a lane, unless the intersection says to go

After fighting with this approach for a long time, I eventually scrapped it in favor of the simpler discrete-event model because:

  • It's fundamentally slow; there's lots of busy work where cars in freeflow with nothing blocking them or stopped in a long queue constantly check to see if anything has changed.
  • Figuring out the acceleration to apply for the next 0.1s in order to satisfy all of the constraints is really complicated. Floating point inaccuracies cause ugly edge cases with speeds that wind up slightly negative and with cars coming to a complete stop slightly past the end of a lane. I wound up storing the "intent" of an action to auto-correct these errors.
  • The realism of having cars accelerate smoothly didn't add value to the core idea in A/B Street, which is to model points of contention like parking capacity and intersections. (This is the same reason why I don't model bike racks for parking bikes -- in Seattle, it's never hard to find something to lock to -- this would be very different if Copenhagen was the target.) Additionally, the kinematics model made silly assumptions about driving anyway -- cars would smash on their accelerators and brakes as hard as possible within all of the constraints.

Travel demand

A/B Street simulates people following a schedule of trips over a day. A single trip has a start and endpoint, a departure time, and a mode. Most trips go between buildings, but the start or endpoint may also be a border intersection to represent something outside the map boundaries. The mode specifies whether the person will walk, bike, drive, or use transit. Without a good set of people and trips, evaluating some changes to a map is hard -- what if the traffic patterns near the change aren't realistic to begin with? This chapter describes where the travel demand data comes from.

Scenarios

A scenario encodes the people and trips taken over a day. See the code.

TODO:

  • talk about vehicle assignment / parked car seeding

Data sources

Seattle: Soundcast

Seattle luckily has the Puget Sound Regional Council, which has produced the Soundcast model. They use census stats, land parcel records, observed vehicle counts, travel diaries, and lots of other things I don't understand to produce a detailed model of the region. We're currently using their 2014 model; the 2018 one should be available sometime in 2020. See the code for importing their data.

TODO:

  • talk about how trips beginning/ending off-map are handled

Berlin

This work is ongoing. See the code. So far, we've found a population count per planning area and are randomly distributing the number of residents to all residential buildings in each area.

Proletariat robot

What if we just want to generate a reasonable model without any city-specific data? One of the simplest approaches is just to spawn people beginning at residential buildings, make them go to some workplace in the morning, then return in the evening. OpenStreetMap building tags can be used to roughly classify building types and distinguish small houses from large apartments. See the proletariat_robot code for an implementation of this.

This is ongoing work spearheaded by Mateusz. Some of the ideas for next steps are to generate different types of people (students, workers), give them a set of activities with durations (go to school for 7 hours, 1 hour lunch break), and then further pick specfic buildings to travel to using more OSM tags.

Custom import

If you have your own data, you can import it. The input format is JSON -- an example:

{
  "scenario_name": "monday",
  "people": [
    {
      "origin": {
	"Position": {
          "longitude": -122.303723,
          "latitude": 47.6372834
	}
      },
      "trips": [
        {
          "departure": 10800.0,
          "destination": {
            "Position": {
              "longitude": -122.3075948,
              "latitude": 47.6394773
	    }
          },
          "mode": "Drive"
        }
      ]
    }
  ]
}

Run the tool:

cargo run --bin import_traffic -- --map=data/system/maps/montlake.bin --input=/path/to/input.json

The tool matches input positions to the nearest building or border intersection, within 100 meters. The departure time is seconds since midnight. The tool will fail if any point doesn't match to a building, or if any of the specified trips can't be created (due to graph connectivity problems, for example). If your requirements are different or you have any trouble using this format/tool, please file a Github issue -- just consider this tool and format a prototype.

Modifying demand

The travel demand model is extremely fixed; the main effect of a different random number seed is currently to initially place parked cars in specific spots. When the player makes changes to the map, exactly the same people and trips are simulated, and we just measure how trip time changes. This is a very short-term prediction. If it becomes much more convenient to bike or bus somewhere, then more people will do it over time. How can we transform the original demand model to respond to these changes?

Right now, there's very preliminary work in sandbox mode for Seattle weekday scenarios. You can cancel all trips for some people (simulating lockdown) or modify the mode for some people (change 50% of all driving trips between 7 and 9am to use transit).

Research

Gridlock

Here "gridlock" refers to the general problem of trips getting permanently stuck, preventing the full simulation from completing. Most of the work is tracked here.

The general lesson is: you can't code your way around all edge cases. The data in OSM often needs manual fixes. It's often useful to spend coding effort on tools to detect and fix OSM problems.

Problems

The choices in the movement model matter. Some gridlock is inherent to any system with queueing and conflicting turns. But in reality, people wiggle around partly blocked turns. And some of this comes from the treatment of the front/back of vehicles.

  • Short roads in OSM causing very weird geometry
  • Intersection geometry being too large, requiring too much time to cross
  • Unrealistic traffic patterns caused by everyone trying to park in one big garage (downtown) or take some alley (the UW soundcast issue)
  • Too many people try to take an unprotected left turn (often at a stop sign)
  • Bad individual traffic signals, usually at 5- or 6-ways
  • Groups of traffic signals logically acting as a single intersection
  • Separate traffic signals along a corridor being unsynchronized
  • Vehicles performing illegal sequences of turns
  • Vehicles are stuck with their plan and not reacting to traffic by changing route
  • Real traffic would result in a gridlock without a deliberate actions to avoid it. Such actions range from individual decisions of drivers to police manually controlling traffic. Intelligent avoidance of gridlock is not simulated and is extremely hard to simulate.
  • Vehicles will wait in lane filled with already waiting vehicles, even if there is a completely empty lane allowing travel in desired direction. It makes easier for entire lane between crossings to fill, contributing to gridlocks. Note that while this and other clearly stupid behaviors are clearly unrealistic, it is not trivial to implement more realistic and more efficient decisions.
  • Issues caused by the unrealistic lane-changing model
    • Two turns that go to the same lane (one going "straight", the other often a lane-change) conflict. The conflict is coarse, at the granularity of the entire intersection. So if vehicles are piled up in two lanes trying to merge into one, then one group is likely to go through as expected, but the second group will wait for the first to completely clear the intersection. Until then, it looks like a conflicting turn is being done.

Solutions

Divide into implemented or not.

  • Synchronizing pairs of signals
  • Uber-turns
    • for interpreting OSM turn restrictions
    • for synchronizing a group of signals
    • for locking turn sequences
      • Once a vehicle starts an uber-turn, prevent others from starting conflicting turns on nearby intersections. Until groups of traffic signals are configured as one, this is necessary to prevent somebody from making it halfway through a sequence then getting blocked.
  • Cycle detector
  • block-the-box protection
    • the manual list of overrides
    • likely shouldn't apply during uber-turns
    • is it always fine to block the box at degenerate intersections?
  • hacks to allow conflicting turns at really broken intersections
  • manually timing signals
  • penalties for lane choice to make lane usage realistic

Not implemented

  • Dynamic rerouting
  • Allow multiple vehicles through intersection at once if there is enough space on lane where given vehicle is going. Currrently vehicles travel through crossings one by one (or, with --disable_block_the_box enabled - will enter crossing even if leaving it will be impossible).
  • Last resort: if someone's waiting on a turn >5m, just go.
  • Uber-turns
    • Group both stop sign and traffic signal intersections when looking for uber-turns. Even a single traffic signal surrounded by tiny roads with stop signs is causing problems.

Strategy for resolving

Because there are so many different causes all tangled together, my approach is to simplify the simulation as much as possible. A problem is much easier to understand and fix when it's isolated. I've been trying this to get the downtown weekday scenario to complete. A list of different techniques to simplify, in no particular order:

  • Use the --infinite_parking flag to just let everyone park directly in their destination buildings. This is useful since downtown has many large parking garages with high capacity, but I don't have a data source describing them.
  • Use the --disable_turn_conflicts flag, which greatly reduces realism, but lets conflicting turns happen simultaneously. (Even with this and other flags, downtown still gridlocks!) It also disables traffic signals, so bad inferred timing isn't an issue.
  • Use the --disable_block_the_box flag to workaround short roads.
  • If you notice problems forming from cars stacking up behind slower cyclists, there's no over-taking implemented yet. Use the scenario modifiers to convert all biking trip to driving: --scenario_modifiers='[{"ChangeMode":{"to_mode":"Drive","pct_ppl":100,"departure_filter":[0.0,86400.0],"from_modes":["Bike"]}}]'
  • If all else fails, use the scenario modifiers to bluntly cancel some percentage of all trips.

Fixing data used in simulation

Give more examples of changesets.

Multi-modal trips

A single trip consists of a sequence of TripLegs -- walking, operating a vehicle (car or bike), and riding the bus. Depending whether a trip begins or ends at a border or building, there are many combinations of these sequences. This is a way to categorize them into three groups. I'm not sure it's the simplest way to express all the state transitons.

Walking-only trips

Trips starting from a border

Trips starting from a building

Live edits

When the player edits the map, there's an efficient process for applying the edits at the map model and rendering layer. In the middle of a simulation, it's less obvious how to apply all edits. Most of the time currently, edits cause the simulation to reset to midnight. Applying edits to the sim without reset is important for running machine learning experiments and for improving the gameplay experience (by having more immediate feedback about the consequences of a change).

The UI has a dirty_from_edits bit to track when changes have been applied without reset. This lets us tell the player that by the end of the day, any score / results are tentative, because their edits might have a different effect earlier in the day.

What works today

Changes to traffic signals are simple -- incremental_edit_traffic_signal happens at the map layer, and then handle_live_edited_traffic_signals at the sim layer just resets the current stage to 0 if the previous configuration had more stages.

TODO: Recalculating paths

Many of the edits will influence routes. For trips that haven't started yet, there's nothing to do immediately. Paths are calculated right before the trip starts, so slight changes to the start/end of the path due to map edits (like where somebody starts biking, for example) are captured naturally.

For currently active trips, in some cases, rerouting would be ideal but not necessary (like if speed limits changed). In other cases -- like changing access restrictions, modifying lane types, closing intersections -- the route must be recomputed. As a simple first attempt, we could just cancel all active trips whose path crosses an edited road or intersection. Later, we can figure out rerouting.

And actually, the only other case to handle is ChangeRouteSchedule, which should just be rescheduling the StartBus commands.

TODO: Parking

What happens if you modify a parking lane while there are cars on it? For now, just delete them. Trips later making use of them will just act as if the car never had room to be spawned at all and get cancelled or fallback to walking.

A better resolution would be to relocate them to other parking spots. If the owner is home, it'd be neat to have them walk outside, move the car, and go back in. But this greatly complicates the simulation -- the edited lane is in a transition state for a while, it modifies schedules, the person might not be around, etc.

Parking

TODO: Fill out the types of parking available, public/private, blackholes, how people pick spots, how seeding works, etc.

Infinite parking

If you pass --infinite_parking on the command line, every building gets unlimited public spots. This effectively removes the effects of parking from the model, since driving trips can always begin or end at their precise building (except for blackhole cases). This is useful if a particular map has poor parking data and you need to get comparative results about speeding up some trips. Often the A/B testing is extremely sensitive, because a parking space close to someone's destination is filled up quickly, slowing down the trip.

Project logistics

This has some background/logistics about the project.

Roadmap

A/B Street has been under active development since June 2018. That's a long time -- what work is happening now and how can you contribute?

Next steps, summer 2020

Afer the alpha launch in June, I plan to focus on:

  • shared biking/walking trails like the Burke Gilman
  • light rail
  • more score functions besides trip time, like safety/comfort
  • changing trip mode choice (if you make a bus route more desirable, switch some trips)
  • web support (so people can try out proposals without installing anything)

Ongoing work

If I had resources to hire a team, this is roughly how I'd organize different roles. If you're interested in helping, these aren't strictly defined positions, just ideas of related tasks.

UI and data visualization

We've got a UX designer, but implementing all of the new designs takes time. Also:

  • improve color schemes for colorblind players, implement night mode, rain effects, etc
  • refactor and clean up the GUI library for other Rust users
  • lots of data viz design / implementation needed

Game design

  • the tutorial mode needs attention
  • many ideas for challenge/story modes, but playtesting, tuning, and game design needed

Map data / GIS

Support more cities:

  • write docs/tools to help people add new cities without programming experience
  • add support for non-OpenStreetMap input: GeoJSON for parking in Perth, other trip demand sources, etc
  • fix bugs for driving on the left side of the road

Improve the quality of map geometry derived from OpenStreetMap:

  • try new algorithms to generate intersection polygons
  • make tools for easily improving relevant data in OSM
  • use ML and lidar/satellite data to get extremely accurate curb / planter / sidewalk geometry

Build tools and organize community mapping:

  • organize an effort to map how traffic signals are timed (partly started)
  • divide and track work for distributed mapathons

Bring in new data to understand more about cities:

  • PM2.5 pollution
  • Tax / land value (is there inequitable access to transit?)

Simulation / modeling

Totally new areas:

  • light rail
  • shared bike/pedestrian paths
  • ridesharing
  • micromobility (scooters, floating bikeshare)
  • more score functions (elevation gain, biking safety)
  • generating trip demand / activity models from scratch or modifying existing ones

Improve existing models:

  • overtaking / lane-changing
  • pedestrian crowds
  • instant vehicle acceleration
  • pedestrians walking on road shoulders (some streets have no sidewalks)
  • buses: transfers, proper schedules, multiple buses per route

Web

A/B Street runs on the web via WASM and WebGL; just waiting on vector text support. Besides that:

  • Share community proposals online, discuss them, vote, etc

Contributing for non-programmers

There's plenty to do besides programming!

  • Mapping, most of which directly contributes to OpenStreetMap:
  • Playtesting by attempting to implement real proposals would also be helpful, to expose where it's awkward for A/B Street to edit the map and to write up problems encountered.
  • Advocacy: I'm not great at finding the right people to to get ideas implemented for real. Maybe you are?

Long-term vision

Longer term, I'd like to take lots of the work in generating and interacting with high-detail OpenStreetMap-based maps and generalize it, possibly as a new OSM viewer/editor.

More generally, I'd like to see how simulation can help individuals understand and explore other policy decisions related to cities. Domains I'm vaguely interested in, but not at all knowledgable about, include land-use / zoning, housing, and supply chains. In late March 2020, a new collaborator started a pandemic model using the existing simulation of people occupying shared spaces. What are other domains could benefit from the rich agent-based model we're building?

Project motivations

I thought it'd be helpful to explain what motivates my work in A/B Street. These are just my personal values; I don't intend to make a careful argument about these here. In no particular order:

  • Transparency and reproducibility: if city government uses data, modeling, or simulation to inform a decision affecting the general public, then anybody ought to be able to repeat that analysis.

    • This means code and data should be open.
    • Businesses like Sidewalk Lab's Replica and Remix still need to generate income, but it's unclear why governments use taxes to pay for something only they see.
    • Decision making should be documented clearly. Why were the 35th Ave bike lanes scrapped? Was the amount of on-street parking on nearby residential roads factored in? Was there analysis of how trip time is impacted by parking in the neighborhood and walking a few blocks to a business on the arterial?
    • I'm personally inspired by approaches like vTaiwan and PDIS

  • Accessibility leads to participation: There's overhead to taking small ideas to advocacy groups or inconveniently timed public meetings. If the planning process is easier to interact with, more people will participate.

  • Short-term changes: ST3 is exciting, but 2040 isn't close. There are much cheaper changes that can be implemented sooner.

    • Most of the edits in A/B Street are inspired by tactical urbanism; they could be prototyped with signs and paint.

  • The US is too dependent on cars: This has an unacceptable impact on the environment. Even ignoring that, many cities are out of room to build more roads. We can't keep scaling population like this.

  • Autonomous vehicles will NOT save the day: They can squeeze more throughput out of existing infrastructure, but only up to a point. They might encourage people to move and tolerate longer commutes. Mass transit and dense land-use patterns handle population growth better.

  • Compromise and trade-offs: I see lots of rhetoric calling for extreme, sudden change. I don't want to ban all cars from downtown Seattle, because that's not realistic. I want to focus on immediate steps forward. I want to come up with estimates about impacting drivers by a median 3 minutes in order to save a bus route 1 minute, and to shift public discourse towards that.

Project history

As of June 2020.

tldr: A/B Street has been in active development since June 2018, but the idea has been festering since I was about 16.

Retrospective

What poor judgments have cost me the most time?

  • UI churn: I should've studied some UX on my own and started with a clear idea of how to organize everything
  • OSM data quality: I should've gained the confidence to upstream fixes earlier
  • Intersection geometry: I should've realized sooner that simulation robustness is more important than nice appearance.
  • Geometry primitives: I sunk too much time into the polyline problem and f64 precision.

Trivia

  • The name was almost "Unstreet" or "Superban" (superb urban)
  • I hope you enjoy and/or are baffled by the release names

Backstory

I originally wanted to tell a much longer story here of how I came to work on A/B Street, but I'm not sure this is the right time yet. So consider this the quick version.

I grew up in Baton Rouge, where driving is effectively the only mode of transport. (I've gone back and made a point of taking long walks to confirm how antagonistically the city is designed towards walking.) Very early on, I fell in love with a Nintendo 64 game called Banjo Kazooie, which led me to the online fan communities of the early 2000's. I wanted to create games too, so I started learning programming via library books and lots of questions on IRC. Because I never had any confidence in art, I wound up working on roguelikes, which led to a fervent interest in pathfinding algorithms and collaborative diffusion. When I started driving in high school, I quickly realized how bad people were at it. I remember being stuck at the intersection of Florida Blvd and Cloud and first wondering if the pathfinding algorithms could help with traffic. Can you see where this is going?

Impatience is a virtue

I moved to Austin for college. One of the first days of class, I shuffled down the stairs of Gearing Hall past a crackly old speaker apocalyptically announcing the weather forecast (details add color, right?) into a seminar demanding a totally open-ended first assignment to do something interesting. After I left, somebody stopped to ask me for directions, but I didn't know campus well yet. I thought about how Google Maps gave really silly walking directions. So I decided I'd hand-draw a map of campus, showing all of the construction, how to cut through the labryinth that is Welch Hall on hot days, and where to find the 24/7 robot coffee machines, and hack together a routing engine to help people find the shortest path between their classes. The feedback I got on this assignment included something along the lines of, "I was really pretty impressed first that you would be so stupid as to actually try to do this..."

Hand-mapping UT Austin

But I did, and that led me to discovering OpenStreetMap, which it turns out was pretty pivotal. (The first version of my campus map was seeded vaguely off an official paper map, but mostly I walked around and invented half-assed surveying methods on the spot.) Next semester, I joined a freshman research stream with somebody who had worked on AIM, UT's demonstration that autonomous vehicles wouldn't need traffic lights. Everything came together, and I started a 3 year journey of building AORTA, a traffic simulator for AVs. Guided by the research lab, I explored the really bizarre idea of letting AVs bid to turn lights green sooner and micro-tolling all roads to disincentivize congestion. Both of these mechanisms would be incredibly unfair to people without the spare cash to back up their high value-of-time, but I brushed this off by saying the currency could be based on carpooling, EVs, etc.

Approximately Orchestrated Routing and Transportation Analyzer

It was great to try research in college; I learned I really dislike munging data and compressing my work into 6 pages of conference paper LaTeX. So I moved to Seattle to work in industry instead, on something completely unrelated to transportation. Lots of things began unravelling for me in Seattle, but one of them was biking. In Austin, I had picked up mountain biking, and all but stopped driving; it was an amazing place to explore and commute by bike. Seattle was different. There were many more cyclists around, but the experience felt more stressful, the drivers more aggressive. I had plenty of near-misses. I kept commuting by bike, but the joy of it was gone. I started noticing how many cars were parked on narrow arterials and wondering why that was a fair use of space. I started paying attention to the public discourse around bike infrastructure in Seattle and feeling like the conversation was... chaotic.

Manhattan took walkability seriously

Fast forward to late 2017. This is where I'll omit chunks of the story. I visited London, my first experience with a city that took public transit seriously. When I returned, lots of latent ideas stopped fermenting and started exploding. I threw together a prototype of A/B Street and started the arduous process at work of open-sourcing it and applying to a program to let me work it on for a few quarters. A few months later, I wound up quitting instead, and began to work on A/B Street in earnest.

Year 1 (June 2018-2019)

I skimmed through git and summarized roughly what I was working on each month, calling out milestones. "UI churn" is pretty much constantly happening.

  • June: polyline geometry and lanes, building paths, protobuf -> serde

  • July: pedestrians, bikes, parked cars, lane edits

  • August: porting AORTA's discrete-time driving model

  • September: multi-leg trips, buses, the first ezgui wizard, randomized scenarios

  • October: A/B test mode (and so per-map plugins), forking RNG for edit-invariance, intersection geometry

  • November: clipping / borders, using blockface for parking, time travel mode, test runner framework

  • December: bezier curves for turns, traffic signal editor, a first attempt at merging intersections, right-click menus, a top menu, modal menus

    • the grand colorscheme refactor: a python script scraped cs.get_def calls at build-time

  • January: careful f64 resolution, ezgui screencapping, synthetic map editor

    • grand refactor: piston to glium

  • February: attempting to use time-space intervals for a new driving model, new discrete-event model instead

    • Feb 19-27: conceiving and cutting over to the new discrete event model

  • March: fleshing out DES model (laggy heads), first attempt to build on windows, gridlock detection

  • April: first public releases, splash screen and rearranging game modes

  • May: fancier agent rendering, attempting to use census tracts, finding real demand data

    • milestone: discovered PSRC Soundcast data, much more realistic trips

Year 2 (June 2019-2020)

Circa October 2019

  • June: contraction hierarchies for pathfinding, stackable game states

  • July: OSM turn restrictions, misc (I think I was in Europe?)

  • August: pedestrian crowds, agent color schemes, parking blackholes, a big raw_data refactor to store Pt2D, attended first hackathon

  • September: offstreet parking, associating parked cars with buildings using Soundcast (before that, anybody could use any car!), implemented texture support for some reason, doing manual MapFixes at scale to fix OSM bugs

    • milestone: got the smallest montlake map to run without gridlock

  • October: parking sim fixes, opportunistic lane-changing, starting challenge modes

  • November: prebaked sim results, time-series plots, undo for edit mode, traffic signal editor grouping turns

    • milestone: Yuwen joins project

  • December: the UI reform begins (flexbox, minimap, trip timelines, cutting over to SVGs, info panels, scrolling), started naming releases sensibly

    • Project leaked to HN, woops

  • January: UI reform continues, the modern tutorial mode appears

  • Feburary: UI and tutorial, all text now pure vectors, port to glow+WASM

  • March: lockdowns start in US, start grouping trips as a person, population heatmap, left-hand driving, info panel and typography overhauls. started engaging with Greenways, started effort to map traffic signals

  • April: Orestis joins and starts the pandemic model, trip tables, the optimize commute challenge, refactor for people's schedules and owned vehicles, trip time dat viz, MAJOR progress fixing gridlock at the sim layer

  • May: gridlock progress, upstreaming fixes in OSM, differential throughput and first real write-up, long-lasting player edits, dedicated parking mapper, maybe vanquished the HiDPI bugs, multi-step turn restrictions, random bios for people, and docs like this to prep for launch ;)

    • milestone: relying on pure OSM, no more MapFixes

Year 3 (June 2020-2021)

  • June: parking lots, real minimap controls, road labels
  • July: loads of bugfixes, map geometry improvements, UI cleanups, access-restricted zones for private neighborhoods and no-through-traffic, better traffic generation between home<->work for new maps, complete overhaul to bus routes and introduction of light rail, commute pattern explorer, importing Krakow and Berlin, smarter lane-changing, walkable shoulders for roads without sidewalks
  • August: Michael joins, multiple traffic signals can be edited together, started a headless JSON API, support for other languages in OSM data, started congestion capping, backwards-compatible and more robust map edits, two-way cycletracks, more cities imported, slurry of bugfixes and performance improvements
  • September: ...
  • October: ...

CHANGELOG

Every time I upload a new binary release, I'll list major changes here.

0.1.0

  • First binary release

0.1.1

  • drawing arrows better
  • start with a splash screen, make it easy to change maps in-game

0.1.2

  • totally revamp GUI by organizing everything into distinct gameplay modes

0.1.3

  • new warp tool that autocompletes street names
  • hideable menus, place context menus better, remove top menu bar, add a simple OSD
  • loading screens reflect what's printed to the terminal
  • depict pedestrians and bikes with more detail
  • tool to scroll through an agent's route
  • make simulation speed controls actually work

0.1.4

  • improve stop sign editor UI (toggle entire roads)
  • better mouseover / selection rendering
  • better traffic signal rendering (show time left, use outlines for yields)
  • make cars actually stop and briefly wait at stop signs
  • improve edit mode diff visualization (cross-hatching)
  • render actual stop signs, not just red lines
  • fix intersection policies confused by conflicting straight turns with lane-changing
  • fix mac scrolling
  • better turn indicators
  • nicer unzoomed view of roads, with different colors for big/small roads

0.1.5

(release file size jumped from ~15MB to ~70MB because of new PSRC trips)

  • improve UX of intersection editors
  • define a better set of maps included by default
  • improve drawing speed by batching more stuff
  • better default traffic signal policies for many cases
  • import and visualize census data
  • fix missing sidewalks on downtown one-ways
  • import and visualize PSRC trip data

0.1.6

  • slider widget for controlling time and speed
  • fixing bad polyline geometry in most cases; visualizing routes should no longer be buggy
  • handle PSRC trips that begin or end out-of-bounds
  • draw agents in unzoomed mode in a way simpler way
  • improve edit mode: detect reverts to original, easier lane type switching
  • lots of fixes for buses: handle edits better, read sequence of stops correctly from GTFS
  • set up A/B tests faster

0.1.7

  • bulk and revert tools in edit mode
  • improve turns and default intersection policies when bike/bus lanes involved
  • new tool to manually hint for short roads and weird intersections. some problems have now been manually fixed
  • scoreboard of trip results for sandbox and A/B test mode
  • reduce lag when sim is running at full speeds, but system is too slow
  • switch to easbar's contraction hierarchy crate, making all pathfinding INSANELY fast
  • remove weird rules about the world freezing when traffic signals are in "overtime"

0.1.8

  • edit mode: convert to a ped scramble cycle, simplify stop sign editor by removing individual turns
  • ui: put labels next to sliders, organize modal menus into sections, add a minimize/maximize icon
  • A/B test mode: savestate, include time controls and agent following/route tools here
  • use more OSM data for turn lanes, turn restrictions from lanes, turn restrictions between entire roads
  • dont attempt to cross a traffic signal if there's absolutely no hope
  • improve bus route UI tools and make routes using transit more sane
  • user-defined shortcuts for jumping between views of a map

0.1.9

  • sliders to pick times in wizards
  • fix hidpi scaling
  • traffic signal diagram scrolls properly
  • easier to instantiate a scenario, show all trips involving a building for a scenario
  • colorschemes to show trip duration or time blocked
  • label buses with route number
  • represent overlapping pedestrians as a labeled crowd
  • massive performance boost via real priority queue
  • prevent cars from "blocking the box"
  • prevent all? aborted trips (due to parking blackholes mostly)
  • smarter roam-around-for-parking router

0.1.10

  • sim
    • parking in off-street garages and on-street lanes on the off-side of oneways now mostly works
    • detect and handle parking blackholes; cars should never get stuck looking for parking now
    • let lower-priority turns happen at traffic signals when higher-priority ones blocked
    • get closer to FCFS ordering at stop signs
    • basic opportunistic lane-changing
    • a bus should be seeded for every route now
  • demand data
    • show trips to/from buildings and borders
    • make PSRC trips seed and attempt to use parked cars
  • UI
    • different heatmap overlays, like parking availability and busiest areas
    • show colorscheme legends when relevant
    • interactively seed parked cars, spawn more types of trips
    • fix major A/B test mode bug (mismatched scenarios and map edits)
    • adjusting sliders, menu placement, dynamic items
    • consolidating different tools into a single info panel for objects
    • bus route explorer shows entire route, current bus location
  • map quality
    • degenerate intersections only have one crosswalk now
    • revamped the map editor for fixing geometry problems, used it in many places
    • nicer yellow center lines (dashed when appropriate)
    • handling OSM turn restriction relations properly
    • fix empty traffic signal phases
    • handling bike lanes on certain sides of the road
    • starting to upstream manually-verified parking lanes into OSM
  • new gameplay: reverse direction of lanes

0.1.11

  • small UI fixes: fixed width traffic signal diagram, skip info phase of menus when empty
  • start drawing (but not using) shared left-turn lanes from OSM
  • fix OSM polylines with redundant points (fixing an issue in ballard)
  • improved traffic signal policies in some cases
  • started upstreaming some sidewalk tags in OSM to fix inference issues
  • fixed misclassified right turns
  • adjusting map colors
  • handling lakes/ocean polygons from OSM way better
  • reorganized sim analytics, added stuff for bus arrivals
  • adding new internal road points to map editor. almost ready to really aggressively use it
  • skipping parking lanes with no nearby sidewalks, since they're unusable
  • fix z-order of bridges/tunnels in unzoomed view
  • allow unzooming indefinitely
  • move lots of sandbox mode controls (and other modes) to menus under buttons and dedicated buttons
  • basic support for marking a lane closed for construction
  • improved geometry of sidewalks at dead-ends

0.1.12

  • reorganize everything as different challenge modes. start implementing 3: optimizing a bus route, speeding up all trips, or causing as much gridlock as possible
  • improved bus route explorer
  • some UI fixes (popup messages in a few places, moving mouse tooltips to the OSD)
  • lots of analytics and time-series plots

0.1.13

  • analytics: prebake baseline results properly. hover over plot series. some new modes to see bus network, throughput of a road/intersection over time
  • log scale for the speed slider
  • add a bulk spawner in freeform mode (F2 on a lane)
  • rendering: nicer routes, crosswalks, zoomed car colors
  • map data: better stop sign and sidewalk heuristics
  • fixed the mac hidpi text rendering issue once and for all?!

0.1.14

  • better crosswalk generation when there's only a sidewalk on one side of a road
  • edit mode UI revamp: paintbrush-style buttons to apply changes to lanes
  • show error messages and prevent edits, like disconnecting sidewalks
  • properly ban bikes from highways (revamped rules for vehicles using a lane)
  • new freeform mode tool to spawn bikes
  • WIP (not working yet): make bikes prefer bike lanes. some debug heatmaps for path cost
  • edit mode has proper undo support

0.1.15

  • minor bugfixes with reverting lane types, preserving stop signs
  • incorporate edits into the challenge splash screen, make sure edits are reset when appropriate
  • starting a new challenge mode, just focused on traffic signals
  • can't leave traffic signal editor with missing turns
  • render pedestrian crowds on building front paths
  • traffic signals support an offset parameter
  • traffic signal visualization and editing revamped to group related turns together
  • can preview traffic using a signal from the editor
  • actually apply priority at intersections, so protected turns get first dibs over yield turns

0.1.16

  • fix Mac crashing with texture limit bug by switching to texture arrays
  • fix crashing simulation when a border intersection was used
  • started to implement a new UI design for starting the game

0.1.17

  • more work on the pre-game UI, with some flexbox layouting
  • prototype a minimap in sandbox mode. doesn't pan or scroll yet.
  • prototype a new speed/time control panel from the mockup
  • nicer time warp loading screen
  • record and show detailed trip timeline, including time to park

0.1.18

  • map data: infer more building addresses
  • some analytics on how long people spend parking and intersection delay over time
  • create an options panel, allowing runtime customization of color scheme, traffic signal rendering, etc
  • internal changes to map building pipeline to make it much easier for new devs to onboard
  • organizing challenges into sub-stages, starting to flesh out specifics for the fix traffic signal track
  • much more realistic pedestrian pathfinding
  • fix minimap on mac (dpi issues)
  • visual tweaks to cars to make front/back easier to distinguish
  • internal change to switch most assets from PNG to SVG

0.1.19

  • some challenge modes show a histogram for counting faster/slower trips
  • new visualization of current demand per direction at a traffic signal
  • implementing some of Yuwen's UI changes: agent counter, split time/speed panel, moved functionality out of the old drop-down menus into a bottom-left tool panel, hiding debug functionality
  • replaced right-click context menus with left click to open info panels
  • fixed random issues reported by people from HN

0.1.20

  • moved some UI functionality around, pulling graphs into info panel
  • interactive legend for the minimap, toggle visibility of different agents
  • nicer colors and shapes for cars
  • misc simulation bugfixes that might help huge_seattle
  • pedestrians choose to use transit more realistically, factoring in time for the bus to drive

0.1.21

  • switch some analytics dashboards to use buttons, not old non-scrolling menus
  • scrollbars... at least a start
  • preview traffic signal changes from live sim as the base
  • traffic signal preview has normal time/speed controls
  • traffic signal editor has undo support
  • minimap has buttons to pan

0.1.22

  • minimap zoom controls
  • traffic signal rendering overhaul
  • heatmap colors improved, heatmap appears on minimap
  • bus info panel, a start to live delay analytics

0.1.23

  • UI revamps: speed panel, minimap controls, heatmap chooser
  • bus timeline
  • hide internal IDs normally
  • limit map zoom
  • fix bugs with crosswalks conflicting with vehicle turns

0.1.24

  • overhaul traffic signal editor UI, and add redo support
  • update main edit mode UI, and add redo support
  • limit max unzoom
  • fix the infamous HiDPI bug once and for all; minimaps should work everywhere
  • almost bug-free support for floating, horizontally and vertically scrolling panels
  • overhaul top-center panel, rename scenarios to be less confusing
  • expose bus analytics outside of challenge mode
  • live info panel can exist during a running simulation
  • consolidated agent route/trip information into info panel

0.1.25

  • overhauled the tutorial
  • tuned top-center panel for sandbox and challenge modes
  • make bike and bus lanes more obvious
  • show map edits as an overlay anywhere
  • tune info panel contents, and show relationships between parked cars and buildings
  • fixes to traffic signal editor, like making all-walk conversion idempotent
  • nicer throughput and delay plots (sliding windows, grid lines)

0.1.26

  • tutorial improved in a few places
  • map data: thinner sidewalks, associate buildings with named amenities
  • traffic model: vehicles can spawn on all lanes from a border
  • much better gameplay speed (previously was too fast)
  • UI tuning: lane editor, minimap, signal editor, heatmap legends don't overwrite minimap
  • traffic signal challenge communicates score more clearly

0.1.27

  • edit mode revamped: click to edit stuff. no more lane paintbrushes. autosaving and save as.
  • tutorial: can quit and resume tutorial now
  • challenge picking flow simplified
  • UI: layouting fixes to full-screen / into stuff, popup menus go beneath buttons, plots improved
  • internal change to render all text using vector graphics. other than a few text layouting issues, shouldn't be noticeable, except now tooltips in plots don't get covered up
  • misc perf improvements (cache SVGs, drawing many circles for unzoomed agents, dont reload prebaked data)
  • upgraded winit, glutin, glium -- hopefully no new bugs introduced on any platforms

0.1.27a

  • patch to fix a crash with empty text dimensions on things like building info panels

0.1.28

  • all info panels revamped
  • some tutorial stages are much more clear, with an updating goal
  • traffic signal scorecard generalized to work for some tutorial too
  • adjust how selected agents look
  • X button on popup menus

0.1.29

  • new tool to convert between stop signs and traffic signals
  • lane editor easier to edit multiple lanes
  • info panels: IDs, mostly avoid horizontal scrolling, better info about trips to/from somewhere, move buttons up
  • traffic signal editor UI overhaul
  • different data in top-right agent meters panel
  • tooltips to communicate keybindings better
  • new jump-to-time panel, showing when rush hours occur
  • speed controls use more useful speeds
  • include ongoing trips in measured trip times
  • jump to next challenge after completing one
  • lots of tutorial tweaks

0.1.30

  • show additional info about traffic patterns and buggy maps
  • revamp tutorial UI to group tasks and messages better
  • handle different mode transitions when info panel open on an agent
  • select entire roads in unzoomed edit mode
  • show total time an agent has spent moving / blocked
  • use 2-phase traffic signals by default, making the 23rd map successfully complete!
  • jump-to-time now optionally points out traffic jams forming
  • challenge splash screen improved

0.1.31

  • overhauled trip timeline in agent info panels
  • overhauled traffic signal details panel and the per-lane turn explorer
  • settings page: show all options at once. add way to scale up text/UI elements for high-DPI displays, and an alternate pan/zoom control scheme
  • traffic signal edits can now be exported and used in any slice of Seattle. will be using this to hand-map many of them.
  • many small tutorial fixes

0.1.32

  • some UI work on giving focus to textboxes, improving dropdown menus
  • road/intersection plots display baseline sim data too
  • start associating people with multiple trips, exposing this a little in the UI
  • bring back elevation data, introduce a new overlay. the elevation data is still really bad.

0.1.33

  • new "population" overlay, showing people (not just current trips). heatmap and dot map to visualize.
  • improved the "delay" overlay to handle roads and intersections
  • removed the confusing and useless alternate color schemes for agents
  • initial left-hand driving side, tested in Perth, also drawing more arrows for all one-way roads
  • loads of internal GUI code refactorings, preparing for a standalone release of the library
  • fixed z-buffering and alpha values for web backend

0.1.34

  • info panels have been totally overhauled again. multiple tabs, way more clear representation of agents, trips, and people.
  • draw people inside of a building
  • applied consistent typography everywhere
  • lots of internal refactoring

0.1.35

  • more info panel work, particularly for trips and buses. change plot settings live.
  • prototype of a SEIR pandemic model based on time spent in shared spaces, by orestis
  • slight heatmap improvements, more coming
  • more typography changes
  • mouse cursor now changes for buttons and dragging!
  • overhaul minimap controls, make layers behavior zoomed in a little better
  • new speed panel and jump-to-time modal

0.1.36

  • overhauled simulation data page, with a table to find slow trips and some initial summary visualizations
  • plots can change windowing and show/hide series
  • layers: fade map to contrast more, better scales/legends
  • show relative trip times in info panels
  • tools to rewind/ffw to watch particular trips
  • refocusing efforts on challenge modes; level 1 of a new one is pretty much ready
  • some simulation fixes around parking and a corner case of cars temporarily forming a cycle
  • orestis improved the population/pandemic heatmaps

0.1.37

  • optimize commute challenge: high score, live sentiment, second stage
  • parked cars are owned by people, not buildings
  • info panel improvements for trips
  • bike layer suggests places where bike lanes could be helpful
  • many improvements to scatter plot
  • a new histogram-ish thing for understanding faster/slower trips
  • handling scenarios longer than 24 hours better (for pandemic model)
  • prototype of commute visualization, grouping buildings by blocks
  • sim bugfixes: crosswalk / vehicle turn conflicts, start bikes in bike lanes from borders

0.1.38

  • major internal changes to ensure people's schedules don't have impossible gaps, to associate fixed bikes/cars to a eprson, handle delayed starts to trips
  • parking changes: show path to closest free spot, utilization of a lane over time, every building includes at least 1 offstreet spot by default
  • progress on removing unrealistic gridlock: detect turn conflict cycles and temporarily allow conflicts, trim last steps of a laggy head
  • internal sim alert system. speeds up debugging, could be used for player-facing "traffic jam!" alerts

0.1.39

  • switched to proper OSM-based maps; no more brittle, manual geometry fixes
  • more sorting and filtering options in trip table and parking overhead tables
  • improve offstreet parking rendering. park closer to destination buildings
  • easier process for importing new cities. introducing Los Angeles, Austin, Barranquilla.
  • new data updater tool so people can opt-in to new cities
  • many internal fixes to prevent gridlock. smarter cycle detection, manual OSM fixes and traffic signal timings

0.1.40

  • differential throughput layer to understand routing diversions
  • map edits now reference longer-lasting OSM IDs, can work cross-map
  • basemap updates: new areas for west seattle, mt baker, lots of upstreamed fixes in OSM and traffic signals, smarter border matching
  • parking: optionally filter on/off-street spots in the layer, allow disconnecting spots via edits
  • render some tunnels with lower opacity
  • new feature to change speed limits and bulk road selection tools
  • first write-up of a real use case (closing lake wash through arboretum)
  • make the traffic signal challenge act like a game, with a failure/win state and scoring

0.1.40a

  • added a mode to map parking

0.1.41

  • new parking mapper tool
  • include a one-shot .osm importer in the release
  • new layer to find different types of amenities / businesses
  • adjust traffic signal rendering style
  • bulk lane editor for changing speed limits and lane types
  • including west seattle and udistrict maps
  • include some OSM buildings that were being skipped
  • dont pause after opening something from sandbox mode
  • adjust turn signals for lane-changing cars
  • lots of fixes for monitors with different DPIs, enabled by default

0.1.42

  • many misc UI bugfixes, especially for high-DPI screens
  • managing turns across multiple nearby intersections: tool to visualize, handling multi-way OSM turn restrictions, using this to ban illegal movements at the pathfinding layer, starting a traffic signal editor variant to edit these
  • rendering improvements: unzoomed agent size, visualizing routes on trip table, transparent roads beneath bridges, draw harbor island
  • overhauled street/address finder
  • parking mapper: shortcut to open bing

0.1.43

  • new map picker!
  • UI polish: traffic signal editor, layers, bus stops, delay plots
  • generate more interesting biographies for people
  • tuned all the map boundaries
  • fleshing out lots of docs in preparation for the alpha release...

0.1.44

  • spawner UI revamped
  • model parking lots! and finally model public/private parking
  • fix up tutorial
  • starting a story map mode

0.1.45

  • overhauled challenge cutscenes and hints
  • traffic signal challenge: fix score detection, add meter, much faster startup, no reset-to-midnight required
  • layers: use gradient for a few, delay comparison, new UI for picker
  • overhauled minimap controls, should be intuitive now
  • edit mode changelist UI started

0.2.0 (alpha launch)

  • road names now shown by default, in a new style
  • all layers now use gradients and show up zoomed in. worst traffic jam layer revamped.
  • scatter and line plot improvements
  • internal UI fixes: proper word wrap
  • bugfixes for following people riding the bus
  • rainbow crosswalks in one neighborhood
  • final polishing for launch

0.2.1

  • busy week due to launch, but many new features in the pipeline
  • many bug fixes
  • edit mode: proper autosave, load proposals, jump between lane/intersection editors
  • very first steps on light rail... importing the tracks
  • starting a new traffic scenario modifier system, to repeat entire scenario or outright cancel trips for some people. many more ideas for filters and actions coming soon.
  • starting to represent private roads
  • add a very simple actuated traffic signal

0.2.2

  • the default traffic signal configuration is much smarter now, handling roads with some sidewalks missing and automatically synchronizing pairs of adjacent lights
  • much faster startup time for large maps
  • better UX for handling unsaved edits
  • access-restricted zones: changing existing zones almost completely works, except for granting new access to pedestrians
  • new sidewalk corner rendering, more rounded
  • ui style standardized for margins, padding
  • Javed got camera panning when your cursor is at the edge of the screen to work; enable it in settings
  • pulling bus stop/route info from OSM, not GTFS. steps towards light rail.
  • experimenting with controls for hiding bridges to see roads underneath; try them in dev mode (ctrl+S)
  • many bug fixes

0.2.3

  • lane geometry is dramatically fixed, especially for one-ways
  • importing lanes from OSM improved
  • UI: bulk select includes select-along-a-route, show all bus routes in the layer, unzoomed zordering for roads/intersections
  • traffic scenario modifier can now convert trip modes
  • slight progress on light rail, although the train only makes one stop
  • vehicles moving through complex intersections with multiple traffic signals will now make it through multiple lights, even if they're unsynchronized
  • new random traffic scenario generator that makes people go between houses and workplaces
  • access-restricted zones: granular editing of individual roads now mostly works
  • removing the hardcoded relative directories, which many people have been having problems with
  • many many bug fixes, and some optimizations to reduce release file size

0.2.4

  • bus/train routes overhauled; they're now one-way, regularly spawn every hour, and may begin or end at a border
  • new commute pattern explorer tool
  • new character art to give cutscenes a bit more personaliy
  • some progress on gridlocking maps, both from manual fixes and an attempt to reduce conflicts in multi-turn sequences
  • misc UI: show cars seeking parking in unzoomed mode, plot arrival rate at border intersections, consolidate bulk selection controls
  • trips modified by an experiment can now be filtered in summaries
  • buses, trains, and passengers on them are now properly distinguished in different stats
  • include krakow and berlin in release
  • buildings with holes in the middle are now rendered properly

0.2.5

  • cars pick lanes better
  • overhaul bus/stop/route info panels
  • UI: better autocomplete, commuter pattern improvements by Michael, toggles instead of checkboxes, contours for heatmaps, edit mode loader revamp
  • internal refactors: turn creation, osm tags, osm parsing
  • import living streets from OSM as restricted-access zones, and other importer tweaks for berlin, krakow, san jose, sydney

0.2.6

  • many roads without sidewalks now have a tiny shoulder lane, still enabling pedestrian movement, but with a penalty
  • bike trips will stop/start at a better position along the sidewalk now
  • support parking lanes on the off-side of a one-way
  • UI: search by building names, commuter patterns shows borders better
  • transit: make people ride off-map, spawn buses on short roads
  • internal cleanups for buttons

0.2.7

  • many intersections with on/off ramps have much better geometry
  • lane-changing banned on turn lanes
  • lots more work matching bus stops/routes to the map. some progress, also some regressions.
  • fixing spawning on tiny borders
  • bus spawn rates from GTFS for seattle. started an editor for the schedule.
  • internal ezgui refactorings

0.2.8

  • multiple traffic signals can now be synchronized and edited together
  • new dashboard for "traffic signal demand" over the entire day and map
  • started experimenting with controlling the headless runner via a JSON API
  • epic ezgui fix by Michael to consolidate handling of HiDPI scaling
  • got a bunch of huge cities importing and loading quickly
  • you can now save the trips you manually spawn in freeform mode, then replay them later

0.2.9

  • import Xi'an, add a Chinese font, and add a tool for that group to import their external demand data
  • control A/B Street through a graphics-less API, with a Python example
  • improve UI for per-direction traffic signal demand
  • on/off ramp geometry fixed in a few more cases
  • fix some missing parking lot aisles, handle parking lots with 0 spots, and extract parking garages from OSM
  • switch road/building language in settings, if OSM data exists
  • congestion capping prototype: declare a max number of vehicles that can pass through a zone per hour, view/edit it, and very simple implementation in the sim layer
  • add custom-drawn trips to the main scenario, for exploring new demand from a new building
  • mkirk fixed up the glow/wasm ezgui backends, letting us remove glium
  • make map edit JSON backwards compatible
  • better lane/turn markings

0.2.10

  • two-way cycletracks and arbitrary direction changes for roads
  • fix map editing for lane reversals, make edits backwards compatible, and massively speed up applying edits
  • fleshing out the headless API and tooling for controlling the simulation from any language
  • import a few more places, redo left-hand driving support so far
  • various bug/performance fixes

0.2.11

  • disabled support for editing the map without resetting the simulation. needs more work, but solid start.
  • improvements to API, activity model, congestion capping
  • small UI tweaks for parking, editing multiple signals
  • fixed last bugs for left-handed driving, should work just as well now
  • lots of graphics experiments from the hackathon, not merged yet

0.2.12

  • new textured color scheme and isometric buildings, in settings
  • new layer to show how far away people parked
  • Massive UI overhauls: jump to time/delay, edit mode, traffic signal editor (now with offsets), lane editor, bulk lane edit, traffic signal demand (individual intersections and all), loading screen
  • the Go API example compares trip times and detects gridlock
  • infinite parking mode
  • show how long a car has been parked in one spot
  • bugfix for some pathfinding costs around uber-turns
  • start to show a trip's purpose

0.2.13

  • alleyways from OSM imported
  • traffic signal minimum time now constrained by crosswalks; thanks Sam!
  • UI changes in progress for trip tables, summaries, bulk edit
  • more API / Python example work for congestion capping
  • bug fixes: isometric buildings, documentation links, dropdown widgets, turn restrictions

0.2.14

  • improve turn generation, with goldenfile tests
  • UI adjustments: unzoomed routes, better delay layer, include reasons for cancelled trips, throughput layer counts
  • small map importing fixes: multipolygon parking lots
  • fix infinite parking and blackholed buildings

0.2.15

  • large internal change allowing asynchronously loading extra files over HTTP for web
  • the release of the first web version!
  • cars looking for parking now have a "thought bubble" showing this, by Michael
  • slow sections of a trip are now shown in the info panel, by Sam
  • fix by Michael for handling window resizing in panels
  • fix original routes on edited maps
  • internal code organization and documentation

0.2.16

  • UI: click unzoomed agents, switch between metric/imperial units, show reason for cancelled trips, new "faded zoom" color scheme based on mapbox, more detailed agent counts in the top-right panel's tooltips
  • started a new dedicated OpenStreetMap viewer, will split out from A/B Street later
  • fix alpha colors on web
  • bugfixes for the new asynchronous map loading
  • some substantial simulation performance gains (168s to 90s on one benchmark!)
  • lots of progress towards editing the map without resetting the simulation to midnight. please test with --live_map_edits and report any issues
  • internal refactoring and code documentation

References

Example use cases

Groups that may be eventually interested

Similar projects

Seattle-specific

SDOT asking for feedback:

Seattlites with opinions and ideas:

Other projects

Notes from related work

SMARTS (https://people.eng.unimelb.edu.au/etanin/tist17.pdf)

  • Split map into sections, simulate in parallel, load-balance
  • has an IDM equation
  • tests against real TomTom data of average speed per link

Games

SimCity, Cities: Skylines https://steamcommunity.com/sharedfiles/filedetails/?id=583429740 https://github.com/fegennari/3DWorld

Open source urban planning

UrbanSim

Proprietary

Sidewalk Labs Model

Maps for people

https://arxiv.org/pdf/1811.01147.pdf

gamma.cs.unc.edu/RoadNetwork/wilkie_TVCG.pdf

section 6.3 talks about offset polylines http://gamma.cs.unc.edu/RoadNetwork

CityBound

https://github.com/aeickhoff/descartes

Discrete Event Simulation papers

  • section 5.1 of Advanced tutorial on microscopic discrete-event traffic simulation refers to some DES systems

    • Florian, Mahut, and Tremblay 2008
    • Sumaryo, Halim, and Ramli 2013
    • Salimifard and Ansari 2013
    • Burghout, Koutsopoulos, and Andreasson 2006
    • Thulasidasan, Kasiviswanathan, Eidenbenz, Galli, Mniszewski, and Romero 2009

  • A Dynamic Traffic Assignment Model for Highly Congested Urban Networks

    • section 2.2 models lanes as a moving and queueing part, references other possibly useful papers
    • dont worry about multiple lanes for the moving part, just the turn queues at the end

Tactical urbanism

Collaborations

We're working with a few different groups on projects using A/B Street.

GLT

Active as of Sept 2020

Green Lights Trading is applying dynamic congestion caps to encourage drivers to find alternate routes, not drive during rush hour, or take public transit instead. I'm actively implementing a proof-of-concept in A/B Street. It could also be used to explore ideas like a downtown congestion charging zone for Seattle.

Xi'an

Active as of Sept 2020

A group (with their own demand data!) is studying traffic signal optimization in Xi'an. They're using the API.

Forecasting group

Active as of Sept 2020

A research group is making a bunch of live map edits and scheduling new trips in the middle of a simulation and using their system to try to predict system-wide effects. They're also using the API.

CityEngine

Stalled as of Sept 2020

Edits in A/B Street could be exported to the shared-row format, then rendered in 3D using ArcGIS CityEngine. Not blocked on us.

Berlin

Stalled as of Sept 2020

A group wants to use A/B Street for various public engagement projects in Berlin. Mostly blocked on getting a reasonable travel demand model.

Manchester

Stalled as of Sept 2020

A group tentatively wants to engage the public about preventing rat runs and making no-through-access zones in Manchester suburbs. Not really blocked on us yet.

Pandemic model

Stalled as of Sept 2020

A group from the Uni of Geneva started a COVID-19 model that figures out how long people spend in shared indoor spaces.