Commit Graph

305 Commits

Author SHA1 Message Date
Dustin Carlino
14927feeae Import part of SF 2021-08-06 10:54:46 -07:00
Dustin Carlino
c98b4f293e Regenerate all maps after last few changes. Split out the actdev scripts to more easily regenerate actdev scenarios. This is necessary if map intersection IDs change, for instance.
Rainier Valley starts crashing in the simulation! Not sure why the
regression happened, disabling for now, will look into it later.
2021-07-29 14:19:46 -07:00
Dustin Carlino
ffcf0a8d66 Fix a bug in transit snapping introduced by 163244ace9. Regenerating all maps, also bringing in a Tempe OSM fix...
The deadend trimming is too enthusiastic, getting rid of some unsnapped
cycleways and things connecting to the map border. Will iterate on it
this week; net benefit for now.
2021-07-27 16:56:11 -07:00
Dustin Carlino
7399f2aff2 Prettyprint durations with one decimal place, not two. Loading screens are a little too intense otherwise. This was changed in #699 because the variable name was wrong. 2021-07-25 10:53:57 -07:00
Dustin Carlino
0e472fcfcb For the Overpass import, include ways that're only partly intersecting the polygon. Without this, the map has no borders. #523, #717
I don't pretend to understand the Overpass query language, but I tried https://wiki.openstreetmap.org/wiki/Overpass_API/Advanced_examples#Completed_ways.2C_but_not_relations and it seems to work!
2021-07-25 10:42:57 -07:00
Dustin Carlino
f35538b460 Grab everything from Overpass, so importing from it looks more complete. #523
Because it's so much faster, make it the default!
2021-07-12 14:21:33 -07:00
Dustin Carlino
66be29d752 Import a new city using Overpass, instead of Geofabrik + clipping. #523
The returned XML seems to be missing lots of stuff, but it's much faster
and wasn't hard to wire up. I'll look into fixing the query...
2021-07-12 14:06:54 -07:00
Dustin Carlino
dd24f3ab89 Make the parallel map importer more useful by skipping places where it doesn't properly work 2021-07-10 17:14:10 -07:00
Dustin Carlino
1539091453 Import Loop 101, adding an option to skip local roads to cut down map
size.
2021-07-08 10:25:36 -07:00
Dustin Carlino
4aa971e34e Add a new importer mode to do Raw->Map in parallel. Occasionally useful
now that my faster machine works again.

Running importer jobs in parallel is actually super hard, because
there's async stuff thrown in there now to download files.
2021-07-06 16:21:42 -07:00
Dustin Carlino
d1a7ae4418 Start importing a piece of Sao Paulo for a collaboration.
No elevation data until the next big round of regeneration; having
trouble getting Docker set up on my resurrected machine.
2021-07-06 10:57:53 -07:00
Trevor Nederlof
3faf9893a9 Add downtown brooklyn as a nyc map 2021-07-01 12:27:58 -07:00
Dustin Carlino
137e9a0c2c Fix up some places affected by the f64 serialization:
- offstreet_parking_length in importer config
- backwards compatibility for map edits
- fixing up the baked-in proposals
- working around a few PolyLine bugs that happen with the new rounding
- Don't regenerate actdev scenarios yet -- the upstream JSON format is
  out of date, will have to fix separately.
2021-06-30 06:35:42 -07:00
Dustin Carlino
8546eb4b5f Make the timer hierarchy for --regen_all or any city with many maps more easy to understand 2021-06-25 15:29:28 -07:00
Dustin Carlino
5161474955 Stop building huge_seattle twice in the importer workflow
I have some fuzzy vision of declaring all importer pieces as a dependency graph, with optional caching of some input, that would express this kind of thing better. But... in the meantime.

Now regenerating everything...
2021-06-25 12:18:04 -07:00
Dustin Carlino
c0ce76a92a Two slight adjustments to speed up the map importer development
workflow.
2021-06-24 12:13:47 -07:00
Dustin Carlino
44171500b7 Get the Green Lake map to complete without gridlock!!!
- Allow blocking the box around two complex intersections in Green Lake.
  This makes the vehicle behavior much more realistic there, by visual
  inspection.
- Amp up offstreet parking to 10 per building. I noticed the simulation
  completes easily with --infinite_parking. This is an approximation of
  that. We make really bad guesses about carpooling and the amount of
  parking available around here, so effectively just remove it from
  consideration for now.
2021-06-23 13:56:42 -07:00
Dustin Carlino
678ed84e4c Fix a broken Dropbox link in the importer pipeline. The PSRC raw data is now just in S3. 2021-06-21 21:41:05 -07:00
Dustin Carlino
03db0675cc Add a Seattle map covering Green Lake 2021-06-20 10:51:30 -07:00
Dustin Carlino
8f199177f5 Upgrade to Rust 1.53
- arrays are now iterable directly
- switch to using BTree{Set,Map}::retain!
- a round of clippy
- regenerate scenarios and prebaked data; not sure why, but there's a
  diff
2021-06-17 23:04:35 -07:00
Dustin Carlino
484720d648 Add code to include pass-through trips (border->border) from Soundcast data. #665
... But leave it disabled, until we can handle such larger file sizes
and levels of traffic.

And although this should be a behavioral no-op, there's a diff in the
scenario files, so recalculate them.
2021-06-14 16:23:14 -07:00
Dustin Carlino
76aba75082 Update links to docs after some reorg in the other repo 2021-06-12 13:38:38 -07:00
Dustin Carlino
fc4f0a8c8e Some Tempe fixes:
- Stop importing rail in Tempe. Not simulating anything on it yet, and it complicates gridlock. #672
- Update the GMNS timing.csv import based on slight format change. #626
2021-06-09 16:35:46 -07:00
Dustin Carlino
60352747bf Get importer and game crates to share dependencies again, by enabling
some features for transitive deps in each crate.
2021-06-09 10:21:25 -07:00
Dustin Carlino
5127a82b54 Import Tehran 2021-06-01 10:19:06 -07:00
Dustin Carlino
5aa00955f9 Clean up the timer.parallelize API, getting rid of the weird enum to
toggle how many CPUs to thrash. We always use all of them, except for
one case, where a separately named method is more clear. Also make that
variation use all but 1 CPU, instead of just half.
2021-05-27 17:47:22 -07:00
Dustin Carlino
9fc79b1089 Use explicit start/endpoints for trips in scenarios. This is necessary
for people that leave one border, then re-enter a different one. #664

Alternative considered: insert a dummy remote trip between the two
borders. We used to do something like this at non-trivial code
complexity expense and having to explain the trip in the UI.

Regenerated all scenarios and prebaked data.

- Modest file size increase, as expected. Montlake scenario goes from
  1.3MB to 1.5, downtown from 37MB to 43MB, all Seattle scenarios from
  280MB to 327MB
- Eyeballed a few of the previously broken trips; they work now!
- Montlake goes from 3127 cancelled trips to just 302!
- No new gridlock, except in Rainier Valley -- disabling that for now
- I discovered missing validation in Poundbury for no-op trips between
  the same building. I'll filter those out and restore prebaked results
  there in a followup.
2021-05-26 14:30:01 -07:00
Dustin Carlino
f1aacb04e7 Import Chorlton for a demo 2021-05-24 10:13:09 -07:00
Vinzent Steinberg
a1899e29f8
Update dependencies (#656)
* Update dependencies

* Use github for polylabel, to make all dependencies use geo 0.18, not a mix of 0.18 and 0.17

* Downgrade lyon to avoid a crash

Co-authored-by: Dustin Carlino <dabreegster@gmail.com>
2021-05-22 16:23:06 -07:00
Dustin Carlino
bbc9adc3b7 Initially import Gilbert, AZ 2021-05-21 12:36:20 -07:00
Dustin Carlino
ef90721741 Fix missing study area polygons on the web. The refactor to Polygon::from_geojson_file introduced filesystem IO to the geom crate. Instead, we need to be explicit about how we do file IO.
Also fix some clippy errors
2021-05-20 14:33:04 -07:00
Dustin Carlino
e876998958 Add new actdev site, Cricklewood in London 2021-05-19 13:35:49 -07:00
Dustin Carlino
5fca901e4c Give up on Docker and AWS Batch to bulk import cities. Switch to static
sharding with GCE instead. #326
2021-05-18 09:50:28 -07:00
Vinzent Steinberg
9d17faa878 Prefer slices over reference to owned type 2021-05-15 14:24:08 -07:00
Dustin Carlino
1edbca6509
Refactor most of the places extracting polygons from geojson files (#644) 2021-05-15 09:05:28 -07:00
Vinzent Steinberg
446a21696d
Fix most clippy warnings (#646)
* More conventional spelling of acronym identifiers

* `new` -> `new_state`

* Remove redundant field name

* Remove needless `collect`

* `to_controls` -> `make_controls`

* Simplify long if statement

* Fix module inception

* Simplify chained if let

* Return directly instead of creating a binding

* Disable clippy warning about `borrow` method

Implementing the `Borrow` trait instead would result in excessive type
annotations.

* Fix most remaining clippy warnings

* Allow clippy::type_complexity

* Fix bad merge from web editor

* Run cargo fmt

* Suppress large_enum_variant warnings

* Rename FYI state to ShowMessage

* Fix upper_case_acronyms warnings

Co-authored-by: Dustin Carlino <dabreegster@gmail.com>
2021-05-14 17:31:32 -07:00
Vinzent Steinberg
fd3b0e2a14
Fix compilation failures and most clippy warnings (#642)
* abstutil: Fix compilation failure

* map_gui: Fix compilation

* traffic_signal_data: Fix compilation failure

* map_model: Fix compilation failure

* abstutil: Fix doctests

* abstio: Fix most clippy warnings

* abstutil: Fix most clippy warnings

* collisions: Fix clippy warning

* convert_osm: Fix clippy warnings

* sim: Fix most clippy warnings

* geom: Fix clippy warnings

* kml: Fix clippy warnings

* map_model: Fix most clippy warnings

* fifteen_min: Fix clippy warnings

* game: Fix many clippy warnings

* Disable some noisy clippy warnings

* headless: Fix clippy warnings

* importer: Fix clippy warnings

* map_editor: Fix clippy warnings

* map_gui: Fix clippy warnings

* osm_viewer: Fix clippy warnings

* parking_mapper: Fix most clippy warnings

* popdat: Fix clippy warnings

* santa: Fix clippy warnings

* sumo: Fix clippy warnings

* traffic_seitan: Fix clippy warning

* updater: Fix clippy warnings

* widgetry: Fix clippy warnings

* tests: Fix some clippy warnings

* Fix compilation on stable Rust

* Simplify unwrapping

* Make use of `Entry` more readable

* Fix formatting

* Fix code that was broken in the refactoring

* Apply cargo +stable fmt

* Fix code that was broken in the refactoring, second try

* Remove `Default` impls that are equivalent to `new`

* Remove obsolete clippy wrapper

* Avoid turbofish

* Prefer `unwrap_or_else` over allowing `clippy::or_fun_call`

* Remove redundant `into_iter`

* Fix typo

* Prefer `&& false` over commenting code out

* Fix some clippy warnings

Co-authored-by: Dustin Carlino <dabreegster@gmail.com>
2021-05-14 08:32:56 -07:00
Dustin Carlino
4192f74050 Import Auckland... even though the coastline is a bit missing 2021-05-11 11:55:54 -07:00
Dustin Carlino
5bac09bf2a Restore popdat.bin and Seattle scenarios and prebaked data. For whatever reason, the GDAL transformation is returning coordinates in a flipped order on my new machine.
Lakeslice prebaked disabled, because it starts gridlocking around 5pm.
I'll fix separately.
2021-05-11 09:34:40 -07:00
Dustin Carlino
5a86a7cef1 Import part of Maryland for a demo 2021-05-10 14:40:53 -07:00
Dustin Carlino
4d3234ea14 Plumb a flag through the importer to try to consolidate all intersections in a map 2021-05-08 11:34:16 -07:00
Dustin Carlino
4ac8c6e30b Tighten the GDAL flag guarding in the importer. This removes some
noise from 'cargo check' and is also a step towards running the importer
in a Docker container without having to rebuild it there. #326
2021-05-06 14:20:00 -07:00
Dustin Carlino
c6e4b6147f Detect when somebody accidentally copies multiple GeoJSON polygons into the importer. 2021-05-06 05:59:37 -07:00
Dustin Carlino
077fef92de Import any mode type from grid2demand 2021-05-06 05:51:07 -07:00
Dustin Carlino
eea136d2ca Import Frýdek-Místek 2021-04-30 12:03:09 -07:00
Dustin Carlino
f495444c42 Consolidate the actdev import script into one step, and import 8 new
sites
2021-04-27 13:13:14 -07:00
Trevor Nederlof
024ea50b72
Add Milwaukee (#625) 2021-04-26 18:54:42 -07:00
Dustin Carlino
599c7b6d6b Change the procgen house tool to also handle taking in a .osm file and doing the quick one-shot conversion. This'll simplify the actdev site import process. 2021-04-20 17:20:57 -07:00
Dustin Carlino
21bfb801d3 Import Hiroshima 2021-04-20 07:58:38 -07:00
Dustin Carlino
7852e1d2b8 Import scenarios from grid2demand's input_agents.csv directly from the UI! #603 [rebuild]
For reference, the input_agents.csv -> abst input is ported from
https://github.com/asu-trans-ai-lab/grid2demand/blob/main/src/demand_to_abst_scenario.py
2021-04-13 13:39:50 -07:00