Commit Graph

64 Commits

Author SHA1 Message Date
Dustin Carlino
007cf0b28d Split the importer into individual commands with structopt. #745 2021-10-25 13:02:00 -07:00
Dustin Carlino
c28dba4b47 Make the importer itself a library, called from the CLI tool. #745 2021-09-17 18:29:23 -07:00
Dustin Carlino
c0ce76a92a Two slight adjustments to speed up the map importer development
workflow.
2021-06-24 12:13:47 -07:00
Dustin Carlino
f78bb8b711 Attempt to display the Timer progress and logs in separate parts of the screen when running the importer. #262
Lots of problems, so disabled by default.
2021-01-21 19:57:38 -08:00
Michael Kirk
6482f8b0a4
util to convert geojson to osmosis polygon (#309) 2020-09-02 15:29:25 -07:00
Dustin Carlino
6919c24bf8 detect the data/ dir more intelligently. fixes #73. still need to
- consider collapsing the directory structure in the releases
- fix wasm

[rebuild]
2020-07-07 11:09:35 -07:00
Dustin Carlino
e082082c32 try again to bundle importer in release for oneshot. disable gdal.
[rebuild]
2020-05-17 09:25:44 -07:00
Dustin Carlino
bf8c35788b moving popdat+precompute into the new unified importer 2020-03-26 10:26:01 -07:00
Dustin Carlino
0133685eb7 converted the import.sh shell script into Rust. towards #27
as part of this, wound up grabbing fresh OSM and GTFS. but actually cant
reimport OSM right now, because fixes reference old OSM IDs and things
break. so sticking with old OSM, but new GTFS. urgh.
2020-03-25 19:54:33 -07:00
Dustin Carlino
1cce49648b some fixes to get Perth to import, part of #27
- filter out indoor corridors
- plumb through a flag for driving on the right/left. don't use it yet.
2020-03-16 15:20:35 -07:00
Dustin Carlino
6e8846bf85 bring back elevation data before the hackathon. manual revert of 35b7c18df9 2020-03-11 11:08:49 -07:00
Dustin Carlino
558a442b79 fix small bug in import script, and skip over bad polylines in
blockface. fixes #4
2019-12-03 19:28:58 -08:00
Dustin Carlino
f6471cf5e4 finalize seeded data script. fixes #3, now the initial dev experience is much easier 2019-12-03 16:36:30 -08:00
Dustin Carlino
d505c3e8a8 make curl follow redirects 2019-12-03 12:29:57 -08:00
Dustin Carlino
c3348957c1 group data/input/osm 2019-12-03 12:25:33 -08:00
Dustin Carlino
db8bc73ddd finally grouping system data too. forgot some data/input things... 2019-12-03 12:22:59 -08:00
Dustin Carlino
2a103d0a21 also group raw_maps in data/input 2019-12-03 12:06:49 -08:00
Dustin Carlino
0e99f40150 merge data/shapes and data/input. stop keeping around raw .kmls. adjust
gitignore to make it easier to spot defunct files in my repo
2019-12-03 11:55:54 -08:00
Dustin Carlino
72c68aa320 remove old census popdat stuff 2019-12-03 11:42:02 -08:00
Dustin Carlino
69d6aca0ef switch from wget to curl, since wget isnt installed by default on mac.
improve developer instructions a bit. don't crash if scrot doesnt exist.
This partly helps with #3, but more work still needed
2019-11-25 11:21:43 -08:00
Dustin Carlino
e881d13014 find all the short lanes, from map_editor 2019-10-26 15:01:27 -07:00
Dustin Carlino
06ebf3cb2c switch from osmosis to osmconvert in import scripts; it's SO much faster 2019-10-26 09:52:36 -07:00
Dustin Carlino
2313b06995 DISABLED: apply sidewalk hints, same as parking. need to sort out what
should happen with parking without sidewalks
2019-10-23 13:00:42 -07:00
Dustin Carlino
9cab5bf903 prep before bringing in king county sidewalks. clean up lane type
calculation, move motorway/parking filter earlier. stop bringing in
street signs.
2019-10-23 09:35:41 -07:00
Dustin Carlino
80106eb3d5 script psrc import 2019-09-28 10:44:46 -07:00
Dustin Carlino
83834f7d40 actually, deferring remove_disconnected has problems.. still need to remove cul-de-sacs early. also, remove the two synthetic maps... will make them again from scratch once the raw_data format is stable. unused in the meantime. 2019-09-17 10:47:47 -07:00
Dustin Carlino
43c2c72e71 importing street sign dataset to override bad parking categories from blockface. the signs look overeager... so disabling 2019-09-08 17:23:53 -07:00
Dustin Carlino
114a76893a now raw_data::Road 2019-08-26 13:59:10 -07:00
Dustin Carlino
1936f5f75e match offstreet parking KML to buildings 2019-08-25 14:00:25 -07:00
Dustin Carlino
8aee410046 download offstreet parking dataset, tweak KML debug rendering 2019-08-24 19:01:37 -07:00
Dustin Carlino
96db77e8b2 stop importing traffic signal data from king county. OSM's data is higher-quality and already attached to specific nodes. 2019-08-15 14:19:11 -07:00
Dustin Carlino
142c34af4e interpret lanes:forward and lanes:backward from OSM 2019-07-02 14:00:54 -05:00
Dustin Carlino
7a49c585cf remove building_type and stop reading the residential permit shapefile 2019-06-18 14:58:57 -07:00
Dustin Carlino
35b7c18df9 remove elevation from map model. never been used, no anticipated need for it 2019-06-12 07:25:31 -07:00
Dustin Carlino
4e2d16af29 consistently using file extensions of .json and .bin, merging duplicate abstutil io functions accordingly 2019-06-11 14:30:11 -07:00
Dustin Carlino
3663c19cdc starting a new crate to import extra population data 2019-05-20 07:48:30 -07:00
Dustin Carlino
366267fcbf try clipping areas a different way. disabled, but close to working well. 2019-04-13 13:25:00 -07:00
Dustin Carlino
d207c3c33a rip out parcels entirely; unused for long time and no anticipated uses 2019-04-11 11:40:37 -07:00
Dustin Carlino
e68d6225c8 clipping roads based on original boundary polygons. buggy, so disabled. 2019-02-11 22:13:49 -08:00
Dustin Carlino
0a3f2676f0 more timer measurements for convert_osm, and make sure to erase
precomputed maps when re-converting a map
2019-02-11 14:22:05 -08:00
Dustin Carlino
8693bbc1e3 import residential building dataset 2019-02-10 19:24:34 -08:00
Dustin Carlino
8eaf09034f handle mulitpolygons with incomplete clipped info 2019-02-09 21:48:21 -08:00
Dustin Carlino
dfe50dcee4 reasonable support for roundabouts... collapse it to a single
intersection, which usually gets treated like a stop sign, and has
reasonable geometry
2019-01-16 15:05:44 -08:00
Dustin Carlino
d95099efbe basic installation instructions, and tweaks to scripts so that things
work better out-of-box
2019-01-11 10:04:56 -08:00
Dustin Carlino
149758d5ae use traffic signal data from new kml, not old shp that's no longer online 2018-11-27 13:10:51 -08:00
Dustin Carlino
5bc9992fc0 using blockface kml in convert_osm to prescribe parking lanes 2018-11-21 22:43:14 -08:00
Dustin Carlino
117adb7f1a consolidating all of the kml parsing code 2018-11-21 13:20:41 -08:00
Dustin Carlino
eabef19ed4 clipping OSM ways in osmosis differently, and hacking around a traffic
signal at a border node
2018-11-08 11:25:32 -08:00
Dustin Carlino
4a1d2c8bb6 making import.sh more magic 2018-10-30 15:07:22 -07:00
Dustin Carlino
23a1adfd95 using timer more thoroughly in convert_osm 2018-10-30 14:55:55 -07:00