a-b-street/abstreet
1
1
Fork 0
mirror of https://github.com/a-b-street/abstreet.git synced 2024-12-26 07:52:05 +03:00
Code Issues Projects Releases Wiki Activity
83bc768e28
abstreet/cloud
History
Dustin Carlino 83bc768e28 Optimize costs of importing in the cloud. #326 
Every time I run an import, 10 GCE workers download about 20GB of
data/input. The S3 outbound charges are uncomfortably high.

Instead, use GCP's S3->GCS transfer tool manually before each run, and
make the GCE VMs read from that instead.

I haven't tested these changes yet, but will soon with the next import.
2021-05-27 08:09:03 -07:00
..
gather_output.sh Polish up the cloud importer, based on its first real use. #326 2021-05-19 22:17:14 -07:00
start_batch_import.sh Optimize costs of importing in the cloud. #326 2021-05-27 08:09:03 -07:00
worker_script.sh Optimize costs of importing in the cloud. #326 2021-05-27 08:09:03 -07:00