Thomas Orozco bf3c2e19f0 mononoke/virtually_sharded_blobstore: a caching blobstore that deduplicates ... Summary: This introduces a caching blobstore that deduplicates reads and writes. The underlying motivation is to improve performance for processes that might find themsleves inadvertently reading the same data concurrently from a bunch of independent callsites (most of Mononoke), or writing the same bit of data over and over again. The latter is particularly useful for things like commit cloud backfilling in WWW, where some logger commits include the same blob being written hundreds or thousands of times, and cause us to overload the underlying Zippy shard in Manifold. This is however a problem we've also encountered in the past in e.g. the deleted files manifest and had to solve there. This blobstore is a little different in the sense that it solves that problem for all writers. This comes at the cost of writes being dropped if they're known to be redundant, which prevents updates through this blobstore. This is desirable for most of Mononoke, but not all (notably, for skiplist updates it's not great). For now, I'm going to add this behind an opt-in flag, and later on I'm planning to make it opt-out and turn it off there (I'm thinking to use the CoreContext for this). Reviewed By: farnz Differential Revision: D22285270 fbshipit-source-id: 4e3502ab2da52a3a0e0e471cd9bc4c10b84a3cc5		2020-07-03 05:53:10 -07:00
..
context	mononoke/virtually_sharded_blobstore: a caching blobstore that deduplicates	2020-07-03 05:53:10 -07:00
monitoring	autocargo: regenerate	2020-05-19 16:08:40 -07:00
repo_listener	push compat() down one level from main	2020-07-03 02:36:09 -07:00
session_id	remove the need to pass mapping to ::derive method	2020-02-12 10:22:39 -08:00
src	push compat() down one level from main	2020-07-03 02:36:09 -07:00
Cargo.toml	push compat() down one level from main	2020-07-03 02:36:09 -07:00