sapling/eden
Stanislau Hlebik f3af697f30 mononoke: introduce BackgroundUnlessTooSlow session class and use it in blobstore
Summary:
Some time ago (see D25910464 (fca761e153)) we've started using Background session class
while deriving data. This was done to avoid overloading blobstore sync queue - if Background
session class is set then multiplex blobstore waits for all blobstores to
finish instead of writing to the blobstore sync queue right away. However if any of the
blobstores fails then we start writing to the blobstore sync queue. In theory it should have avoided the problem of overloading blobstore sync queue while having the same multiplex reliability (i.e. if only a single blobstore fails the whole multiplex put doesn't fail)

Unfortunately there was a flaw - if blobstore put of a single blobstore wasn't
failing but was just too slow, then the whole multiplexed put operation becomes
too slow. This diff fixes this flaw by adding a timeout - if multiplexed put is
taking too long then we fallback to writing entries to the blobstore sync
queue.

Note that I added a new session class - BackgroundUnlessTooSlow -
because I figured that in some cases we are ok with waiting a long time but not
writing to the sync queue. Skiplist builder might be a good example of that -
since it's doing overwrites then we don't want to write to the blobstore sync
queue at all, because healer doesn't process overwrites correctly.

Reviewed By: farnz

Differential Revision: D30892377

fbshipit-source-id: 69ac1795002b124e11daac13d8bfe59895191168
2021-09-13 08:09:12 -07:00
..
fs pretty print rage sections and normalize underlining 2021-09-10 23:54:48 -07:00
hg-server third-party/rust: bump tracing versions in preparation for patching 2021-09-09 22:38:25 -07:00
integration pretty print rage sections and normalize underlining 2021-09-10 23:54:48 -07:00
locale
mononoke mononoke: introduce BackgroundUnlessTooSlow session class and use it in blobstore 2021-09-13 08:09:12 -07:00
scm HGPLAIN version of createremote 2021-09-13 06:15:04 -07:00
test_support test_support: canonicalize the temporary directory path 2021-08-16 16:08:45 -07:00
test-data fix fsck snapshot integration tests 2021-07-14 16:20:04 -07:00
.gitignore
Eden.project.toml