sapling/eden/mononoke/tunables
Stanislau Hlebik 8716b24d75 mononoke: make gettreepack buffer size configurable and change default to 1000
Summary:
The goal of this diff is to make gettreepack requests that need to fetch a lot
of entries from blobstore faster.

Some of the gettreepack requests take really long time if they need to fetch,
say, 60K blobs. While this number of blobs is relatively high, it certainly
shouldn't take 1.5 mins to finish.

Upon closer inspection it looks like the reason for this slowness is a
combination of tokio coop mechanism and extremely large buffer size. Disabling
tokio coop and/or decreasing the buffer size makes requests significantly
faster (though I have to admin I didn't dig deeper to understand why they cause
this slowness).

To mitigate the issue I decided to decrease the buffer size first. This is a small
and predictable change that will be very easy to revert with tunable. Besides
having a buffer size of >1K doesn't make a lot of sense anyway, because we have
1K cachelib shards. Also this 10K buffer size was set a long time ago
(D8610582 (3dcf0048d8)), and there weren't a lot of measurements done to justify this size.
From the summary it seems that the problem was in slow manifold fetches from
remote regions, but as I said above, setting it highger than 1K is not helpful
anyway.

Reviewed By: krallin

Differential Revision: D28536686

fbshipit-source-id: 5f0ea04af8ce0c54c79d9668d95f2a1ece10f047
2021-05-19 07:30:06 -07:00
..
src mononoke: make gettreepack buffer size configurable and change default to 1000 2021-05-19 07:30:06 -07:00
tunables-derive rust: remove patch for async-compression 2021-05-05 12:20:34 -07:00
Cargo.toml rust: remove patch for async-compression 2021-05-05 12:20:34 -07:00