sapling/eden/mononoke/repo_client
Kostia Balytskyi cd9479ae54 repo_client: sample small gettreepack logging
Summary:
`gettreepack` accounts for ~6B logged scuba rows a day (https://fburl.com/scuba/mononoke_test_perf/vpnsn1ny) out of ~10B totally logged rows (https://fburl.com/scuba/mononoke_test_perf/qw78ecxe), so 60% of rows. For the vast majority of `gettreepack` instances we log 3 log tags: "Start processing", "Gettreepack params" and "Command processed". Similarly, the vast majority requests just 1 mfnode: https://fburl.com/scuba/mononoke_test_perf/3xwotsgq. If we sample logging for these commands by a factor of 100, we'll be able to save almost all of these 60% of rows (it's not entirely clear how that will actually influence our retention, but likely pretty significantly).

What do we lose if we do this sampling?
There are a few perf counters, like GettreepackResponseSize, GettreepackNumTreepacks, GettreepackDirectories, GettreepackDesignatedNodes, that will lose their aggregation accuracy. Given that we're only sampling single-mfnode gettreepacks, these values are not likely to be very interesting. However, we are still leaving a possibility to turn verbose logging back on and get full amount of logging.

Reviewed By: mitrandir77, krallin

Differential Revision: D26148453

fbshipit-source-id: a8521364bb5323d41c6c0c7d82d50508c0eda068
2021-02-04 13:51:26 -08:00
..
getbundle_response mononoke: getbundle optimization for many heads with low gen number 2021-02-03 03:55:46 -08:00
mononoke_repo mononoke/repo_client: use commit sync config from Mononoke API 2021-02-04 10:40:02 -08:00
obsolete Add version specificiation to internal dependencies 2021-01-25 22:10:24 -08:00
remotefilelog Add version specificiation to internal dependencies 2021-01-25 22:10:24 -08:00
repo_read_write_status Add version specificiation to internal dependencies 2021-01-25 22:10:24 -08:00
reverse_filler_queue Add version specificiation to internal dependencies 2021-01-25 22:10:24 -08:00
schemas reversefillerqueue: change bundle from varchar to varbinary 2020-05-20 07:34:22 -07:00
scribe_commit_queue Add version specificiation to internal dependencies 2021-01-25 22:10:24 -08:00
src repo_client: sample small gettreepack logging 2021-02-04 13:51:26 -08:00
streaming_clone Add version specificiation to internal dependencies 2021-01-25 22:10:24 -08:00
unbundle mononoke/repo_client: use commit sync config from Mononoke API 2021-02-04 10:40:02 -08:00
wirepack Add version specificiation to internal dependencies 2021-01-25 22:10:24 -08:00
Cargo.toml repo_client: sample small gettreepack logging 2021-02-04 13:51:26 -08:00