Thomas Orozco 1ee93bdcfb mononoke/rendezvous: use in-flight connection count to decide when to batch ... Summary: After doing some local benchmarking (using MononokeApi instantiation as the benchmark), one thing that's apparent is that we have quite a few parameters here and that tuning them is likely to be a challenge. One parameter in particular is the batch "objective", which controls how many requests we want to see in the last batching interval before we choose to batch (this is `rendezvous_dispatch_min_threshold`). The problem with this is this is that there is no good, real-world, metric to set it based on. This in contrast to the other parameters we have, which do have some reasonable metric to compare to: - rendezvous_dispatch_delay_ms: this is overhead we add to queries, so it should be small & on the order of query execution latency (i.e. a few ms). - rendezvous_dispatch_max_threshold: this controls how big our batches get, so it should be on the order of what makes a SQL query too big (i.e. less than a hundred records). In contrast, we want to set `rendezvous_dispatch_min_threshold` such that batching kicks in before we start using too many concurrent connections (which is what query batching seeks to reduce), but the problem is that those two numbers aren't directly connected. One clear problem, for example, is that if our DB is in-region vs. out of-region, then for a given query execution time, and a desired concurrency level before batching kicks in, we'd need different values of `rendezvous_dispatch_min_threshold` (it would have to kick in faster for the out-of-region workload). So, this diff updates rendez vou to actually track concurrent connection count before we force batching. This is the actual metric we care about here, and it has a pretty natural "real world" values we can look at to decide where to set it (our connection pool — which is limited at 100 concurrent connections —, and our open connection baseline). Note: I set this at 5 because that's more or less what servers look like outside of spikes for Bonsai hg mapping, and of Changesets where I'm planning to introduce this in the future: - bonsai: https://fburl.com/ods/6d4a9qb5 - changesets: https://fburl.com/ods/kuq5x1vw (note: to make sense of this, focus on just one server, otherwise the constnat spikes we get sort of hide the big picture). Reviewed By: farnz Differential Revision: D27792603 fbshipit-source-id: 1a9189f6b50d48444b3373bd1cb14dc51b85a6d2		2021-04-16 10:27:44 -07:00
..
src	mononoke/rendezvous: use in-flight connection count to decide when to batch	2021-04-16 10:27:44 -07:00
tunables-derive	autocargo v1: changes to match autocargo v2 generation results.	2021-02-19 11:03:55 -08:00
Cargo.toml	third-party/rust: update futures	2021-03-04 06:42:55 -08:00