Stefan Filip 84017abe21 segmented_changelog: update OnDemandUpdateDag to have smaller critical sections ... Summary: The on demand update code we have is the most basic logic that we could have. The main problem is that it has long and redundant write locks. This change reduces the write lock strictly to the section that has to update the in memory IdDag. Updating the Dag has 3 phases: * loading the data that is required for the update; * updating the IdMap; * updating the IdDag; The Dag can function well for serving requests as long as the commits involved have been built so we want to have easy read access to both the IdMap and the IdDag. The IdMap is a very simple structure and because it's described as an Arc<dyn IdMap> we push the update locking logic to the storage. The IdDag is a complicated structure that we ask to update itself. Those functions take mutable references. Updating the storage of the iddag to hide the complexities of locking is more difficult. We deal with the IdDag directly by wrapping it in a RwLock. The RwLock allows for easy read access which we expect to be the predominant access pattern. Updates to the dag are not completely stable so racing updates can have conflicting results. In case of conflics one of the update processes would have to restart. It's easier to reason about the process if we just allow one "thread" to start an update process. The update process is locked by a sync mutex. The "threads" that fail the race to update are asked to wait until the ongoing update is complete. The waiters will poll on a shared future that tracks the ongoing dag update. After the update is complete the waiters will go back to checking if the data they have is available in the dag. It is possible that the dag is updated in between determining that the an update is needed and acquiring the ongoing_update lock. This is fine because the update building process checks the state of dag before the dag and updates only what is necessary if necessary. Reviewed By: krallin Differential Revision: D26508430 fbshipit-source-id: cd3bceed7e0ffb00aee64433816b5a23c0508d3c		2021-02-22 18:17:21 -08:00
..
src	load_limiter: static, sliced rate limitting	2021-02-15 09:56:56 -08:00
Cargo.toml	segmented_changelog: update OnDemandUpdateDag to have smaller critical sections	2021-02-22 18:17:21 -08:00