sapling/eden/fs/store/hg
Matt Glazar 3b9a0310a1 Fix deadlock when restarting during RocksDbLocalStore::get()
Summary:
If TreeInode::startLoadingInode() is in progress, and EdenServer::startTakeoverShutdown() is called, edenfs can deadlock:

1. Thread A: A FUSE request calls TreeInode::readdir() -> TreeInode::prefetch() -> TreeInode::startLoadingInode() on the children TreeInode-s -> RocksDbLocalStore::getFuture().
2. Thread B: A takeover request calls EdenServer::performTakeoverShutdown() -> InodeMap::shutdown().
3. Thread C: RocksDbLocalStore::getFuture() (called in step 1) completes -> TreeInode::inodeLoadComplete(). (The inodeLoadComplete continuation was registered by TreeInode::registerInodeLoadComplete().)
4. Thread C: After TreeInode::inodeLoadComplete() returns, the TreeInode's InodePtr is destructed, dropping the reference count to 0.
5. Thread C: InodeMap::onInodeUnreferenced() -> InodeMap::shutdownComplete() -> EdenMount::shutdown() (called in step 2) completes -> EdenServer::performTakeoverShutdown().
6. Thread C: EdenServer::performTakeoverShutdown() -> localStore_.reset() -> RocksDbLocalStore::~RocksDbLocalStore().
7. Thread C: RocksDbLocalStore::~RocksDbLocalStore() signals the thread pool to exit and waits for the pool's threads to exit. Because thread C is one of the threads managed by RocksDbLocalStore's thread pool, the signal is never handled and RocksDbLocalStore::~RocksDbLocalStore() never finishes.

Fix this deadlock by executing EdenServer::shutdown()'s callback (in EdenServer::performTakeoverShutdown()) on a different thread.

Reviewed By: simpkins

Differential Revision: D14337058

fbshipit-source-id: 1d63b4e7d8f5103a2dde31e329150bf763be3db7
2019-03-12 19:29:35 -07:00
..
test pass ReloadableConfig down to HgBackingStore ctor 2018-10-31 11:50:38 -07:00
CMakeLists.txt Use MononokeCurlBackingStore in HgBackingStore 2019-03-11 14:34:09 -07:00
hg_import_helper.py Fix hg_import_helper import path 2019-01-30 14:49:01 -08:00
HgBackingStore.cpp Use MononokeCurlBackingStore in HgBackingStore 2019-03-11 14:34:09 -07:00
HgBackingStore.h Use MononokeCurlBackingStore in HgBackingStore 2019-03-11 14:34:09 -07:00
HgImporter.cpp improve error messages if the hg import helper fails to start 2019-02-27 13:46:36 -08:00
HgImporter.h Change the Windows pipe functions to not return until all the bytes are read/written or error. 2018-11-26 08:19:47 -08:00
HgImportPyError.cpp include the exception type name in hg_import_helper.py errors 2017-11-15 13:31:58 -08:00
HgImportPyError.h include the exception type name in hg_import_helper.py errors 2017-11-15 13:31:58 -08:00
HgManifestImporter.cpp move folly/experimental/logging to folly/logging/ 2018-04-30 21:29:29 -07:00
HgManifestImporter.h split RocksDbLocalStore out from LocalStore 2018-02-07 11:54:16 -08:00
HgProxyHash.cpp Future<T>::then Future<T>::then() -> Future<T>::thenValue or thenTry. 2018-09-14 17:10:57 -07:00
HgProxyHash.h move HgProxyHash to its own top-level file 2018-06-14 22:02:38 -07:00
proxy_import_helper.py eden: hg_import_helper.py execs hg debugedenimporthelper 2019-02-14 22:27:00 -08:00
tester.cpp Fix deadlock when restarting during RocksDbLocalStore::get() 2019-03-12 19:29:35 -07:00