mirror of
https://github.com/facebook/sapling.git
synced 2024-10-06 23:07:18 +03:00
1c83391f6e
Summary: Re-implement the BfsIter using a thread pool. This fixes two important problems with the async stream version: 1. With the old BfsIter, each level of search would wait 1+1=2ms if there wasn't enough data to fill the chunks (see chunks_timeout). This meant using an ExactMatcher to bulk fetch a short list of paths would have unavoidable overhead of 20ms+, depending on the depth of the paths. 2. The old BfsIter stream pipeline was not being torn down correctly. It was leaking threads that were keeping references to the store alive, which in turn caused the TreeStore to not flush its caches since it never got dropped. I also think it is responsible for some of the file descriptor leaks. The new implementation seems to also be faster for big walks. I tested using mf.walk() with a large sparse profile matcher. The old BfsIter was ~27s, and the new is ~21s. Both take around 4s with caches hot. For the ExactMatcher case, in my repo w/ caches hot the old BfsIter took ~25ms and the new takes ~1ms. This will shave off a good chunk from the native status time, which hits the BfsIter for files not found in the dirstate. Reviewed By: quark-zju Differential Revision: D43622477 fbshipit-source-id: 5abde6025405d06170b31ed65ccef4990d9f5474 |
||
|---|---|---|
| .. | ||
| benches | ||
| src | ||
| Cargo.toml | ||