sapling/eden/scm/lib/revisionstore
Xavier Deguillard 5ea2e581c6 revisionstore: store LFS blobs in an IndexedLog
Summary:
In the legacy lfs extension, LFS blobs were stored as loosefiles on disk, and
as we saw with loosefiles for remotefilelog, they can incur a significant
overhead to maintain. Due to LFS blobs being large by definition, the number of
loose LFS blobs should be reasonable for repack to walk over all of them to
chose which one to throw away.

A different approach would be to simply store the blobs in an on-disk format
that allows automatic size management, and simple indexing. That format is an
IndexedLog. This of course doesn't come without drawbacks, the main one being
that the IndexedLog API mandate that the full blob is present on insertion,
preventing streaming writes to it, the solution is to simply chunk the blobs
before writing them to it. While proper streaming is not done just yet, the
storage format no longer prevent it from being implemented.

Reviewed By: DurhamG

Differential Revision: D20633783

fbshipit-source-id: 37a88331e747cf22511aa348da2d30edfa481a60
2020-03-30 14:45:46 -07:00
..
src revisionstore: store LFS blobs in an IndexedLog 2020-03-30 14:45:46 -07:00
Cargo.toml revisionstore: feature gate the Mononoke LFS tests 2020-03-30 08:40:43 -07:00
CMakeLists.txt Merge fb-mercurial sources into the eden repository 2019-11-13 20:20:32 -08:00
RevisionStore.cpp Merge fb-mercurial sources into the eden repository 2019-11-13 20:20:32 -08:00
RevisionStore.h Move fb-mercurial sources into an eden/scm subdirectory. 2019-11-13 16:04:48 -08:00