sapling/eden
Xavier Deguillard 5ea2e581c6 revisionstore: store LFS blobs in an IndexedLog
Summary:
In the legacy lfs extension, LFS blobs were stored as loosefiles on disk, and
as we saw with loosefiles for remotefilelog, they can incur a significant
overhead to maintain. Due to LFS blobs being large by definition, the number of
loose LFS blobs should be reasonable for repack to walk over all of them to
chose which one to throw away.

A different approach would be to simply store the blobs in an on-disk format
that allows automatic size management, and simple indexing. That format is an
IndexedLog. This of course doesn't come without drawbacks, the main one being
that the IndexedLog API mandate that the full blob is present on insertion,
preventing streaming writes to it, the solution is to simply chunk the blobs
before writing them to it. While proper streaming is not done just yet, the
storage format no longer prevent it from being implemented.

Reviewed By: DurhamG

Differential Revision: D20633783

fbshipit-source-id: 37a88331e747cf22511aa348da2d30edfa481a60
2020-03-30 14:45:46 -07:00
..
benchmarks Require callers to pass NetworkSocket to TAsyncSocket 2020-03-11 23:45:16 -07:00
docs documentation for paths 2020-01-17 12:08:42 -08:00
fs use TEnumTraits 2020-03-27 15:27:02 -07:00
integration exit if the EdenFS lock file ever becomes invalid 2020-03-26 20:17:12 -07:00
locale add a copyright header to glibc_en.po 2019-04-26 14:38:27 -07:00
mononoke resolver.rs: make Bundle2Resolver contain refs to ctx and repo 2020-03-30 12:21:09 -07:00
scm revisionstore: store LFS blobs in an IndexedLog 2020-03-30 14:45:46 -07:00
scripts Tidy up license headers 2019-10-11 05:28:23 -07:00
test_support eden: remove hypothesis test bits 2020-01-23 11:37:27 -08:00
test-data enable treemanifest in snapshots 2019-08-28 18:46:03 -07:00
third-party fix some macOS build warnings 2020-01-23 17:37:39 -08:00
win flip EDEN_WIN_NOMONONOKE to EDEN_HAVE_MONONOKE 2019-08-27 17:04:19 -07:00
.gitignore eden: wire up mac contbuild 2019-02-05 21:52:30 -08:00
Eden.project.toml Eden.project.toml file for Nuclide 2018-04-26 11:05:23 -07:00