Commit Graph

4 Commits

Author SHA1 Message Date
Xavier Deguillard
16777c3d81 merge: remove the rustworkers option
Summary: It's now the default.

Reviewed By: DurhamG

Differential Revision: D21981563

fbshipit-source-id: 9e27e65ea8e47019f1542a505f5b7726c2cbeba2
2020-06-10 20:10:29 -07:00
Xavier Deguillard
1ece50fc62 tests: fix test-fb-hgext-remotefilelog-rust-lfs.t on Windows
Summary:
The IndexedLog code uses a lock file to lock a directory on Windows, make
sure we account for that.

Reviewed By: DurhamG

Differential Revision: D20818882

fbshipit-source-id: 7e9aa255354d36899ad57168311a4276d448dc07
2020-04-06 10:18:05 -07:00
Xavier Deguillard
5ea2e581c6 revisionstore: store LFS blobs in an IndexedLog
Summary:
In the legacy lfs extension, LFS blobs were stored as loosefiles on disk, and
as we saw with loosefiles for remotefilelog, they can incur a significant
overhead to maintain. Due to LFS blobs being large by definition, the number of
loose LFS blobs should be reasonable for repack to walk over all of them to
chose which one to throw away.

A different approach would be to simply store the blobs in an on-disk format
that allows automatic size management, and simple indexing. That format is an
IndexedLog. This of course doesn't come without drawbacks, the main one being
that the IndexedLog API mandate that the full blob is present on insertion,
preventing streaming writes to it, the solution is to simply chunk the blobs
before writing them to it. While proper streaming is not done just yet, the
storage format no longer prevent it from being implemented.

Reviewed By: DurhamG

Differential Revision: D20633783

fbshipit-source-id: 37a88331e747cf22511aa348da2d30edfa481a60
2020-03-30 14:45:46 -07:00
Xavier Deguillard
bb68ce52ff remotefilelog: provide fast path for cmp, size and isbinary
Summary: For LFS blobs, these can be obtained very easily by querying the ContentStore.

Reviewed By: DurhamG

Differential Revision: D20504235

fbshipit-source-id: 937ef20184d6524b1355565f9ab81e40b56d7ab0
2020-03-19 16:36:41 -07:00