sapling/remotefilelog
Durham Goode 7227563c61 repack: add incremental repack
Summary:
This adds a --incremental flag to the hg repack command. This flag causes repack
to look at the distribution of pack files in the repo and performs the most
minimal repack to keep the repo in good shape. Currently it's only implemented
for datapacks.

The new remotefilelog.datagenerations config contains a list of the sizes for
the different generations of pack files. For instance:

  [remotefilelog]
  datagenerations=1GB
    100MB
    1MB

Designates 4 generations - packs over 1GB, packs over 100MB, packs over 1MB, and
implicitly packs undex 1MB. The incremental algorithm will try to keep each
generation to less than 3 pack files (prioritizing the larger generations
first). When performing a repack it will grab at least 2 packs, and will grab
more if the total pack size is less than 100MB (since repacking at that level is
pretty cheap).

I have no idea if this is a good algorithm. We'll how to see and iterate.

Test Plan: Adds a test

Reviewers: mitrandir, #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D3306523

Signature: t1:3306523:1463697129:c87f4a397ef357b5ca4a80d01e9a6ca4d61f9d3d
2016-05-20 09:31:25 -07:00
..
__init__.py repack: add incremental repack 2016-05-20 09:31:25 -07:00
basestore.py windows: chmod file to be writable before deleting 2016-05-20 08:37:49 -07:00
cacheclient.py checkcode: fix errors in cacheclient and debugcommands 2016-04-26 13:00:31 -07:00
constants.py store: basic wire protocol for bundle delivery 2016-05-16 10:59:09 -07:00
contentstore.py store: force remote stores to always check the shared cache 2016-05-16 10:59:09 -07:00
datapack.py packs: prevent creation of empty packs 2016-05-20 09:31:19 -07:00
debugcommands.py packs: add debug commands to view pack contents 2016-05-16 10:59:09 -07:00
fileserverclient.py windows: grp module not supported 2016-05-20 08:08:57 -07:00
historypack.py packs: prevent creation of empty packs 2016-05-20 09:31:19 -07:00
metadatastore.py store: force remote stores to always check the shared cache 2016-05-16 10:59:09 -07:00
remotefilectx.py checkcode: fix remotefilectx and remotefilelog 2016-04-26 13:00:31 -07:00
remotefilelog.py checkcode: fix remotefilectx and remotefilelog 2016-04-26 13:00:31 -07:00
remotefilelogserver.py store: basic wire protocol for bundle delivery 2016-05-16 10:59:09 -07:00
repack.py repack: add incremental repack 2016-05-20 09:31:25 -07:00
shallowbundle.py shallowbundle: fix broken fallback orig call 2016-04-28 17:34:08 -07:00
shallowrepo.py repack: don't require complete history during data repack 2016-05-16 10:59:09 -07:00
shallowstore.py Add more comments explaining various complexities 2013-11-05 17:19:59 -08:00
shallowutil.py util: close temporary file descriptor 2016-05-20 08:41:04 -07:00
shallowverifier.py remotefilelog: removing filelog check from verification process 2016-01-07 16:57:39 -08:00