mirror of
https://github.com/facebook/sapling.git
synced 2024-10-12 01:39:21 +03:00
baacd19749
Summary: Once `remotefilelog.fetchpacks` is enabled, `hg gc` will no longer be able to limit the size of the hgcache. This will be particularly challenging for Sandcastle/Quicksand as they already see hgcache over 100GB. The long-term plan is switching to IndexedLog based stores with a log rotate functionality to control the cache size. In the meantime, we can implement a basic logic to enforce the size of the hgcache that simply remove packfiles once the cache is over the configured size. One complication of this method is that several concurrent Mercurial processes could be running and accessing the packfiles being removed. In this case, we can split the packfiles in 2 categories: ones created a while back, and new ones. Removing packfiles from the first case, lookups will simply raise a KeyError and data will be re-fetched from Memcache/Mononoke, ie: failure is acceptable. The second category belongs to ones that were just created by downloading them from Memcache/Mononoke, and the code strongly assume that they will stick around. A failure at this point will not be recovered. One way of fixing this would be to handle these failures properly and simply retry, the other is to not remove new packfiles. A time of 10 minutes was chosen to categorize the packfiles. Reviewed By: quark-zju Differential Revision: D15014076 fbshipit-source-id: 014eea0251ea3a630aaaa75759cd492271a5c5cd
56 lines
1.6 KiB
Raku
56 lines
1.6 KiB
Raku
|
|
$ . "$TESTDIR/library.sh"
|
|
|
|
$ cat >> $HGRCPATH <<EOF
|
|
> [format]
|
|
> userustdatapack=True
|
|
> [remotefilelog]
|
|
> fetchpacks=True
|
|
> EOF
|
|
|
|
$ hginit master
|
|
$ cd master
|
|
$ cat >> .hg/hgrc <<EOF
|
|
> [remotefilelog]
|
|
> server=True
|
|
> serverexpiration=-1
|
|
> EOF
|
|
$ echo x > x
|
|
$ hg commit -qAm x
|
|
$ echo x >> x
|
|
$ hg commit -qAm x2
|
|
$ cd ..
|
|
|
|
$ hgcloneshallow ssh://user@dummy/master shallow -q
|
|
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob)
|
|
|
|
$ cd shallow
|
|
$ cat >> .hg/hgrc <<EOF
|
|
> [remotefilelog]
|
|
> userustrepack=True
|
|
> EOF
|
|
$ cd ..
|
|
|
|
$ cd shallow
|
|
$ find $CACHEDIR | sort
|
|
$TESTTMP/hgcache
|
|
$TESTTMP/hgcache/master
|
|
$TESTTMP/hgcache/master/packs
|
|
$TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histidx
|
|
$TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histpack
|
|
$TESTTMP/hgcache/master/packs/887690f1138ae5b99c50d754ed02262874bf8ecb.dataidx
|
|
$TESTTMP/hgcache/master/packs/887690f1138ae5b99c50d754ed02262874bf8ecb.datapack
|
|
|
|
$ touch -m -t 200001010000 $TESTTMP/hgcache/master/packs/887690f1138ae5b99c50d754ed02262874bf8ecb.datapack
|
|
|
|
# Cleanup the old over the limit packfiles
|
|
$ hg repack --config remotefilelog.cleanoldpacks=True --config remotefilelog.cachelimit="10B"
|
|
|
|
$ find $CACHEDIR | sort
|
|
$TESTTMP/hgcache
|
|
$TESTTMP/hgcache/master
|
|
$TESTTMP/hgcache/master/packs
|
|
$TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histidx
|
|
$TESTTMP/hgcache/master/packs/276d308429d0303762befa376788300f0310f90e.histpack
|
|
$TESTTMP/hgcache/master/packs/repacklock
|