Summary: Implemented logic for fetching data from the scmmemcache and the server in pack files format. By default `prefetch` will still use loose files, unless the `remotefilelog.fetchpacks` config flag will be set to true.
Differential Revision: D13905733
fbshipit-source-id: cc559f5bc600067199a91efe6cf44b1bde6bf69b
Summary:
D13646642 used the `--date` flag for the `touch` command which
unfortunately does not work on our prod OSX hosts. Therefore, lets switch to
the more compatible `-t` flag.
Reviewed By: DurhamG
Differential Revision: D13673021
fbshipit-source-id: 6f9e0234e7fd3d4c345bedecdd2197949d71c420
Summary:
We've observed some users with very large hgcache directories that were filled
with temporary pack files that for some reasons were not removed/renamed on a
previous repack.
These files can appear due to a variety of reasons, such as forcibly killing
hg, or a host power-off, or simply due to a bug in mercurial. It is likely that
the later case is what causes some of the hgcache directories to grow and this
patch doesn't attempt on finding the underlying mercurial issue. Rather, let's
alleviate the issue by simply removing the temporary files older than 24h.
Reviewed By: ikostia
Differential Revision: D13646642
fbshipit-source-id: faa0605e322d440a75187e2517cbbcb13031dae0
Summary:
This fixes LFS compatibility with packlocaldata.
Also drop the config as the default version is 1 now.
Reviewed By: DurhamG
Differential Revision: D13469486
fbshipit-source-id: 1dc4a1051667419d7aab97bf95f93cacd166468a
Summary:
It turns out data packs contain the exact mercurial file blob,
including the rename metadata header. This is different from how loose files work, which
contain the non-headered text and instead store the rename in the history at the
end of the file.
Ideally we'd change datapacks to not store the metadata version of the file, but
unfortunately this is how it has always been, so instead we must change the
commit-straight-to-pack code to write the headered blob.
Differential Revision: D12936247
fbshipit-source-id: 3f909d2964d7f7200ac0e31e47bd195b0c9b03e5
Summary:
The incremental repack implementation omits repacking local data - it was
recently added for full repack, but not incremental repack. Add it to
incremental repack.
We also don't run `_deletebigpacks` for non-shared data, as that data is
only stored locally.
Reviewed By: DurhamG
Differential Revision: D10525957
fbshipit-source-id: 012d7e8d455ac607773fc7013dcc4df0881665e8
Summary:
Loose files don't contain the original filename inside them, but rather the
directory name is based on the sha1 hash of the filename. This means that
given a remotefilelog data file, we can't find out the name of the file that
it contains.
Repack needs to know this (plus it's also useful for debug purposes), so add
an additional file in the directory, `filename`, which contains the original
filename of all the data files in the directory.
It follows that `sha1(read(filename)) == basename(dirname(filename))`
Reviewed By: DurhamG
Differential Revision: D10516851
fbshipit-source-id: aeeb1a116e0d50a134c274c3bb31c7c26aa8015c
Summary: Packing loose files for local data into `.hg/store/packs`. Existing pack files for local data can also be repacked.
Reviewed By: DurhamG
Differential Revision: D9181106
fbshipit-source-id: 4d044e97178b4263b87404357445b43eb99017b3
Summary:
It can sometimes be nice to be able to just repack loose data and avoid existing packs.
This came up during building the megarepo, which creates many loose files, but including the packs added too much repack time.
Differential Revision: D7209296
fbshipit-source-id: 10afed40e409733e0ee004f025013cf86f3f7bf6
Summary:
Now they are unnecessary since `run-tests.py` will set up `PYTONPATH`
correctly.
Differential Revision: D6865042
fbshipit-source-id: ca95314f725968e14349a9d916434aa832c596f9
Summary:
Moves the remotefilelog extension into hgext/ and it's tests into
tests/.
I did not fix up all the check-module errors, since it's a ton of work for
very little impact at this point.
Test Plan: make local && ./run-tests.py
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D6680030
