sapling/tests/test-lfs.t
Jun Wu de84869150 lfs: cleanup user-facing messages
Summary:
The diff cleans up messages shown to the user. It makes verbose messages
gated by `if ui.verbose`, and simplifies some words.

The resulting user experience is, when there is no large file involved, lfs
shows nothing. When there are largefiles being downloaded or uploaded, show
progress bar if it takes long. The progress bar is the only user visible
output from lfs by default.

Test Plan: `rt test-lfs.t`

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4813297

Signature: t1:4813297:1491213089:c9ec363eb65df4c85282c90b230da3321b93b5e0
2017-04-05 15:58:56 -07:00

269 lines
13 KiB
Perl

# Initial setup
$ cat >> $HGRCPATH << EOF
> [extensions]
> lfs=$TESTDIR/../hgext3rd/lfs/
> [lfs]
> threshold=1000B
> chunksize=1000B
> blobstore=cache/localblobstore
> remotestore=dummy
> remotepath=$TESTTMP/dummy-remote/
> EOF
$ LONG=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
# Prepare server and enable extension
$ hg init server
$ hg clone -q server client
$ cd client
# Commit small file
$ echo s > smallfile
$ hg commit -Aqm "add small file"
# Commit large file
$ echo $LONG > largefile
$ hg commit --traceback -Aqm "add large file"
# Ensure metadata is stored
$ hg debugdata largefile 0
version https://git-lfs.github.com/spec/chunking
chunks d7dbc611df1fe7dfacfe267a2bfd32ba8fc27ad16aa72af7e6c553a120b92f18:1000,ed0f071aa4ff28ab9863b6cfc5f407e915612d70502422e4ab9b09f3dfec4a74:501
hashalgo sha256
size 1501
# Check the blobstore is populated
$ find .hg/cache/localblobstore | sort
.hg/cache/localblobstore
.hg/cache/localblobstore/d7
.hg/cache/localblobstore/d7/dbc611df1fe7dfacfe267a2bfd32ba8fc27ad16aa72af7e6c553a120b92f18
.hg/cache/localblobstore/ed
.hg/cache/localblobstore/ed/0f071aa4ff28ab9863b6cfc5f407e915612d70502422e4ab9b09f3dfec4a74
# Check the blob stored contains the actual contents of the file
$ cat .hg/cache/localblobstore/d7/dbc611df1fe7dfacfe267a2bfd32ba8fc27ad16aa72af7e6c553a120b92f18
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB (no-eol)
$ cat .hg/cache/localblobstore/ed/0f071aa4ff28ab9863b6cfc5f407e915612d70502422e4ab9b09f3dfec4a74
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
# Push changes to the server
$ hg push -v | egrep -v '^(uncompressed| )'
pushing to $TESTTMP/server (glob)
searching for changes
lfs: computing set of blobs to upload
lfs: uploading blobs to the remote (2 chunk(s), 1.47 KB)
2 changesets found
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files
# Unknown remotestore
$ hg push --config lfs.remotestore=404
abort: lfs: unknown remotestore: 404
[255]
$ cd ../
# Initialize new client (not cloning) and setup extension
$ hg init client2
$ cd client2
$ cat >> .hg/hgrc <<EOF
> [paths]
> default = $TESTTMP/server
> EOF
# Pull from server
$ hg pull default
pulling from $TESTTMP/server (glob)
requesting all changes
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files
(run 'hg update' to get a working copy)
# Check the blobstore is not yet populated
$ [ -f .hg/cache/localblobstore ]
[1]
# Update to the last revision containing the large file
$ hg update
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
# Check the blobstore has been populated on update
$ find .hg/cache/localblobstore | sort
.hg/cache/localblobstore
.hg/cache/localblobstore/d7
.hg/cache/localblobstore/d7/dbc611df1fe7dfacfe267a2bfd32ba8fc27ad16aa72af7e6c553a120b92f18
.hg/cache/localblobstore/ed
.hg/cache/localblobstore/ed/0f071aa4ff28ab9863b6cfc5f407e915612d70502422e4ab9b09f3dfec4a74
# Check the contents of the file are fetched from blobstore when requested
$ hg cat -r . largefile
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
# Check the file has been copied in the working copy
$ cat largefile
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
$ cd ..
# Check blobstore could be an absolute path outside repo vfs:
$ mkdir -p $TESTTMP/global-store
$ cat >> $HGRCPATH <<EOF
> [lfs]
> blobstore=$TESTTMP/global-store
> EOF
$ hg init repo2
$ cd repo2
$ echo $LONG > largefile
$ hg add largefile
$ hg commit -m initcommit
$ [ -f $TESTTMP/global-store/ed/0f071aa4ff28ab9863b6cfc5f407e915612d70502422e4ab9b09f3dfec4a74 ]
$ [ -f $TESTTMP/global-store/d7/dbc611df1fe7dfacfe267a2bfd32ba8fc27ad16aa72af7e6c553a120b92f18 ]
$ cd ..
# Check rename, and switch between large and small files
$ hg init repo3
$ cd repo3
$ cat >> .hg/hgrc << EOF
> [lfs]
> threshold=10B
> chunksize=10B
> EOF
$ echo LONGER-THAN-TEN-BYTES-WILL-TRIGGER-LFS > large
$ echo SHORTER > small
$ hg add . -q
$ hg commit -m 'commit with lfs content'
$ hg mv large l
$ hg mv small s
$ hg commit -m 'renames'
$ echo SHORT > l
$ echo BECOME-LARGER-FROM-SHORTER > s
$ hg commit -m 'large to small, small to large'
$ echo 1 >> l
$ echo 2 >> s
$ hg commit -m 'random modifications'
$ echo RESTORE-TO-BE-LARGE > l
$ echo SHORTER > s
$ hg commit -m 'switch large and small again'
# Push and pull the above repo
$ hg --cwd .. init repo4
$ hg push ../repo4
pushing to ../repo4
searching for changes
adding changesets
adding manifests
adding file changes
added 5 changesets with 10 changes to 4 files
$ hg --cwd .. init repo5
$ hg --cwd ../repo5 pull ../repo3
pulling from ../repo3
requesting all changes
adding changesets
adding manifests
adding file changes
added 5 changesets with 10 changes to 4 files
(run 'hg update' to get a working copy)
$ cd ..
# Test clone
$ hg init repo6
$ cd repo6
$ cat >> .hg/hgrc << EOF
> [lfs]
> threshold=30B
> chunksize=20B
> EOF
$ echo LARGE-BECAUSE-IT-IS-MORE-THAN-30-BYTES > large
$ echo SMALL > small
$ hg commit -Aqm 'create a lfs file' large small
$ cd ..
$ hg clone repo6 repo7
updating to branch default
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd repo7
$ cat large
LARGE-BECAUSE-IT-IS-MORE-THAN-30-BYTES
$ cat small
SMALL
# Test bypass
$ hg cat -r tip large --config lfs.remotepath=$TESTTMP/404 --config lfs.blobstore=cache/404
abort: No such file or directory: $TESTTMP/404/2e81056070ae365867a5e7f804abe39c12e07bdf11c098994d1bc0ab9981910a
[255]
$ hg cat -r tip large --config lfs.remotepath=$TESTTMP/404 --config lfs.blobstore=cache/404 --config lfs.bypass=1
version https://git-lfs.github.com/spec/chunking
chunks 2e81056070ae365867a5e7f804abe39c12e07bdf11c098994d1bc0ab9981910a:20,903f301311488de7befd35be124433f73d3b7b6bce1d75440474907ff2ae7591:19
hashalgo sha256
size 39
$ cd ..
# Verify the repos
$ cat > $TESTTMP/dumpflog.py << EOF
> # print raw revision sizes, flags, and hashes for certain files
> import hashlib
> from mercurial import revlog
> from mercurial.node import short
> def hash(rawtext):
> h = hashlib.sha512()
> h.update(rawtext)
> return h.hexdigest()[:4]
> def reposetup(ui, repo):
> # these 2 files are interesting
> for name in ['l', 's']:
> fl = repo.file(name)
> if len(fl) == 0:
> continue
> sizes = [revlog.revlog.rawsize(fl, i) for i in fl]
> texts = [fl.revision(i, raw=True) for i in fl]
> flags = [fl.flags(i) for i in fl]
> hashes = [hash(t) for t in texts]
> print(' %s: rawsizes=%r flags=%r hashes=%r'
> % (name, sizes, flags, hashes))
> EOF
$ for i in client client2 server repo3 repo4 repo5 repo6 repo7; do
> echo 'repo:' $i
> hg --cwd $i verify --config extensions.dumpflog=$TESTTMP/dumpflog.py -q
> done
repo: client
repo: client2
repo: server
repo: repo3
l: rawsizes=[828, 6, 8, 216] flags=[8192, 0, 0, 8192] hashes=['2791', '948c', 'cc88', '5f1a']
s: rawsizes=[623, 283, 283, 8] flags=[8192, 8192, 8192, 0] hashes=['db08', 'ff24', '2217', '826b']
repo: repo4
l: rawsizes=[828, 6, 8, 216] flags=[8192, 0, 0, 8192] hashes=['2791', '948c', 'cc88', '5f1a']
s: rawsizes=[623, 283, 283, 8] flags=[8192, 8192, 8192, 0] hashes=['db08', 'ff24', '2217', '826b']
repo: repo5
l: rawsizes=[828, 6, 8, 216] flags=[8192, 0, 0, 8192] hashes=['2791', '948c', 'cc88', '5f1a']
s: rawsizes=[623, 283, 283, 8] flags=[8192, 8192, 8192, 0] hashes=['db08', 'ff24', '2217', '826b']
repo: repo6
repo: repo7