2019-12-10 02:24:31 +03:00
|
|
|
#chg-compatible
|
|
|
|
|
2020-01-20 13:42:49 +03:00
|
|
|
$ disable treemanifest
|
clone: add a new clone function with less tech-debt
Summary:
The current `clone --shallow` command has some issues:
- It fetches *all* remote bookmarks, since selectivepull does not work with
streamclone, then remove most remote bookmarks in a second transaction.
- It goes through remotenames, which is racy, and D20703268 does not fix the
clone case. Possible cause of T65349853.
- Too many wrappers (ex. in remotefilelog, remotenames, fastdiscovery) wtih
many configurations (ex. narrow-heads on/off) makes it hard to reason about.
Instead of bandaidding the clone function, this diff adds a new clone implementation
that aims to solve the issues:
- Use streamclone, but do not pull all remote names.
- Pull selectivepull names explicitly with a working "discovery" strategy
(repo heads should be non-empty with narrow-heads on or off).
- Do clone in one transaction. Outside world won't see an incomplete state.
- Use `repo.pull` API, which is not subject to race conditions.
- Eventually, this might be the only supported "clone" after Mononoke becoming
the single source of truth.
Note: the code path still goes through bookmarks.py and remotenames.py.
They will be cleaned up in upcoming diffs.
Reviewed By: DurhamG
Differential Revision: D21011401
fbshipit-source-id: d8751ac9bd643e9661e58c87b683be285f0dc925
2020-05-12 20:20:16 +03:00
|
|
|
$ setconfig remotenames.selectivepull=1
|
|
|
|
|
2017-09-21 00:02:58 +03:00
|
|
|
$ LFSPATH=$TESTTMP/lfs
|
|
|
|
$ export LFSPATH
|
|
|
|
$ mkdir $LFSPATH
|
|
|
|
|
|
|
|
$ . "$TESTDIR/library.sh"
|
|
|
|
|
|
|
|
$ hginit master
|
|
|
|
$ cd master
|
|
|
|
$ cat >> $HGRCPATH <<EOF
|
|
|
|
> [extensions]
|
2018-01-09 05:58:08 +03:00
|
|
|
> lfs=
|
2017-09-21 00:02:58 +03:00
|
|
|
> [lfs]
|
|
|
|
> url=file://$LFSPATH
|
2018-08-23 10:22:04 +03:00
|
|
|
> verify=existance
|
2017-09-21 00:02:58 +03:00
|
|
|
> EOF
|
|
|
|
$ cat >> .hg/hgrc <<EOF
|
|
|
|
> [remotefilelog]
|
|
|
|
> server=True
|
|
|
|
> EOF
|
|
|
|
$ echo x > x
|
|
|
|
$ echo z > z
|
|
|
|
$ hg commit -qAm x
|
|
|
|
$ echo x2 > x
|
|
|
|
$ echo y > y
|
|
|
|
$ hg commit -qAm y
|
|
|
|
$ echo large > large
|
|
|
|
$ hg --config 'lfs.threshold=1' commit -qAm y
|
|
|
|
$ hg bookmark foo
|
|
|
|
$ hg debuglfsupload -r tip
|
|
|
|
|
|
|
|
$ cd ..
|
|
|
|
|
|
|
|
# prefetch a revision
|
|
|
|
|
|
|
|
$ hgcloneshallowlfs ssh://user@dummy/master shallow file://$LFSPATH --noupdate
|
clone: add a new clone function with less tech-debt
Summary:
The current `clone --shallow` command has some issues:
- It fetches *all* remote bookmarks, since selectivepull does not work with
streamclone, then remove most remote bookmarks in a second transaction.
- It goes through remotenames, which is racy, and D20703268 does not fix the
clone case. Possible cause of T65349853.
- Too many wrappers (ex. in remotefilelog, remotenames, fastdiscovery) wtih
many configurations (ex. narrow-heads on/off) makes it hard to reason about.
Instead of bandaidding the clone function, this diff adds a new clone implementation
that aims to solve the issues:
- Use streamclone, but do not pull all remote names.
- Pull selectivepull names explicitly with a working "discovery" strategy
(repo heads should be non-empty with narrow-heads on or off).
- Do clone in one transaction. Outside world won't see an incomplete state.
- Use `repo.pull` API, which is not subject to race conditions.
- Eventually, this might be the only supported "clone" after Mononoke becoming
the single source of truth.
Note: the code path still goes through bookmarks.py and remotenames.py.
They will be cleaned up in upcoming diffs.
Reviewed By: DurhamG
Differential Revision: D21011401
fbshipit-source-id: d8751ac9bd643e9661e58c87b683be285f0dc925
2020-05-12 20:20:16 +03:00
|
|
|
fetching changelog
|
2019-08-31 00:56:12 +03:00
|
|
|
3 files to transfer, * bytes of data (glob)
|
2018-02-10 00:08:38 +03:00
|
|
|
transferred * bytes in * seconds (*) (glob)
|
clone: add a new clone function with less tech-debt
Summary:
The current `clone --shallow` command has some issues:
- It fetches *all* remote bookmarks, since selectivepull does not work with
streamclone, then remove most remote bookmarks in a second transaction.
- It goes through remotenames, which is racy, and D20703268 does not fix the
clone case. Possible cause of T65349853.
- Too many wrappers (ex. in remotefilelog, remotenames, fastdiscovery) wtih
many configurations (ex. narrow-heads on/off) makes it hard to reason about.
Instead of bandaidding the clone function, this diff adds a new clone implementation
that aims to solve the issues:
- Use streamclone, but do not pull all remote names.
- Pull selectivepull names explicitly with a working "discovery" strategy
(repo heads should be non-empty with narrow-heads on or off).
- Do clone in one transaction. Outside world won't see an incomplete state.
- Use `repo.pull` API, which is not subject to race conditions.
- Eventually, this might be the only supported "clone" after Mononoke becoming
the single source of truth.
Note: the code path still goes through bookmarks.py and remotenames.py.
They will be cleaned up in upcoming diffs.
Reviewed By: DurhamG
Differential Revision: D21011401
fbshipit-source-id: d8751ac9bd643e9661e58c87b683be285f0dc925
2020-05-12 20:20:16 +03:00
|
|
|
fetching selected remote bookmarks
|
2017-09-21 00:02:58 +03:00
|
|
|
$ cd shallow
|
|
|
|
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg prefetch -r 'desc(x)'
|
2020-01-09 18:44:55 +03:00
|
|
|
2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over *s (glob) (?)
|
2017-09-21 00:02:58 +03:00
|
|
|
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg cat -r 'desc(x)' x
|
2017-09-21 00:02:58 +03:00
|
|
|
x
|
|
|
|
|
|
|
|
# prefetch a range of revisions
|
|
|
|
|
|
|
|
$ clearcache
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg prefetch -r 'desc(x)'::109c3a557a73b62e762a1a928070e84f52c9e2c6
|
2020-01-09 18:44:55 +03:00
|
|
|
4 files fetched over 1 fetches - (4 misses, 0.00% hit ratio) over *s (glob) (?)
|
2017-09-21 00:02:58 +03:00
|
|
|
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg cat -r 'desc(x)' x
|
2017-09-21 00:02:58 +03:00
|
|
|
x
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg cat -r 109c3a557a73b62e762a1a928070e84f52c9e2c6 x
|
2017-09-21 00:02:58 +03:00
|
|
|
x2
|
|
|
|
|
|
|
|
# prefetch certain files
|
|
|
|
|
|
|
|
$ clearcache
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg prefetch -r 109c3a557a73b62e762a1a928070e84f52c9e2c6 x
|
2020-01-09 18:44:55 +03:00
|
|
|
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob) (?)
|
2017-09-21 00:02:58 +03:00
|
|
|
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg cat -r 109c3a557a73b62e762a1a928070e84f52c9e2c6 x
|
2017-09-21 00:02:58 +03:00
|
|
|
x2
|
|
|
|
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg cat -r 109c3a557a73b62e762a1a928070e84f52c9e2c6 y
|
2017-09-21 00:02:58 +03:00
|
|
|
y
|
2020-01-09 18:44:55 +03:00
|
|
|
1 files fetched over 1 fetches - (1 misses, 0.00% hit ratio) over *s (glob) (?)
|
2017-09-21 00:02:58 +03:00
|
|
|
|
|
|
|
# prefetch large file
|
|
|
|
|
2020-07-07 00:00:16 +03:00
|
|
|
$ hg prefetch -r 'max(desc(y))'
|
2020-01-09 18:44:55 +03:00
|
|
|
2 files fetched over 1 fetches - (2 misses, 0.00% hit ratio) over *s (glob) (?)
|
2017-09-21 00:02:58 +03:00
|
|
|
|
|
|
|
# prefetch on pull when configured
|
|
|
|
|
|
|
|
$ printf "[remotefilelog]\npullprefetch=bookmark()\n" >> .hg/hgrc
|
2019-02-25 14:50:46 +03:00
|
|
|
$ hg debugstrip tip
|
2017-09-21 00:02:58 +03:00
|
|
|
|
|
|
|
$ clearcache
|
|
|
|
$ hg pull
|
|
|
|
pulling from ssh://user@dummy/master
|
|
|
|
searching for changes
|
|
|
|
adding changesets
|
|
|
|
adding manifests
|
|
|
|
adding file changes
|
|
|
|
added 1 changesets with 0 changes to 0 files
|
clone: add a new clone function with less tech-debt
Summary:
The current `clone --shallow` command has some issues:
- It fetches *all* remote bookmarks, since selectivepull does not work with
streamclone, then remove most remote bookmarks in a second transaction.
- It goes through remotenames, which is racy, and D20703268 does not fix the
clone case. Possible cause of T65349853.
- Too many wrappers (ex. in remotefilelog, remotenames, fastdiscovery) wtih
many configurations (ex. narrow-heads on/off) makes it hard to reason about.
Instead of bandaidding the clone function, this diff adds a new clone implementation
that aims to solve the issues:
- Use streamclone, but do not pull all remote names.
- Pull selectivepull names explicitly with a working "discovery" strategy
(repo heads should be non-empty with narrow-heads on or off).
- Do clone in one transaction. Outside world won't see an incomplete state.
- Use `repo.pull` API, which is not subject to race conditions.
- Eventually, this might be the only supported "clone" after Mononoke becoming
the single source of truth.
Note: the code path still goes through bookmarks.py and remotenames.py.
They will be cleaned up in upcoming diffs.
Reviewed By: DurhamG
Differential Revision: D21011401
fbshipit-source-id: d8751ac9bd643e9661e58c87b683be285f0dc925
2020-05-12 20:20:16 +03:00
|
|
|
adding remote bookmark foo
|
|
|
|
prefetching file contents
|
2020-06-24 04:45:45 +03:00
|
|
|
4 files fetched over * fetches - (4 misses, 0.00% hit ratio) over *s (glob) (?)
|
2017-09-21 00:02:58 +03:00
|
|
|
|
|
|
|
$ hg up tip
|
|
|
|
4 files updated, 0 files merged, 0 files removed, 0 files unresolved
|