facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-06 23:07:18 +03:00
Code Issues Projects Releases Wiki Activity
4d4274ce53
sapling/eden/scm/tests/test-bundle2-exchange.t

1121 lines
42 KiB
Perl
Raw Normal View History

mark all tests requiring Python 2 Summary: This diff marks **ALL** mercurial tests requiring Python 2 feature. After you fixes some tests, simply remove the `py2` feature requirement and that tests will be continuously run after your diff is landed. To bypass this feature requirement, run the tests command with `HGTEST_FORCE_PY2=1`. For example: ``` HGTEST_FORCE_PY2=1 buck test //eden/scm/tests:hg_run_tests ``` or ``` HGTEST_FORCE_PY2=1 python run-tests.py ``` ---- Basically this diff are created with the following commands: ``` $ sed -i 's/import feature\(.*\)$/import feature\1\n\nfeature.require(["py2"])/' test-*-t.py $ sed -i '1s/^/#require py2\n/' test-*.t $ ls | grep -P "^test.*(?<\!-t)\.py$" > list && vim -p $(cat list) # manually adding feature requires for these Python tests. ``` (Note: this ignores all push blocking failures!) ignore-conflict-markers Reviewed By: singhsrb Differential Revision: D19655148 fbshipit-source-id: 985e3ccb4010cc559049f1d89f8909bc2d9b5e20
2020-01-31 05:40:49 +03:00
#require py2
tests: enable chg for 572 tests Summary: Add `#chg-compatible` to 572 tests that seem to pass with chg enabled. This should make them run faster. Reviewed By: xavierd Differential Revision: D18870507 fbshipit-source-id: fe895e733efffc9286cd3d17c7a156c803124395
2019-12-10 02:24:31 +03:00
#chg-compatible
tests: use disable treemanifest in place of setconfig Summary: Use `disable treemanifest` in place of `setconfig extensions.treemanifest=!` Reviewed By: quark-zju Differential Revision: D19457678 fbshipit-source-id: 47c8b3c4f4ed5cfc97275ca67afab4a86ffe7f0d
2020-01-20 13:42:49 +03:00
$ disable treemanifest
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
Test exchange of common information using bundle2
$ getmainid() {
> hg -R main log --template '{node}\n' --rev "$1"
> }
enable obsolescence
bundle2: allow lazily acquiring the lock In the external pushrebase extension, it is valuable to be able to do some work without taking the lock (like running expensive hooks). This enables significantly higher commit throughput. This patch adds an option to lazily acquire the lock. It means that all bundle2 part handlers that require writing to the repo must first call op.gettransction(), when in this mode.
2015-10-06 02:19:54 +03:00
$ cp $HGRCPATH $TESTTMP/hgrc.orig
test-bundle2-exchange: create temp script in $TESTTMP, not $TESTDIR The bundle2-pushkey-hook.sh script is currently created in $TESTTMP, and leaves an untracked file in that directory (tests/) after running.
2015-01-08 01:30:40 +03:00
$ cat > $TESTTMP/bundle2-pushkey-hook.sh << EOF
tests: make a multi-statement hook in bundle2-exchange Windows compatible This is similar to the fix in ab91e34fea50.
2014-12-24 20:07:59 +03:00
> echo pushkey: lock state after \"\$HG_NAMESPACE\"
> hg debuglock
> EOF
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cat >> $HGRCPATH << EOF
> [experimental]
config: use 'experimental.evolution.create-markers' Extract 'experimental.evolution' = createmarkers as 'experimental.evolution.createmarkers'. We keep the new option in the 'experimental.evolution' namespace in order to stay coherent with other options ('experimental.evolution.bundle-obsmarker' and 'experimental.evolution.track-operation') ease the renaming as possibly 'evolution.createmarkers'. Differential Revision: https://phab.mercurial-scm.org/D1149
2017-09-28 20:19:06 +03:00
> evolution.createmarkers=True
config: use 'experimental.evolution.exchange' Extract 'experimental.evolution' = exchange as 'experimental.evolution.exchange'. We keep the new option in the 'experimental.evolution' namespace in order to stay coherent with other options ('experimental.evolution.bundle-obsmarker' and 'experimental.evolution.track-operation') ease the renaming as possibly 'evolution.exchange'. Differential Revision: https://phab.mercurial-scm.org/D1151
2017-09-28 23:07:58 +03:00
> evolution.exchange=True
bundle2: disable ouput capture unless we use http (issue4613 issue4615) The current bundle2 processing was capturing all output. This is nice as it provide better meta data about what output what, but this was changing two things: 1) adding a prefix "remote: " to "other" output during local push (issue4613) 2) local and ssh push does not provide real time output anymore (issue4615) As we are unsure about what form should be used in (1) and how to solve (2) we disable output capture in this two cases. Output capture can be forced using an experimental option.
2015-04-29 03:38:02 +03:00
> bundle2-output-capture=True
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
> [ui]
tests: quote $PYTHON for Windows When unquoted, MSYS sees the colon between the drive letter and path as a Unix path separator and unhelpfully splits on it, feeding only the drive letter as the command. Much chaos ensues. I vaguely remember trying to get the test runner to use /letter/path/to/exe syntax the last time this happened, without success. I doubt a check-code rule would work, since sometimes it is quoted, and sometimes the quotes are escaped.
2017-07-07 08:05:20 +03:00
> ssh="$PYTHON" "$TESTDIR/dummyssh"
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
> logtemplate={rev}:{node|short} {phase} {author} {bookmarks} {desc|firstline}
> [web]
> push_ssl = false
> allow_push = *
> [phases]
> publish=False
test-bundle2: test that we got appropriate hook called with appropriate data We can notice that this transaction wide hook is only happening during push and it is missing changegroup-related information. We'll want to fix this but this is not what this patch is about.
2014-10-14 01:47:36 +04:00
> [hooks]
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
> pretxnclose.tip = hg log -r tip -T "pre-close-tip:{node|short} {phase} {bookmarks}\n"
> txnclose.tip = hg log -r tip -T "postclose-tip:{node|short} {phase} {bookmarks}\n"
tests: simplify printenv calls Make printenv executable so that we don't need python, TESTDIR, or quoting.
2015-06-08 23:10:15 +03:00
> txnclose.env = sh -c "HG_LOCAL= printenv.py txnclose"
test-bundle2-exchange: create temp script in $TESTTMP, not $TESTDIR The bundle2-pushkey-hook.sh script is currently created in $TESTTMP, and leaves an untracked file in that directory (tests/) after running.
2015-01-08 01:30:40 +03:00
> pushkey= sh "$TESTTMP/bundle2-pushkey-hook.sh"
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
> EOF
The extension requires a repo (currently unused)
$ hg init main
$ cd main
$ touch a
$ hg add a
$ hg commit -m 'a'
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:3903775176ed draft
postclose-tip:3903775176ed draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_PHASES_MOVED=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=commit
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg unbundle $TESTDIR/bundles/rebase.hg
adding changesets
adding manifests
adding file changes
changegroup: do not show "(+/- ? heads)" Summary: In an upcoming change we'll replace `cl.heads()` with `repo.heads()`. In changegroup code path the repo can be in an inconsistent state (ex. a bookmark refers to an unknown commit) which breaks `repo.heads()`. Since the whole purpose of the heads calculation is just to show `(+/- ? heads)`, which does not affect any real features, let's just remove it. Reviewed By: markbt Differential Revision: D18366735 fbshipit-source-id: 893be2cec0c32b64a80b3ef4ca65b69f8ed76b27
2019-11-07 21:50:10 +03:00
added 8 changesets with 7 changes to 7 files
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NODE=cd010b8cd998f3981a5a8115f94f8da4ab506089 HG_NODE_LAST=02de42196ebee42ef284b6780a87cdc96e8eaab6 HG_PHASES_MOVED=1 HG_SOURCE=unbundle HG_TXNID=TXN:$ID$ HG_TXNNAME=unbundle
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
bundle:*/tests/bundles/rebase.hg HG_URL=bundle:*/tests/bundles/rebase.hg (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cd ..
Real world exchange
=====================
Add more obsolescence information
$ hg -R main debugobsolete -d '0 0' 1111111111111111111111111111111111111111 `getmainid 9520eea781bc`
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=debugobsolete
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main debugobsolete -d '0 0' 2222222222222222222222222222222222222222 `getmainid 24b6387c8c8c`
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=debugobsolete
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
clone --pull
$ hg -R main phase --public cd010b8cd998
changelog: fix readpending if no pending data exist (issue4609) Since transaction are used for more than just changesets, it is possible to have a transaction without new changesets at all. In this case no ''00changelog.i.a' are written. In all cases the 'changelog.readpending' method is called if the repository has any pending data. The 'revlog' logic provides empty content if the file is missing, so the whole operation resulted in an empty changelog. We now skip reading the pending file if it is missing.
2015-04-20 18:16:22 +03:00
pre-close-tip:02de42196ebe draft
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_PHASES_MOVED=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=phase
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg clone main other --pull --rev 9520eea781bc
adding changesets
adding manifests
adding file changes
added 2 changesets with 2 changes to 2 files
1 new obsolescence markers
phases: read pending data when appropriate If we are called by a hook and pending data exists, read those.
2014-10-18 09:23:06 +04:00
pre-close-tip:9520eea781bc draft
test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.
2014-11-12 19:54:57 +03:00
postclose-tip:9520eea781bc draft
clone: protect writes with locks and transaction Summary: This resolves a ProgrammingError added later. Reviewed By: xavierd Differential Revision: D18538146 fbshipit-source-id: fd659491e77e2b21f876f16a61860806acbc1480
2019-11-27 04:46:35 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_NODE=cd010b8cd998f3981a5a8115f94f8da4ab506089 HG_NODE_LAST=9520eea781bcca16c1e15acc0ba14335a0e8e5ba HG_PHASES_MOVED=1 HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_TXNNAME=clone HG_URL=file:$TESTTMP/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
updating to branch default
2 files updated, 0 files merged, 0 files removed, 0 files unresolved
clone: protect writes with locks and transaction Summary: This resolves a ProgrammingError added later. Reviewed By: xavierd Differential Revision: D18538146 fbshipit-source-id: fd659491e77e2b21f876f16a61860806acbc1480
2019-11-27 04:46:35 +03:00
pre-close-tip:9520eea781bc draft
postclose-tip:9520eea781bc draft
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=clone
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other log -G
@ 1:9520eea781bc draft Nicolas Dumazet <nicdumz.commits@gmail.com> E
|
o 0:cd010b8cd998 public Nicolas Dumazet <nicdumz.commits@gmail.com> A
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
pull
$ hg -R main phase --public 9520eea781bc
changelog: fix readpending if no pending data exist (issue4609) Since transaction are used for more than just changesets, it is possible to have a transaction without new changesets at all. In this case no ''00changelog.i.a' are written. In all cases the 'changelog.readpending' method is called if the repository has any pending data. The 'revlog' logic provides empty content if the file is missing, so the whole operation resulted in an empty changelog. We now skip reading the pending file if it is missing.
2015-04-20 18:16:22 +03:00
pre-close-tip:02de42196ebe draft
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_PHASES_MOVED=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=phase
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other pull -r 24b6387c8c8c
tests: remove (glob) annotations that were only for '\' matches # skip-blame because this was mechanically rewritten the following script. I ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t ones appear to be, and they run without addition failures on both Windows and Linux. import argparse import os import re ap = argparse.ArgumentParser() ap.add_argument('path', nargs='+') opts = ap.parse_args() globre = re.compile(r'^(.*) \(glob\)(.*)$') for p in opts.path: tmp = p + '.tmp' with open(p, 'rb') as src, open(tmp, 'wb') as dst: for line in src: m = globre.match(line) if not m or '$LOCALIP' in line or '*' in line: dst.write(line) continue if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'): dst.write(line) continue dst.write(m.group(1) + m.group(2) + '\n') os.unlink(p) os.rename(tmp, p)
2017-12-11 06:50:57 +03:00
pulling from $TESTTMP/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
searching for changes
adding changesets
adding manifests
adding file changes
changegroup: do not show "(+/- ? heads)" Summary: In an upcoming change we'll replace `cl.heads()` with `repo.heads()`. In changegroup code path the repo can be in an inconsistent state (ex. a bookmark refers to an unknown commit) which breaks `repo.heads()`. Since the whole purpose of the heads calculation is just to show `(+/- ? heads)`, which does not affect any real features, let's just remove it. Reviewed By: markbt Differential Revision: D18366735 fbshipit-source-id: 893be2cec0c32b64a80b3ef4ca65b69f8ed76b27
2019-11-07 21:50:10 +03:00
added 1 changesets with 1 changes to 1 files
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
1 new obsolescence markers
phases: read pending data when appropriate If we are called by a hook and pending data exists, read those.
2014-10-18 09:23:06 +04:00
pre-close-tip:24b6387c8c8c draft
test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.
2014-11-12 19:54:57 +03:00
postclose-tip:24b6387c8c8c draft
pull: update remote/master unconditionally Summary: This is inteneded to solve multiple problems: - Mitigate issues where our CI system does not enable remotenames in some random code paths, which leads to missing or stale `remote/master`. This further mitigates bad cases with narrow-heads' phase calculation where a stale `remote/master` can cause many commits to become draft incorrectly. - Mitigate issues in `remotenames` where race condition can happen for pushrebase (remotenames does a "listnames" after the push operation). See D18601035 for a hacky workaround. Reviewed By: DurhamG Differential Revision: D19380940 fbshipit-source-id: 4481ff114d35be37c331c72ac561c2a0894206cb
2020-01-17 21:11:25 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_NODE=24b6387c8c8cae37178880f3fa95ded3cb1cf785 HG_NODE_LAST=24b6387c8c8cae37178880f3fa95ded3cb1cf785 HG_PHASES_MOVED=1 HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_TXNNAME=pull HG_URL=file:$TESTTMP/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other log -G
o 2:24b6387c8c8c draft Nicolas Dumazet <nicdumz.commits@gmail.com> F
|
| @ 1:9520eea781bc draft Nicolas Dumazet <nicdumz.commits@gmail.com> E
|/
o 0:cd010b8cd998 public Nicolas Dumazet <nicdumz.commits@gmail.com> A
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
pull empty (with phase movement)
$ hg -R main phase --public 24b6387c8c8c
changelog: fix readpending if no pending data exist (issue4609) Since transaction are used for more than just changesets, it is possible to have a transaction without new changesets at all. In this case no ''00changelog.i.a' are written. In all cases the 'changelog.readpending' method is called if the repository has any pending data. The 'revlog' logic provides empty content if the file is missing, so the whole operation resulted in an empty changelog. We now skip reading the pending file if it is missing.
2015-04-20 18:16:22 +03:00
pre-close-tip:02de42196ebe draft
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_PHASES_MOVED=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=phase
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other pull -r 24b6387c8c8c
tests: remove (glob) annotations that were only for '\' matches # skip-blame because this was mechanically rewritten the following script. I ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t ones appear to be, and they run without addition failures on both Windows and Linux. import argparse import os import re ap = argparse.ArgumentParser() ap.add_argument('path', nargs='+') opts = ap.parse_args() globre = re.compile(r'^(.*) \(glob\)(.*)$') for p in opts.path: tmp = p + '.tmp' with open(p, 'rb') as src, open(tmp, 'wb') as dst: for line in src: m = globre.match(line) if not m or '$LOCALIP' in line or '*' in line: dst.write(line) continue if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'): dst.write(line) continue dst.write(m.group(1) + m.group(2) + '\n') os.unlink(p) os.rename(tmp, p)
2017-12-11 06:50:57 +03:00
pulling from $TESTTMP/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
no changes found
changelog: fix readpending if no pending data exist (issue4609) Since transaction are used for more than just changesets, it is possible to have a transaction without new changesets at all. In this case no ''00changelog.i.a' are written. In all cases the 'changelog.readpending' method is called if the repository has any pending data. The 'revlog' logic provides empty content if the file is missing, so the whole operation resulted in an empty changelog. We now skip reading the pending file if it is missing.
2015-04-20 18:16:22 +03:00
pre-close-tip:24b6387c8c8c public
test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.
2014-11-12 19:54:57 +03:00
postclose-tip:24b6387c8c8c public
pull: update remote/master unconditionally Summary: This is inteneded to solve multiple problems: - Mitigate issues where our CI system does not enable remotenames in some random code paths, which leads to missing or stale `remote/master`. This further mitigates bad cases with narrow-heads' phase calculation where a stale `remote/master` can cause many commits to become draft incorrectly. - Mitigate issues in `remotenames` where race condition can happen for pushrebase (remotenames does a "listnames" after the push operation). See D18601035 for a hacky workaround. Reviewed By: DurhamG Differential Revision: D19380940 fbshipit-source-id: 4481ff114d35be37c331c72ac561c2a0894206cb
2020-01-17 21:11:25 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=0 HG_PHASES_MOVED=1 HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_TXNNAME=pull HG_URL=file:$TESTTMP/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other log -G
o 2:24b6387c8c8c public Nicolas Dumazet <nicdumz.commits@gmail.com> F
|
| @ 1:9520eea781bc draft Nicolas Dumazet <nicdumz.commits@gmail.com> E
|/
o 0:cd010b8cd998 public Nicolas Dumazet <nicdumz.commits@gmail.com> A
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
pull empty
$ hg -R other pull -r 24b6387c8c8c
tests: remove (glob) annotations that were only for '\' matches # skip-blame because this was mechanically rewritten the following script. I ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t ones appear to be, and they run without addition failures on both Windows and Linux. import argparse import os import re ap = argparse.ArgumentParser() ap.add_argument('path', nargs='+') opts = ap.parse_args() globre = re.compile(r'^(.*) \(glob\)(.*)$') for p in opts.path: tmp = p + '.tmp' with open(p, 'rb') as src, open(tmp, 'wb') as dst: for line in src: m = globre.match(line) if not m or '$LOCALIP' in line or '*' in line: dst.write(line) continue if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'): dst.write(line) continue dst.write(m.group(1) + m.group(2) + '\n') os.unlink(p) os.rename(tmp, p)
2017-12-11 06:50:57 +03:00
pulling from $TESTTMP/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
no changes found
test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.
2014-11-12 19:54:57 +03:00
pre-close-tip:24b6387c8c8c public
postclose-tip:24b6387c8c8c public
pull: update remote/master unconditionally Summary: This is inteneded to solve multiple problems: - Mitigate issues where our CI system does not enable remotenames in some random code paths, which leads to missing or stale `remote/master`. This further mitigates bad cases with narrow-heads' phase calculation where a stale `remote/master` can cause many commits to become draft incorrectly. - Mitigate issues in `remotenames` where race condition can happen for pushrebase (remotenames does a "listnames" after the push operation). See D18601035 for a hacky workaround. Reviewed By: DurhamG Differential Revision: D19380940 fbshipit-source-id: 4481ff114d35be37c331c72ac561c2a0894206cb
2020-01-17 21:11:25 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=0 HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_TXNNAME=pull HG_URL=file:$TESTTMP/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other log -G
o 2:24b6387c8c8c public Nicolas Dumazet <nicdumz.commits@gmail.com> F
|
| @ 1:9520eea781bc draft Nicolas Dumazet <nicdumz.commits@gmail.com> E
|/
o 0:cd010b8cd998 public Nicolas Dumazet <nicdumz.commits@gmail.com> A
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
add extra data to test their exchange during push
$ hg -R main bookmark --rev eea13746799a book_eea1
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:02de42196ebe draft
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main debugobsolete -d '0 0' 3333333333333333333333333333333333333333 `getmainid eea13746799a`
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft
postclose-tip:02de42196ebe draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=debugobsolete
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main bookmark --rev 02de42196ebe book_02de
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main debugobsolete -d '0 0' 4444444444444444444444444444444444444444 `getmainid 02de42196ebe`
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=debugobsolete
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main bookmark --rev 42ccdea3bb16 book_42cc
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main debugobsolete -d '0 0' 5555555555555555555555555555555555555555 `getmainid 42ccdea3bb16`
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=debugobsolete
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main bookmark --rev 5fddd98957c8 book_5fdd
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main debugobsolete -d '0 0' 6666666666666666666666666666666666666666 `getmainid 5fddd98957c8`
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=debugobsolete
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main bookmark --rev 32af7686d403 book_32af
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main debugobsolete -d '0 0' 7777777777777777777777777777777777777777 `getmainid 32af7686d403`
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=debugobsolete
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other bookmark --rev cd010b8cd998 book_eea1
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:24b6387c8c8c public
postclose-tip:24b6387c8c8c public
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other bookmark --rev cd010b8cd998 book_02de
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:24b6387c8c8c public
postclose-tip:24b6387c8c8c public
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other bookmark --rev cd010b8cd998 book_42cc
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:24b6387c8c8c public
postclose-tip:24b6387c8c8c public
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other bookmark --rev cd010b8cd998 book_5fdd
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:24b6387c8c8c public
postclose-tip:24b6387c8c8c public
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other bookmark --rev cd010b8cd998 book_32af
bookmarks: change bookmark within a transaction For some time, bookmark can and should be moved in the transaction. This changeset migrates the 'hg bookmarks' commands to use a transaction. Tests regarding rollback and transaction hooks are impacted for obvious reasons. Some have to be slightly updated to keep testing the same things. Some can just be dropped because they do not make sense anymore.
2014-09-28 11:49:36 +04:00
pre-close-tip:24b6387c8c8c public
postclose-tip:24b6387c8c8c public
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=bookmark
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main phase --public eea13746799a
changelog: fix readpending if no pending data exist (issue4609) Since transaction are used for more than just changesets, it is possible to have a transaction without new changesets at all. In this case no ''00changelog.i.a' are written. In all cases the 'changelog.readpending' method is called if the repository has any pending data. The 'revlog' logic provides empty content if the file is missing, so the whole operation resulted in an empty changelog. We now skip reading the pending file if it is missing.
2015-04-20 18:16:22 +03:00
pre-close-tip:02de42196ebe draft book_02de
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_PHASES_MOVED=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=phase
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
push
$ hg -R main push other --rev eea13746799a --bookmark book_eea1
pushing to other
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
changegroup: do not show "(+/- ? heads)" Summary: In an upcoming change we'll replace `cl.heads()` with `repo.heads()`. In changegroup code path the repo can be in an inconsistent state (ex. a bookmark refers to an unknown commit) which breaks `repo.heads()`. Since the whole purpose of the heads calculation is just to show `(+/- ? heads)`, which does not affect any real features, let's just remove it. Reviewed By: markbt Differential Revision: D18366735 fbshipit-source-id: 893be2cec0c32b64a80b3ef4ca65b69f8ed76b27
2019-11-07 21:50:10 +03:00
remote: added 1 changesets with 0 changes to 0 files
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
remote: 1 new obsolescence markers
bundle2: capture output issue during transaction close We were capturing all output issue during bundle2 processing, and all output issue during transaction rollback in case of failure. However, the output issue during transaction commit was still roaming the land freely. It is now put back in line. This let the user see output from 'pretxnclose' and 'txnclose' (and related) in the right order.
2015-04-23 16:54:45 +03:00
remote: pre-close-tip:eea13746799a public book_eea1
remote: postclose-tip:eea13746799a public book_eea1
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
remote: txnclose hook: HG_BOOKMARK_MOVED=1 HG_BUNDLE2=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_NODE=eea13746799a9e0bfd88f29d3c2e9dc9389f524f HG_NODE_LAST=eea13746799a9e0bfd88f29d3c2e9dc9389f524f HG_PHASES_MOVED=1 HG_SOURCE=push HG_TXNID=TXN:$ID$ HG_TXNNAME=push HG_URL=file:$TESTTMP/other
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
updating bookmark book_eea1
push: elevate phase transaction to cover entire operation This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Most pushes already open a transaction in order to sync phase information. This diff replaces that transaction with one that spans the entire push operation. This transaction will be used in a later patch to guard repository changes made during the reply handler.
2014-11-22 02:06:38 +03:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_SOURCE=push-response HG_TXNID=TXN:$ID$ HG_TXNNAME=push-response
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
file:/*/$TESTTMP/other HG_URL=file:$TESTTMP/other (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other log -G
o 3:eea13746799a public Nicolas Dumazet <nicdumz.commits@gmail.com> book_eea1 G
|\
| o 2:24b6387c8c8c public Nicolas Dumazet <nicdumz.commits@gmail.com> F
| |
@ | 1:9520eea781bc public Nicolas Dumazet <nicdumz.commits@gmail.com> E
|/
o 0:cd010b8cd998 public Nicolas Dumazet <nicdumz.commits@gmail.com> book_02de book_32af book_42cc book_5fdd A
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
3333333333333333333333333333333333333333 eea13746799a9e0bfd88f29d3c2e9dc9389f524f 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
pull over ssh
$ hg -R other pull ssh://user@dummy/main -r 02de42196ebe --bookmark book_02de
pulling from ssh://user@dummy/main
searching for changes
adding changesets
adding manifests
adding file changes
changegroup: do not show "(+/- ? heads)" Summary: In an upcoming change we'll replace `cl.heads()` with `repo.heads()`. In changegroup code path the repo can be in an inconsistent state (ex. a bookmark refers to an unknown commit) which breaks `repo.heads()`. Since the whole purpose of the heads calculation is just to show `(+/- ? heads)`, which does not affect any real features, let's just remove it. Reviewed By: markbt Differential Revision: D18366735 fbshipit-source-id: 893be2cec0c32b64a80b3ef4ca65b69f8ed76b27
2019-11-07 21:50:10 +03:00
added 1 changesets with 1 changes to 1 files
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
1 new obsolescence markers
updating bookmark book_02de
phases: read pending data when appropriate If we are called by a hook and pending data exists, read those.
2014-10-18 09:23:06 +04:00
pre-close-tip:02de42196ebe draft book_02de
test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.
2014-11-12 19:54:57 +03:00
postclose-tip:02de42196ebe draft book_02de
pull: use transaction Summary: Without this change, the next diff will fail tests with metalog programming errors (write outside transaction). Some test changes are caused by the being deprecated `rollback` command. Reviewed By: DurhamG Differential Revision: D19380939 fbshipit-source-id: 7c893d3025bb697102835670b8a38f8fb9a624c8
2020-01-15 08:00:48 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_NODE=02de42196ebee42ef284b6780a87cdc96e8eaab6 HG_NODE_LAST=02de42196ebee42ef284b6780a87cdc96e8eaab6 HG_PHASES_MOVED=1 HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_TXNNAME=pull HG_URL=ssh://user@dummy/main
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
3333333333333333333333333333333333333333 eea13746799a9e0bfd88f29d3c2e9dc9389f524f 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
4444444444444444444444444444444444444444 02de42196ebee42ef284b6780a87cdc96e8eaab6 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
pull over http
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R main -p 0 --port-file $TESTTMP/.port -d --pid-file=main.pid -E main-error.log
$ HGPORT=`cat $TESTTMP/.port`
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cat main.pid >> $DAEMON_PIDS
$ hg -R other pull http://localhost:$HGPORT/ -r 42ccdea3bb16 --bookmark book_42cc
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pulling from http://localhost:$HGPORT/ (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
searching for changes
adding changesets
adding manifests
adding file changes
changegroup: do not show "(+/- ? heads)" Summary: In an upcoming change we'll replace `cl.heads()` with `repo.heads()`. In changegroup code path the repo can be in an inconsistent state (ex. a bookmark refers to an unknown commit) which breaks `repo.heads()`. Since the whole purpose of the heads calculation is just to show `(+/- ? heads)`, which does not affect any real features, let's just remove it. Reviewed By: markbt Differential Revision: D18366735 fbshipit-source-id: 893be2cec0c32b64a80b3ef4ca65b69f8ed76b27
2019-11-07 21:50:10 +03:00
added 1 changesets with 1 changes to 1 files
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
1 new obsolescence markers
updating bookmark book_42cc
phases: read pending data when appropriate If we are called by a hook and pending data exists, read those.
2014-10-18 09:23:06 +04:00
pre-close-tip:42ccdea3bb16 draft book_42cc
test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.
2014-11-12 19:54:57 +03:00
postclose-tip:42ccdea3bb16 draft book_42cc
pull: use transaction Summary: Without this change, the next diff will fail tests with metalog programming errors (write outside transaction). Some test changes are caused by the being deprecated `rollback` command. Reviewed By: DurhamG Differential Revision: D19380939 fbshipit-source-id: 7c893d3025bb697102835670b8a38f8fb9a624c8
2020-01-15 08:00:48 +03:00
txnclose hook: HG_BOOKMARK_MOVED=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_NODE=42ccdea3bb16d28e1848c95fe2e44c000f3f21b1 HG_NODE_LAST=42ccdea3bb16d28e1848c95fe2e44c000f3f21b1 HG_PHASES_MOVED=1 HG_SOURCE=pull HG_TXNID=TXN:$ID$ HG_TXNNAME=pull HG_URL=http://localhost:$HGPORT/ (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cat main-error.log
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
3333333333333333333333333333333333333333 eea13746799a9e0bfd88f29d3c2e9dc9389f524f 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
4444444444444444444444444444444444444444 02de42196ebee42ef284b6780a87cdc96e8eaab6 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
5555555555555555555555555555555555555555 42ccdea3bb16d28e1848c95fe2e44c000f3f21b1 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
push over ssh
$ hg -R main push ssh://user@dummy/other -r 5fddd98957c8 --bookmark book_5fdd
pushing to ssh://user@dummy/other
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: 1 new obsolescence markers
bookmark: read pending data when appropriate If we are called by a hook and pending data exists, read it.
2014-09-29 08:27:48 +04:00
remote: pre-close-tip:5fddd98957c8 draft book_5fdd
test-bundle2: check visible data in pre/post-transaction hooks We are about to make bookmarks and phases available for hooks. Therefore we need a witness for this new availability. We introduce the new hooks in a distinct changeset to reduce the noise in the ones with actual changes.
2014-11-12 19:54:57 +03:00
remote: postclose-tip:5fddd98957c8 draft book_5fdd
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
remote: txnclose hook: HG_BOOKMARK_MOVED=1 HG_BUNDLE2=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_NODE=5fddd98957c8a54a4d436dfe1da9d87f21a1b97b HG_NODE_LAST=5fddd98957c8a54a4d436dfe1da9d87f21a1b97b HG_SOURCE=serve HG_TXNID=TXN:$ID$ HG_TXNNAME=serve HG_URL=remote:ssh:$LOCALIP
bundle2: capture output issue during transaction close We were capturing all output issue during bundle2 processing, and all output issue during transaction rollback in case of failure. However, the output issue during transaction commit was still roaming the land freely. It is now put back in line. This let the user see output from 'pretxnclose' and 'txnclose' (and related) in the right order.
2015-04-23 16:54:45 +03:00
updating bookmark book_5fdd
push: elevate phase transaction to cover entire operation This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Most pushes already open a transaction in order to sync phase information. This diff replaces that transaction with one that spans the entire push operation. This transaction will be used in a later patch to guard repository changes made during the reply handler.
2014-11-22 02:06:38 +03:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_SOURCE=push-response HG_TXNID=TXN:$ID$ HG_TXNNAME=push-response
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
ssh://user@dummy/other HG_URL=ssh://user@dummy/other
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R other log -G
o 6:5fddd98957c8 draft Nicolas Dumazet <nicdumz.commits@gmail.com> book_5fdd C
|
o 5:42ccdea3bb16 draft Nicolas Dumazet <nicdumz.commits@gmail.com> book_42cc B
|
| o 4:02de42196ebe draft Nicolas Dumazet <nicdumz.commits@gmail.com> book_02de H
| |
| | o 3:eea13746799a public Nicolas Dumazet <nicdumz.commits@gmail.com> book_eea1 G
| |/|
| o | 2:24b6387c8c8c public Nicolas Dumazet <nicdumz.commits@gmail.com> F
|/ /
| @ 1:9520eea781bc public Nicolas Dumazet <nicdumz.commits@gmail.com> E
|/
o 0:cd010b8cd998 public Nicolas Dumazet <nicdumz.commits@gmail.com> book_32af A
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
3333333333333333333333333333333333333333 eea13746799a9e0bfd88f29d3c2e9dc9389f524f 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
4444444444444444444444444444444444444444 02de42196ebee42ef284b6780a87cdc96e8eaab6 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
5555555555555555555555555555555555555555 42ccdea3bb16d28e1848c95fe2e44c000f3f21b1 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
6666666666666666666666666666666666666666 5fddd98957c8a54a4d436dfe1da9d87f21a1b97b 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
push over http
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R other -p 0 --port-file $TESTTMP/.port -d --pid-file=other.pid -E other-error.log
$ HGPORT2=`cat $TESTTMP/.port`
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cat other.pid >> $DAEMON_PIDS
$ hg -R main phase --public 32af7686d403
changelog: fix readpending if no pending data exist (issue4609) Since transaction are used for more than just changesets, it is possible to have a transaction without new changesets at all. In this case no ''00changelog.i.a' are written. In all cases the 'changelog.readpending' method is called if the repository has any pending data. The 'revlog' logic provides empty content if the file is missing, so the whole operation resulted in an empty changelog. We now skip reading the pending file if it is missing.
2015-04-20 18:16:22 +03:00
pre-close-tip:02de42196ebe draft book_02de
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_PHASES_MOVED=1 HG_TXNID=TXN:$ID$ HG_TXNNAME=phase
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg -R main push http://localhost:$HGPORT2/ -r 32af7686d403 --bookmark book_32af
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: 1 new obsolescence markers
bundle2: capture output issue during transaction close We were capturing all output issue during bundle2 processing, and all output issue during transaction rollback in case of failure. However, the output issue during transaction commit was still roaming the land freely. It is now put back in line. This let the user see output from 'pretxnclose' and 'txnclose' (and related) in the right order.
2015-04-23 16:54:45 +03:00
remote: pre-close-tip:32af7686d403 public book_32af
remote: postclose-tip:32af7686d403 public book_32af
tests: fix missing (glob) annotations in test-bundle2-exchange.t
2017-04-04 01:55:55 +03:00
remote: txnclose hook: HG_BOOKMARK_MOVED=1 HG_BUNDLE2=1 HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_NEW_OBSMARKERS=1 HG_NODE=32af7686d403cf45b5d95f2d70cebea587ac806a HG_NODE_LAST=32af7686d403cf45b5d95f2d70cebea587ac806a HG_PHASES_MOVED=1 HG_SOURCE=serve HG_TXNID=TXN:$ID$ HG_TXNNAME=serve HG_URL=remote:http:$LOCALIP: (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
updating bookmark book_32af
push: elevate phase transaction to cover entire operation This patch series is intended to allow bundle2 push reply part handlers to make changes to the local repository; it has been developed in parallel with an extension that allows the server to rebase incoming changesets while applying them. Most pushes already open a transaction in order to sync phase information. This diff replaces that transaction with one that spans the entire push operation. This transaction will be used in a later patch to guard repository changes made during the reply handler.
2014-11-22 02:06:38 +03:00
pre-close-tip:02de42196ebe draft book_02de
postclose-tip:02de42196ebe draft book_02de
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_SOURCE=push-response HG_TXNID=TXN:$ID$ HG_TXNNAME=push-response
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
http://localhost:$HGPORT2/ HG_URL=http://localhost:$HGPORT2/ (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cat other-error.log
Check final content.
$ hg -R other log -G
o 7:32af7686d403 public Nicolas Dumazet <nicdumz.commits@gmail.com> book_32af D
|
o 6:5fddd98957c8 public Nicolas Dumazet <nicdumz.commits@gmail.com> book_5fdd C
|
o 5:42ccdea3bb16 public Nicolas Dumazet <nicdumz.commits@gmail.com> book_42cc B
|
| o 4:02de42196ebe draft Nicolas Dumazet <nicdumz.commits@gmail.com> book_02de H
| |
| | o 3:eea13746799a public Nicolas Dumazet <nicdumz.commits@gmail.com> book_eea1 G
| |/|
| o | 2:24b6387c8c8c public Nicolas Dumazet <nicdumz.commits@gmail.com> F
|/ /
| @ 1:9520eea781bc public Nicolas Dumazet <nicdumz.commits@gmail.com> E
|/
o 0:cd010b8cd998 public Nicolas Dumazet <nicdumz.commits@gmail.com> A
$ hg -R other debugobsolete
1111111111111111111111111111111111111111 9520eea781bcca16c1e15acc0ba14335a0e8e5ba 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
2222222222222222222222222222222222222222 24b6387c8c8cae37178880f3fa95ded3cb1cf785 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
3333333333333333333333333333333333333333 eea13746799a9e0bfd88f29d3c2e9dc9389f524f 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
4444444444444444444444444444444444444444 02de42196ebee42ef284b6780a87cdc96e8eaab6 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
5555555555555555555555555555555555555555 42ccdea3bb16d28e1848c95fe2e44c000f3f21b1 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
6666666666666666666666666666666666666666 5fddd98957c8a54a4d436dfe1da9d87f21a1b97b 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
7777777777777777777777777777777777777777 32af7686d403cf45b5d95f2d70cebea587ac806a 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
transaction: write pending generated files Such file are generated with a .pending prefix. It is up to the reader to implement the necessary logic for reading pending files. We add a test to ensure pending files are properly cleaned-up in both success and error cases.
2014-10-18 09:19:05 +04:00
(check that no 'pending' files remain)
$ ls -1 other/.hg/store/phaseroots*
other/.hg/store/phaseroots
$ ls -1 other/.hg/store/00changelog.i*
other/.hg/store/00changelog.i
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
Error Handling
==============
Check that errors are properly returned to the client during push.
Setting up
$ cat > failpush.py << EOF
> """A small extension that makes push fails when using bundle2
>
> used to test error handling in bundle2
> """
>
codemod: import from the edenscm package Summary: D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`. That has nasty problems if both `import mercurial` and `import edenscm.mercurial` are used, because Python would think `mercurial.foo` and `edenscm.mercurial.foo` are different modules so code like `try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)` would fail to handle the `edenscm.mercurial` version. There are also some module-level states (ex. `extensions._extensions`) that would cause trouble if they have multiple versions in a single process. Change imports to use the `edenscm` so ideally the `mercurial` is no longer imported at all. Add checks in extensions.py to catch unexpected extensions importing modules from the old (wrong) locations when running tests. Reviewed By: phillco Differential Revision: D13868981 fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
2019-01-30 03:25:33 +03:00
> from edenscm.mercurial import error
> from edenscm.mercurial import bundle2
> from edenscm.mercurial import exchange
> from edenscm.mercurial import extensions
> from edenscm.mercurial import registrar
configitems: register the test 'failpush.reason' config
2017-10-14 00:10:49 +03:00
> cmdtable = {}
> command = registrar.command(cmdtable)
>
> configtable = {}
> configitem = registrar.configitem(configtable)
> configitem('failpush', 'reason',
> default=None,
> )
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
>
> def _pushbundle2failpart(pushop, bundler):
configitems: register the test 'failpush.reason' config
2017-10-14 00:10:49 +03:00
> reason = pushop.ui.config('failpush', 'reason')
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
> part = None
> if reason == 'abort':
> bundler.newpart('test:abort')
> if reason == 'unknown':
bundle2: lowercase part types Since the capitalization no longer carries any meaning (previous diff), this patch normalizes all of the bundle2 part type strings to lower case.
2014-12-18 02:11:26 +03:00
> bundler.newpart('test:unknown')
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
>
> @bundle2.parthandler("test:abort")
> def handleabort(op, part):
error: get Abort from 'error' instead of 'util' The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be confused about that and gives all the credit to 'util' instead of the hardworking 'error'. In a spirit of equity, we break the cycle of injustice and give back to 'error' the respect it deserves. And screw that 'util' poser. For great justice.
2015-10-08 22:55:45 +03:00
> raise error.Abort('Abandon ship!', hint="don't panic")
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
>
> def uisetup(ui):
> exchange.b2partsgenmapping['failpart'] = _pushbundle2failpart
> exchange.b2partsgenorder.insert(0, 'failpart')
>
> EOF
$ cd main
$ hg up tip
3 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo 'I' > I
$ hg add I
$ hg ci -m 'I'
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
pre-close-tip:e7ec4e813ba6 draft
postclose-tip:e7ec4e813ba6 draft
hook: add hook name information to external hook While we are here, we can also add the hook name information to external hook.
2017-03-31 12:53:56 +03:00
txnclose hook: HG_HOOKNAME=txnclose.env HG_HOOKTYPE=txnclose HG_TXNID=TXN:$ID$ HG_TXNNAME=commit
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ hg id
tags: remove tip tag Summary: In a future diff we'll be removing tags. The most prevalent tag is 'tip', which shows up in a ton of test output. Let's drop that tag first, so we can safely update the tests before we drop tags entirely. Reviewed By: xavierd Differential Revision: D18995058 fbshipit-source-id: 8c63710cd4ed567ea24e32724b8660f9006a61f1
2019-12-18 00:45:17 +03:00
e7ec4e813ba6
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cd ..
$ cat << EOF >> $HGRCPATH
> [extensions]
> failpush=$TESTTMP/failpush.py
> EOF
tests: drop DAEMON_PIDS from killdaemons calls
2015-06-08 22:55:40 +03:00
$ killdaemons.py
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R other -p 0 --port-file $TESTTMP/.port -d --pid-file=other.pid -E other-error.log
$ HGPORT2=`cat $TESTTMP/.port`
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cat other.pid >> $DAEMON_PIDS
Doing the actual push: Abort error
$ cat << EOF >> $HGRCPATH
> [failpush]
> reason = abort
> EOF
$ hg -R main push other -r e7ec4e813ba6
pushing to other
searching for changes
abort: Abandon ship!
(don't panic)
[255]
$ hg -R main push ssh://user@dummy/other -r e7ec4e813ba6
pushing to ssh://user@dummy/other
searching for changes
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: Abandon ship!
bundle2: keep hint close to the primary message when remote abort The remote hint message was ignored when reporting the remote error and passed to the local generic abort error. I think I might initially have tried to avoid reimplementing logic controlling the hint display depending of the verbosity level. However, first, there does not seems to have such verbosity related logic and second the resulting was wrong as the primary error and the hint were split apart. We now properly print the hint as remote output.
2017-02-10 19:56:47 +03:00
remote: (don't panic)
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
abort: push failed on remote
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
[255]
$ hg -R main push http://localhost:$HGPORT2/ -r e7ec4e813ba6
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
searching for changes
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: Abandon ship!
bundle2: keep hint close to the primary message when remote abort The remote hint message was ignored when reporting the remote error and passed to the local generic abort error. I think I might initially have tried to avoid reimplementing logic controlling the hint display depending of the verbosity level. However, first, there does not seems to have such verbosity related logic and second the resulting was wrong as the primary error and the hint were split apart. We now properly print the hint as remote output.
2017-02-10 19:56:47 +03:00
remote: (don't panic)
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
abort: push failed on remote
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
[255]
Doing the actual push: unknown mandatory parts
$ cat << EOF >> $HGRCPATH
> [failpush]
> reason = unknown
> EOF
$ hg -R main push other -r e7ec4e813ba6
pushing to other
searching for changes
abort: missing support for test:unknown
[255]
$ hg -R main push ssh://user@dummy/other -r e7ec4e813ba6
pushing to ssh://user@dummy/other
searching for changes
abort: missing support for test:unknown
[255]
$ hg -R main push http://localhost:$HGPORT2/ -r e7ec4e813ba6
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
searching for changes
abort: missing support for test:unknown
[255]
Doing the actual push: hook abort
$ cat << EOF >> $HGRCPATH
> [failpush]
> reason =
> [hooks]
test-bundle2-exchange: make hooks compatible with Windows The cmd.exe process doesn't fail the hook when "; false" is appended, and its echo command prints out the quote characters.
2015-04-26 22:10:09 +03:00
> pretxnclose.failpush = sh -c "echo 'You shall not pass!'; false"
> txnabort.failpush = sh -c "echo 'Cleaning up the mess...'"
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
> EOF
tests: drop DAEMON_PIDS from killdaemons calls
2015-06-08 22:55:40 +03:00
$ killdaemons.py
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R other -p 0 --port-file $TESTTMP/.port -d --pid-file=other.pid -E other-error.log
$ HGPORT2=`cat $TESTTMP/.port`
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
$ cat other.pid >> $DAEMON_PIDS
$ hg -R main push other -r e7ec4e813ba6
pushing to other
searching for changes
bundle2-localpeer: properly propagate the server output on error (issue4594) In case of errors, output parts salvaged from the reply bundle need to be processed for outputting their content. This concludes our quest for fixing issue4594.
2015-04-16 11:01:51 +03:00
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
bundle2: capture output issue during transaction close We were capturing all output issue during bundle2 processing, and all output issue during transaction rollback in case of failure. However, the output issue during transaction commit was still roaming the land freely. It is now put back in line. This let the user see output from 'pretxnclose' and 'txnclose' (and related) in the right order.
2015-04-23 16:54:45 +03:00
remote: pre-close-tip:e7ec4e813ba6 draft
remote: You shall not pass!
bundle2: capture transaction rollback message output (issue4614) The output from the transaction rollback was not included into the reply bundle. It was eventually caught by the usual 'unbundle' output capture and sent to the client but the result was out of order on the client side. We now capture the output for the transaction release and transmit it the same way as all other output. We should probably rethink the whole output capture things but this would not be appropriate for stable. The is still multiple cases were output failed to be properly capture, they will be fixed in later changesets.
2015-04-23 16:20:36 +03:00
remote: transaction abort!
bundle2: also capture hook output during processing External hook used to directly write on stdout and stderr. As a result their output was not captured by the bundle2 processing. This resulted in confusing out of order output on the client side. We are now capturing hooks output in this context.
2015-04-23 19:03:58 +03:00
remote: Cleaning up the mess...
bundle2: capture transaction rollback message output (issue4614) The output from the transaction rollback was not included into the reply bundle. It was eventually caught by the usual 'unbundle' output capture and sent to the client but the result was out of order on the client side. We now capture the output for the transaction release and transmit it the same way as all other output. We should probably rethink the whole output capture things but this would not be appropriate for stable. The is still multiple cases were output failed to be properly capture, they will be fixed in later changesets.
2015-04-23 16:20:36 +03:00
remote: rollback completed
bundle2: test hooking using the new transaction-level hook There is no strong reason to keep a bundle2-level hook as we can use the transaction-level hook.
2015-03-12 03:53:50 +03:00
abort: pretxnclose.failpush hook exited with status 1
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
[255]
$ hg -R main push ssh://user@dummy/other -r e7ec4e813ba6
pushing to ssh://user@dummy/other
searching for changes
bundle2-wireproto: properly propagate the server output on error (issue4594) In case of errors, output parts salvaged from the reply bundle are re-injected into the bundle carrying the exception. We still need to fix the situation for non-wireprotocol push.
2015-04-16 10:17:37 +03:00
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
bundle2: capture output issue during transaction close We were capturing all output issue during bundle2 processing, and all output issue during transaction rollback in case of failure. However, the output issue during transaction commit was still roaming the land freely. It is now put back in line. This let the user see output from 'pretxnclose' and 'txnclose' (and related) in the right order.
2015-04-23 16:54:45 +03:00
remote: pre-close-tip:e7ec4e813ba6 draft
remote: You shall not pass!
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
remote: transaction abort!
bundle2: also capture hook output during processing External hook used to directly write on stdout and stderr. As a result their output was not captured by the bundle2 processing. This resulted in confusing out of order output on the client side. We are now capturing hooks output in this context.
2015-04-23 19:03:58 +03:00
remote: Cleaning up the mess...
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
remote: rollback completed
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: pretxnclose.failpush hook exited with status 1
abort: push failed on remote
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
[255]
$ hg -R main push http://localhost:$HGPORT2/ -r e7ec4e813ba6
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
searching for changes
bundle2-wireproto: properly propagate the server output on error (issue4594) In case of errors, output parts salvaged from the reply bundle are re-injected into the bundle carrying the exception. We still need to fix the situation for non-wireprotocol push.
2015-04-16 10:17:37 +03:00
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
bundle2: capture output issue during transaction close We were capturing all output issue during bundle2 processing, and all output issue during transaction rollback in case of failure. However, the output issue during transaction commit was still roaming the land freely. It is now put back in line. This let the user see output from 'pretxnclose' and 'txnclose' (and related) in the right order.
2015-04-23 16:54:45 +03:00
remote: pre-close-tip:e7ec4e813ba6 draft
remote: You shall not pass!
bundle2: capture transaction rollback message output (issue4614) The output from the transaction rollback was not included into the reply bundle. It was eventually caught by the usual 'unbundle' output capture and sent to the client but the result was out of order on the client side. We now capture the output for the transaction release and transmit it the same way as all other output. We should probably rethink the whole output capture things but this would not be appropriate for stable. The is still multiple cases were output failed to be properly capture, they will be fixed in later changesets.
2015-04-23 16:20:36 +03:00
remote: transaction abort!
bundle2: also capture hook output during processing External hook used to directly write on stdout and stderr. As a result their output was not captured by the bundle2 processing. This resulted in confusing out of order output on the client side. We are now capturing hooks output in this context.
2015-04-23 19:03:58 +03:00
remote: Cleaning up the mess...
bundle2: capture transaction rollback message output (issue4614) The output from the transaction rollback was not included into the reply bundle. It was eventually caught by the usual 'unbundle' output capture and sent to the client but the result was out of order on the client side. We now capture the output for the transaction release and transmit it the same way as all other output. We should probably rethink the whole output capture things but this would not be appropriate for stable. The is still multiple cases were output failed to be properly capture, they will be fixed in later changesets.
2015-04-23 16:20:36 +03:00
remote: rollback completed
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: pretxnclose.failpush hook exited with status 1
abort: push failed on remote
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
[255]
transaction: write pending generated files Such file are generated with a .pending prefix. It is up to the reader to implement the necessary logic for reading pending files. We add a test to ensure pending files are properly cleaned-up in both success and error cases.
2014-10-18 09:19:05 +04:00
(check that no 'pending' files remain)
$ ls -1 other/.hg/store/phaseroots*
other/.hg/store/phaseroots
$ ls -1 other/.hg/store/00changelog.i*
other/.hg/store/00changelog.i
bundle2: split test in two We split the test between the one dedicated to the binary format and the one dedicated to checking the exchange of data using pull and push.
2014-10-02 08:55:22 +04:00
bundle2: also save output when error happens during part processing Until this changeset, we were only able to save output if an error happened during the 'transaction.close()' phase. If the 'processbundle' call raised an exception, the 'bundleoperation' object was never returned, so the reply bundle was never accessible and no output could be salvaged. We introduce a quick (but not very elegant) fix to gain access to any reply created during the processing. This conclude this output related series. We should hopefully be able client-side to see the whole server output, in a proper order. The code is now complex enough that a refactoring of it would make sense on default.
2015-04-23 18:36:18 +03:00
Check error from hook during the unbundling process itself
$ cat << EOF >> $HGRCPATH
test-bundle2-exchange: make hooks compatible with Windows The cmd.exe process doesn't fail the hook when "; false" is appended, and its echo command prints out the quote characters.
2015-04-26 22:10:09 +03:00
> pretxnchangegroup = sh -c "echo 'Fail early!'; false"
bundle2: also save output when error happens during part processing Until this changeset, we were only able to save output if an error happened during the 'transaction.close()' phase. If the 'processbundle' call raised an exception, the 'bundleoperation' object was never returned, so the reply bundle was never accessible and no output could be salvaged. We introduce a quick (but not very elegant) fix to gain access to any reply created during the processing. This conclude this output related series. We should hopefully be able client-side to see the whole server output, in a proper order. The code is now complex enough that a refactoring of it would make sense on default.
2015-04-23 18:36:18 +03:00
> EOF
tests: drop DAEMON_PIDS from killdaemons calls
2015-06-08 22:55:40 +03:00
$ killdaemons.py # reload http config
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R other -p 0 --port-file $TESTTMP/.port -d --pid-file=other.pid -E other-error.log
$ HGPORT2=`cat $TESTTMP/.port`
bundle2: also save output when error happens during part processing Until this changeset, we were only able to save output if an error happened during the 'transaction.close()' phase. If the 'processbundle' call raised an exception, the 'bundleoperation' object was never returned, so the reply bundle was never accessible and no output could be salvaged. We introduce a quick (but not very elegant) fix to gain access to any reply created during the processing. This conclude this output related series. We should hopefully be able client-side to see the whole server output, in a proper order. The code is now complex enough that a refactoring of it would make sense on default.
2015-04-23 18:36:18 +03:00
$ cat other.pid >> $DAEMON_PIDS
$ hg -R main push other -r e7ec4e813ba6
pushing to other
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: Fail early!
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
abort: pretxnchangegroup hook exited with status 1
[255]
$ hg -R main push ssh://user@dummy/other -r e7ec4e813ba6
pushing to ssh://user@dummy/other
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: Fail early!
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: pretxnchangegroup hook exited with status 1
abort: push failed on remote
bundle2: also save output when error happens during part processing Until this changeset, we were only able to save output if an error happened during the 'transaction.close()' phase. If the 'processbundle' call raised an exception, the 'bundleoperation' object was never returned, so the reply bundle was never accessible and no output could be salvaged. We introduce a quick (but not very elegant) fix to gain access to any reply created during the processing. This conclude this output related series. We should hopefully be able client-side to see the whole server output, in a proper order. The code is now complex enough that a refactoring of it would make sense on default.
2015-04-23 18:36:18 +03:00
[255]
$ hg -R main push http://localhost:$HGPORT2/ -r e7ec4e813ba6
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: also save output when error happens during part processing Until this changeset, we were only able to save output if an error happened during the 'transaction.close()' phase. If the 'processbundle' call raised an exception, the 'bundleoperation' object was never returned, so the reply bundle was never accessible and no output could be salvaged. We introduce a quick (but not very elegant) fix to gain access to any reply created during the processing. This conclude this output related series. We should hopefully be able client-side to see the whole server output, in a proper order. The code is now complex enough that a refactoring of it would make sense on default.
2015-04-23 18:36:18 +03:00
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: Fail early!
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: pretxnchangegroup hook exited with status 1
abort: push failed on remote
bundle2: also save output when error happens during part processing Until this changeset, we were only able to save output if an error happened during the 'transaction.close()' phase. If the 'processbundle' call raised an exception, the 'bundleoperation' object was never returned, so the reply bundle was never accessible and no output could be salvaged. We introduce a quick (but not very elegant) fix to gain access to any reply created during the processing. This conclude this output related series. We should hopefully be able client-side to see the whole server output, in a proper order. The code is now complex enough that a refactoring of it would make sense on default.
2015-04-23 18:36:18 +03:00
[255]
bundle2: disable ouput capture unless we use http (issue4613 issue4615) The current bundle2 processing was capturing all output. This is nice as it provide better meta data about what output what, but this was changing two things: 1) adding a prefix "remote: " to "other" output during local push (issue4613) 2) local and ssh push does not provide real time output anymore (issue4615) As we are unsure about what form should be used in (1) and how to solve (2) we disable output capture in this two cases. Output capture can be forced using an experimental option.
2015-04-29 03:38:02 +03:00
Check output capture control.
(should be still forced for http, disabled for local and ssh)
$ cat >> $HGRCPATH << EOF
> [experimental]
> bundle2-output-capture=False
> EOF
$ hg -R main push other -r e7ec4e813ba6
pushing to other
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
Fail early!
transaction abort!
Cleaning up the mess...
rollback completed
abort: pretxnchangegroup hook exited with status 1
[255]
$ hg -R main push ssh://user@dummy/other -r e7ec4e813ba6
pushing to ssh://user@dummy/other
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: Fail early!
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: pretxnchangegroup hook exited with status 1
abort: push failed on remote
bundle2: disable ouput capture unless we use http (issue4613 issue4615) The current bundle2 processing was capturing all output. This is nice as it provide better meta data about what output what, but this was changing two things: 1) adding a prefix "remote: " to "other" output during local push (issue4613) 2) local and ssh push does not provide real time output anymore (issue4615) As we are unsure about what form should be used in (1) and how to solve (2) we disable output capture in this two cases. Output capture can be forced using an experimental option.
2015-04-29 03:38:02 +03:00
[255]
$ hg -R main push http://localhost:$HGPORT2/ -r e7ec4e813ba6
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: disable ouput capture unless we use http (issue4613 issue4615) The current bundle2 processing was capturing all output. This is nice as it provide better meta data about what output what, but this was changing two things: 1) adding a prefix "remote: " to "other" output during local push (issue4613) 2) local and ssh push does not provide real time output anymore (issue4615) As we are unsure about what form should be used in (1) and how to solve (2) we disable output capture in this two cases. Output capture can be forced using an experimental option.
2015-04-29 03:38:02 +03:00
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: Fail early!
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: pretxnchangegroup hook exited with status 1
abort: push failed on remote
bundle2: disable ouput capture unless we use http (issue4613 issue4615) The current bundle2 processing was capturing all output. This is nice as it provide better meta data about what output what, but this was changing two things: 1) adding a prefix "remote: " to "other" output during local push (issue4613) 2) local and ssh push does not provide real time output anymore (issue4615) As we are unsure about what form should be used in (1) and how to solve (2) we disable output capture in this two cases. Output capture can be forced using an experimental option.
2015-04-29 03:38:02 +03:00
[255]
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
Check abort from mandatory pushkey
$ cat > mandatorypart.py << EOF
codemod: import from the edenscm package Summary: D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`. That has nasty problems if both `import mercurial` and `import edenscm.mercurial` are used, because Python would think `mercurial.foo` and `edenscm.mercurial.foo` are different modules so code like `try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)` would fail to handle the `edenscm.mercurial` version. There are also some module-level states (ex. `extensions._extensions`) that would cause trouble if they have multiple versions in a single process. Change imports to use the `edenscm` so ideally the `mercurial` is no longer imported at all. Add checks in extensions.py to catch unexpected extensions importing modules from the old (wrong) locations when running tests. Reviewed By: phillco Differential Revision: D13868981 fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
2019-01-30 03:25:33 +03:00
> from edenscm.mercurial import exchange
> from edenscm.mercurial import pushkey
> from edenscm.mercurial import node
> from edenscm.mercurial import error
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
> @exchange.b2partsgenerator('failingpuskey')
> def addfailingpushey(pushop, bundler):
> part = bundler.newpart('pushkey')
py3: fix pushkey encoding by removing local encoding Summary: Now that bundle2 has strings everywhere, pushkey needs to also handle encoding correctly. Previously it used local encoding, which seems strange. Let's drop the encoding entirely since we're only dealing with since we're only dealing with utf8 string and bytes. Reviewed By: farnz Differential Revision: D19665894 fbshipit-source-id: ed054ab25636ac52de646329f26681ae843b6de3
2020-02-01 00:09:53 +03:00
> part.addparam('namespace', 'phases')
> part.addparam('key', pushop.repo['cd010b8cd998'].hex())
> part.addparam('old', str(0)) # successful update
> part.addparam('new', str(0))
push: catch and process PushkeyFailed error We add a way to register "pushkey failure callback" that will be used if the push is aborted by a pushkey failure. A part generator adding mandatory pushkey parts should register a failure callback for all of them. The callback will be in charge of generating a meaningful abort if this part fails. If no callback is registered, the error is propagated. Catch PushkeyFailed error in exchange.
2015-06-06 02:30:11 +03:00
> def fail(pushop, exc):
> raise error.Abort('Correct phase push failed (because hooks)')
> pushop.pkfailcb[part.id] = fail
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
> EOF
$ cat >> $HGRCPATH << EOF
> [hooks]
> pretxnchangegroup=
> pretxnclose.failpush=
> prepushkey.failpush = sh -c "echo 'do not push the key !'; false"
> [extensions]
> mandatorypart=$TESTTMP/mandatorypart.py
> EOF
$ "$TESTDIR/killdaemons.py" $DAEMON_PIDS # reload http config
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R other -p 0 --port-file $TESTTMP/.port -d --pid-file=other.pid -E other-error.log
$ HGPORT2=`cat $TESTTMP/.port`
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
$ cat other.pid >> $DAEMON_PIDS
(Failure from a hook)
$ hg -R main push other -r e7ec4e813ba6
pushing to other
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
do not push the key !
pushkey-abort: prepushkey.failpush hook exited with status 1
transaction abort!
Cleaning up the mess...
rollback completed
push: catch and process PushkeyFailed error We add a way to register "pushkey failure callback" that will be used if the push is aborted by a pushkey failure. A part generator adding mandatory pushkey parts should register a failure callback for all of them. The callback will be in charge of generating a meaningful abort if this part fails. If no callback is registered, the error is propagated. Catch PushkeyFailed error in exchange.
2015-06-06 02:30:11 +03:00
abort: Correct phase push failed (because hooks)
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
[255]
$ hg -R main push ssh://user@dummy/other -r e7ec4e813ba6
pushing to ssh://user@dummy/other
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: do not push the key !
remote: pushkey-abort: prepushkey.failpush hook exited with status 1
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
bundle2: convey PushkeyFailed error over the wire We add a way to convey the precise exception. This will allow better error message on the server.
2015-06-10 20:10:53 +03:00
abort: Correct phase push failed (because hooks)
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
[255]
$ hg -R main push http://localhost:$HGPORT2/ -r e7ec4e813ba6
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: do not push the key !
remote: pushkey-abort: prepushkey.failpush hook exited with status 1
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
bundle2: convey PushkeyFailed error over the wire We add a way to convey the precise exception. This will allow better error message on the server.
2015-06-10 20:10:53 +03:00
abort: Correct phase push failed (because hooks)
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
[255]
(Failure from a the pushkey)
$ cat > mandatorypart.py << EOF
codemod: import from the edenscm package Summary: D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`. That has nasty problems if both `import mercurial` and `import edenscm.mercurial` are used, because Python would think `mercurial.foo` and `edenscm.mercurial.foo` are different modules so code like `try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)` would fail to handle the `edenscm.mercurial` version. There are also some module-level states (ex. `extensions._extensions`) that would cause trouble if they have multiple versions in a single process. Change imports to use the `edenscm` so ideally the `mercurial` is no longer imported at all. Add checks in extensions.py to catch unexpected extensions importing modules from the old (wrong) locations when running tests. Reviewed By: phillco Differential Revision: D13868981 fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
2019-01-30 03:25:33 +03:00
> from edenscm.mercurial import exchange
> from edenscm.mercurial import pushkey
> from edenscm.mercurial import node
> from edenscm.mercurial import error
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
> @exchange.b2partsgenerator('failingpuskey')
> def addfailingpushey(pushop, bundler):
> part = bundler.newpart('pushkey')
py3: fix pushkey encoding by removing local encoding Summary: Now that bundle2 has strings everywhere, pushkey needs to also handle encoding correctly. Previously it used local encoding, which seems strange. Let's drop the encoding entirely since we're only dealing with since we're only dealing with utf8 string and bytes. Reviewed By: farnz Differential Revision: D19665894 fbshipit-source-id: ed054ab25636ac52de646329f26681ae843b6de3
2020-02-01 00:09:53 +03:00
> part.addparam('namespace', 'phases')
> part.addparam('key', pushop.repo['cd010b8cd998'].hex())
> part.addparam('old', str(4)) # will fail
> part.addparam('new', str(3))
push: catch and process PushkeyFailed error We add a way to register "pushkey failure callback" that will be used if the push is aborted by a pushkey failure. A part generator adding mandatory pushkey parts should register a failure callback for all of them. The callback will be in charge of generating a meaningful abort if this part fails. If no callback is registered, the error is propagated. Catch PushkeyFailed error in exchange.
2015-06-06 02:30:11 +03:00
> def fail(pushop, exc):
> raise error.Abort('Clown phase push failed')
> pushop.pkfailcb[part.id] = fail
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
> EOF
$ cat >> $HGRCPATH << EOF
> [hooks]
> prepushkey.failpush =
> EOF
$ "$TESTDIR/killdaemons.py" $DAEMON_PIDS # reload http config
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R other -p 0 --port-file $TESTTMP/.port -d --pid-file=other.pid -E other-error.log
$ HGPORT2=`cat $TESTTMP/.port`
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
$ cat other.pid >> $DAEMON_PIDS
$ hg -R main push other -r e7ec4e813ba6
pushing to other
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
transaction abort!
Cleaning up the mess...
rollback completed
pushkey: lock state after "phases"
locks: make sure debuglocks knows about malformed locks and undolog lock Summary: Let's report that this lock cannot be read by Mercurial Reviewed By: markbt Differential Revision: D7653196 fbshipit-source-id: c5b7889cdde9c0ecc03a8c961aeba92f426648b1
2018-04-18 03:16:48 +03:00
lock: free
wlock: free
debuglock: also report infinitepushbackup and prefetch locks Summary: To be able to investigate problems with dangling lock files "infinitepushbackup" and "prefetch". - "infinitepushbackup", file "infinitepushbackup.lock" in repo.sharedvfs - "prefetchlock", file "prefetchlock" in repo.svfs Also some small fixes around. - Crate "undolog" directory if it doesn't exist. - `report` function was restructed. Now if lock checking method is not passed lock will be checked with `lockmod.lock`. Reviewed By: markbt Differential Revision: D14949807 fbshipit-source-id: 02143ff923145e67e88c5627cf3355a834823b6d
2019-04-29 14:31:28 +03:00
undolog/lock: absent
prefetchlock: free
infinitepushbackup.lock: free
push: catch and process PushkeyFailed error We add a way to register "pushkey failure callback" that will be used if the push is aborted by a pushkey failure. A part generator adding mandatory pushkey parts should register a failure callback for all of them. The callback will be in charge of generating a meaningful abort if this part fails. If no callback is registered, the error is propagated. Catch PushkeyFailed error in exchange.
2015-06-06 02:30:11 +03:00
abort: Clown phase push failed
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
[255]
$ hg -R main push ssh://user@dummy/other -r e7ec4e813ba6
pushing to ssh://user@dummy/other
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
remote: pushkey: lock state after "phases"
locks: make sure debuglocks knows about malformed locks and undolog lock Summary: Let's report that this lock cannot be read by Mercurial Reviewed By: markbt Differential Revision: D7653196 fbshipit-source-id: c5b7889cdde9c0ecc03a8c961aeba92f426648b1
2018-04-18 03:16:48 +03:00
remote: lock: free
remote: wlock: free
debuglock: also report infinitepushbackup and prefetch locks Summary: To be able to investigate problems with dangling lock files "infinitepushbackup" and "prefetch". - "infinitepushbackup", file "infinitepushbackup.lock" in repo.sharedvfs - "prefetchlock", file "prefetchlock" in repo.svfs Also some small fixes around. - Crate "undolog" directory if it doesn't exist. - `report` function was restructed. Now if lock checking method is not passed lock will be checked with `lockmod.lock`. Reviewed By: markbt Differential Revision: D14949807 fbshipit-source-id: 02143ff923145e67e88c5627cf3355a834823b6d
2019-04-29 14:31:28 +03:00
remote: undolog/lock: absent
remote: prefetchlock: free
remote: infinitepushbackup.lock: free
bundle2: convey PushkeyFailed error over the wire We add a way to convey the precise exception. This will allow better error message on the server.
2015-06-10 20:10:53 +03:00
abort: Clown phase push failed
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
[255]
$ hg -R main push http://localhost:$HGPORT2/ -r e7ec4e813ba6
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT2/ (glob)
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: transaction abort!
remote: Cleaning up the mess...
remote: rollback completed
remote: pushkey: lock state after "phases"
locks: make sure debuglocks knows about malformed locks and undolog lock Summary: Let's report that this lock cannot be read by Mercurial Reviewed By: markbt Differential Revision: D7653196 fbshipit-source-id: c5b7889cdde9c0ecc03a8c961aeba92f426648b1
2018-04-18 03:16:48 +03:00
remote: lock: free
remote: wlock: free
debuglock: also report infinitepushbackup and prefetch locks Summary: To be able to investigate problems with dangling lock files "infinitepushbackup" and "prefetch". - "infinitepushbackup", file "infinitepushbackup.lock" in repo.sharedvfs - "prefetchlock", file "prefetchlock" in repo.svfs Also some small fixes around. - Crate "undolog" directory if it doesn't exist. - `report` function was restructed. Now if lock checking method is not passed lock will be checked with `lockmod.lock`. Reviewed By: markbt Differential Revision: D14949807 fbshipit-source-id: 02143ff923145e67e88c5627cf3355a834823b6d
2019-04-29 14:31:28 +03:00
remote: undolog/lock: absent
remote: prefetchlock: free
remote: infinitepushbackup.lock: free
bundle2: convey PushkeyFailed error over the wire We add a way to convey the precise exception. This will allow better error message on the server.
2015-06-10 20:10:53 +03:00
abort: Clown phase push failed
bundle2: abort when a mandatory pushkey part fails So far, result of a pushkey operation had no consequence on the transaction (beside the change). We makes it respect the 'mandatory' flag of part so that failed pushkey call abort the whole transaction. This will allow rejecting changes (primary target: changesets) regarding phases or bookmark criteria in the future (when we will push such data in a mandatory part). We currently raise an abort error because all clients support it. We'll introduce a more precise error in the next changesets.
2015-05-27 15:28:40 +03:00
[255]
bundle2: allow lazily acquiring the lock In the external pushrebase extension, it is valuable to be able to do some work without taking the lock (like running expensive hooks). This enables significantly higher commit throughput. This patch adds an option to lazily acquire the lock. It means that all bundle2 part handlers that require writing to the repo must first call op.gettransction(), when in this mode.
2015-10-06 02:19:54 +03:00
Test lazily acquiring the lock during unbundle
$ cp $TESTTMP/hgrc.orig $HGRCPATH
$ cat >> $HGRCPATH <<EOF
> [ui]
tests: quote $PYTHON for Windows When unquoted, MSYS sees the colon between the drive letter and path as a Unix path separator and unhelpfully splits on it, feeding only the drive letter as the command. Much chaos ensues. I vaguely remember trying to get the test runner to use /letter/path/to/exe syntax the last time this happened, without success. I doubt a check-code rule would work, since sometimes it is quoted, and sometimes the quotes are escaped.
2017-07-07 08:05:20 +03:00
> ssh="$PYTHON" "$TESTDIR/dummyssh"
bundle2: allow lazily acquiring the lock In the external pushrebase extension, it is valuable to be able to do some work without taking the lock (like running expensive hooks). This enables significantly higher commit throughput. This patch adds an option to lazily acquire the lock. It means that all bundle2 part handlers that require writing to the repo must first call op.gettransction(), when in this mode.
2015-10-06 02:19:54 +03:00
> EOF
$ cat >> $TESTTMP/locktester.py <<EOF
> import os
codemod: import from the edenscm package Summary: D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`. That has nasty problems if both `import mercurial` and `import edenscm.mercurial` are used, because Python would think `mercurial.foo` and `edenscm.mercurial.foo` are different modules so code like `try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)` would fail to handle the `edenscm.mercurial` version. There are also some module-level states (ex. `extensions._extensions`) that would cause trouble if they have multiple versions in a single process. Change imports to use the `edenscm` so ideally the `mercurial` is no longer imported at all. Add checks in extensions.py to catch unexpected extensions importing modules from the old (wrong) locations when running tests. Reviewed By: phillco Differential Revision: D13868981 fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
2019-01-30 03:25:33 +03:00
> from edenscm.mercurial import extensions, bundle2, util
bundle2: allow lazily acquiring the lock In the external pushrebase extension, it is valuable to be able to do some work without taking the lock (like running expensive hooks). This enables significantly higher commit throughput. This patch adds an option to lazily acquire the lock. It means that all bundle2 part handlers that require writing to the repo must first call op.gettransction(), when in this mode.
2015-10-06 02:19:54 +03:00
> def checklock(orig, repo, *args, **kwargs):
> if repo.svfs.lexists("lock"):
> raise util.Abort("Lock should not be taken")
> return orig(repo, *args, **kwargs)
> def extsetup(ui):
> extensions.wrapfunction(bundle2, 'processbundle', checklock)
> EOF
$ hg init lazylock
$ cat >> lazylock/.hg/hgrc <<EOF
> [extensions]
> locktester=$TESTTMP/locktester.py
> EOF
$ hg clone -q ssh://user@dummy/lazylock lazylockclient
$ cd lazylockclient
$ touch a && hg ci -Aqm a
$ hg push
pushing to ssh://user@dummy/lazylock
searching for changes
bundle2: attribute remote failures to remote (issue4788) Before bundle2, hook output from hook failures was prefixed with "remote: ". Up to this point with bundle2, the output was converted to the message to print in an Abort exception. This had 2 implications: 1) It was unclear whether an error message came from the local repo or the remote 2) The exit code changed from 1 to 255 This patch changes the handling of error:abort bundle2 parts during push to prefix the error message with "remote: ". This restores the old behavior. We still preserve the behavior of raising an Abort during bundle2 application failure. This is a regression from pre-bundle2 because the exit code changed. Because we no longer raise an Abort with the remote's message, we needed to insert a message for the new Abort. So, I invented a new error message for that. This is another change from pre-bundle2. However, I like the new error message because it states unambiguously who aborted the push failed, which I think is important for users so they can decide what's next.
2015-10-24 02:39:22 +03:00
remote: Lock should not be taken
abort: push failed on remote
bundle2: allow lazily acquiring the lock In the external pushrebase extension, it is valuable to be able to do some work without taking the lock (like running expensive hooks). This enables significantly higher commit throughput. This patch adds an option to lazily acquire the lock. It means that all bundle2 part handlers that require writing to the repo must first call op.gettransction(), when in this mode.
2015-10-06 02:19:54 +03:00
[255]
$ cat >> ../lazylock/.hg/hgrc <<EOF
> [experimental]
> bundle2lazylocking=True
> EOF
$ hg push
pushing to ssh://user@dummy/lazylock
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
$ cd ..
Servers can disable bundle1 for clone/pull operations
$ killdaemons.py
$ hg init bundle2onlyserver
$ cd bundle2onlyserver
$ cat > .hg/hgrc << EOF
> [server]
> bundle1.pull = false
> EOF
$ touch foo
$ hg -q commit -A -m initial
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -p 0 --port-file $TESTTMP/.port -d --pid-file=hg.pid
$ HGPORT=`cat $TESTTMP/.port`
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
$ cat hg.pid >> $DAEMON_PIDS
tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.
2016-08-02 16:23:03 +03:00
$ hg --config devel.legacy.exchange=bundle1 clone http://localhost:$HGPORT/ not-bundle2
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
requesting all changes
abort: remote error:
incompatible Mercurial client; bundle2 required
(see https://www.mercurial-scm.org/wiki/IncompatibleClient)
[255]
$ killdaemons.py
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
$ cd ..
bundle1 can still pull non-generaldelta repos when generaldelta bundle1 disabled
$ hg --config format.usegeneraldelta=false init notgdserver
$ cd notgdserver
$ cat > .hg/hgrc << EOF
> [server]
> bundle1gd.pull = false
hg: remove bundlev1 from the supported outgoing versions Summary: Removes bundlev1 from the supported outgoing versions, but keeps a flag around to force enable it if tests need it. Reviewed By: quark-zju Differential Revision: D7591176 fbshipit-source-id: 280cbbbe87848e3d6c9d448ce4f87c5eadeff720
2018-04-12 02:04:53 +03:00
> [format]
> allowbundle1=True
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
> EOF
$ touch foo
$ hg -q commit -A -m initial
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -p 0 --port-file $TESTTMP/.port -d --pid-file=hg.pid
$ HGPORT=`cat $TESTTMP/.port`
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
$ cat hg.pid >> $DAEMON_PIDS
tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.
2016-08-02 16:23:03 +03:00
$ hg --config devel.legacy.exchange=bundle1 clone http://localhost:$HGPORT/ not-bundle2-1
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ killdaemons.py
$ cd ../bundle2onlyserver
bundle1 pull can be disabled for generaldelta repos only
$ cat > .hg/hgrc << EOF
> [server]
> bundle1gd.pull = false
hg: remove bundlev1 from the supported outgoing versions Summary: Removes bundlev1 from the supported outgoing versions, but keeps a flag around to force enable it if tests need it. Reviewed By: quark-zju Differential Revision: D7591176 fbshipit-source-id: 280cbbbe87848e3d6c9d448ce4f87c5eadeff720
2018-04-12 02:04:53 +03:00
> [format]
> allowbundle1=True
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
> EOF
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -p 0 --port-file $TESTTMP/.port -d --pid-file=hg.pid
$ HGPORT=`cat $TESTTMP/.port`
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
$ cat hg.pid >> $DAEMON_PIDS
tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.
2016-08-02 16:23:03 +03:00
$ hg --config devel.legacy.exchange=bundle1 clone http://localhost:$HGPORT/ not-bundle2
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
requesting all changes
abort: remote error:
incompatible Mercurial client; bundle2 required
(see https://www.mercurial-scm.org/wiki/IncompatibleClient)
[255]
$ killdaemons.py
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
Verify the global server.bundle1 option works
bundle-tests: operate from outside a repository We are about to add a test for ssh pull/cloning being denied because of bundle1 usage. For this, it is cleaner to not operate from the clone using http. So we update the test beforehand for clarity. This is more churns that what I'm happy to see on stable, but the rests of the series is worth it in my opinion.
2017-02-10 20:06:12 +03:00
$ cd ..
$ cat > bundle2onlyserver/.hg/hgrc << EOF
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
> [server]
> bundle1 = false
hg: remove bundlev1 from the supported outgoing versions Summary: Removes bundlev1 from the supported outgoing versions, but keeps a flag around to force enable it if tests need it. Reviewed By: quark-zju Differential Revision: D7591176 fbshipit-source-id: 280cbbbe87848e3d6c9d448ce4f87c5eadeff720
2018-04-12 02:04:53 +03:00
> [format]
> allowbundle1=True
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
> EOF
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R bundle2onlyserver -p 0 --port-file $TESTTMP/.port -d --pid-file=hg.pid
$ HGPORT=`cat $TESTTMP/.port`
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
$ cat hg.pid >> $DAEMON_PIDS
tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.
2016-08-02 16:23:03 +03:00
$ hg --config devel.legacy.exchange=bundle1 clone http://localhost:$HGPORT not-bundle2
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
requesting all changes
abort: remote error:
incompatible Mercurial client; bundle2 required
(see https://www.mercurial-scm.org/wiki/IncompatibleClient)
[255]
$ killdaemons.py
tests_: simplify bundle2 requirement testing Differential Revision: https://phabricator.intern.facebook.com/D4599415 (grafted from 35efb60b54b45e4bf8a281da35377ceabd9a2d0d) (grafted from c976227665e29759c3bfd62a78b653815999f33c) (grafted from ce81c51c90c6a7e67377b8368ff056be747ba808) (grafted from f8ebd7782719ffb9f6c9bbdc740af4d628bc4c38) (grafted from b683d8dc9677c2e4f24395f2ae849129bd4d6c81) (grafted from 98300e5ba307c80f257e9d28b3afa7ef30d9a109) (grafted from 6baa745cb3ea1a59c5e194b28d1db7e6a14da8fa) (grafted from 0bd1e681e370164b2b58ed6329dcceab8a45ba1a) (grafted from b88e051e45b3b34eebfe68b579d40e0848dd9164) (grafted from d7d3ea27b8e834ca6a7ee86d67109f68dff8b627) (grafted from 07f80547629b090986e880005f17af3470927034) (grafted from f3ecb29d9b68d2adcf28a1986c070ceb85da8421) (grafted from 23c77af0834657915d69f06623b8d831fc03dfbc) (grafted from 6c0111953ffaacad3deb12f81bf05454f4c8bfc1) (grafted from e6564dd98f0ab63736ba70500d60db0def7025f8) (grafted from 3228fe1cc79cea64fa4cc674bf71f24123433ec7) (grafted from 5a8c522c3db09d88bc6d45393c40256139ab4d42) (grafted from 8d3f1b10a2bcda17dc55cdc97712bdc456acc406) (grafted from 1f994f75a2a4f57c0f19981f71254e22963eaa04) (grafted from 9bf91f3db246c40475f57657d82bd3f50e571e84) (grafted from 2f88c5c88fd55b48fc0c30aa2abd701b2b7b9d7d) (grafted from fab2b4f5d6e855c657f69dccb08f720bab4a82b1) (grafted from e2fc694c13c74f7948680d653568e62d7010c4f4) (grafted from d474fcb2eb58404a689024cc503abe28136c8395) (grafted from 8b2233600993b96119186ddbfb16232c48247fe1) (grafted from ba5486b8c838e72c35cb437cbdf0add5b2100c6b) (grafted from 42699d89bd0da2e204075246eb4c27baa642fc2a) (grafted from 02d0e17b6b7f938ba58f26d84191ef347bcdbd17) (grafted from 9f9bba61993924c1c96a5dd15c0c50895ec8e525) (grafted from c3bc9b3844d852ba7365574d3fbd4aa8e2763aa0) (grafted from fc5d31337891e9eb218f100b177f8ead2f6353da) (grafted from baad4585f877267a3264dc4c70e8cd75bfc6688e) (grafted from 6d46603cea8fc9d04628817ae1e239b38e62ce62) (grafted from c54d6c0b3b3dae086b8ad186e178a1e137b57cae) (grafted from 1755e00c824c7d5b30a7daff573c3ab1e2f1fd2c) (grafted from 81826a9db3f5f9c8e760d2b09afb06d9f74a1609) (grafted from 73bce05fdb8cb8f8739bd2c90741ac7d422e24a4) (grafted from c942744744e071464d3a476e57fd74f922fca497) (grafted from 77b797a46b52249b26da6d30a0c63e381d1af0fe) (grafted from 9b09bef68b1aa1ef3804aec330e793dcc7ada944) (grafted from fefea5a4a24f0c8219b4b0e2b0cc70b4dfadfde2) (grafted from 3428e6ab54999e70b38a5ee62291c8774bae7e4a) (grafted from dd57d190fa75589d800beed188c5a381678cca3c) (grafted from 9005ba0d5e6420ed8d048c08fc38a9682e536a26) (grafted from daeb4bf87b68309c4c3c608853affe2adeb9fcbd) (grafted from 6ad984397b22666ecebc2ff737b0d52ffb95b9b7) (grafted from bd1f17920c18d88b6c4dd451f76699f6f665bc32) (grafted from 31a92a48004ab6a4bfc389f674cacf549703e2b9) (grafted from 7e80636f1529b8902292980912586f1db8598a83) (grafted from 55ecf6a9f9bc89fd0330c4684f225e213be18033) (grafted from 3361f627a0b8cb7666a529494ed80a0e1d7a365c) (grafted from f51abda49b2d33a936567b786ce785c97e4b9c64) (grafted from 5c01ed3bf394351e116fcece1fbd9e1b19f57ea4) (grafted from 9503cdb335a95950235b7871978bcb6cb3e87222) (grafted from 220d31489a377997e71d5766f9f29b30fd1662e1) (grafted from 97483c98cf95113e95d03bf69f3307f92ef232a4) (grafted from b3e9f8f43b4dd76422c20234acf76ec82a9c5f5b) (grafted from 831d9f1b2940b6944fc1015c57f348a22de7a149) (grafted from 051e1081a45e85944ea4414174a452e208904371) (grafted from 9fb7c4c494166dee97bb04826cf71b0fa774cd2b) (grafted from bfb4cb9c08c582323ceb82eb5cd2bcaeaf70329c) (grafted from ab548e4f700a65bd24fbbbdb43f971eb5da73be5) (grafted from 8870c382cfecbcbd4f8cec3ca0cb8759a43cf31c) (grafted from 8e09e732b846b42beb49866d3785572df89f1e38) (grafted from 951064f2a2980a00d085fce0d207c0225447b5d5) (grafted from fce58e29903c2ef95b0481c11d59e5ef4fb56d15) (grafted from 391f610601eeb542b9aa2bed3f541609772b8599) (grafted from 681ea78b8ae1c29589a49a5bb74e0f3dff30e1d7) (grafted from add76a9d94ee5d7b829be52458d1dfb65585c2d3) (grafted from 475ae856da634e53637131717723fe895be0def6) (grafted from b22d9651b8a7957c8275301d79f6ba62fc5a38f4) (grafted from 3bf8c57dfd2fb432f4d499014dd9ab4ced26133d) (grafted from 47718758ce30722c7a6e4d4dc324bdef20d45163) (grafted from 59270eeea34cc381933ce50600983819e8a35794) (grafted from 5397fdfa10d831ba358925e4a32faa23fed18173) (grafted from de9cc0df3d097d25a8a218010e4a8370373f5068) (grafted from e7f5a21e9dd2f2a4527943f5dad0d597c0f2b7a4) (grafted from 5c383ebe8f57ccabf1ede74905a951631c7e32de) (grafted from bf4ad8ffd761b575a843fe5f8370fb4a5c1a2eb9) (grafted from 8523ff03bfa8708eefb667d9df64fba58e5a9f8c) (grafted from 5220db13df5a745d113ebba89185ac42f4f890d1) (grafted from 8a66c7352d03e545e8cbe80a1f84122624f2035c) (grafted from 94cf7cdbce472e0694582b18acd6f2fc3f46e839) (grafted from 29575e1ce294a235a1df9136a6b807b8947a20cd) (grafted from d23b69b215e4201e90b600797b41ab362e7c7044) (grafted from 386af1481e70f160f6b3d0e08ba8934372446b68) (grafted from fc69c78396e9ab15fd961540204db004db933799) (grafted from 8a8e35573518b72937af626999b885c6f6491683) (grafted from 9fb083f031e7358fdb73ef6fac5278f841f2b084) (grafted from aa20fd817539bdd8d15adb340ef714121dd273b2) (grafted from 2b68df547651d1727b4320f2ffbb8fded3ab49ac) (grafted from ff3bf5623756d977c88a25d546b34160229177d3) (grafted from fd5f39d43bac7fa2c8a1e5785e3dc060f539277c) (grafted from 8871ae64560384cf7693a5c32333b90e6774ba2a) (grafted from 3a01e81a1a78138af5892129f447bee53fb85a73) (grafted from ca1bc04c027005d42b4dc5eee52c3874a13790df) (grafted from c68f7302ce65f055e205ed701490529571aaaa32) (grafted from c7e264107eb95c8ab01804699775374b43bcbd84) (grafted from 3748e7898899c994a4b5019b66f705e2c7a8d2eb) (grafted from fe2d08b837334f9b3b1bb3a18a28f010a02c0340) (grafted from dd29c6c56f0de76550d2f61671eb05da6a03fbc3)
2018-01-03 16:35:56 +03:00
$ hg --config devel.legacy.exchange=bundle1 clone ssh://user@dummy/bundle2onlyserver not-bundle2-ssh 2>&1 | grep "remote:"
bundle1: fix bundle1-denied reporting for pull over ssh Changeset a0966f529e1b introduced a config option to have the server deny pull using bundle1. The original protocol has not really been design to allow that kind of error reporting so some hack was used. It turned the hack only works on HTTP and that ssh server hangs forever when this is used. After further digging, there is no way to report the error in a unified way. Using `ooberror` freeze ssh and raising 'Abort' makes HTTP return a HTTP-500 without further details. So with sadness we implement a version that dispatch according to the protocol used. Now the error is properly reported, but we still have ungraceful abort after that. The protocol do not allow anything better to happen using bundle1.
2017-02-10 20:06:08 +03:00
remote: abort: incompatible Mercurial client; bundle2 required
remote: (see https://www.mercurial-scm.org/wiki/IncompatibleClient)
bundle-tests: operate from outside a repository We are about to add a test for ssh pull/cloning being denied because of bundle1 usage. For this, it is cleaner to not operate from the clone using http. So we update the test beforehand for clarity. This is more churns that what I'm happy to see on stable, but the rests of the series is worth it in my opinion.
2017-02-10 20:06:12 +03:00
$ cat > bundle2onlyserver/.hg/hgrc << EOF
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
> [server]
> bundle1gd = false
> EOF
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -R bundle2onlyserver -p 0 --port-file $TESTTMP/.port -d --pid-file=hg.pid
$ HGPORT=`cat $TESTTMP/.port`
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
$ cat hg.pid >> $DAEMON_PIDS
tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.
2016-08-02 16:23:03 +03:00
$ hg --config devel.legacy.exchange=bundle1 clone http://localhost:$HGPORT/ not-bundle2
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
requesting all changes
abort: remote error:
incompatible Mercurial client; bundle2 required
(see https://www.mercurial-scm.org/wiki/IncompatibleClient)
[255]
$ killdaemons.py
bundle-tests: operate from outside a repository We are about to add a test for ssh pull/cloning being denied because of bundle1 usage. For this, it is cleaner to not operate from the clone using http. So we update the test beforehand for clarity. This is more churns that what I'm happy to see on stable, but the rests of the series is worth it in my opinion.
2017-02-10 20:06:12 +03:00
$ cd notgdserver
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
$ cat > .hg/hgrc << EOF
> [server]
> bundle1gd = false
hg: remove bundlev1 from the supported outgoing versions Summary: Removes bundlev1 from the supported outgoing versions, but keeps a flag around to force enable it if tests need it. Reviewed By: quark-zju Differential Revision: D7591176 fbshipit-source-id: 280cbbbe87848e3d6c9d448ce4f87c5eadeff720
2018-04-12 02:04:53 +03:00
> [format]
> allowbundle1=True
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
> EOF
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -p 0 --port-file $TESTTMP/.port -d --pid-file=hg.pid
$ HGPORT=`cat $TESTTMP/.port`
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
$ cat hg.pid >> $DAEMON_PIDS
tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.
2016-08-02 16:23:03 +03:00
$ hg --config devel.legacy.exchange=bundle1 clone http://localhost:$HGPORT/ not-bundle2-2
wireproto: support disabling bundle1 only if repo is generaldelta I recently implemented the server.bundle1* options to control whether bundle1 exchange is allowed. After thinking about Mozilla's strategy for handling generaldelta rollout a bit more, I think server operators need an additional lever: disable bundle1 if and only if the repo is generaldelta. bundle1 exchange for non-generaldelta repos will not have the potential for CPU explosion that generaldelta repos do. Therefore, it makes sense for server operators to continue to allow bundle1 exchange for non-generaldelta repos without having to set a per-repo hgrc option to change the policy depending on whether the repo is generaldelta. This patch introduces a new set of options to control bundle1 behavior for generaldelta repos. These options enable server operators to limit bundle1 restrictions to the class of repos that can be performance issues. It also allows server operators to tie bundle1 access to store format. In many server environments (including Mozilla's), legacy repos will not be generaldelta and new repos will or might be. New repos often aren't bound by legacy access requirements, so setting a global policy that disallows access to new/generaldelta repos via bundle1 could be a reasonable policy in many server environments. This patch makes this policy very easy to implement (modify global hgrc, add options to existing generaldelta repos to grandfather them in).
2015-12-20 22:56:24 +03:00
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ killdaemons.py
$ cd ../bundle2onlyserver
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
Verify bundle1 pushes can be disabled
$ cat > .hg/hgrc << EOF
> [server]
> bundle1.push = false
> [web]
> allow_push = *
> push_ssl = false
> EOF
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
$ hg serve -p 0 --port-file $TESTTMP/.port -d --pid-file=hg.pid -E error.log
$ HGPORT=`cat $TESTTMP/.port`
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
$ cat hg.pid >> $DAEMON_PIDS
$ cd ..
$ hg clone http://localhost:$HGPORT bundle2-only
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cd bundle2-only
$ echo commit > foo
$ hg commit -m commit
tests: use 'legacy.exchange' option in 'test-bundle2-exchange.t' The new option will stay around. The experimental option was only meant to be temporary.
2016-08-02 16:23:03 +03:00
$ hg --config devel.legacy.exchange=bundle1 push
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT/ (glob)
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
searching for changes
hg: add develwarn for bundle1 format Summary: We want to deprecate the bundlev1 format, so let's start by adding a develwarn. Later diffs will update the tests to not use v1, then remove v1 as a supported outgoing bundle entirely. Reviewed By: quark-zju Differential Revision: D7591166 fbshipit-source-id: 143ad029bfe4d141f91d6d5077342dfa44ad2944
2018-04-12 02:04:30 +03:00
devel-warn: using deprecated bundlev1 format
at: */changegroup.py:* (makechangegroup) (glob)
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
abort: remote error:
incompatible Mercurial client; bundle2 required
(see https://www.mercurial-scm.org/wiki/IncompatibleClient)
[255]
bundle1: fix bundle1-denied reporting for push over ssh Changeset a0966f529e1b introduced a config option to have the server deny push using bundle1. The original protocol has not really be design to allow such kind of error reporting so some hack was used. It turned the hack only works on HTTP and that ssh wire peer hangs forever when the same hack is used. After further digging, there is no way to report the error in a unified way. Using 'ooberror' freeze ssh and raising 'Abort' makes HTTP return a HTTP500 without further details. So with sadness we implement a version that dispatch according to the protocol used. We also add a test for pushing over ssh to make sure we won't regress in the future. That test show that the hint is missing, this is another bug fixed in the next changeset.
2017-02-10 19:56:59 +03:00
(also check with ssh)
$ hg --config devel.legacy.exchange=bundle1 push ssh://user@dummy/bundle2onlyserver
pushing to ssh://user@dummy/bundle2onlyserver
searching for changes
hg: add develwarn for bundle1 format Summary: We want to deprecate the bundlev1 format, so let's start by adding a develwarn. Later diffs will update the tests to not use v1, then remove v1 as a supported outgoing bundle entirely. Reviewed By: quark-zju Differential Revision: D7591166 fbshipit-source-id: 143ad029bfe4d141f91d6d5077342dfa44ad2944
2018-04-12 02:04:30 +03:00
devel-warn: using deprecated bundlev1 format
at: */changegroup.py:* (makechangegroup) (glob)
bundle1: fix bundle1-denied reporting for push over ssh Changeset a0966f529e1b introduced a config option to have the server deny push using bundle1. The original protocol has not really be design to allow such kind of error reporting so some hack was used. It turned the hack only works on HTTP and that ssh wire peer hangs forever when the same hack is used. After further digging, there is no way to report the error in a unified way. Using 'ooberror' freeze ssh and raising 'Abort' makes HTTP return a HTTP500 without further details. So with sadness we implement a version that dispatch according to the protocol used. We also add a test for pushing over ssh to make sure we won't regress in the future. That test show that the hint is missing, this is another bug fixed in the next changeset.
2017-02-10 19:56:59 +03:00
remote: abort: incompatible Mercurial client; bundle2 required
bundle1: display server abort hint during unbundle The code was printing the abort message but not the hint. This is now fixed.
2017-02-10 19:56:52 +03:00
remote: (see https://www.mercurial-scm.org/wiki/IncompatibleClient)
bundle1: fix bundle1-denied reporting for push over ssh Changeset a0966f529e1b introduced a config option to have the server deny push using bundle1. The original protocol has not really be design to allow such kind of error reporting so some hack was used. It turned the hack only works on HTTP and that ssh wire peer hangs forever when the same hack is used. After further digging, there is no way to report the error in a unified way. Using 'ooberror' freeze ssh and raising 'Abort' makes HTTP return a HTTP500 without further details. So with sadness we implement a version that dispatch according to the protocol used. We also add a test for pushing over ssh to make sure we won't regress in the future. That test show that the hint is missing, this is another bug fixed in the next changeset.
2017-02-10 19:56:59 +03:00
[1]
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
$ hg push
serve: assign hg serve ports dynamically in tests Summary: Previously `hg server` uses `HGPORT` that might be in use. This patch uses `-p 0 --port-file ...` so `hg server` always gets assigned a free port. The change was first made by the following Ruby script: ``` re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/ Dir['*.t'].each do |path| old = File.read(path) new = old.lines.map do |l| next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/] "#{l.chomp} (glob)\n" end.join.gsub re, <<-'EOS'.chomp $ hg serve\1 -p 0 --port-file $TESTTMP/.port \3 $ \2=`cat $TESTTMP/.port` $ EOS File.write(path, new) if old != new end ``` Then there are some manual changes: run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since it does not know the assigned value in tests. test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use explicit paths instead of relying on `.hgrc` since the test restarts the server and `.hg/hgrc` having an outdated URL. test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it. Changed the order so the correct `$HGPORT` is written. test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`. Reviewed By: DurhamG Differential Revision: D6925398 fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
2018-02-08 02:18:29 +03:00
pushing to http://localhost:$HGPORT/ (glob)
wireproto: config options to disable bundle1 bundle2 is the new and preferred wire protocol format. For various reasons, server operators may wish to force clients to use it. One reason is performance. If a repository is stored in generaldelta, the server must recompute deltas in order to produce the bundle1 changegroup. This can be extremely expensive. For mozilla-central, bundle generation typically takes a few minutes. However, generating a non-gd bundle from a generaldelta encoded mozilla-central requires over 30 minutes of CPU! If a large repository like mozilla-central were encoded in generaldelta and non-gd clients connected, they could easily flood a server by cloning. This patch gives server operators config knobs to control whether bundle1 is allowed for push and pull operations. The default is to support legacy bundle1 clients, making this patch backwards compatible.
2015-12-05 02:12:11 +03:00
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
Reference in New Issue Copy Permalink