sapling/eden/mononoke/tests/integration/test-push-protocol.t

493 lines
13 KiB
Perl
Raw Normal View History

# Copyright (c) Facebook, Inc. and its affiliates.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License found in the LICENSE file in the root
# directory of this source tree.
mononoke: rebuild test framework Summary: Our test framework as it stands right now is a light passthrough to the hg `run-tests.py` test framework, which attempts to place all the files it needs to run (including tests) into a `python_binary`, then runs the hg test runner from that directory. It heavily relies on how Buck works to offer functionality: - It expects that all the sources it registers for its master binary will all be in the same directory when it builds - It expects that the sources will be symlinks to the real files so that `--interactive` can work. This has a few problems: - It doesn't work in `mode/opt`. The archive that gets built in `mode/opt` doesn't actually have all the sources we registered, so it's impossible to run tests. - To add a new test, you must rebuild everything. We don't do that very often, but it'd be nice if we didn't have to. - Iterating on the runner itself is painful, because as far as Buck is concerned, it depends on the entire world. This means that every change to the runner has to scan a lot more stuff than necessary. There's some functionality I'd like to get into the runner (like reporting test timings) that hasn't been easy to add as a result. This diff attempts to solve these problems by separating concerns a little more: - The runner is now just a simple `python_binary`, so it's easier to make changes to it. - The runner now provides the logic of working from local files when needed (this means you can add a new test and it'll work immediately), - All the binaries we need are dependencies of the integration test target, not the runner's. However, to make it possible to run the runner incrementally while iterating on something, there's a manifest target that points at all the various paths the runner needs to work. This will also help integrate the test runner with other build frameworks if necessary (e.g. for open-sourcing). - We have separate targets for various assets we need to run the tests (e.g. the hg test framework). - The runner now controls whether to use the network blackhole. This was necessary because the network blackhole breaks PAR archives (because tmp is no longer owned by the right owner, because we use a user namespace). We should be able to bring this back at some point if we want to by using a proper chroot for opt tests. I included a README to explain this new design as well. There are some things that could yet stand to be improved here (notably, I think we should put assets and tests in different directories for the sake of clarity), but so far I've been aiming at providing a 1-1 translation of the old system into the new one. I am planning to make further improvements in followup diffs. Reviewed By: farnz Differential Revision: D15921732 fbshipit-source-id: 09052591c419acf97f7e360b1e88ef1f412da6e5
2019-06-25 18:30:05 +03:00
$ . "${TEST_FIXTURES}/library.sh"
setup configuration
$ setup_common_config
$ cd $TESTTMP
setup repo
$ hginit_treemanifest repo-hg
$ cd repo-hg
$ echo "a file content" > a
$ hg add a
$ hg ci -ma
setup master bookmarks
$ hg bookmark master_bookmark -r 'tip'
verify content
$ hg log
commit: 0e7ec5675652
bookmark: master_bookmark
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: a
$ cd $TESTTMP
$ blobimport repo-hg/.hg repo
setup two repos: one will be used to push from, another will be used
to pull these pushed commits
$ hgclone_treemanifest ssh://user@dummy/repo-hg repo2
$ hgclone_treemanifest ssh://user@dummy/repo-hg repo3
$ cd repo2
$ hg pull ../repo-hg
pulling from ../repo-hg
searching for changes
no changes found
start mononoke
$ mononoke
$ wait_for_mononoke
BEGIN Creation of new commits
create new commits in repo2 and check that they are seen as outgoing
$ mkdir b_dir
$ echo "new a file content" > a
$ echo "b file content" > b_dir/b
$ hg add b_dir/b
$ hg ci -mb
$ echo "updated b file content" > b_dir/b
$ mkdir c_dir
$ echo "c file content" > c_dir/c
$ hg add c_dir/c
$ hg ci -mc
create a commit that makes identical change to file b
$ hg update '.^'
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ echo "updated b file content" > b_dir/b
$ mkdir d_dir
$ echo "d file content" > d_dir/d
$ hg add d_dir/d
$ hg ci -md
create a commit that reverts files a and b to older version
$ echo "a file content" > a
$ echo "b file content" > b_dir/b
$ hg ci -me
create a commit that sets content of some files to content of other files
$ echo "b file content" > a
$ echo "a file content" > b_dir/b
$ mkdir c_dir
$ echo "a file content" > c_dir/c
$ hg add c_dir/c
$ echo "b file content" > d_dir/d
$ hg ci -mf
create a commit that renames, copy and deletes some files
$ hg rm b_dir/b
$ hg mv a b_dir/b
$ mkdir e_dir
$ hg mv c_dir/c e_dir/e
$ mkdir a_dir
$ hg mv d_dir/d a_dir/a
$ echo "a file content" > a_dir/a
$ hg cp a_dir/a b_dir/a_bis
$ hg ci -mg
END Creation of new commits
move master bookmarks
$ hg bookmark -f master_bookmark -r 'tip'
$ hg log -r "reverse(all())" --stat
commit: 634de738bb0f
bookmark: master_bookmark
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: g
(re)
a | 1 -
a_dir/a | 1 +
b_dir/a_bis | 1 +
b_dir/b | 2 +-
c_dir/c | 1 -
d_dir/d | 1 -
e_dir/e | 1 +
7 files changed, 4 insertions(+), 4 deletions(-)
commit: 8315ea53ef41
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: f
a | 2 +-
b_dir/b | 2 +-
c_dir/c | 1 +
d_dir/d | 2 +-
4 files changed, 4 insertions(+), 3 deletions(-)
commit: 30da5bf63484
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: e
a | 2 +-
b_dir/b | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
commit: fbd6b221382e
parent: bb0985934a0f
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: d
b_dir/b | 2 +-
d_dir/d | 1 +
2 files changed, 2 insertions(+), 1 deletions(-)
commit: f40c09205504
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: c
b_dir/b | 2 +-
c_dir/c | 1 +
2 files changed, 2 insertions(+), 1 deletions(-)
commit: bb0985934a0f
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: b
a | 2 +-
b_dir/b | 1 +
2 files changed, 2 insertions(+), 1 deletions(-)
commit: 0e7ec5675652
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: a
a | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
(re)
$ hgmn outgoing ssh://user@dummy/repo
comparing with ssh://user@dummy/repo
searching for changes
commit: bb0985934a0f
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: b
(re)
commit: f40c09205504
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: c
(re)
commit: fbd6b221382e
parent: bb0985934a0f
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: d
(re)
commit: 30da5bf63484
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: e
(re)
commit: 8315ea53ef41
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: f
(re)
commit: 634de738bb0f
bookmark: master_bookmark
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: g
(re)
push to Mononoke
$ hgmn push --force --config treemanifest.treeonly=True --debug ssh://user@dummy/repo
pushing to ssh://user@dummy/repo
mononoke: rebuild test framework Summary: Our test framework as it stands right now is a light passthrough to the hg `run-tests.py` test framework, which attempts to place all the files it needs to run (including tests) into a `python_binary`, then runs the hg test runner from that directory. It heavily relies on how Buck works to offer functionality: - It expects that all the sources it registers for its master binary will all be in the same directory when it builds - It expects that the sources will be symlinks to the real files so that `--interactive` can work. This has a few problems: - It doesn't work in `mode/opt`. The archive that gets built in `mode/opt` doesn't actually have all the sources we registered, so it's impossible to run tests. - To add a new test, you must rebuild everything. We don't do that very often, but it'd be nice if we didn't have to. - Iterating on the runner itself is painful, because as far as Buck is concerned, it depends on the entire world. This means that every change to the runner has to scan a lot more stuff than necessary. There's some functionality I'd like to get into the runner (like reporting test timings) that hasn't been easy to add as a result. This diff attempts to solve these problems by separating concerns a little more: - The runner is now just a simple `python_binary`, so it's easier to make changes to it. - The runner now provides the logic of working from local files when needed (this means you can add a new test and it'll work immediately), - All the binaries we need are dependencies of the integration test target, not the runner's. However, to make it possible to run the runner incrementally while iterating on something, there's a manifest target that points at all the various paths the runner needs to work. This will also help integrate the test runner with other build frameworks if necessary (e.g. for open-sourcing). - We have separate targets for various assets we need to run the tests (e.g. the hg test framework). - The runner now controls whether to use the network blackhole. This was necessary because the network blackhole breaks PAR archives (because tmp is no longer owned by the right owner, because we use a user namespace). We should be able to bring this back at some point if we want to by using a proper chroot for opt tests. I included a README to explain this new design as well. There are some things that could yet stand to be improved here (notably, I think we should put assets and tests in different directories for the sake of clarity), but so far I've been aiming at providing a 1-1 translation of the old system into the new one. I am planning to make further improvements in followup diffs. Reviewed By: farnz Differential Revision: D15921732 fbshipit-source-id: 09052591c419acf97f7e360b1e88ef1f412da6e5
2019-06-25 18:30:05 +03:00
running * 'user@dummy' '$TESTTMP/mononoke_hgcli -R repo serve --stdio' (glob)
sending hello command
sending between command
remote: * (glob)
mononoke: fix of the discovery problem Summary: This is the test to cover tricky case in the discovery logic. Previously Mononoke's known() wireproto method returned `true` for both public and draft commits. The problem was in that it affects pushrebase. There are a few problems with the current setup. A push command like `hg push -r HASH --to BOOK` may actually do two things - it can either move a bookmark on the server or do a pushrebase. What it does depends on how discovery phase of the push finishes. Each `hg push` starts with a discovery algorithm that tries to figure out what commits to send to the server. If client decides that server already has all the commits then it'll just move the bookmark, otherwise it'll run the pushrebase. During discovery client sends wireproto `known()` method to the server with a list of commit hashes, and server returns a list of booleans telling if a server knows the commit or not. Before this diff Mononoke returned true for both draft commits and public commits, while Mercurial returned true it only for public commits. So if Mononoke already has a draft commit (it might have it because the commit was created via `hg pushbackup` or was created in the previous unsuccessful push attempt), then hg client discovery will decide to move a bookmark instead of pushrebasing, which in the case of master bookmark might have disastrous consequences. To fix it let's return false for draft commits, and also implement `knownnodes` which return true for draft commits (a better name for these methods would be `knownpublic` and `known`). Note though that in order to trigger the problem the position of the bookmark on the server should be different from the position of the bookmark on the client. This is because of short-circuting in the hg client discovery logic (see https://fburl.com/s5r76yle). The potential downside of the change is that we'll fetch bookmarks more often, but we'll add bookmark cache later if necessary. Reviewed By: ikostia Differential Revision: D14560355 fbshipit-source-id: b943714199576e14a32e87f325ae8059d95cb8ed
2019-03-25 12:17:27 +03:00
remote: capabilities: * (glob)
remote: 1
sending clienttelemetry command
query 1; heads
sending batch command
searching for changes
all remote heads known locally
checking for updated bookmarks
preparing listkeys for "bookmarks"
sending listkeys command
received listkey for "bookmarks": 57 bytes
6 changesets found
list of changesets:
bb0985934a0f8a493887892173b68940ceb40b4f
f40c09205504d8410f8c8679bf7a85fef25f9337
fbd6b221382efa5d5bc53130cdaccf06e04c97d3
30da5bf63484d2d6572edafb3ea211c17cd8c005
8315ea53ef41d34f56232c88669cc80225b6e66d
634de738bb0ff135e32d48567718fb9d7dedf575
sending unbundle command
bundle2-output-bundle: "HG20", 4 parts total
bundle2-output-part: "replycaps" * bytes payload (glob)
bundle2-output-part: "changegroup" (params: 1 mandatory) streamed payload
bundle2-output-part: "pushkey" (params: 4 mandatory) empty payload
bundle2-output-part: "b2x:treegroup2" (params: 3 mandatory) streamed payload
bundle2-input-bundle: 1 params no-transaction
bundle2-input-part: "reply:changegroup" (params: 2 mandatory) supported
bundle2-input-part: "reply:pushkey" (params: 2 mandatory) supported
bundle2-input-bundle: 1 parts total
updating bookmark master_bookmark
Now pull what was just pushed
$ cd ../repo3
$ hgmn log -r "reverse(all())" --stat
commit: 0e7ec5675652
bookmark: master_bookmark
user: test
date: Thu Jan 01 00:00:00 1970 +0000
summary: a
(re)
a | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
(re)
$ hgmn pull -q
Because the revision numbers are assigned nondeterministically we cannot
compare output of the entire tree. Instead we compare only linear histories
$ hgmn log --graph --template '{node} {bookmarks}' -r "::f40c09205504"
pulling 'f40c09205504' from 'ssh://user@dummy/repo'
o f40c09205504d8410f8c8679bf7a85fef25f9337
|
o bb0985934a0f8a493887892173b68940ceb40b4f
|
@ 0e7ec5675652a04069cbf976a42e45b740f3243c
(re)
$ hgmn log --graph --template '{node} {bookmarks}' -r "::634de738bb0f"
o 634de738bb0ff135e32d48567718fb9d7dedf575 master_bookmark
|
o 8315ea53ef41d34f56232c88669cc80225b6e66d
|
o 30da5bf63484d2d6572edafb3ea211c17cd8c005
|
o fbd6b221382efa5d5bc53130cdaccf06e04c97d3
|
o bb0985934a0f8a493887892173b68940ceb40b4f
|
@ 0e7ec5675652a04069cbf976a42e45b740f3243c
(re)
This last step is verifying every commit one by one, it is done in a single
command, but the output of this command is long
$ for commit in `hg log --template '{node} ' -r '0e7ec567::634de738'` f40c09205504d8410f8c8679bf7a85fef25f9337; do \
$ if [ "`hg export -R $TESTTMP/repo2 ${commit}`" == "`hgmn export ${commit} 2> /dev/null`" ]; then echo "${commit} comparison SUCCESS"; fi; hgmn export ${commit}; echo; echo; done
0e7ec5675652a04069cbf976a42e45b740f3243c comparison SUCCESS
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID 0e7ec5675652a04069cbf976a42e45b740f3243c
# Parent 0000000000000000000000000000000000000000
a
(re)
diff -r 000000000000 -r 0e7ec5675652 a
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/a Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+a file content
(re)
(re)
bb0985934a0f8a493887892173b68940ceb40b4f comparison SUCCESS
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID bb0985934a0f8a493887892173b68940ceb40b4f
# Parent 0e7ec5675652a04069cbf976a42e45b740f3243c
b
diff -r 0e7ec5675652 -r bb0985934a0f a
--- a/a Thu Jan 01 00:00:00 1970 +0000
+++ b/a Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-a file content
+new a file content
diff -r 0e7ec5675652 -r bb0985934a0f b_dir/b
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/b_dir/b Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+b file content
fbd6b221382efa5d5bc53130cdaccf06e04c97d3 comparison SUCCESS
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID fbd6b221382efa5d5bc53130cdaccf06e04c97d3
# Parent bb0985934a0f8a493887892173b68940ceb40b4f
d
diff -r bb0985934a0f -r fbd6b221382e b_dir/b
--- a/b_dir/b Thu Jan 01 00:00:00 1970 +0000
+++ b/b_dir/b Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-b file content
+updated b file content
diff -r bb0985934a0f -r fbd6b221382e d_dir/d
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/d_dir/d Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+d file content
30da5bf63484d2d6572edafb3ea211c17cd8c005 comparison SUCCESS
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID 30da5bf63484d2d6572edafb3ea211c17cd8c005
# Parent fbd6b221382efa5d5bc53130cdaccf06e04c97d3
e
diff -r fbd6b221382e -r 30da5bf63484 a
--- a/a Thu Jan 01 00:00:00 1970 +0000
+++ b/a Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-new a file content
+a file content
diff -r fbd6b221382e -r 30da5bf63484 b_dir/b
--- a/b_dir/b Thu Jan 01 00:00:00 1970 +0000
+++ b/b_dir/b Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-updated b file content
+b file content
8315ea53ef41d34f56232c88669cc80225b6e66d comparison SUCCESS
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID 8315ea53ef41d34f56232c88669cc80225b6e66d
# Parent 30da5bf63484d2d6572edafb3ea211c17cd8c005
f
diff -r 30da5bf63484 -r 8315ea53ef41 a
--- a/a Thu Jan 01 00:00:00 1970 +0000
+++ b/a Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-a file content
+b file content
diff -r 30da5bf63484 -r 8315ea53ef41 b_dir/b
--- a/b_dir/b Thu Jan 01 00:00:00 1970 +0000
+++ b/b_dir/b Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-b file content
+a file content
diff -r 30da5bf63484 -r 8315ea53ef41 c_dir/c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/c_dir/c Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+a file content
diff -r 30da5bf63484 -r 8315ea53ef41 d_dir/d
--- a/d_dir/d Thu Jan 01 00:00:00 1970 +0000
+++ b/d_dir/d Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-d file content
+b file content
634de738bb0ff135e32d48567718fb9d7dedf575 comparison SUCCESS
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID 634de738bb0ff135e32d48567718fb9d7dedf575
# Parent 8315ea53ef41d34f56232c88669cc80225b6e66d
g
diff -r 8315ea53ef41 -r 634de738bb0f a
--- a/a Thu Jan 01 00:00:00 1970 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +0,0 @@
-b file content
diff -r 8315ea53ef41 -r 634de738bb0f a_dir/a
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/a_dir/a Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+a file content
diff -r 8315ea53ef41 -r 634de738bb0f b_dir/a_bis
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/b_dir/a_bis Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+a file content
diff -r 8315ea53ef41 -r 634de738bb0f b_dir/b
--- a/b_dir/b Thu Jan 01 00:00:00 1970 +0000
+++ b/b_dir/b Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-a file content
+b file content
diff -r 8315ea53ef41 -r 634de738bb0f c_dir/c
--- a/c_dir/c Thu Jan 01 00:00:00 1970 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +0,0 @@
-a file content
diff -r 8315ea53ef41 -r 634de738bb0f d_dir/d
--- a/d_dir/d Thu Jan 01 00:00:00 1970 +0000
+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +0,0 @@
-b file content
diff -r 8315ea53ef41 -r 634de738bb0f e_dir/e
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/e_dir/e Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+a file content
f40c09205504d8410f8c8679bf7a85fef25f9337 comparison SUCCESS
# HG changeset patch
# User test
# Date 0 0
# Thu Jan 01 00:00:00 1970 +0000
# Node ID f40c09205504d8410f8c8679bf7a85fef25f9337
# Parent bb0985934a0f8a493887892173b68940ceb40b4f
c
diff -r bb0985934a0f -r f40c09205504 b_dir/b
--- a/b_dir/b Thu Jan 01 00:00:00 1970 +0000
+++ b/b_dir/b Thu Jan 01 00:00:00 1970 +0000
@@ -1,1 +1,1 @@
-b file content
+updated b file content
diff -r bb0985934a0f -r f40c09205504 c_dir/c
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/c_dir/c Thu Jan 01 00:00:00 1970 +0000
@@ -0,0 +1,1 @@
+c file content