Before this patch, transplant can't restore dirstate as expected at
failure other than one while patching. This causes:
- unexpected file status
- dirstate refers already rollback-ed parent
(only at failure of transplanting the 2nd or later revision)
To restore dirstate correctly also at unexpected failure, this patch
encloses scope of store lock and transaction by 'dirstateguard'.
This is temporary fixing for stable branch. See
DirstateTransactionPlan wiki page for detail about the future plan to
treat dirstate consistently around scope boundary of transaction.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
This patch also adds 'if lock' examination for safety
'lock.release()', because creating 'dirstateguard' object may fail
unexpectedly (e.g. IOError for saving dirstate).
BTW, in the test script, putting section header '[extensions]' into
'.hg/hgrc' is needed to fix incomplete disabling 'abort' extension at
d0d06f4ca862.
To detect change of a file without redundant comparison of file
content, dirstate recognizes a file as certainly clean, if:
(1) it is already known as "normal",
(2) dirstate entry for it has valid (= not "-1") timestamp, and
(3) mode, size and timestamp of it on the filesystem are as same as
ones expected in dirstate
This works as expected in many cases, but doesn't in the corner case
that changing a file keeps mode, size and timestamp of it on the
filesystem.
The timetable below shows steps in one of typical such situations:
---- ----------------------------------- ----------------
timestamp of "f"
----------------
dirstate file-
time action mem file system
---- ----------------------------------- ---- ----- -----
* *** ***
- 'hg transplant REV1 REV2 ...'
- transplanting REV1
....
N
- change "f", but keep size N
(via 'patch.patch()')
- 'dirstate.normal("f")' N ***
(via 'repo.commit()')
- transplanting REV2
- change "f", but keep size N
(via 'patch.patch()')
- aborted while patching
N+1
- release wlock
- 'dirstate.write()' N N N
- 'hg status' shows "r1" as "clean" N N N
---- ----------------------------------- ---- ----- -----
The most important point is that 'dirstate.write()' is executed at N+1
or later. This causes writing dirstate timestamp N of "f" out
successfully. If it is executed at N, 'parsers.pack_dirstate()'
replaces timestamp N with "-1" before actual writing dirstate out.
This issue can occur when 'hg transplant' satisfies conditions below:
- multiple revisions to be transplanted change the same file
- those revisions don't change mode and size of the file, and
- the 2nd or later revision of them fails after changing the file
The root cause of this issue is that files are changed without
flushing in-memory dirstate changes via 'repo.commit()' (even though
omitting 'dirstate.normallookup()' on files changed by 'patch.patch()'
for efficiency also causes this issue).
To detect changes of files correctly, this patch writes in-memory
dirstate changes out explicitly after marking files as clean in
'committablectx.markcommitted()', which is invoked via
'repo.commit()'.
After this change, timetable is changed as below:
---- ----------------------------------- ----------------
timestamp of "f"
----------------
dirstate file-
time action mem file system
---- ----------------------------------- ---- ----- -----
* *** ***
- 'hg transplant REV1 REV2 ...'
- transplanting REV1
....
N
- change "f", but keep size N
(via 'patch.patch()')
- 'dirstate.normal("f")' N ***
(via 'repo.commit()')
----------------------------------- ---- ----- -----
- 'dirsttate.write()' -1 -1
----------------------------------- ---- ----- -----
- transplanting REV2
- change "f", but keep size N
(via 'patch.patch()')
- aborted while patching
N+1
- release wlock
- 'dirstate.write()' -1 -1 N
- 'hg status' shows "r1" as "clean" -1 -1 N
---- ----------------------------------- ---- ----- -----
To reproduce this issue in tests certainly, this patch emulates some
timing critical actions as below:
- change "f" at N
'patch.patch()' with 'fakepatchtime.py' explicitly changes mtime
of patched files to "2000-01-01 00:00" (= N).
- 'dirstate.write()' via 'repo.commit()' at N
'fakedirstatewritetime.py' forces 'pack_dirstate()' to use
"2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked
via 'committablectx.markcommitted()'.
- 'dirstate.write()' via releasing wlock at N+1 (or "not at N")
'pack_dirstate()' via releasing wlock uses actual timestamp at
runtime as "now", and it should be different from the "2000-01-01
00:00" of "f".
BTW, this patch doesn't test cases below, even though 'patch.patch()'
is used similarly in these cases:
1. failure of 'hg import' or 'hg qpush'
2. success of 'hg import', 'hg qpush' or 'hg transplant'
Case (1) above doesn't cause this kind of issue, because:
- if patching is aborted by conflicts, changed files are committed
changed files are marked as CLEAN, even though they are partially
patched.
- otherwise, dirstate are fully restored by 'dirstateguard'
For example in timetable above, timestamp of "f" in .hg/dirstate
is restored to -1 (or less than N), and subsequent 'hg status' can
detect changes correctly.
Case (2) always causes 'repo.status()' invocation via 'repo.commit()'
just after changing files inside same wlock scope.
---- ----------------------------------- ----------------
timestamp of "f"
----------------
dirstate file-
time action mem file system
---- ----------------------------------- ---- ----- -----
N *** ***
- make file "f" clean N
- execute 'hg foobar'
....
- 'dirstate.normal("f")' N ***
(e.g. via dirty check
or previous 'repo.commit()')
- change "f", but keep size N
- 'repo.status()' (*1)
(via 'repo.commit()')
---- ----------------------------------- ---- ----- -----
At a glance, 'repo.status()' at (*1) seems to cause similar issue (=
"changed files are treated as clean"), but actually doesn't.
'dirstate._lastnormaltime' should be N at (*1) above, because
'dirstate.normal()' via dirty check is finished at N.
Therefore, "f" changed at N (= 'dirstate._lastnormaltime') is forcibly
treated as "unsure" at (*1), and changes are detected as expected (see
'dirstate.status()' for detail).
If 'hg import' is executed with '--no-commit', 'repo.status()' isn't
invoked just after changing files inside same wlock scope.
But preceding 'dirstate.normal()' is invoked inside another wlock
scope via 'cmdutil.bailifchanged()', and in-memory changes should be
flushed at the end of that scope.
Therefore, timestamp N of clean "f" should be replaced by -1, if
'dirstate.write()' is invoked at N. It means that condition of this
issue isn't satisfied.
Transplant is apparently allowing using revision numbers when transplanting
through http. I call this nonsense and update the test to use an explicit hash.
This "feature" will break in the next change fixing another bug.
If resolving a merge conflict result in an empty changesets, we now properly
skip the changeset instead of crashing.
Original patch from Robert Collins <robertc@robertcollins.net>.
$TESTDIR is added to the path, so this is superfluous. Also,
inconsistent use of quotes means we might have broken on tests with
paths containing spaces.
Before this patch, a part of "test-transplant.t" fails unexpectedly on
Windows environment, because semicolon (";") in HGEDITOR isn't
recognized as the command separator by "cmd.exe".
This patch newly introduces "checkeditform-n-cat.sh" script to invoke
"cat" in it safely anywhere.
The tests often set ui.interactive to control normally interactive prompts from
stdin. That gave an output where it was non-obvious what prompts got which
which response, and the output lacked the newline users would see after input.
Instead, if the input not is a tty, write the selection and a newline.
"editform" argument for "getcommiteditor" is decided according to the
format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes in COMMAND
This patch newly adds "normal" and "merge" as ROUTE, to distinguish
merge commits from other.
This patch adds 4 test patterns to test combination of "merge"(x2) and
"--continue"(x2).
Before this patch, transplant extension shows the list of available
responses by specific string, even though the prompt string passed to
"ui.promptchoice()" has enough (maybe i18n-ed) information.
This patch uses "ui.extractchoices()" to show the list of available
responses.
Before this patch, transplant extension uses "ui.prompt()" for
interactive transplant, and has to check whether user response
returned by "ui.prompt()" is valid or not in own code.
In addition to it, transplant extension uses response characters
(e.g. "y", "n", and so on) directly in own code, and this disallows to
use another response characters by translation, even though the help
shown by '?' typing is translatable.
This patch uses "ui.promptchoice()" instead of "ui.prompt()" to
resolve problems above.
Before this patch, transplant with "--merge" option fails with
traceback unexpectedly, if it causes pull from the source repository
on the local host.
"discovery.findcommonincoming()" invokes "capable()" method on the
object given from "localrepository.pull()", but it is
"localrepository" object in this case and doesn't have such method.
This patch uses peer object of source repository as "remote" argument
for "localrepository.pull()" invocation like other invocations of it
in transplant.py.
Problem:
getremotechanges would return the 'other' repo if nothing was incoming and
there thus wasn't any bundle to base the repo on. The 'other' could be a http
peer which only implement the functionality available over the http protocol.
Transplant could thus fail with
TypeError: argument of type 'httppeer' is not iterable
Solution:
Return the local repo instead of the remote peer if there is no reason to place
a bundlerepo on top of the local repo.
If patch.patch() reports patched files when applying a changeset and the
following commit says nothing changed, transplant used to abort with a
RuntimeError, assuming something went wrong with patching.
The mismatch is patch.patch() reports patched files, not changed ones.
It could be modified to report changed files but it means duplicating
work from status, may be expensive in the case of binary files, and is
probably not that useful at API level. For instance, if two patches are
applied on the working directory, the outcome may be nothing changed
while each call would have returned modified files. The caller would
have to call status() itself again.
This patch fixes the issue by trusting patching code: if the patch
succeeded and commit reports nothing changed, then nothing changed,
patch() did not "dropped changes on the floor".
This predicate is used to find csets that were created because of a graft,
transplant or rebase --keep. An optional revset can be supplied, in which case
the result will be limited to those copies which specified one of the revs as
the source for the command.
hg log -r destination() # csets copied from anywhere
hg log -r destination(branch(default)) # all csets copied from default
hg log -r origin(x) or destination(origin(x)) # all instances of x
This predicate will follow a cset through different types of copies. Given a
repo with a cset 'S' that is grafted to create G(S), which itself is
transplanted to become T(G(S)):
o-S
/
o-o-G(S)
\
o-T(G(S))
hg log -r destination( S ) # { G(S), T(G(S)) }
hg log -r destination( G(S) ) # { T(G(S)) }
The implementation differences between the three different copy commands (see
the origin() predicate) are not intentionally exposed, however if the
transplant was a graft instead:
hg log -r destination( G(S) ) # {}
because the 'extra' field in G(G(S)) is S, not G(S). The implementation cannot
correct this by following sources before G(S) and then select the csets that
reference those sources because the cset provided to the predicate would also
end up selected. If there were more than two copies, sources of the argument
would also get selected.
Note that the convert extension does not currently update the 'extra' map in its
destination csets, and therefore copies made prior to the convert will be
missing from the resulting set.
Instead of the loop over 'subset', the following almost works, but does not
select a transplant of a transplant. That is, 'destination(S)' will only
select T(S).
dests = set([r for r in subset if _getrevsource(repo, r) in args])
Some tests ended up in a directory several directories deeper than $TESTTMP,
usually because some 'cd ..' had been forgotten between different test cases.
Add 'cd ..' where they are missing so the tests get back where they started.
Otherwise, all transplanted revisions are gone and the failing one cannot be
fixed (unless it is the first one).
I do not know what is the expected behaviour with rollback, probably something
pull-like. Non-conflicting cases should work as previously. But something like:
$ hg transplant r1 r2
commiting r1 as c1
failing r2
$ hg transplant --continue
committing r2 as c2
$ hg rollback
would reset the repository to its state before the "transplant --continue"
instead of the whole transplant session. To fix this we might need a way to
open an existing journal file, not sure this is worth the pain.
This change permits the transplant extension to operate on merge
changesets by way of --parent. This is particularly useful for
workflows which cherrypick branch merges rather than each commit
within a branch.
When issuing `hg pull -r REV` in a repo with no common ancestor with the
remote repo, the message 'requesting all changes' is printed, even though only
the changese that are ancestors of REV are actually requested. This can be
confusing for users (see
http://www.selenic.com/pipermail/mercurial/2010-October/035508.html).
This silences the message if (and only if) the '-r' option was passed.
This adds a " (glob)" marker that works like a simpler version of
(re): "*" is converted to ".*", and "?" is converted to ".".
Both special characters can be escaped using "\", and the backslash
itself can be escaped as well.
Other glob-style syntax, like "**", "[chars]", or "[!chars]", isn't
supported.
Consider this test:
$ hg glog --template '{rev}:{node|short} "{desc}"\n'
@ 2:20c4f79fd7ac "3"
|
| o 1:38f24201dcab "2"
|/
o 0:2a18120dc1c9 "1"
Because each line beginning with "|" can be compiled as a regular
expression (equivalent to ".*|"), they will match any output.
Similarly:
$ echo foo
The blank output line can be compiled as a regular expression and will
also match any output.
With this patch, none of the above output lines will be matched as
regular expressions. A line must end in " (re)" in order to be matched
as one.
Lines are still matched literally first, so the following will pass:
$ echo 'foo (re)'
foo (re)
Many tests fixed the commit date of their changesets at '1000000 0' or
similar. However testing with "Mon Jan 12 13:46:40 1970 +0000" is not
better than testing with "Thu Jan 01 00:00:00 1970 +0000", which is
the default run-tests.py installs.
Removing the unnecessary flag removes some clutter and will hopefully
make it clearer what the tests are really trying to test. Some tests
did not even change their output when the dates were changed, in which
case the -d flag was truly irrelevant.
Dates used in sequence (such as '0 0', '1 0', etc...) were left alone
since they may make the test easier to understand.
