This will be useful when reusing submerge() to improve the handling of subrepos
on mq.
# HG changeset patch
# User Angel Ezquerra <angel.ezquerra@gmail.com>
# Date 1377117244 -7200
# Wed Aug 21 22:34:04 2013 +0200
# Node ID 2defb5453f223c3027eb2f7788fbddd52bbb3352
# Parent a5c90acff5e61aae714ba6c9457d766c54b4f124
subrepo: make submerge() return the merged substate
This will be useful when reusing submerge() to improve the handling of subrepos
on mq.
Before this patch, "subrepo._calcfilehash()" opens files by "open()"
without any mode specification. This implies "text mode" on Windows.
When target file contains '\x00' byte, "read()" in "text mode" reads
file contents in without data after '\x00'.
This causes invalid SHA1 hash calculation in "subrepo._calcfilehash()".
This patch opens files in 'rb' mode to read exact data in.
Mercurial handles git subrepos by incorrect way.
If the mercurial repo has a git sub-repo and somebody started
a new branch in the subrepo and push it into git, the next one
who will clone the whole repo will get incorrect branch name in the git
subrepo.
Note that we could raise this exception even if no pattern were specified, but
the revision contained no files. However this should not happen in practice
since in that case commands.py/archive would exit earlier with an "no working
directory: please specify a revision" error message instead.
The tracebacks in subrepos are truncated at the point where the original
exception is caught and SubrepoAbort is raised in its place since 6c419dfc848c.
That hides the most relevant subrepo methods when an error occurs. Python 2.x
doesn't support chaining exceptions, so it is manually done here for manual
printing later.
This patch stops mercurial from pushing unmodified subrepos. An unmodified
subrepo is one whose store is "clean" versus a given target subrepo.
Note that subrepos may have a clean store versus a target repo but not versus another. This patch handles this scenario by individually keeping track of the state of the store versus all push targets.
Tests will be added on the following revision.
The mercurial subrepo "storeclean" method works by calculating a "store hash" of
the repository state and comparing it to a cached store hash. The store hash is
always cached when the repository is cloned from or pushed to a remote
repository, but also on pull as long as the repository already had a clean
store. If the hashes match the store is "clean" versus the selected repository.
Note that this method is currenty unused, but it will be used by a later patch.
The store hash is calculated by hashing several key repository files, such as
the bookmarks file the phaseroots file and the changelog. Note that the hash
comparison is done file by file so that we can exit early if a pair of hashes
do not match. Also the hashes are calculated starting with the file that is
most likely to be smaller upto the file that is more likely to be larger.
Currently this method is unused and it is not implemented for any specific
subrepo type (it always returns False). It receives a remote repository path
because a repository may have a clean store versus a given repository but not
versus another.
Consider a bookmark B that exists both locally and remotely. If B is updated
remotely, and then a pull is performed where the pull set contains the new
location of B, the bookmark is updated locally. However, if remote B is
updated in the middle of a pull to a location not in the pull set, the
bookmark won't be updated locally at all.
To fix this, list bookmarks before pulling in changesets, not after. This
still leaves a race open if B gets moved in between listing bookmarks and
pulling in changesets, but the race window is much smaller. Fixing the race
properly would require a bundle format upgrade.
test-hook.t's output changes because we no longer do two listkeys calls during
pull, just one.
test-pull-http.t's output changes because we now search for bookmarks before
searching for changes.
Publishing server may contains draft changeset when they are created locally. As
publishing is the default, it is actually fairly common. Because of this
"inconsistency" phases synchronization may be done even to publishing server.
This may cause severe issues for subrepo. It is possible to reference read-only
repository as subrepo. Push in a super repo recursively push subrepo. Those
pushes to potential read only repo are not optional, they are "suffered" not
"choosed". This does not break because as the repo is untouched the push is
supposed to be empty. If the reference repo locally contains draft changesets, a
courtesy push is triggered to turn them public. As the repo is read only, the
push fails (after possible prompt asking for credential). Failure of the
sub-push aborts the whole subrepo push. This force the user to define a custom
default-push for such subrepo.
This changeset introduce a prevention of this error client side by skipping the
courtesy phase synchronisation in problematic situation. The phases
synchronisation is skipped when four conditions are gathered:
- this is a subrepo push, (normal push to read-only repo)
- and remote support phase
- and remote is publishing
- and no changesets was pushed (if we pushed changesets, repo is not read only)
The internal config option used in this version is not definitive. It is here to
demonstrate a working fix to the issue.
In the future we probably wants to track subrepo changes and avoid pushing to
untouched one. That will prevent any attempt to push to read-only or unreachable
subrepo.
Another fix to prevent courtesy push from older clients to push to newer server
is also still needed.
This is an alternative fix for issue3518, enabling sharing of repositories with
subrepos, without unconditionally setting the default path in the resulting
repo's hgrc file. Better test coverage is added here, but won't prove this code
is working until f48752441ca0 is backed out.
The problem with the original fix is, if a default path is not available to be
copied over from the share source, the default path on the resulting repo is set
to the source location. Since that's where the actual repository is stored, the
path is essentially self-referential, so push, pull, incoming and outgoing
effectively operate on itself. While incoming and outgoing make it look like
nothing was changed, push currently hangs (see issue3657). In this case where
there is not a real default path, these operations should abort with
"default(-push) not found", like the source repo would. Note this problem with
the original fix affected repos without subrepos too.
Add sorted() in places found by testing with PYTHONHASHSEED=random and code
inspection.
An alternative to sprinkling sorted() all over would be to change substate to a
custom dict with sorted iterators...
This patch is meant to make it easier for tools that wrap the mercurial
output (such as TortoiseHg) to find the "in subrepo MYSUBREPO" string that
(since 6c419dfc848c) is appended after subrepo error messages, particularly when
the mercurial output is translated to a non-English language.
The message remains the same but the '%s' that was used to prepend the original
error message in front of the 'in subrepo' string has been moved out of the
translatable string.
As an example of the usefulness of making it easy to look for "in subrepo
MYSUBREPO" strings, TortoiseHg looks for these strings in error messages in
order to "linkify them" (i.e. convert "MYSUBREPO" into alink to the
corresponding subrepo).
The original string made it hard for a tool such as TortoiseHg to look for the
translated string on mercurial's output because the translated string contained
the error message itself. This meant that a regular expression was required to
ignore the error message part. With this change TortoiseHg can just get the
translated "(in subrepo %s)" string, substitute %s for the subrepo path (which
it gets from the subrepo exception) and simply search for the resulting string
(no regular expression needed, or at least a much simpler regular expression
could be used).
Additionaly, the existing string could lead a translator mistakenly assume that
it was possible invert the order of the %s (error and subrepo path) fields,
which would not work because the string interpolation was position based.
This change appends the subrepo path to subrepo errors. That is, when there
is an error performing an operation a subrepo, rather than displaying a message
such as:
pushing subrepo MYSUBREPO to PATH
searching for changes
abort: push creates new remote head HEADHASH!
hint: did you forget to merge? use push -f to force
mercurial will show:
pushing subrepo MYSUBREPO to PATH
searching for changes
abort: push creates new remote head HEADHASH! (in subrepo MYSUBREPO)
hint: did you forget to merge? use push -f to force
The rationale for this change is that the current error messages make it hard
for TortoiseHg (and similar tools) to tell the user which subrepo caused the
push failure.
The "(in subrepo MYSUBREPO)" message has been added to those subrepo methods
were it made sense (by using a decorator). We avoid appending "(in subrepo XXX)"
multiple times when subrepos are nexted by throwing a "SubrepoAbort" exception
after the extra message is appended. The decorator will then "ignore" (i.e. just
re-raise) the exception and never add the message again.
A small drawback of this method is that part of the exception trace is lost when
the exception is catched and re-raised by the annotatesubrepoerror decorator.
Also, because the state() function already printed the subrepo path when it
threw an error, that error has been changed to avoid duplicating the subrepo
path in the error message.
Note that I have also updated several subrepo related tests to reflect these
changes.
Color extension achieves colorization by overriding the class of
"ui" object just before command execution.
Before this patch, "diff()" of abstractsubrepo and classes
derived from it has no "ui" argument, so "diff()" of hgsubrepo
uses "self._repo.ui" to invoke "cmdutil.diffordiffstat()".
For separation of configuration between repositories, revision
1498948ee815 changed the initialization source of "self._repo.ui"
from "ui"(overridden) to "baseui"(plain) of parent repository.
And this caused break of colorization.
This patch adds "ui" argument to "diff()" of abstractsubrepo and
classes derived from it to pass "ui" object of caller side.
Files in a subrepo were overwritten on update. But this should only happen on a
clean update (example: -C is specified).
Use the overwrite parameter introduced for svn subrepos in e3640daa4703 to
decide whether to merge changes (as update) or remove them (as clean).
The new function hg.updaterepo is intruduced to keep all update calls in hg.
test-subrepo.t is extended to test if an untracked file is overwritten
(issue3276). (Update -C is already tested in many places.)
The first two chunks are debugging output which has changed. (Because overwrite
is not always true anymore for subrepos)
All other tests still pass without any change.
Do not pass ui because it contains the configuration of the repo. It is the
same object as repo.ui.
When a repo is passed to hg.peer, the global configuration is read from
repo.baseui.
Before this patch, repository local configurations are not isolated
between repositories in subrepo tree, because "localrepository"
objects for each subrepositories are created with "ui" instance of the
parent of each ones.
So, local configuration of the parent or higher repositories are
visible also in children or lower ones.
This patch uses "baseui" instead of "ui" to create repository object:
the former contains only global configuration.
This patch also copies 'ui.commitsubrepos' configuration to commit
recursively in subrepo tree, because it may be set in not
"repo.baseui" but "repo.ui".
svn --version --quiet is implemented since svn 0.14.1 (August 2002)
and prints just the version number, not the long output (21 lines)
of "svn --version".
Additionally I expect this output format to be more stable, at least
it is not changed with different translations.
For example LC_ALL=de_DE.utf-8 would cause the version check to fail,
because "svn, Version 1.6.12 (r955767)" with a capital "V" will be printed.
Using "svn --version --quiet" would only print the version number, but then
matching other messages, e.g. "Committed revision" would fail.
Subversion 1.7 changes its XML output to include an explicit encoding tag:
<?xml version="1.0" encoding="UTF-8"?>
This triggers xml.dom.minidom to always return unicode strings, causing
other parts of the code to explode.
We unconditionally encode path names before handing them back, which
works with both str (actually a no-op) and unicode values.
This introduces a peer method into all repository classes, which currently
simply returns self. It also changes hg.repository so it now raises an
exception if the supplied paths does not resolve to a localrepo or descendant.
Finally, all call sites are changed to use the peer and local methods as
appropriate, where peer is used whenever the code is dealing with a remote
repository (even if it's on local disk).
Add a match object to subrepo.archive(). This will allow the -X and -I
options to be honored inside subrepos when archiving. They formerly
only affect the top level repo.
dbc9f6aea991 introduced "svn info TARGET@REV" to determine if a certain
path exists in the specified revision, but in svn 1.5 the error message
"Not a valid URL" yields exit code 0 so the error is not caught.
Use "svn list TARGET@REV" instead which works with svn 1.5 and is even
faster in some situations.
Popen does not consider "foo.cmd" equivalent to "foo" on Windows.
Unfortunately, the default MSYS Git installation installs only "git.cmd" into
the path by default. This patch probes for both possible names on Windows.
Until now, when calling subrepo.subrepo with a path at which there is no
subrepo, a "nullstate" tuple would be returned. However, this is not very
useful (the tuple can't really be used for creating a subrepo), so we'd just as
soon have the function just fail, and leave it up to the caller to decide what
to do.
The motivation for doing this now is to simplify the solution for (issue3056).
Note that aborting in subrepo.state() prevents "repairing" commands like revert
to be issued. The user will have to edit the .hgsubstate manually (but he
probably had already otherwise this would not be failing). The same behaviour
already happens with invalid .hgsub entries.
The initial version was to take the "Revision" field from svn info. It works
but produces false positive when parent paths are being moved or unrelated
changes are being committed, causing it to change while the svn checkout itself
remains the same. To avoid spurious commit, we took "Revision" and "Last
Changed Rev" for general comparison and kept the latter to answer "what is your
revision?" question. This is better but fails when the subrepo path exists at
"Revision" but not at "Last Changed Rev". This patch adds a check for this, and
returns "Revision" if the path does not exist. We try to avoid doing this as
much as possible at it implies an extra, *remote* call.
Previous code was printing a traceback because it expected some error output
from svn. But sometimes our definition of "changed" differs with the subversion
one. For instance, subversion ignores missing files when committing. And when
there are only missing files, svn commit will be a successful no-op with no
output. Still, we should stick to our definition including missing files in
changes as doing otherwise could cause surprising behaviour for the user.
