Previously, revert was only possible if the '--no-backup'
switch was specified.
Now, to support backups, we explicitly go over all modified
files in the subrepo.
Up until now we compared the "path" and "sharedpath" repository properties to
check if a repository is shared. This was relying an implementation detail of
shared repositories. In order to make it possible to change the way shared
repositories are implemented, we encapsulate this check into its own localrepo
method, called shared.
This new method returns None if the repository is shared, and something else
(for now a string describing the short of share) otherwise.
The reason why I did not call this method "isshared" and made it return a
boolean is that I plan to introduce a new type of shared repository soon.
# NOTES:
This is the first patch in a series whose purpose is to add support for
creating "full repository shares", which are repositories that share everything
with the repository source except their current revision, branch and bookmark.
This series is RFC because I am not very sure of some of the solutions I took.
Comments are welcome!
This no longer needs to be explicitly passed because the subrepo object tracks
the 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in revert is to emit status and warning messages, and to
check the verbose flag prior to printing the action to be performed on a file.
The local repo's ui was already being used to print a warning message in
wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status()
call. Unlike other methods where the matcher is passed along and narrowed, a
new matcher is created in each repo, and therefore the bad() method already used
the local repo's ui.
This no longer needs to be explicitly passed because the subrepo object tracks
the 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in remove is to emit status and warning messages, and to
check the verbose flag prior to printing files to be removed. The bad() method
on the matcher still uses the root repo's ui, because narrowing the matcher
doesn't change the ui object.
The local repo's ui was already being used to print a warning message in
wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status()
call.
This no longer needs to be explicitly passed because the subrepo object tracks
the 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in forget is to emit status and warning messages, and to
check the verbose flag prior to printing files to be forgotten. The bad()
method on the matcher still uses the root repo's ui, because narrowing the
matcher doesn't change the ui object.
The local repo's ui was already being used to print a warning message in
wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status()
call.
This no longer needs to be explicitly passed because the subrepo object tracks
a 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in cat is to emit a status message when a subrepo is
missing. The bad() method on the matcher still uses the root repo's ui, because
narrowing the matcher doesn't change the ui object.
The current state of subrepo methods is to pass a 'ui' object to some methods,
which has the effect of overriding the subrepo configuration since it is the
root repo's 'ui' that is passed along as deep as there are subrepos. Other
subrepo method are *not* passed the root 'ui', and instead delegate to their
repo object's 'ui'. Even in the former case where the root 'ui' is available,
some methods are inconsistent in their use of both the root 'ui' and the local
repo's 'ui'. (Consider hg._incoming() uses the root 'ui' for path expansion
and some status messages, but also calls bundlerepo.getremotechanges(), which
eventually calls discovery.findcommonincoming(), which calls
setdiscovery.findcommonheads(), which calls status() on the local repo 'ui'.)
This inconsistency with respect to the configured output level is probably
always hidden, because --verbose, --debug and --quiet, along with their 'ui.xxx'
equivalents in the global and user level hgrc files are propagated from the
parent repo to the subrepo via 'baseui'. The 'ui.xxx' settings in the parent
repo hgrc file are not propagated, but that seems like an unusual thing to set
on a per repo config file. Any 'ui.xxx' options changed by --config are also
not propagated, because they are set on repo.ui by dispatch.py, not repo.baseui.
The goal here is to cleanup the subrepo methods by dropping the 'ui' parameter,
which in turn prevents mixing subtly different 'ui' instances on a given subrepo
level. Some methods use more than just the output level settings in 'ui' (add
for example ends up calling scmutil.checkportabilityalert() with both the root
and local repo's 'ui' at different points). This series just goes for the low
hanging fruit and switches methods that only use the output level.
If we really care about not letting a subrepo config override the root repo's
output level, we can propagate the verbose, debug and quiet settings to the
subrepo in the same way 'ui.commitsubrepos' is in hgsubrepo.__init__.
Archive only uses the 'ui' object to call its progress() method, and gitsubrepo
calls status().
Creation of the subrepo's '_repo' object creates a new 'ui' by combining the
parent repo's 'baseui' and reading in the subrepo's hgrc file. This simply
avoids 'self.ui' and 'self._repo.ui' pointing to different objects, which seems
like a potential source of bugs.
Git and Svn subrepos are unchanged, because they don't have their own ui, and
have always used their parent's for their configuration.
The localrepository class has a 'ui' member, so keeping the names the same will
allow for duck typing with subrepo instances when accessing 'ui'. Changing this
is easier than finding all of the localrepository instance uses and renaming
that to '_ui'.
Previously, git subrepos did not support reverting.
This change adds basic support for reverting
when '--no-backup' is specified.
A warning is given (and the current state is kept)
when a revert is done without the '--no-backup' flag.
Since addremove on the top of a directory tree will recursively handle sub
directories, it should be the same with deep subrepos, once the user has
explicitly asked to process a subrepo. This really only has an effect when a
path that is a subrepo (or is in a subrepo) is given, since -S causes all
subrepos to be processed already. An addremove without a path that crosses into
a subrepo, will still not enter any subrepos, per backward compatibility rules.
The recursive addremove operation occurs completely before the first subrepo is
committed. Only hg subrepos support the addremove operation at the moment- svn
and git subrepos will warn and abort the commit.
This will be used in the next patch to print a warning from the base class. It
seems better than having to explicitly pass it to a new method, since a lot of
existing methods also require it.
So far, git subrepositories were silently ignored for diffs.
This patch adds support for git subrepositories,
with the remark that --include and --exclude are not supported.
If --include or --exclude are used, the subrepo is ignored.
This patch also replaces "self._getstorehashcachepath" (building
absolute path up) by "self._getstorehashcachename" (building relative
path up), because "vfs.writelines" requires relative path.
This patch also replaces "self._getstorehashcachepath" (building
absolute path up) by "self._getstorehashcachename" (building relative
path up), because "vfs.tryreadlines" requires relative path.
This patch makes "_readstorehashcache()" return "[]" (returned by
"vfs.tryreadlines()"), when cache file doesn't exist, even though
"_readstorehashcache()" returned '' (empty string) in such case before
this patch.
"_readstorehashcache()" is invoked only by the code path below in
"_storeclean()":
for filehash in self._readstorehashcache(path):
if filehash != itercache.next():
clean = False
break
In this case, "[]" and '' don't differ from each other, because both
of them cause avoiding iteration of "for loop".
This "vfs" object will be used by subsequent patches to handle cache
store hash files without direct file APIs.
This patch decorates "_cachestorehashvfs" with "@propertycache" to
delay vfs creation, because it is used only for cooperation with other
repositories.
In this patch, "/" is used as the path separator, even though
"self._repo.join" uses platform specific path separator (e.g. "\\" on
Windows). But it is reasonable enough, because "store" and other
management file handling already include such implementation, and they
work well.
"_calcfilehash" can be completely replaced by simple "vfs.tryread"
invocation.
def _calcfilehash(filename):
data = ''
if os.path.exists(filename):
fd = open(filename, 'rb')
data = fd.read()
fd.close()
return util.sha1(data).hexdigest()
Building absolute path "absname" up by "self._repo.join" for files in
"filelist" is avoided, because "vfs.tryread" does so internally.
Existance of specified "path" should be examined by "exists" via wvfs
of the parent repository, because the working directory of the parent
repository may be in UTF-8 mode. Wide API should be used via wvfs in
such case.
In this patch, "/" is used as the path separator, even though "path"
uses platform specific path separator (e.g. "\\" on Windows). But it
is reasonable enough, because "store" and other management file
handling already include such implementation, and they work well.
"util.makedirs" for the (sub-)repository root of "hgsubrepo" is also
executed in the constructor of "localrepository", if "create" is True
and ".hg" of it doesn't exist.
This patch avoids redundant "util.makedirs" invocation in the
constructor of "hgsubrepo".
Like 'forget', git and svn subrepos are currently not supported. Unfortunately
the name 'remove' is already used in the subrepo classes, so we break the
convention of naming the subrepo function after the command.
We do modify the lists that make up the status in several places, so
it seems risky to use the same instance of a list for several
different status types. Use a separate empty list for each type
instead.
Before this patch, "close()" for the file object opened in
"_initrepo()" may not be executed, if unexpected exception is raised,
because it isn't executed in "finally" clause.
This patch ensures "close()" execution at the end of "_initrepo()" by
moving it into "finally" clause.
This patch puts configuration lines into "lines" array and write them
out at once, to narrow the scope of "try"/"finally" for review-ability.
This patch doesn't use "vfs.write()", because:
- current "vfs.write()" implementation doesn't take "mode" argument
to open file in "text" mode
- writing hgrc file out in binary mode may break backward compatibility
Before this patch, "close()" for the file object opened in
"_cachestorehash()" may not be executed, if unexpected exception is
raised, because it isn't executed in "finally" clause.
This patch ensures "close()" execution at the end of
"_cachestorehash()" by moving it into "finally" clause.
Before this patch, "close()" for the file object opened in
"_readstorehashcache()" may not be executed, if unexpected exception
is raised, because it isn't executed in "finally" clause.
This patch ensures "close()" execution at the end of
"_readstorehashcache()" by moving it into "finally" clause.
Before this patch, "close()" for the file object opened in
"_calcfilehash()" may not be executed, if unexpected exception is
raised, because it isn't executed in "finally" clause.
This patch ensures "close()" execution at the end of "_calcfilehash()"
by moving it into "finally" clause.
Before this patch, "lock.release()" for "self._repo" in
"_cachestorehash()" of "hgsubrepo" may not be executed, if unexpected
exception is raised, because it isn't executed in "finally" clause.
This patch ensures "lock.release()" execution at the end of
"_cachestorehash()" by moving it into "finally" clause.
Before this patch, "lock.release()" for "self._repo" in "storeclean()"
of "hgsubrepo" may not be executed, if unexpected exception is raised,
because it isn't executed in "finally" clause.
This patch ensures "lock.release()" execution at the end of
"storeclean()" by moving it into "finally" clause.
This patch chooses moving almost all lines in "storeclean()" into
"_storeclean()" instead of indenting them for "try/finally" clauses,
to keep diff simple for review-ability.
Before this patch, "reporelpath()" uses "rstrip(os.sep)" to trim
"os.sep" at the end of "parent.root" path.
But it doesn't work correctly with some problematic encodings on
Windows, because some multi-byte characters in such encodings contain
'\\' (0x5c) as the tail byte of them.
In such cases, "reporelpath()" leaves unexpected '\\' at the beginning
of the path returned to callers.
"lcalrepository.root" seems not to have tail "os.sep", because it is
always normalized by "os.path.realpath()" in "vfs.__init__()", but in
fact it has tail "os.sep", if it is a root (of the drive): path
normalization trims tail "os.sep" off "/foo/bar/", but doesn't trim
one off "/".
So, just avoiding "rstrip(os.sep)" in "reporelpath()" causes
regression around issue3033 fixed by e3dfde137fa5.
This patch introduces "pathutil.normasprefix" to normalize specified
path in the specific way for problematic encodings without regression
around issue3033.
Before this patch, sanitizing ".hg/hgrc" scans directories and files
also in meta data area for non-hg subrepos: under ".svn" for
Subversion subrepo, for example.
This may cause not only performance impact (especially in large scale
subrepos) but also unexpected removing meta data files.
This patch avoids sanitizing ".hg/hgrc" in meta data area for non-hg
subrepos.
This patch stops checking "ignore" target at the first
(case-insensitive) appearance of it, because continuation of scanning
is meaningless in almost all cases.
Before this patch, "hg update" doesn't sanitize ".hg/hgrc" in non-hg
subrepos correctly, if "hg update" is executed not at the root of the
parent repository.
"_sanitize()" takes relative path to subrepo from the root of the
parent repository, and passes it to "os.walk()". In this case,
"os.walk()" expects CWD to be equal to the root of the parent
repository.
So, "os.walk()" can't find specified path (or may scan unexpected
path), if CWD isn't equal to the root of the parent repository.
Non-hg subrepo under nested hg-subrepos may cause same problem, too:
CWD may be equal to the root of the outer most repository, or so.
This patch makes "_sanitize()" take absolute path to the root of
subrepo to sanitize correctly in such cases.
This patch doesn't normalize the path to hostile files as the one
relative to CWD (or the root of the outer most repository), to fix the
problem in the simple way suitable for "stable".
Normalizing should be done in the future: maybe as a part of the
migration to vfs.
Before this patch, sanitizing ".hg/hgrc" in git subrepo doesn't work,
when the working directory is updated by "git merge --ff".
"_sanitize()" is not invoked after checking target revision out into
the working directory in this case, even though it is invoked
indirectly via "checkout" (or "rawcheckout") in other cases.
This patch invokes "_sanitize()" explicitly also after "git merge
--ff" execution.
"_sanitize()" was introduced by 5131f2755f60 on "stable" branch, but
it has done nothing for sanitizing since 5131f2755f60.
"_sanitize()" assumes "Visitor" design pattern:
"os.walk()" should invoke specified function ("v" in this case)
for each directory elements under specified path
but "os.walk()" assumes "Iterator" design pattern:
callers of it should drive loop to scan each directory elements
under specified path by themselves with the returned generator
object
Because of this mismatching, "_sanitize()" just discards the generator
object returned by "os.walk()" and does nothing for sanitizing.
This patch makes "_sanitize()" work.
This patch also changes the format of warning message to show each
unlinked files, for multiple appearances of "potentially hostile
.hg/hgrc".
The cat command with an explicit path into a subrepo is now handled by invoking
cat on the file, from that subrepo. The previous behavior was to complain that
the file didn't exist in the revision (of the top most repo). Now when the file
is actually missing, the revision of the subrepo is named instead (though it is
probably desirable to continue naming the top level repo).
The documented output formatters %d and %p reflect the path from the top level
repo, since the purpose of this is to give the illusion of a unified repository.
Support for the undocumented (for cat) formatters %H, %R, %h, %m and %r was
added long ago (I tested back as far as 0.5), but unfortunately these will
reflect the subrepo node instead of the parent context.
The previous implementation was a bit loose with the return value, i.e. it would
return 0 if _any_ file requested was cat'd successfully. This maintains that
behavior.
Before this patch, "contrib/check-code.py" can't detect these
problems, because the regexp pattern to detect "% inside _()" doesn't
suppose the case that format string consists of multiple string
components concatenated implicitly or explicitly,
This patch does below for that regexp pattern to detect "% inside _()"
problems in such case.
- put "+" into separator part ("[ \t\n]") for explicit concatenation
("...." + "...." style)
- enclose "component and separator" part by "(?:....)+" for
concatenation itself ("...." "...." or "...." + "....")
Before this patch, "contrib/check-code.py" can't detect these
problems, because the regexp pattern to detect "% inside _()" doesn't
suppose the case that the format string and "%" aren't placed in the
same line.
This patch replaces "\s" in that regexp pattern with "[ \t\n]" to
detect "% inside _()" problems in such case.
"[\s\n]" can't be used in this purpose, because "\s" is automatically
replaced with "[ \t]" by "_preparepats()" and "\s" in "[]" causes
nested "[]" unexpectedly.
Some extensions set configuration settings that showed up in 'hg showconfig
--debug' with 'none' as source. That was confusing.
Instead, they will now tell which extension they come from.
This change tries to be consistent and specify a source everywhere - also where
it perhaps is less relevant.
When a subrepo revision was hidden it was considered missing and mercurial was
unable to update to the corresponding parent revision. Instead warn the user of
the problem and let it choose what to do (the default is to udpate anyway).
This revision has no behaviour change. It simply removes an unnecessary else
that follows an if / return block. The change looks big because a big chunk of
code has been unindented one level.
If a subrepo revision is hidden (because it was amended, for example) it does
not make sense to try to "get" it from the remote subrepository.
Note that in order to avoid making the change look bigger than it is, this adds
an unnecessary else clause. This will be removed on a follow up patch.
Before this patch, phase of newly created commit is determined by
"phases.new-commit" configuration regardless of phase of state in each
subrepositories.
For example, this may cause the "public" revision in the parent
repository referring the "secret" one in subrepository.
This patch checks phase of state in each subrepositories before
committing in the parent, and aborts or changes phase of newly created
commit if subrepositories have more restricted phase than the parent.
This patch uses "follow" as default value of "phases.checksubrepos"
configuration, because it can keep consistency between phases of the
parent and subrepositories without breaking existing tool chains.
When a subrepo has changed on the local and remote revisions, prompt the user
whether it wants to merge those subrepo revisions, keep the local revision or
keep the remote revision.
Up until now mercurial would always perform a merge on a subrepo that had
changed on the local and the remote revisions. This is often inconvenient. For
example:
- You may want to perform the actual subrepo merge after you have merged the
parent subrepo files.
- Some subrepos may be considered "read only", in the sense that you are not
supposed to add new revisions to them. In those cases "merging a subrepo" means
choosing which _existing_ revision you want to use on the merged revision. This
is often the case for subrepos that contain binary dependencies (such as DLLs,
etc).
This new prompt makes mercurial better cope with those common scenarios.
Notes:
- The default behavior (which is the one that is used when ui is not
interactive) remains unchanged (i.e. merge is the default action).
- This prompt will be shown even if the ui --tool flag is set.
- I don't know of a way to test the "keep local" and "keep remote" options (i.e.
to force the test to choose those options).
# HG changeset patch
# User Angel Ezquerra <angel.ezquerra@gmail.com>
# Date 1378420708 -7200
# Fri Sep 06 00:38:28 2013 +0200
# Node ID 2fb9cb0c7b26303ac3178b7739975e663075857d
# Parent 796d34e1b749b79834321ef1181ed8433a5515d9
merge: let the user choose to merge, keep local or keep remote subrepo revisions
When a subrepo has changed on the local and remote revisions, prompt the user
whether it wants to merge those subrepo revisions, keep the local revision or
keep the remote revision.
Up until now mercurial would always perform a merge on a subrepo that had
changed on the local and the remote revisions. This is often inconvenient. For
example:
- You may want to perform the actual subrepo merge after you have merged the
parent subrepo files.
- Some subrepos may be considered "read only", in the sense that you are not
supposed to add new revisions to them. In those cases "merging a subrepo" means
choosing which _existing_ revision you want to use on the merged revision. This
is often the case for subrepos that contain binary dependencies (such as DLLs,
etc).
This new prompt makes mercurial better cope with those common scenarios.
Notes:
- The default behavior (which is the one that is used when ui is not
interactive) remains unchanged (i.e. merge is the default action).
- This prompt will be shown even if the ui --tool flag is set.
- I don't know of a way to test the "keep local" and "keep remote" options (i.e.
to force the test to choose those options).
This will be useful when reusing submerge() to improve the handling of subrepos
on mq.
# HG changeset patch
# User Angel Ezquerra <angel.ezquerra@gmail.com>
# Date 1377117244 -7200
# Wed Aug 21 22:34:04 2013 +0200
# Node ID 2defb5453f223c3027eb2f7788fbddd52bbb3352
# Parent a5c90acff5e61aae714ba6c9457d766c54b4f124
subrepo: make submerge() return the merged substate
This will be useful when reusing submerge() to improve the handling of subrepos
on mq.
Before this patch, "subrepo._calcfilehash()" opens files by "open()"
without any mode specification. This implies "text mode" on Windows.
When target file contains '\x00' byte, "read()" in "text mode" reads
file contents in without data after '\x00'.
This causes invalid SHA1 hash calculation in "subrepo._calcfilehash()".
This patch opens files in 'rb' mode to read exact data in.
Mercurial handles git subrepos by incorrect way.
If the mercurial repo has a git sub-repo and somebody started
a new branch in the subrepo and push it into git, the next one
who will clone the whole repo will get incorrect branch name in the git
subrepo.
Note that we could raise this exception even if no pattern were specified, but
the revision contained no files. However this should not happen in practice
since in that case commands.py/archive would exit earlier with an "no working
directory: please specify a revision" error message instead.
The tracebacks in subrepos are truncated at the point where the original
exception is caught and SubrepoAbort is raised in its place since 6c419dfc848c.
That hides the most relevant subrepo methods when an error occurs. Python 2.x
doesn't support chaining exceptions, so it is manually done here for manual
printing later.
We were always using only the first 12 characters of the subrepo revision id
when generating the "subrepo diverged" promptchoice. This is not necessarily
correct for non mercurial subrepos.
This method takes an "id" (e.g. a revision id) and returns a "short" version
(e.g. a short revision id).
This will be used on the next revision to fix a small bug in the way that the
text on the promptchoice shown when a subrepo diverges is generated.
This patch stops mercurial from pushing unmodified subrepos. An unmodified
subrepo is one whose store is "clean" versus a given target subrepo.
Note that subrepos may have a clean store versus a target repo but not versus another. This patch handles this scenario by individually keeping track of the state of the store versus all push targets.
Tests will be added on the following revision.
The mercurial subrepo "storeclean" method works by calculating a "store hash" of
the repository state and comparing it to a cached store hash. The store hash is
always cached when the repository is cloned from or pushed to a remote
repository, but also on pull as long as the repository already had a clean
store. If the hashes match the store is "clean" versus the selected repository.
Note that this method is currenty unused, but it will be used by a later patch.
The store hash is calculated by hashing several key repository files, such as
the bookmarks file the phaseroots file and the changelog. Note that the hash
comparison is done file by file so that we can exit early if a pair of hashes
do not match. Also the hashes are calculated starting with the file that is
most likely to be smaller upto the file that is more likely to be larger.
Currently this method is unused and it is not implemented for any specific
subrepo type (it always returns False). It receives a remote repository path
because a repository may have a clean store versus a given repository but not
versus another.
Consider a bookmark B that exists both locally and remotely. If B is updated
remotely, and then a pull is performed where the pull set contains the new
location of B, the bookmark is updated locally. However, if remote B is
updated in the middle of a pull to a location not in the pull set, the
bookmark won't be updated locally at all.
To fix this, list bookmarks before pulling in changesets, not after. This
still leaves a race open if B gets moved in between listing bookmarks and
pulling in changesets, but the race window is much smaller. Fixing the race
properly would require a bundle format upgrade.
test-hook.t's output changes because we no longer do two listkeys calls during
pull, just one.
test-pull-http.t's output changes because we now search for bookmarks before
searching for changes.
Publishing server may contains draft changeset when they are created locally. As
publishing is the default, it is actually fairly common. Because of this
"inconsistency" phases synchronization may be done even to publishing server.
This may cause severe issues for subrepo. It is possible to reference read-only
repository as subrepo. Push in a super repo recursively push subrepo. Those
pushes to potential read only repo are not optional, they are "suffered" not
"choosed". This does not break because as the repo is untouched the push is
supposed to be empty. If the reference repo locally contains draft changesets, a
courtesy push is triggered to turn them public. As the repo is read only, the
push fails (after possible prompt asking for credential). Failure of the
sub-push aborts the whole subrepo push. This force the user to define a custom
default-push for such subrepo.
This changeset introduce a prevention of this error client side by skipping the
courtesy phase synchronisation in problematic situation. The phases
synchronisation is skipped when four conditions are gathered:
- this is a subrepo push, (normal push to read-only repo)
- and remote support phase
- and remote is publishing
- and no changesets was pushed (if we pushed changesets, repo is not read only)
The internal config option used in this version is not definitive. It is here to
demonstrate a working fix to the issue.
In the future we probably wants to track subrepo changes and avoid pushing to
untouched one. That will prevent any attempt to push to read-only or unreachable
subrepo.
Another fix to prevent courtesy push from older clients to push to newer server
is also still needed.
This is an alternative fix for issue3518, enabling sharing of repositories with
subrepos, without unconditionally setting the default path in the resulting
repo's hgrc file. Better test coverage is added here, but won't prove this code
is working until f48752441ca0 is backed out.
The problem with the original fix is, if a default path is not available to be
copied over from the share source, the default path on the resulting repo is set
to the source location. Since that's where the actual repository is stored, the
path is essentially self-referential, so push, pull, incoming and outgoing
effectively operate on itself. While incoming and outgoing make it look like
nothing was changed, push currently hangs (see issue3657). In this case where
there is not a real default path, these operations should abort with
"default(-push) not found", like the source repo would. Note this problem with
the original fix affected repos without subrepos too.
Add sorted() in places found by testing with PYTHONHASHSEED=random and code
inspection.
An alternative to sprinkling sorted() all over would be to change substate to a
custom dict with sorted iterators...
This patch is meant to make it easier for tools that wrap the mercurial
output (such as TortoiseHg) to find the "in subrepo MYSUBREPO" string that
(since 6c419dfc848c) is appended after subrepo error messages, particularly when
the mercurial output is translated to a non-English language.
The message remains the same but the '%s' that was used to prepend the original
error message in front of the 'in subrepo' string has been moved out of the
translatable string.
As an example of the usefulness of making it easy to look for "in subrepo
MYSUBREPO" strings, TortoiseHg looks for these strings in error messages in
order to "linkify them" (i.e. convert "MYSUBREPO" into alink to the
corresponding subrepo).
The original string made it hard for a tool such as TortoiseHg to look for the
translated string on mercurial's output because the translated string contained
the error message itself. This meant that a regular expression was required to
ignore the error message part. With this change TortoiseHg can just get the
translated "(in subrepo %s)" string, substitute %s for the subrepo path (which
it gets from the subrepo exception) and simply search for the resulting string
(no regular expression needed, or at least a much simpler regular expression
could be used).
Additionaly, the existing string could lead a translator mistakenly assume that
it was possible invert the order of the %s (error and subrepo path) fields,
which would not work because the string interpolation was position based.
This change appends the subrepo path to subrepo errors. That is, when there
is an error performing an operation a subrepo, rather than displaying a message
such as:
pushing subrepo MYSUBREPO to PATH
searching for changes
abort: push creates new remote head HEADHASH!
hint: did you forget to merge? use push -f to force
mercurial will show:
pushing subrepo MYSUBREPO to PATH
searching for changes
abort: push creates new remote head HEADHASH! (in subrepo MYSUBREPO)
hint: did you forget to merge? use push -f to force
The rationale for this change is that the current error messages make it hard
for TortoiseHg (and similar tools) to tell the user which subrepo caused the
push failure.
The "(in subrepo MYSUBREPO)" message has been added to those subrepo methods
were it made sense (by using a decorator). We avoid appending "(in subrepo XXX)"
multiple times when subrepos are nexted by throwing a "SubrepoAbort" exception
after the extra message is appended. The decorator will then "ignore" (i.e. just
re-raise) the exception and never add the message again.
A small drawback of this method is that part of the exception trace is lost when
the exception is catched and re-raised by the annotatesubrepoerror decorator.
Also, because the state() function already printed the subrepo path when it
threw an error, that error has been changed to avoid duplicating the subrepo
path in the error message.
Note that I have also updated several subrepo related tests to reflect these
changes.
Color extension achieves colorization by overriding the class of
"ui" object just before command execution.
Before this patch, "diff()" of abstractsubrepo and classes
derived from it has no "ui" argument, so "diff()" of hgsubrepo
uses "self._repo.ui" to invoke "cmdutil.diffordiffstat()".
For separation of configuration between repositories, revision
1498948ee815 changed the initialization source of "self._repo.ui"
from "ui"(overridden) to "baseui"(plain) of parent repository.
And this caused break of colorization.
This patch adds "ui" argument to "diff()" of abstractsubrepo and
classes derived from it to pass "ui" object of caller side.
Files in a subrepo were overwritten on update. But this should only happen on a
clean update (example: -C is specified).
Use the overwrite parameter introduced for svn subrepos in e3640daa4703 to
decide whether to merge changes (as update) or remove them (as clean).
The new function hg.updaterepo is intruduced to keep all update calls in hg.
test-subrepo.t is extended to test if an untracked file is overwritten
(issue3276). (Update -C is already tested in many places.)
The first two chunks are debugging output which has changed. (Because overwrite
is not always true anymore for subrepos)
All other tests still pass without any change.
Do not pass ui because it contains the configuration of the repo. It is the
same object as repo.ui.
When a repo is passed to hg.peer, the global configuration is read from
repo.baseui.
Before this patch, repository local configurations are not isolated
between repositories in subrepo tree, because "localrepository"
objects for each subrepositories are created with "ui" instance of the
parent of each ones.
So, local configuration of the parent or higher repositories are
visible also in children or lower ones.
This patch uses "baseui" instead of "ui" to create repository object:
the former contains only global configuration.
This patch also copies 'ui.commitsubrepos' configuration to commit
recursively in subrepo tree, because it may be set in not
"repo.baseui" but "repo.ui".
svn --version --quiet is implemented since svn 0.14.1 (August 2002)
and prints just the version number, not the long output (21 lines)
of "svn --version".
Additionally I expect this output format to be more stable, at least
it is not changed with different translations.
For example LC_ALL=de_DE.utf-8 would cause the version check to fail,
because "svn, Version 1.6.12 (r955767)" with a capital "V" will be printed.
Using "svn --version --quiet" would only print the version number, but then
matching other messages, e.g. "Committed revision" would fail.
Subversion 1.7 changes its XML output to include an explicit encoding tag:
<?xml version="1.0" encoding="UTF-8"?>
This triggers xml.dom.minidom to always return unicode strings, causing
other parts of the code to explode.
We unconditionally encode path names before handing them back, which
works with both str (actually a no-op) and unicode values.
This introduces a peer method into all repository classes, which currently
simply returns self. It also changes hg.repository so it now raises an
exception if the supplied paths does not resolve to a localrepo or descendant.
Finally, all call sites are changed to use the peer and local methods as
appropriate, where peer is used whenever the code is dealing with a remote
repository (even if it's on local disk).
Add a match object to subrepo.archive(). This will allow the -X and -I
options to be honored inside subrepos when archiving. They formerly
only affect the top level repo.
dbc9f6aea991 introduced "svn info TARGET@REV" to determine if a certain
path exists in the specified revision, but in svn 1.5 the error message
"Not a valid URL" yields exit code 0 so the error is not caught.
Use "svn list TARGET@REV" instead which works with svn 1.5 and is even
faster in some situations.
Popen does not consider "foo.cmd" equivalent to "foo" on Windows.
Unfortunately, the default MSYS Git installation installs only "git.cmd" into
the path by default. This patch probes for both possible names on Windows.
Until now, when calling subrepo.subrepo with a path at which there is no
subrepo, a "nullstate" tuple would be returned. However, this is not very
useful (the tuple can't really be used for creating a subrepo), so we'd just as
soon have the function just fail, and leave it up to the caller to decide what
to do.
The motivation for doing this now is to simplify the solution for (issue3056).
Note that aborting in subrepo.state() prevents "repairing" commands like revert
to be issued. The user will have to edit the .hgsubstate manually (but he
probably had already otherwise this would not be failing). The same behaviour
already happens with invalid .hgsub entries.
The initial version was to take the "Revision" field from svn info. It works
but produces false positive when parent paths are being moved or unrelated
changes are being committed, causing it to change while the svn checkout itself
remains the same. To avoid spurious commit, we took "Revision" and "Last
Changed Rev" for general comparison and kept the latter to answer "what is your
revision?" question. This is better but fails when the subrepo path exists at
"Revision" but not at "Last Changed Rev". This patch adds a check for this, and
returns "Revision" if the path does not exist. We try to avoid doing this as
much as possible at it implies an extra, *remote* call.
Previous code was printing a traceback because it expected some error output
from svn. But sometimes our definition of "changed" differs with the subversion
one. For instance, subversion ignores missing files when committing. And when
there are only missing files, svn commit will be a successful no-op with no
output. Still, we should stick to our definition including missing files in
changes as doing otherwise could cause surprising behaviour for the user.
The warning is similar to the warning that was shown before hgsubrepo revert
support was added, with the exception that now the subrepo type is shown.
For example, when trying to revert a git subrepo located in "include/mygitsub",
the warning message would be:
include/mygitsub: reverting git subrepos is unsupported
- _svncommand() in files() returns a tuple since 1ca3bbcf0c2a not a string.
- _svncommand() in filedata() returns a tuple not a string.
- "svn list" returns files but also directories.
- "svn list" is not recursive by default.
I have no idea what happens to svn:externals possibly embedded in the svn
subrepository.
When a subrepo is reverted but --no-backup is not set, call revert on the
subrepo that is being reverted prior to updating it to the revision specified
in the parent repo's .hgsubstate file.
The --all flag is passed down to the subrepo when it is being reverted. If the
--all flag is not set, all files that are modified on the subrepo will be
reverted.
Reverting a subrepo is done by updating it to the revision that is selected on
the parent repo .hgsubstate file.
* ISSUES/TODO:
- reverting added and removed subrepos is not supported yet.
- reverting subrepos is only supported if the --no-backup flag is used (this
limitation will be removed on another patch).
- The behavior of the --all flag has been changed. It now reverts subrepos as
well. Note that this may lead to data loss if the user has a dirty subrepo.
Merging ancestors with children is allowed if they are on different
named branches. This did not work for subrepo merges before. To fix
this inconsistency, the mergefunc() will now use the simple update
path only if both versions are on the same named branch. If not, they
get merged into a new changeset, just as if you did the merge from the
subrepo's root directly.
When support for handling explicit paths in subrepos was added to the forget
command (155b89136ae7), subrepo recursion wasn't taken into account. This
change fixes that by pulling the majority of the logic of commands.forget into
cmdutil.forget, which can then be called from both there and subrepo.forget.
When support for handling explicit paths in subrepos was added to the add
command (825c4cefde4b), subrepo recursion wasn't taken into account. This
change adds an explicitonly argument to cmdutil.add to allow controlling which
levels of recursion should include only explicit paths versus all matched
paths.
path to subrepo is used to identify or check location of subrepo.
it should be normalized (in "/" delimiter form), because it is also
used with narrowmatcher which uses only normalized path even on
Windows environment.
this patch applies "util.pconvert()" on path to subrepo (called
"subpath") to normalize it.
for this patch, referers of below were checked.
- subrepo.state()
- subrepo.itersubrepos()
- subrepo.subrepo()
- context.sub()
- context.substate()
typical usecase is:
for subpath in ctx.substate:
sub = ctx.sub(subpath)
... ctx.substate[subpath] ....
in this case, normalization has no side effect, because keys given
from substate are used as key itself.
other cases shown below also seem to require subpath to be normalized.
- path components are joined by "/", in "commands.forget()":
for subpath in ctx.substate:
subforget[subpath + '/' + fsub] = (fsub, sub)
- normalized "file" is used to check below condition, in
"commands.revert()", "localrepository.commit()", and
"localrepository._checknested()"
file in ctx.substate
- substate.keys() is passed to dirstate.walk()/status() which use
only normalized pathes
Before, we deleted foo when we determined that there were zero
changesets in the foo subrepo. Any files in foo was deleted too. We
now delete foo/.hg instead, and the normal checks in hg.clone will
then abort if there are untracked files in foo.
Before, running 'hg archive -S' could result in
abort: unknown revision '65903cebad86f1a84bd4f1134f62fa7dcb7a1c98'!
if a subrepo was missing completely or had missing changesets. Now,
the missing changesets will be pulled or cloned as appropriate.
This make Mercurial subrepos match Git subrepos which already took
care to fetch any missing commits before starting the archive.
Up until now the all the push command options were ignored when pushing
subrepos. In particular, the fact that the --new-branch command was not passed
down to subrepos made it not possible to push a repo when any of its
subrepos had a new branch, even if you used the --new-branch option of the push
command.
In addition the error message was confusing since it showed the following hint:
"--new-branch hint: use 'hg push --new-branch' to create new remote branches".
However using the --new_branch flag did not fix the problem, as it was ignored
when pushing subrepos.
This patch passes the --new-branch and --ssh flags to every subrepo that is
pushed.
Issues/Limitations:
- All subrepo types get these flags, but only the mercurial subrepos use them.
- It is no longer possible to _not_ pass down these flags to subrepos when
pushing:
* An alternative would be to introduce a --subrepos flag that should be
used to pass down these flags to the subrepos.
* If we did this, it could make sense to make the --force flag respect this
new --subrepos flag as well for consistency's sake.
- Matt suggested that the ssh related flags could also be passed down to
subrepos during pull and clone. However it seems that it would be the "update"
command that would need to get those, since subrepos are only pulled on update.
In any case I'd prefer to leave that for a later patch.
Change the behavior of the forget command such that explicit paths in
subrepos are handled by forgetting the file in the subrepo. This eliminates the
previous behavior where if you called "hg forget" for an explicit path in a
subrepo, it would state that the file is already untracked.
Change the behavior of the add command such that explicit paths in
subrepos are always added. This eliminates the previous behavior
where if you called "hg add" for an explicit path in a subrepo
without specifying the -S option, it would be silently ignored.
If you specify patterns, or no arguments at all, the -S option
will still be needed to activate recursion into subrepos.
This fixes a regression introduced with the new url handling in 1.9.
This should perhaps be fixed in the url class instead, but that might be too
invasive for a stable bugfix.
