In the next patch, I'm gonna removing the global command aliases that force
the epoch date but some tests either fail or their output change after that.
Instead I'm copying the needed aliases in the test files that will otherwise
change.
Update test-rebase-obsolete.t because a revision hash is based on the epoch
date after a 'commit --amend' and the output will change after removing date
aliases.
Update test-subrepo-git.t as the git subrepo doesn't use traditional date
mechanisms. I'm not sure that updating the git subrepo to support default-date
make sense. Add the commit alias to the test in order for making it pass after
removing the date aliases globally.
Rather than sometimes using a complicated shell construct to dump pwned.txt
(if it wasn't expected to exist, but might, if something were broken) or
just cat (if it was expected to exist), just use the "f" utility, which
will be consistent in its behavior across different platforms.
Also make sure that *something* gets put into pwned.txt, even if we ended
up typoing the message variable.
Having a single "pwned" message which may or may not be emitted during the
tests for CVE-2016-3068 leads to extra confusion. Allow each test to emit
a more detailed message based on what the expectations are.
In both cases, we expect a version of git which has had the vulnerability
plugged, as well as a version of mercurial which also knows about
GIT_ALLOW_PROTOCOL. For the first test, we make sure GIT_ALLOW_PROTOCOL is
unset, meaning that the ext-protocol subrepo should be ignored; if it
isn't, there's either a problem with mercurial or the installed copy of
git.
For the second test, we explicitly allow ext-protocol subrepos, which means
that the subrepo will be accessed and a message emitted confirming that
this was, in fact, our intention.
The "pwned" message from this test gets gets sent to stderr, and so may get
emitted in different places from run to run in the rest of mercurial's
output. This patch forces the message to go to a specific file instead,
whose existence and contents we can examine at a stable point in the test's
execution.
CVE-2016-3068 (1/1)
Git's git-remote-ext remote helper provides an ext:: URL scheme that
allows running arbitrary shell commands. This feature allows
implementing simple git smart transports with a single shell shell
command. However, git submodules could clone arbitrary URLs specified
in the .gitmodules file. This was reported as CVE-2015-7545 and fixed
in git v2.6.1.
However, if a user directly clones a malicious ext URL, the git client
will still run arbitrary shell commands.
Mercurial is similarly effected. Mercurial allows specifying git
repositories as subrepositories. Git ext:: URLs can be specified as
Mercurial subrepositories allowing arbitrary shell commands to be run
on `hg clone ...`.
The Mercurial community would like to thank Blake Burkhart for
reporting this issue. The description of the issue is copied from
Blake's report.
This commit changes submodules to pass the GIT_ALLOW_PROTOCOL env
variable to git commands with the same list of allowed protocols that
git submodule is using.
When the GIT_ALLOW_PROTOCOL env variable is already set, we just pass it
to git without modifications.
Before this patch, some tests using external "diff" command via
extdiff extension fail on Solaris, because system standard "diff" (=
/usr/bin/diff) on Solaris always formats chunk header in the style
below:
@@ -X.x +Y.y @@
even though "diff" on Linux sometimes omits ".x" and/or ".y" in it.
This patch makes chunk header of external diff glob-ed for portability
of tests, and adds check-code.py rules to detect such diff output in
tests.
This patch also changes "hg diff" output in test-subrepo-git to
simplify detection rules, even though it is certainly portable because
these lines are generated by "git" command.
This patch is a part of making tests using external "diff" portable,
and tests below aren't yet portable even after this patch.
test-largefiles-update.t
test-subrepo-deep-nested-change.t
d6f3cca7f3a5 is adding a new tar call but forgot to filter out the
pax_global_header file that some tar versions write. This is similar to what
happened in 686775e575f3.
Unix utilities like tar will happily prefix the files it packs with './', but
annoyingly, Windows Explorer will not show these packed files when it opens the
archive. Since there doesn't seem to be a point in including './' in the path
names, just drop it. The default 'hg archive' prefix is the basename of the
archive, so specifying '.' allows for that default to be disabled completely.
'git diff-index' by default does not recurse into subdirectories
when changes are found. As a result, the directory is shown as changed,
rather than the files in this directory.
Adding '-r' results in recursing until the blobs themselves are checked.
The previous behavior when archiving a subrepo 's' on Windows was to internally
name the file under it 's\file', due to the use of vfs.reljoin(). When printing
the file list from the archive on Windows or Linux, the file was named
's\\file'. The archive extracted OK on Windows, but if the archive was brought
to a Linux system, it created a file named 's\file' instead of a directory 's'
containing 'file'.
*.zip format achives seemed not to have the problem, but this was definitely an
issue with *.tgz archives.
Largefiles actually got this right, but a test is added to keep this from
regressing. The subrepo-deep-nested-change.t test was repurposed to archive to
a file, since there are several subsequent tests that archive to a directory.
The output change is losing the filesystem prefix '../archive_lf' and not
listing the directories 'sub1' and 'sub1/sub2'.
One case where that would happen is while trying to resolve a subrepo, if the
path to the subrepo was actually a broken symlink. This bug was exposed by an
hg-git test.
In git 2.2.0 and higher, removing files and directories is changed:
removing an object that does not exist returns success rather than failure.
As a result, even though .hg/hgrc does not exist, success is returned
and the .hg/ directory is removed.
To handle this correctly, use 'rm -rf' to allow successful removing
for all git versions.
The exact changeset where this was introduced in git:
1054af7d04aef64378d69a0496b45cdbf6a0bef2
wrapper.c: remove/unlink_or_warn: simplify, treat ENOENT as success
This follows normal Mercurial rules, and the message is lifted from
workingctx.add(). The file is printed with abs() to be consistent with how it
is printed in workingctx, even though that is inconsistent with how added files
are printed in verbose mode. Further, the 'already tracked' notifications come
after all of the files that are added are printed, like in Mercurial.
As a side effect, we now have the reject list to return to the caller, so that
'hg add' exits with the proper code. It looks like an abort occurs if git fails
to add the file. Prior to touching 'snake.python' in the test, this was the
result of attempting to add the file after a 'git rm':
fatal: pathspec 'snake.python' did not match any files
abort: git add error 128 in s (in subrepo s)
I'm not sure what happens when git is a deep subrepo, but the 'in s' and
'in subrepo s' from @annotatesubrepoerror are redundant here. Maybe we should
stat the files before invoking git to catch this case and print out the prettier
hg message? The other thing missing from workingctx.add() is the call to
scmutil.checkportable(), but that would need to borrow the parent's ui object.
The previous test gave a false success because only an hg-ignored pattern was
specified. Therefore match.files() was empty, and it fell back to the files
unknown to git. The simplest fix is to always consider what is unknown to git,
as well as anything specified explicitly. Files that are ignored by git can
only be introduced by an explicit mention in match.files().
V2: use 'self._ctx.node()' instead of 'rev' in makefileobj.
As Matt Harbison mentioned, using 'rev' does not make sense,
since we'd be passing a git revision to the top-level
Mercurial repository.
Previously, git subrepo diffs did not have a newline at the end.
This caused multiple subrepo diffs to be joined on the same line.
Additionally, the command prompt after the diff still contained
a part of the diff.
Previously, revert was only possible if the '--no-backup'
switch was specified.
Now, to support backups, we explicitly go over all modified
files in the subrepo.
'git diff --stat' output changed with regard to the amount
of changes/insertions/deletions shown.
In older git versions (1.7.7.6), output was shown as:
2 files changed, 2 insertions(+), 1 deletions(-)
In newer versions, output is shown as:
2 files changed, 2 insertions(+), 1 deletion(-)
This patch uses a regex to handle both cases.
Previously, git subrepos did not support reverting.
This change adds basic support for reverting
when '--no-backup' is specified.
A warning is given (and the current state is kept)
when a revert is done without the '--no-backup' flag.
The output of 'git diff --stat' changed in git 1.7.10 and 1.7.11.
To ensure the tests work with earlier versions of git as well,
the output is now wrapped with a whitespace regex.
So far, git subrepositories were silently ignored for diffs.
This patch adds support for git subrepositories,
with the remark that --include and --exclude are not supported.
If --include or --exclude are used, the subrepo is ignored.
Before this patch, sanitizing ".hg/hgrc" scans directories and files
also in meta data area for non-hg subrepos: under ".svn" for
Subversion subrepo, for example.
This may cause not only performance impact (especially in large scale
subrepos) but also unexpected removing meta data files.
This patch avoids sanitizing ".hg/hgrc" in meta data area for non-hg
subrepos.
This patch stops checking "ignore" target at the first
(case-insensitive) appearance of it, because continuation of scanning
is meaningless in almost all cases.
Before this patch, "hg update" doesn't sanitize ".hg/hgrc" in non-hg
subrepos correctly, if "hg update" is executed not at the root of the
parent repository.
"_sanitize()" takes relative path to subrepo from the root of the
parent repository, and passes it to "os.walk()". In this case,
"os.walk()" expects CWD to be equal to the root of the parent
repository.
So, "os.walk()" can't find specified path (or may scan unexpected
path), if CWD isn't equal to the root of the parent repository.
Non-hg subrepo under nested hg-subrepos may cause same problem, too:
CWD may be equal to the root of the outer most repository, or so.
This patch makes "_sanitize()" take absolute path to the root of
subrepo to sanitize correctly in such cases.
This patch doesn't normalize the path to hostile files as the one
relative to CWD (or the root of the outer most repository), to fix the
problem in the simple way suitable for "stable".
Normalizing should be done in the future: maybe as a part of the
migration to vfs.
Before this patch, sanitizing ".hg/hgrc" in git subrepo doesn't work,
when the working directory is updated by "git merge --ff".
"_sanitize()" is not invoked after checking target revision out into
the working directory in this case, even though it is invoked
indirectly via "checkout" (or "rawcheckout") in other cases.
This patch invokes "_sanitize()" explicitly also after "git merge
--ff" execution.
"_sanitize()" was introduced by 5131f2755f60 on "stable" branch, but
it has done nothing for sanitizing since 5131f2755f60.
"_sanitize()" assumes "Visitor" design pattern:
"os.walk()" should invoke specified function ("v" in this case)
for each directory elements under specified path
but "os.walk()" assumes "Iterator" design pattern:
callers of it should drive loop to scan each directory elements
under specified path by themselves with the returned generator
object
Because of this mismatching, "_sanitize()" just discards the generator
object returned by "os.walk()" and does nothing for sanitizing.
This patch makes "_sanitize()" work.
This patch also changes the format of warning message to show each
unlinked files, for multiple appearances of "potentially hostile
.hg/hgrc".
When a subrepo has changed on the local and remote revisions, prompt the user
whether it wants to merge those subrepo revisions, keep the local revision or
keep the remote revision.
Up until now mercurial would always perform a merge on a subrepo that had
changed on the local and the remote revisions. This is often inconvenient. For
example:
- You may want to perform the actual subrepo merge after you have merged the
parent subrepo files.
- Some subrepos may be considered "read only", in the sense that you are not
supposed to add new revisions to them. In those cases "merging a subrepo" means
choosing which _existing_ revision you want to use on the merged revision. This
is often the case for subrepos that contain binary dependencies (such as DLLs,
etc).
This new prompt makes mercurial better cope with those common scenarios.
Notes:
- The default behavior (which is the one that is used when ui is not
interactive) remains unchanged (i.e. merge is the default action).
- This prompt will be shown even if the ui --tool flag is set.
- I don't know of a way to test the "keep local" and "keep remote" options (i.e.
to force the test to choose those options).
# HG changeset patch
# User Angel Ezquerra <angel.ezquerra@gmail.com>
# Date 1378420708 -7200
# Fri Sep 06 00:38:28 2013 +0200
# Node ID 2fb9cb0c7b26303ac3178b7739975e663075857d
# Parent 796d34e1b749b79834321ef1181ed8433a5515d9
merge: let the user choose to merge, keep local or keep remote subrepo revisions
When a subrepo has changed on the local and remote revisions, prompt the user
whether it wants to merge those subrepo revisions, keep the local revision or
keep the remote revision.
Up until now mercurial would always perform a merge on a subrepo that had
changed on the local and the remote revisions. This is often inconvenient. For
example:
- You may want to perform the actual subrepo merge after you have merged the
parent subrepo files.
- Some subrepos may be considered "read only", in the sense that you are not
supposed to add new revisions to them. In those cases "merging a subrepo" means
choosing which _existing_ revision you want to use on the merged revision. This
is often the case for subrepos that contain binary dependencies (such as DLLs,
etc).
This new prompt makes mercurial better cope with those common scenarios.
Notes:
- The default behavior (which is the one that is used when ui is not
interactive) remains unchanged (i.e. merge is the default action).
- This prompt will be shown even if the ui --tool flag is set.
- I don't know of a way to test the "keep local" and "keep remote" options (i.e.
to force the test to choose those options).
The test didn't break when I originally committed it (git version
1.7.12.4 (Apple Git-37)), and still doesn't on that machine, but the
output changes elsewhere with "Warning: you are leaving N commits
behind..."
This change sets a ref on the commit we're updating the subrepo _away_
from to quash the warning.
