I ran into a case when adding a test where there were cryptic hg command line
errors. I eventually traced it back to 'hg id' printing debug messages before
the hash:
invalid branchheads cache (served): tip differs <hash>
This method should eliminate any other output except the node.
There are currently no tests for change/remove conflicts of subrepos,
and it's pretty broken. Add some tests demonstrating some of the
breakages and fix the most obvious one (a KeyError when trying to look
up a subrepo in the wrong context).
Even if largefiles extension is enabled in a repository, "repo"
object, which isn't "largefiles.reposetup()"-ed, is passed to
overridden functions in the cases below unexpectedly, because
extensions are enabled for each repositories strictly.
(1) clone without -U:
(2) pull with -U:
(3) pull with --rebase:
combination of "enabled@src", "disabled@dst" and
"not-required@src" cause this situation.
largefiles requirement
@src @dst @src result
-------- -------- --------------- --------------------
enabled disabled not-required aborted unexpectedly
required requirement error (intentional)
-------- -------- --------------- --------------------
enabled enabled * success
-------- -------- --------------- --------------------
disabled enabled * success (only for "pull")
-------- -------- --------------- --------------------
disabled disabled not-required success
required requirement error (intentional)
-------- -------- --------------- --------------------
(4) update/revert with a subrepo disabling largefiles
In these cases, overridden functions cause accessing to largefiles
specific fields of not "largefiles.reposetup()"-ed "repo" object, and
execution is aborted.
- (1), (2), (4) cause accessing to "_lfstatuswriters" in
"getstatuswriter()" invoked via "updatelfiles()"
- (3) causes accessing to "_lfcommithooks" in "overriderebase()"
For safe accessing to these fields, this patch examines whether passed
"repo" object is "largefiles.reposetup()"-ed or not before accessing
to them.
This patch chooses examining existence of newly introduced
"_largefilesenabled" instead of "_lfcommithooks" and
"_lfstatuswriters" directly, because the former is better name for the
generic "largefiles is enabled in this repo" mark than the latter.
In the future, all other overridden functions should avoid largefiles
specific processing for efficiency, and "_largefilesenabled" is better
also for such purpose.
BTW, "lfstatus" can't be used for such purpose, because some code
paths set it forcibly regardless of existence of it in specified
"repo" object.
The default joinfmt, "x.values()[0]", can't be used here because it picks
either 'bookmark' or 'current' randomly.
I got wrong result with PYTHONHASHSEED=1 on my amd64 machine.
My Mac test machine has 'echo' in '/opt/local/libexec/gnubin/echo' since, well,
GNU is not BSD.
Also, I feel it need to be said about using regexes:
Some people, when confronted with a problem, think "I know, I'll use regular
expressions." Now they have two problems.
Before this patch, revset predicate "tag()" and "named('tags')" differ
from each other, because the former doesn't include "tip" but the
latter does.
For equivalence, "named('tags')" shouldn't include the revision
corresponded to "tip". But just removing "tip" from the "tags"
namespace causes breaking backward compatibility, even though "tip"
itself is planned to be eliminated, as mentioned below.
http://selenic.com/pipermail/mercurial-devel/2015-February/066157.html
To mask specific names ("tip" in this case) for "named()" predicate,
this patch introduces "deprecated" into "namespaces", and makes
"named()" predicate examine whether each names are masked by the
namespace, to which they belong.
"named()" will really work correctly after 3.3.1 (see a3c326a7f57a for
detail), and fixing this on STABLE before 3.3.1 can prevent initial
users of "named()" from expecting "named('tags')" to include "tip".
It is reason why this patch is posted for STABLE, even though problem
itself isn't so serious.
This may have to be flagged as "(BC)", if applied on DEFAULT.
Before this patch, revset predicate "named()" uses each nodes gotten
from target namespaces directly.
This causes problems below:
- combination of other predicates doesn't work correctly, because
they assume that revisions are listed up in number
- "hg log" doesn't show any revisions for "named()" result, because:
- "changeset_printer" stores formatted output for each revisions
into dict with revision number (= ctx.rev()) as a key of them
- "changeset_printer.flush(rev)" writes stored output for
the specified revision, but
- "commands.log" invokes it with the node, gotten from "named()"
- "hg debugrevspec" shows nodes (= may be binary) directly
Difference between revset predicate "tag()" and "named('tags')" in
tests is fixed in subsequent patch.
Before this patch, "bookmark()", "named()" and "tag()" predicates
raise "Abort", when the specified pattern doesn't match against
existing ones.
This prevents "present()" predicate from continuing the query, because
it only catches "RepoLookupError".
This patch raises "RepoLookupError" instead of "Abort", to make
"present()" predicate continue the query, even if "bookmark()",
"named()" or "tag()" in the sub-query of it are aborted.
This patch doesn't contain raising "RepoLookupError" for "re:" pattern
in "tag()", because "tag()" treats it differently from others. Actions
of each predicates at failure of pattern matching can be summarized as
below:
predicate "literal:" "re:"
---------- ----------- ------------
bookmark abort abort
named abort abort
tag abort continue (*1)
branch abort continue (*2)
---------- ----------- ------------
"tag()" may have to abort in the (*1) case for similarity, but this
change may break backward compatibility of existing revset queries. It
seems to have to be changed on "default" branch (with "BC" ?).
On the other hand, (*2) seems to be reasonable, even though it breaks
similarity, because "branch()" in this case doesn't check exact
existence of branches, but does pick up revisions of which branch
matches against the pattern.
This patch also adds tests for "branch()" to clarify behavior around
"present()" of similar predicates, even though this patch doesn't
change "branch()".
The previous code was adding standin files to the matcher's file list when
neither the standin file nor the original existed in the context. Somehow, this
was confusing the logging code into behaving differently from when the extension
wasn't loaded.
It seems that this was an attempt to support naming a directory that only
contains largefiles, as a test fails if the else clause is dropped entirely.
Therefore, only append the "standin" if it is a directory. This was found by
running the test suite with --config extensions.largefiles=.
The first added test used to log an additional cset that wasn't logged normally.
The only relation it had to file 'a' is that 'a' was the source of a move, but
it isn't clear why having '.hglf/a' in the list causes this change:
@@ -47,6 +47,11 @@
Make sure largefiles doesn't interfere with logging a regular file
$ hg log a --config extensions.largefiles=
+ changeset: 3:2ca5ba701980
+ user: test
+ date: Thu Jan 01 00:00:04 1970 +0000
+ summary: d
+
changeset: 0:9161b9aeaf16
user: test
date: Thu Jan 01 00:00:01 1970 +0000
The second added test used to complain about a file not being in the parent
revision:
@@ -1638,10 +1643,8 @@
Ensure that largefiles doesn't intefere with following a normal file
$ hg --config extensions.largefiles= log -f d -T '{desc}' -G
- @ c
- |
- o a
-
+ abort: cannot follow file not in parent revision: ".hglf/d"
+ [255]
$ hg log -f d/a -T '{desc}' -G
@ c
|
Note that there is still something fishy with the largefiles code, because when
using a glob pattern like this:
$ hg log 'glob:sub/*'
the pattern list would contain '.hglf/glob:sub/*'. None of the tests show this
(this test lives in test-largefiles.t at 1349), it was just something that I
noticed when the code was loaded up with print statements.
The conditional was a bit too narrow and produced buggy result when a node was
present in both common and heads (because it pleased the discovery) and it was
locally known but filtered.
This resulted in buggy getbundle request and server side crash.
Before this patch, failure of updating subrepos may cause inconsistent
".hgsubstate". For example:
1. dirstate entry for ".hgsubstate" of the parent repo is filled
with valid size/date (via "hg state" or so)
2. "hg update" is invoked at the parent repo
3. ".hgsubstate" of the parent repo is updated on the filesystem as
a part of "g"(et) action in "merge.applyupdates"
4. it is assumed that size/date of ".hgsubstate" on the filesystem
aren't changed from ones at (1)
this is not so difficult condition, because just changing hash
ids (every ids are same in length) in ".hgsubstate" doesn't
change the file size of it
5. "subrepo.submerge()" is invoked to update subrepos
6. failure of updating in one of subrepos raises exception
(e.g. "untracked file differs")
7. "hg update" is aborted without updating dirstate of the parent repo
dirstate entry for ".hgsubstate" still holds size/date at (1)
Then, ".hgsubstate" of the parent repo is treated as "CLEAN"
unexpectedly, because updating ".hgsubstate" at (3) doesn't change
size/date of it on the filesystem: see assumption at (4).
This inconsistent ".hgsubstate" status causes unexpected behavior, for
example:
- "hg revert" forgets to revert ".hgsubstate"
- "hg update" misunderstands that (not yet updated) subrepos diverge
(then, it shows the prompt to confirm user's decision)
To avoid inconsistent ".hgsubstate" status above, this patch marks
".hgsubstate" as possibly dirty before "submerge" invocation.
"normallookup"-ed (= dirty) dirstate should be written out, even if
processing is aborted by failure.
This patch marks ".hgsubstate" as possibly dirty before "submerge",
also when it is removed or merged while merging, for safety. This
should prevent Mercurial from misunderstanding inconsistent
".hgsubstate" as clean.
To satisfy conditions at (1) and (4) above, this patch uses "hg status
--config debug.dirstate.delaywrite=2" (to fill valid size/date into
dirstate) and "touch" (to fix date of the file).
Before this changeset rebase was getting very confused if any revision in the
rebase set became hidden. This was fairly easy to achieve if a rebased revision
was made visible by the working copy location. The rebase process would update
somewhere else and the revision would become hidden.
To work around this issue, we ensure rebased revisions remain visible for the
whole process.
This is a simple change suitable for stable. More subtle usage of unfiltered
repository in rebase may solve this issue more cleanly.
b8552020f458 broke things ... and the following cleanups didn't fix all issues.
It didn't work with the diffargs shipped in mergetools.rc with explicit
quoting. Parameters would end up with being quoted twice - especially if they
really needed quoting.
To work around that, look for explicit quotes around the variables that will be
substituted with proper quoting. Also accept an additional prefix so we can
handle both
--foo='$parent'
and
'--foo=$parent'
It will however still fail if the user intentionally place the variable inside
a quoted string, as in
'parent $parent is on the left'
There is currently no good way to handle that, short of knowing exactly which
quoting mechanism will be used.
This effectively reverts 9fc565fa1621, which caused some normal file copies to
not be displayed as copies. Other normal file copies could be displayed- the
exact reason isn't clear. This also adds two tests that were failing prior to
this backout, so that this can be sorted out next cycle.
The difference between copy cases that worked and those that didn't seemed to be
in copies.pathcopies(). When largefiles isn't enabled for the changed test, or
lfstatus is not set in the commands.status() override, 'y.ancestor(x) == x'.
That wasn't true otherwise, which fell through to the _chain() method. In this
case, the copy is removed in the criss cross loop.
'y.ancestor(x)' returns a context.changectx type, while 'x' is a lfilesctx type
in the failing case. I tried adding the ancestor method to the lfilesctx class
to change the type of the ancestor context, however the context when printed as
a string then gains a '+'. This points to it being a context.committablectx,
which clearly isn't correct for an ancestor. Possibly the problem is the
lfilesctx needs to subclass context.committablectx in some cases, but
context.changectx in others, within the same invocation? I'm not sure how to
pull that off, and backing out this change is safer during the freeze.
As to the status changing when a path is specified, I haven't looked into it
yet.
We use -T consistently elsewhere to refer to the --template option.
The old -t option is now renamed to --oldtemplate so that -t still
works. This has the benign side effect of introducing and immediately
deprecating a new long option.
We also test with both -t and -T options.
1. This is consistent with pushing.
2. This allows to see the URL of the other repo in case accessing the repo
fails, e.g. wrong ssh path or issues with the https certificate, without
using --debug or showconfig paths.
Additionally add test for this in the context of ssh with a wrong path.
The issue is titled "filtered revision 'XXX' (not in 'served' subset)" and that
is the error message you sometimes get when trying to look at a file (/file or
/annotate) in hgweb. For example:
http://hg.intevation.org/mercurial/crew/file/8414f8487b33/mercurial/cmdutil.py
This happens when a parent revision for a file is hidden, thus it is
not 'served' and isn't accessible in hgweb by default. When hgweb tries to
access such changeset, it produces the error and HTTP status code 404.
Another detail is that the parents() function, that is used in multiple places
in hgweb, sometimes returned changesets that were obsoleted by the current
changeset for the file. For example, when using rebase with evolve and rebasing
a divergent changeset that introduces a file on top of current branch. Or
grafting a change and making the new grafted changeset obsolete the source
(shown in the test case). The result is the same - the obsoleted changeset was
mistakingly returned from parents(), even though it's not a parent and the only
link to the new changeset is an obsoletion marker (and rebase/graft metadata?
not sure it matters).
The problem is fixed by using introrev() instead of linkrev() for finding
parents. This prevents parents() function from returning unrelated obsolete
changesets.
The test case prepares a separate repo because (afaict) all other test cases
never reuse file names, so there are no files that were changed in multiple
changesets. So no previously available files have obsolete changesets in their
history.
Since the point of --dry-run is to show what will happen, the output with and
without it should agree. And since revert wasn't being called on subrepos with
--dry-run before, revert in the subrepo had to be defanged in this case.
Previously, when a largefile is forgotten and then reverted, a warning was
issued:
$ hg revert -R subrepo subrepo/large.txt
file not managed: subrepo/large.txt (glob)
This was purely cosmetic as the file itself actually was reverted.
The problem was even with all of the matcher patching, the largefile pattern
given on the command line wasn't converted to a standin because the standin was
neither in ctx nor wctx. This causes the named largefile to be added to the
'names' dict in cmdutil.revert() in the repo walk at line 2550. The warning was
printed out when the 'names' dict is iterated, because the file was specified
exactly.
Since core revert recurses into subrepos and largefiles only overrides the
revert method in commands.py, it doesn't work properly when reverting a subrepo.
However, it still will recurse into the subrepo and call the installed matcher
method, so lfdirstate is reopened for the current repo level to prevent any new
problems.
It was introduced at 1e92106a20bc to warn that "hg resolve" did nothing
meaningful. The warning seems not good for "hg resolve -l" because it is
rather like "hg status" or "hg files".
Mercurial uses a synopsis string and the docstring of a command for the
command's help output. Today there is no way for an extension that adds
functionality to a command to extend either of these help strings.
This patch enables appending to both the doctring and the synopsis, and adds
a test for this functionality. Example usage is shown in the test and is also
described in the docstring of extensions.wrapcommand().
debugsetparents is a debug command and does not provide the same
guarantees as non-debug commands do. In particular, when the user sets
a different parent, any clean files will remain clean in the dirstate
even though the new parent might have a different version of the file
(so it should appear modified compared to the new parent). Let's
instead achieve the same effect by updating to the new parent and
reverting the contents back to what they were.
This fix can be tested by passing '--config
debug.dirstate.delaywrite=2' to the 'hg update' command in the
beforemerge().
This can simplify the conversion from numeric revision to string. Without it,
we have to handle -1 specially because repo['-1'] != repo[-1].
The -1 revision is not officially documented, but this change makes sense
assuming that "rev(%d)" exists for scripting or third-party tools.
Before this patch, test-tag.t can't run successfully on Windows,
because:
- quoted hg command ('"hg"') prevents "hg.bat" from working correctly
(only at testing with pure Python build)
"%~f0" and "%~dp0hg" in "hg.bat" cause unexpected result in this
case. BTW, quoted "\path\to\hg" works correctly.
- "`pwd`" in the command line is expanded unexpectedly
not "C:\path\to\TESTTMP" but "C;C:\path\to\TESTTMP"
When running "hg log 'set:added()'", we create two matchers: one used
for producing the revset and one used for finding files to match. In
185b6b930e8c (graphlog: evaluate FILE/-I/-X filesets on the working
dir, 2012-02-26), we started passing a revision argument along from
what's currently in cmdutil._makelogrevset() to
revset._matchfiles(). When the revision was an empty string, it
referred to the working copy. This was subtly done with "repo[rev or
None]". Then, in 5ff5c5c9e69f (revset: avoid recalculating filesets,
2014-10-22), that conversion from empty string to None was lost. Note
that repo[''] is equivalent to repo['.'], not repo[None].
The consequence of this, to the user, is that when running "hg log
'set:added()'", the file matcher matches files added in the working
copy, while the revset matcher matches revisions that touch files
added in the parent of the working copy. As a result, only revisions
that touch any files added in the parent of the working copy will be
considered, but they will only be included if they also touch files
added in the working copy.
Fix the bug by converting '' to None again, but make it a little more
explicit this time (plus, we now have tests for it).
On IRC, rom1dep reported a traceback[1] from setting
experimental.strip-bundle2-version to True. This diff catches
unexpected values and falls back to the non-experimental bundle1
implementation after issuing a warning.
[1] http://gist.tamytro.org/_admin/gists/qXcdQLwtApgy6e3NwWgl
Previously, git subrepo diffs did not have a newline at the end.
This caused multiple subrepo diffs to be joined on the same line.
Additionally, the command prompt after the diff still contained
a part of the diff.
Before this patch, "test-run-tests.t" doesn't test "run-tests.py" with
"--pure", even if outer "run-tests.py" is executed with it.
This patch uses not "HG_RUN_TESTS_PURE" but "HGTEST_RUN_TESTS_PURE",
because "HG_" prefixed environments are forcibly dropped in "_getenv()".
This is also useful to run "run-tests.py" successfully by
"run-tests.py --pure" on Windows without any compilation tools (like
VisualStudio).
When "hg.bat" is invoked via interactive shell "cmd.exe" on Windows,
it can store own exit code into ERRORLEVEL correctly, regardless of
explicit "exit" statement in it: "cmd.exe" seems to hold ERRORLEVEL
updated by the last command in the batch file (= "python hg", in
"hg.bat" case).
On the other hand, "hg.bat" is invoked indirectly via
"subprocess.Popen" (e.g. shell alias, hooks, hgclient and so on), the
parent process always receives exit code 0 from spawned "hg.bat":
batch files on Windows seem not to be really spawned like as shell
scripts on UNIX, but to be executed in the "cmd.exe" process.
This patch returns exit code explicitly for indirect invocation.
"/b" should be specified for "exit" to prevent "cmd.exe" from being
terminated when "hg.bat" is invoked interactively from it.
Before this patch, "run-tests.py" executes "hghave" process without
any modifications for environment variables, even though actual tests
are executed with LC_ALL, LANG and LANGUAGE explicitly assigned "C".
When "run-tests.py" is executed:
- with non-"C" locale environment variables on any platforms, or
- without any explicit locale environment setting on Windows
(only for "outer-repo" feature using "hg root")
external commands indirectly executed by "hghave" may show translated
messages.
This causes incorrect "hghave" result and skipping tests, because some
regexp matching of "hghave" expect external commands to show
un-translated messages.
To prevent external commands from showing translated messages, this
patch makes "run-tests.py" execute "hghave" with same environment
variables as ones for actual tests.
This patch doesn't make "hghave" execute external commands forcibly
with LC_ALL, LANG and LANGUAGE explicitly assigned "C", because
changing "run-tests.py" is cheaper than changing "hghave":
- "os.popen" should be replaced by "subprocess.Popen" or so, and
- setting up environment variables should be newly added
Previously, we had weird, nonsensical behavior when committing a file move
with a missing source. This removes that weird logic and tests that the bug
this strange behavior caused is fixed. Also adds a longish comment to prevent
some poor soul from accidentally re-implementing the bug in the future.
When a binary source has been copied or renamed into a non-binary
file, we don't check whether the copy source was binary. There is a
code comment explaining that a git mode will be forced anyway in order
to capture the copy record (i.e. losedatafn() will be called). This is
true, but forcing git mode is not the only effect binary files have:
when git mode was already requested, we use the binary-ness to tell us
whether to use a regular unified diff or a git binary diff. The user
sees this as a "Binary file $file has changed" instead of the binary
When a directory is named in the commit file list, the previous behavior was to
walk the list, and if no normal files in the directory were also named, add the
corresponding standin for each largefile in that directory. The directory is
then dropped from the list, so that committing a directory with no normal file
changes works. It then added the corresponding standin directory for the first
largefile seen, by prefixing it with '.hglf/'.
The latter is unnecessary since each affected largefile is explicitly referenced
by its standin in the list. It also caused an abort if there were no changed
largefiles in the directory, because none of its standins changed:
abort: .hglf/foo/bar: no match under directory!
This list of files is used to tweak a matcher in lfutil.updatestandinsbymatch(),
which is what is passed to commit().
The status() call that is ultimately done in the commit code with this matcher
seems to have some OS specific differences. It is not necessary to append '.'
for Windows to run the largefiles tests cleanly. But if '.' is not added to the
list, the match function isn't called on Linux, so status() would miss any
normal files that were also in a named directory. The commit then proceeds
without those normal files, or says "nothing changed" if there were no changed
largefiles in the directory. This is not filesystem specific, as VFAT on Linux
had the same behavior as when run on ext4. It is also not an issue with
lfilesrepo.status(), since that only calls the overridden implementation when
paths are passed to commit. I dont have access to an OS X machine ATM to test
there.
Maybe there's a better way to do this. But since the standin directory for the
first largefile was previously being added, and that caused the same walk in
status(), there's no preformance change to this. There is no danger of
erroneously committing files in '.', because the original match function is
called, and if it fails, the lfutil.updatestandinsbymatch() tweaked matcher only
indicates a match if the file is in the list of standins- and '.' never is. The
added tests confirm this.
