If p1 is null, ':.' is translated as '0:null'. But rangeset can't handle null,
only revision 0 was visible. Because 'null' should not be listed implicitly,
"log --follow" (without -r) should be empty if p1 is null.
Test of "hg grep -f" is added for cmdutil.walkchangerevs().
Some templates in paper style use <tbody> elements inside <table> to assign a
class to "body" part of that table (in this case, to make rows striped). The
problem is that the <tbody> is preceded by <tr> element, which browsers
understand as an implicit start of table body, so the following exlicit <tbody>
will actually be "nested", which is not valid.
Since that first <tr> contains table headers, wrapping it in <thead> is both
semantically correct and follows the advertised XHTML 1.1 doctype.
If the gettext utilities aren't installed, there is no way to make the
translations. This causes the gettext client to fall back to the untranslated
message, which triggers "** NOTHING TRANSLATED **" errors and a test failure.
The <p> elements were used to create an empty space between the diffstat link
and the diffstat table, but they don't have any semantic meaning, so it is
better to use css instead.
Default margins for <p> elements can differ depending on the browser, but
usually the margin is 1em (exceptions are IE 6 and 7 with 14pt, which is
comparable). The css rule sets top margin to 1em.
This change is a "better version" of e028c221db4e, where <p> elements were
simply properly closed.
The <p> elements were used to create an empty space between the last menu item
(i.e. "help") and the atom feed icon, but they don't have any semantic meaning,
so it is better to use css instead.
The css rule uses top margin of 10px, which is equal to the top margin of the
menu blocks ("help", "changeset, browse", etc). Previously, with <p> elements,
the margin wasn't set explicitly and was browser-dependent.
This change is a "better version" of e028c221db4e, where <p> elements were
simply properly closed.
When cloning a repo that requires largefiles, the user had to either enable the
extension on the command line and then manually edit the local hgrc file after
the clone, or just enable it globally for the user. Since it is a feature of
last resort, and materially affects even repos without any largefiles when it is
enabled, we should make it easier to not have it enabled globally.
This simply adds the enabling statement to the local hgrc if the requires file
mandates its use (which only happens after the first largefile is committed).
That means that a user who works with a mix of largefile and normal repos can
always clone with '--config extensions.largefiles=', and the extension is
permanently enabled or not as appropriate.
The change in test-largefiles.t is simply because the order of loading rebase
and largefiles changed. The same change occurs if the order is flipped in the
hgrc file at the top of the test.
Adds a configuration setting for allowing users to specify the default behavior
of 'hg histedit' without arguments. This saves users from having to manually
figure out the bottom commit or a complicated revset. My current revset of
choice is "only(.) & draft() - ::merge()"
The commits that histedit can work with is usually quite limited, so if this
feature ends up working well, we may want to consider making "only(.) & draft()
- ::merge()" the default behavior for everyone.
The destination is validated by pathutil.canonpath() for illegal components, and
that it is in the repository. The logic for creating the standin directory tree
was calling this before cmdutil.copy(), but without the destination file name
component. The cmdutil.copy() logic also calls pathutil.canonpath(), but with
the file name component. By always calling the core logic first, the error
message is always consistent. Specifically, the old behavior for these tests
was to say '.hg' contains an illegal component, and '..' is not under root.
A user wasn't likely to notice the discrepancy, but this eliminates a needless
difference when running the test suite with --config extensions.largefiles=.
Previously histedit only stored the short version of the rule nodes in the
state. This meant that later we couldn't resolve a rule node to its full
form if the commit had been deleted from the repo.
Let's store the full form from the beginning.
Before this patch, referring alias arguments is parsed by string base
operation "str.replace".
This causes problems below (see the previous patch introducing
"_parsealiasdefn" for detail)
- the shorter name argument breaks referring the longer name
- argument names in the quoted string are broken
This patch replaces parsing alias definition by "_parsealiasdefn" to
parse strictly.
V2: use 'self._ctx.node()' instead of 'rev' in makefileobj.
As Matt Harbison mentioned, using 'rev' does not make sense,
since we'd be passing a git revision to the top-level
Mercurial repository.
commitctx already showed notes with filenames but didn't provide any context.
It is just as relevant to know when manifest or changelog is committed.
So, in addition to filenames, also show headlines 'committing files:',
'committing manifest' and 'committing changelog'.
This is kind of similar to the debugbundle command but gives summarized actual
uncompressed number of bytes when creating the bundle. The numbers are as
usable as the bundle format is efficient. Hopefully bundle2 will make it a
better indicator of actual entropy.
This is useful when accepting pull requests to assess whether the repo size
increase seems reasonable for the diff before pushing stuff upstream, It has
helped me catching large files that should have been committed as largefiles
but was committed as regular files in intermediate changesets.
This output doesn't combine well with debug output so we only enable it when
verbose without debug.
Before this patch, there is no way to concatenate strings at runtime.
For example, to search for the issue ID "1234" in descriptions against
all of "issue 1234", "issue:1234", issue1234" and "bug(1234)"
patterns, the revset below should be written fully from scratch for
each issue ID.
grep(r"\bissue[ :]?1234\b|\bbug\(1234\)")
This patch introduces new infix operator "##" to concatenate
strings/symbols at runtime. Operator symbol "##" comes from the same
one of C pre-processor. This concatenation allows parametrizing a part
of strings in revset queries.
In the case of example above, the definition of the revset alias using
operator "##" below can search issue ID "1234" in complicated patterns
by "issue(1234)" simply:
issue($1) = grep(r"\bissue[ :]?" ## $1 ## r"\b|\bbug\(" ## $1 ## r"\)")
"##" operator does:
- concatenate not only strings but also symbols into the string
Exact distinction between strings and symbols seems not to be
convenience, because it is tiresome for users (and
"revset.getstring" treats both similarly)
For example of revset alias "issue()", "issue(1234)" is easier
than "issue('1234')".
- have higher priority than any other prefix, infix and postfix
operators (like as "##" of C pre-processor)
This patch (re-)assigns the priority 20 to "##", and 21 to "(",
because priority 19 is already assigned to "-" as prefix "negate".
This is consistent with addlargefiles(), and will make it easier to get the
paths that are printed correct when recursing into subrepos or invoking from
outside the repository. It also now restricts the path that the addremove is
performed on if a path is given, as is done with normal files.
The repo.status() call needs to exclude clean files when performing an
addremove, because the addremove override method calling this used to pass the
list of files to delete, which caused the matcher to only consider those files
in building the status list. Now the matcher is restricted only to the extent
that the caller requested- usually directories if at all. There's no reason for
addremove to care about clean files anyway- we don't want them deleted.
Tracked files that are deleted should always be reported as such, no
matter what their state was in earlier revisions. This is encoded in
in two conditions in the loop in basectx._buildstatus() for modified
and added files, but the check is missing for clean files. We should
check for clean files too, but instead of adding the check in a third
place, move it earlier and skip most of the loop body for deleted
files.
When calculating status involving the working copy and a revision
other than the parent of the working copy, the files that are not in
the working context manifest ('mf2' in the basectx._buildstatus())
will be reported as removed (note that deleted files _are_ in the
working context manifest). However, if the file is reported as deleted
in the dirstate, it will get that status too (as shown by failing
tests).
Fix by removing deleted files from the 'removed' list after the main
loop in _buildstatus().
The fix for linkrev pointing to hidden revision was crashing when the file was
missing from head's manifest. We now properly handle this case.
(yes I feel silly)
Avoid spending too much time adding (glob) after running run-tests -i. This
doesn't handle all cases but it helps.
The run-tests tests add a bit of escaping of trailing (glob) in the output to
avoid interference from the outer test runner.
The regexp for matching the output lines contains a group for making multiline
substitute in a way that works with Python before 2.7.
Before this patch, a problematic revset alias aborts execution
immediately, even if it isn't referred in the specified revset.
If old "hg" may be used too (for example, bisecting Mercurial itself),
it is also difficult to write alias definitions using features newly
introduced by newer "hg" into configuration files, because such alias
definitions cause unexpected abortion at parsing revset aliases with
old "hg".
This patch delays showing parse error for the revset alias until it is
actually referred at runtime.
This patch detects referring problematic aliases in "_expandaliases"
by examination of "revsetalias.error", which is initialized with the
error message only when parsing fails.
For usability, this patch also warns about problematic aliases, even
if they aren't referred at runtime. This should help users to know
potential problems in their alias definitions earlier.
If the file revision with a filtered linkrev does not have any
(unfiltered) children, we cannot use it to bound the search for
another introduction. Instead, we have to look at the file revision
used by each head changeset. If one of them uses this file revision, we
know there is another occurrence and we have a starting point. See
inline comments for details.
Adding some kind of permanent reference of all the introductions of a
file revision instead of just the first one would be much better. But
this is more difficult. I hope to take that into account in the next
repository format.
This revset is used by 'hg log FILENAME'. This prevent bugs when used on
a repository with hidden revisions.
Instead of just discarding file revisions whose linkrevs point to filtered
revisions, we put them aside and post-process them trying to find a non-filtered
introduction. See inline documentation for details about how it works.
This only fixes some of the problems. Once again, more will be needed when we can
cannot rely on child revisions of a file to find linkrev-shadowned revisions.
A test is added for 'hg log' catching such cases.
Before this patch, the result of "status()" on "workingcommitctx" may
incorrectly contain files other than ones to be committed, because
"workingctx._dirstatestatus()" returns the result of
"dirstate.status()" directly.
For correct matching, this patch overrides "_dirstatestatus" in
"workingcommitctx" and makes it return matched files only in
"self._status".
This patch uses empty list for "deleted", "unknown" and "ignored" of
status, because status between "changectx"s also makes them empty.
Before this patch, "status()" on "workingcommitctx" with "always
match" object causes breaking "self._status" in
"workingctx._buildstatus()", because "workingctx._buildstatus()"
caches the result of "dirstate.status()" into "self._status" for
efficiency, even though it should be fixed at construction for
committing.
For example, template function "diff()" without any patterns in
"committemplate" implies "status()" on "workingcommitctx" with "always
match" object, via "basectx.diff()" and "patch.diff()".
Then, broken "self._status" causes committing unexpected files.
To avoid breaking already fixed "self._status" at "ctx.status()", this
patch overrides "_buildstatus" in "workingcommitctx".
This patch doesn't write out the result of template function "diff()"
in "committemplate" in "test-commit.t", because matching against files
to be committed still has an issue fixed in subsequent patch.
Before this patch, "workingctx.status" caches the result of
"dirstate.status" directly into "self._status".
But "dirstate.status" is invoked with False "list*" arguments in
normal "self._status" accessing route, and this makes
"unknown"/"ignored"/"clean" of status empty.
This may cause unexpected result of code paths internally accessing to
them (accessors for external usage are already removed by previous patch).
This patch makes "unknown"/"ignored"/"clean" of cached status empty
for equivalence. Making them empty is executed only when at least one
of "unknown", "ignored" or "clean" has files, for efficiency.
The annotate logic now use the new 'introrev' method to bootstrap its traversal.
This catches issues from linkrev-shadowing of the changeset introducing the
version of a file in source changeset.
More tests have been added to display pathological cases.
The follow revset (used by `hg log --follow`) now uses the new 'introrev'
method to bootstrap its traversal. This catches issues from linkrev-shadowing of
the changesets introducing the version of a file in source changeset.
A new test has been added to display pathological cases.
Another test is affected because it was meant to test this behavior but actually
failed to do so for half of the output. The output are now similar.
Because of the way filenodes are computed, you can have multiple changesets
"introducing" the same file revision. For example, in the changeset graph
below, changeset 2 and 3 both change a file -to- and -from- the same content.
o 3: content = new
|
| o 2: content = new
|/
o 1: content = old
In such cases, the file revision is create once, when 2 is added, and just reused
for 3. So the file change in '3' (from "old" to "new)" has no linkrev pointing
to it). We'll call this situation "linkrev-shadowing". As the linkrev is used for
optimization purposes when walking a file history, the linkrev-shadowing
results in an unexpected jump to another branch during such a walk.. This leads to
multiple bugs with log, annotate and rename detection.
One element to fix such bugs is to ensure that walking the file history sticks on
the same topology as the changeset's history. For this purpose, we extend the
logic in 'basefilectx.parents' so that it always defines the proper changeset
to associate the parent file revision with. This "proper" changeset has to be an
ancestor of the changeset associated with the child file revision.
This logic is performed in the '_adjustlinkrev' function. This function is
given the starting changeset and all the information regarding the parent file
revision. If the linkrev for the file revision is an ancestor of the starting
changeset, the linkrev is valid and will be used. If it is not, we detected a
topological jump caused by linkrev shadowing, we are going to walk the
ancestors of the starting changeset until we find one setting the file to the
revision we are trying to create.
The performance impact appears acceptable:
- We are walking the changelog once for each filelog traversal (as there should
be no overlap between searches),
- changelog traversal itself is fairly cheap, compared to what is likely going
to be perform on the result on the filelog traversal,
- We only touch the manifest for ancestors touching the file, And such
changesets are likely to be the one introducing the file. (except in
pathological cases involving merge),
- We use manifest diff instead of full manifest unpacking to check manifest
content, so it does not involve applying multiple diffs in most case.
- linkrev shadowing is not the common case.
Tests for fixed issues in log, annotate and rename detection have been
added.
But this changeset does not solve all problems. It fixes -ancestry-
computation, but if the linkrev-shadowed changesets is the starting one, we'll
still get things wrong. We'll have to fix the bootstrapping of such operations
in a later changeset. Also, the usage of `hg log FILE` without --follow still
has issues with linkrev pointing to hidden changesets, because it relies on the
`filelog` revset which implement its own traversal logic that is still to be
fixed.
Thanks goes to:
- Matt Mackall: for nudging me in the right direction
- Julien Cristau and Rémi Cardona: for keep telling me linkrev bug were an
evolution show stopper for 3 years.
- Durham Goode: for finding a new linkrev issue every few weeks
- Mads Kiilerich: for that last rename bug who raise this topic over my
anoyance limit.
Before this patch, "workingctx.status" always replaces "self._status"
by the recent result, even though:
- status isn't calculated against the parent of the working directory, or
- specified "match" isn't "always" one
(status is only visible partially)
If "workingctx" object is shared between some procedures indirectly
referring "ctx._status", this incorrect caching may cause unexpected
result: for example, "ctx._status" is used via "manifest()", "files()"
and so on.
To cache "self._status" correctly at "workingctx.status", this patch
overwrites "self._status" in "workingctx._buildstatus" only when:
- status is calculated against the parent of the working directory, and
- specified "match" is "always" one
This patch can be applied (and effective) only on default branch,
because procedure around "basectx.status" is much different between
stable and default: for example, overwriting "self._status" itself is
executed not in "workingctx._buildstatus" but in
"workingctx._poststatus", on stable branch.
The full path is propagated to the original match object since this is often
used directly for printing a file name to the user. This is cleaner than
requiring each caller to join the prefix with the file name prior to calling it,
and will lead to not having to pass the prefix around separately. It is also
consistent with the bad() and abs() methods in terms of the required input. The
uipath() method now inherits this path building property.
There is no visible change in path style for rel() because it ultimately calls
util.pathto(), which returns an os.sep based path. (The previous os.path.join()
was violating the documented usage of util.pathto(), that its third parameter be
'/' separated.) The doctest needed to be normalized to '/' separators to avoid
test differences on Windows, now that a full path is returned for a short
filename.
The test changes are to drop globs that are no longer necessary when printing an
absolute file in a subrepo, as returned from match.uipath(). Previously when
os.path.join() was used to add the prefix, the absolute path to a file in a
subrepo was printed with a mix of '/' and '\'. The absolute path for a file not
in a subrepo, as returned from match.uipath(), is still purely '/' based.
This patch makes "posix.shellquote" examine the specified string and
quote it only when it may have to be quoted for safety, like as the
previous patch for "windows.shellquote".
In fact, on POSIX environment, quoting itself doesn't cause issues
like issue4463. But (almost) equivalent quoting policy can avoid
examining test result differently on POSIX and Windows (even though
showing command line with "%r" causes such examination in
"test-extdiff.t").
The last hunk for "test-extdiff.t" in this patch isn't needed for the
previous patch for "windows.shellquote", because the code path of it
is executed only "#if execbit" (= avoided on Windows).
Before this patch, "windows.shellquote" (as used as "util.shellquote")
always quotes specified strings with double quotation marks, for
external process invocation.
But some problematic applications can't work correctly, when command
line arguments are quoted: see issue4463 for detail.
On the other hand, quoting itself is needed to specify arguments
containing whitespaces and/or some special characters exactly.
This patch makes "windows.shellquote" examine the specified string and
quote it only when it may have to be quoted for safety.
Before this patch, all command line arguments for external tools are
quoted by the combination of "shlex.split" and "util.shellquote". But
this causes some problems.
- some problematic commands can't work correctly with quoted arguments
For example, 'WinMerge /r ....' is OK, but 'WinMerge "/r" ....' is
NG. See also below for detail about this problem.
https://bitbucket.org/tortoisehg/thg/issue/3978/
- quoting itself may change semantics of arguments
For example, when the environment variable CONCAT="foo bar baz':
- mydiff $CONCAT => mydiff foo bar baz (taking 3 arguments)
- mydiff "$CONCAT" => mydiff "foo bar baz" (taking only 1 argument)
For another example, single quoting (= "util.shellquote") on POSIX
environment prevents shells from expanding environment variables,
tilde, and so on:
- mydiff "$HOME" => mydiff /home/foobar
- mydiff '$HOME' => mydiff $HOME
- "shlex.split" can't handle some special characters correctly
It just splits specified command line by whitespaces.
For example, "echo foo;echo bar" is split into ["echo",
"foo;echo", "bar"].
On the other hand, if quoting itself is omitted, users can't specify
options including space characters with "--option" at runtime.
The root cause of this issue is that "shlex.split + util.shellquote"
combination loses whether users really want to quote each command line
elements or not, even though these can be quoted arbitrarily in
configurations.
To resolve this problem, this patch does:
- prevent configurations from being processed by "shlex.split" and
"util.shellquote"
only (possibly) "findexe"-ed or "findexternaltool"-ed command path
is "util.shellquote", because it may contain whitespaces.
- quote options specified by "--option" via command line at runtime
This patch also makes "dodiff()" take only one "args" argument instead
of "diffcmd" and "diffopts. It also omits applying "util.shellquote"
on "args", because "args" should be already stringified in "extdiff()"
and "mydiff()".
The last hunk for "test-extdiff.t" replaces two whitespaces by single
whitespace, because change of "' '.join()" logic causes omitting
redundant whitespaces.
Previously, revert was only possible if the '--no-backup'
switch was specified.
Now, to support backups, we explicitly go over all modified
files in the subrepo.
In the body of the loop in trydiff(), there are conditions like:
addedset or (f in modifiedset and to is None)
The second half of that expression is to account for the fact that
merge-in additions appear as additions. By instead fixing up the sets
of modified and added files to compensate for this fact, we can
simplify the body of the loop. It also fixes one case where the
addedset was checked without the additional check (the "have we
already reported a copy above?" case in the code, also see fixed test
case).
The similar condition with 'removedset' in it seems to have served no
purpose even before this change, so it could have been simplified even
before.
There is no --after for addremove, so the printing for addremove can be hoisted
out of the 'not after' check. The difference between the two remove messages
reflects the existing difference between core remove and core addremove styles
for printing the file.
There are still some pre-existing issues here. Core addremove only prints on
inexact matches or when verbose. But since the largefiles that are being
removed are passed to removelargefiles() as a pattern list, there is never an
inexact match, which would keep the largefiles from being printed at all unless
verbose is specified. Therefore, the output is a little more aggressive than
core. The addremove print style here is also inconsistent with core- it should
use matcher.uipath(f) instead of f. These can be fixed once a matcher is passed
in.
When a directory was renamed and a new untracked file was added in the
new directory and the remote directory added a file by the same name
in the old directory, the local untracked file gets overwritten, as
demonstrated by the broken test case in test-rename-dir-merge.
Fix by checking for unknown files for 'dg' actions too. Since
_checkunknownfile() currently expects the same filename in both
contexts, we need to add a new parameter for the remote filename to
it.
The pushkey operation used to be in its own wireprotocol command and (in
practice) always be lock free when running the hook. With bundle2, it happen in
a greater scheme and a hook running locking command would get stuck. We now run
such hooks after the lock release as similar hook do.
Bundle2 test are altered to ensure we are lockfree at hook running time.
This is different from latesttagdistance in that while latesttagdistance is
defined to be the length of the longest path to the latest tag,
changessincelatesttag is the number of changes contained in @ that aren't
contained in the latest tag. So, if 't' is the latest tag in the repository
below:
t
|
v
--o--o----o
\ \
..o..o..@
then latesttagdistance is 2, but changessincelatesttag is 4.
Note that changessincelatesttag is always greater than or equal to the
latesttagdistance -- that's because changessincelatesttag counts all the
changes in the longest path since the latest tag, and possibly others. This is
an important fact that we'll take advantage of in upcoming patches.
By moving the conversion from the file->action dict after the bid
merge code, bid merge can be simplified a little.
A few tests are affected by this change. Where we used to iterate over
the actions first in order of the action type ('g', 'r', etc.) [1], we
now iterate in order of filename. This difference affects the order of
debug log statements.
[1] And then in the non-deterministic order of files in the manifest
dictionary (the order returned from manifest.diff()).
This patch makes bundrepo retract the phase boundary for new commits to 'draft'
status, which is consistent with the behavior of 'hg unbundle'. The old
behavior was for commits to appear with the same phase as their nearest
ancestor in the base repository.
This affects several classes of operation:
* Inspecting a bundle with the -B flag
* Treating a bundle file as a peer (old: everything public, new: everything draft)
* Incoming command (neither old or new behavior is sensible -- fixed in next patch)
Previously these would be considered to be relative to the current working
directory. That behavior is both undocumented and doesn't really make sense.
There are two reasonable options for how to resolve relative paths:
- relative to the repo root
- relative to the config file
Resolving these files relative to the repo root matches existing behavior with
hooks. An earlier discussion about this is available at
http://mercurial.markmail.org/thread/tvu7yhzsiywgkjzl.
Thanks to Isaac Jurado <diptongo@gmail.com> for the initial patchset that
spurred the discussion.
The test case doesn't only check the commit message, but also the
patch, which can result in confusing output like
+ Revision df6f06d17100 does not comply to commit message rules
+ ------------------------------------------------------
+ 32: adds double empty line
+
+
even when there are no double blank lines in the commit message. Drop
the "commit message" part to make it less confusing.
Merged files are considered modified at commit time even if only 1 parent
differs. In this case we must use the change context of this parent for
expansion.
The issue went unnoticed for long because it is only apparent until the next
update to the merge revision - except in test-keyword where it was always
staring us in the face.
Mercurial backout command makes a commmit by default only when the backed out
revision is the parent of working directory and doesn't commit in any other
case.
The --commit option changes behaviour of backout to make a commit whenever
possible (i.e. there is no unresolved conflicts). This behaviour seems more
intuitive to many use (especially git users migrating to hg).
This patch adds the -B/--bookmarks option to the share command added by the
share extension. All it does for now is create a marker, 'bookmarks.shared',
that will be used by future code to implement the sharing functionality.
Because:
- the test to avoid regression for issue4470 was already added to
test-commit-amend.t by previous patch
It is also a part of test series about manifest calculation issues
of memctx in test-commit-amend.t.
- this is the only test using "commit --amend" in test-commit.t
Before this patch, "memctx._manifest" tries to get (and use normally)
filectx also for newly-removed files, even though "memctx.filectx()"
returns None for such files.
To calculate manifest correctly even with newly-removed files, this
patch does:
- replace "man.iteritems()" for the loop by "self._status.modified"
to avoid accessing itself to newly removed files
this also reduces loop cost for large manifest.
- remove files in "self._status.removed" from the manifest
In this patch, amending is confirmed twice to examine both (1) newly
removed files and (2) ones already removed in amended revision.
Before this patch, "memctx._manifest" calculates the manifest
according to the 1st parent. This causes the disappearance
of newly added files from the manifest.
For example, if newly added files aren't listed up in manifest of
memctx, they aren't listed up in "added" field of "status" returned by
"ctx.status()", and "{diff()}" (= "patch.diff") in "committemplate"
shows nothing for them.
To calculate manifest including newly added files correctly, this
patch puts newly added files (= ones in "self._status.added") into the
manifest.
Some details of changes for "test-commit-amend.t" in this patch:
- "touch foo" is replaced by "echo foo > foo", because newly added
empty file can't be shown in "diff()" output without "diff.git"
configuration
- amending is confirmed twice to examine both (1) newly added files
and (2) ones already added in amended revision
Before this patch, "memctx._status" is initialized by "(files, [], [],
[], [], [], [])" and this causes "memctx.modified" to include not
only modified files but also added and removed ones incorrectly.
This patch adds "_status" method to calculate exact status being
committed according to "files" specified at construction time.
Exact "_status" is useful to share/reuse logic of committablectx.
This patch is also preparation for issues fixed by subsequent patches.
Some details of changes for tests in this patch:
- some filename lines are omitted in "test-convert-svn-encoding.t",
because they are correctly listed up as "removed" files
those lines are written out in "localrepository.commitctx" for
"modified" and "added" files by "ui.note".
- "| fixbundle" filterring in "test-histedit-fold.t" is omitted to
check lines including "added" correctly
"fixbundle" discards all lines including "added".
filectxfn returns None for removed files, so we have to check for None
before computing the new file content hash for the manifest.
Includes a test that proves this works, by demonstrating that we can
show the diff of an amended commit in the committemplate.
We have our own fast-path logic to see if something passes a sniff
test for being a Subversion repository, but it's possible for a user
to svnsync a repo using svn 1.8 and then use svn 1.7 bindings (as in
the bug) to try and convert the repo. If we at least tell the user the
version of libsvn that we used, they might get enough of a hint to
check on their own for format incompatibilities between their
svn{admin,sync} and the libsvn used by hg.
Instead of just bootstrapping the algorithm with the first revision we
see, allow callers to pass revs that should be displayed first. All
branches are retained until we can display such revision.
Expected usage is to display the current working copy parent first.
The algorithm now works when some revisions are skipped. We now use "first
included ancestors" instead of just "parent" to link changesets with each other.
We add an experimental config option to use the topological sorting. I first
tried to hook the 'groupbranchiter' function in the 'sort' revset but this was useless
because graphlog enforces revision number sorting :(
As the goal is to advance on the topological iteration logic, I see this
experimental option as a good way to move forward.
We have to use turn the iterator into a list because the graphlog is apparently
not ready for pure iterator input yet.
'git diff --stat' output changed with regard to the amount
of changes/insertions/deletions shown.
In older git versions (1.7.7.6), output was shown as:
2 files changed, 2 insertions(+), 1 deletions(-)
In newer versions, output is shown as:
2 files changed, 2 insertions(+), 1 deletion(-)
This patch uses a regex to handle both cases.
Previously, git subrepos did not support reverting.
This change adds basic support for reverting
when '--no-backup' is specified.
A warning is given (and the current state is kept)
when a revert is done without the '--no-backup' flag.
This does not cause any behavioral change unless a 'bookmarks.shared' marker
file exists. A future change will add UI to create this file when a repository
is shared.
Without this patch, if the server sets preferuncompressed, there's no way for
clients to override that and force a non-streaming clone. With this patch, we
extend the meaning of --pull to also override preferuncompressed and force a
non-streaming clone.
The output of 'git diff --stat' changed in git 1.7.10 and 1.7.11.
To ensure the tests work with earlier versions of git as well,
the output is now wrapped with a whitespace regex.
By moving the cd/dc prompts out of calculateupdates(), we let
largefiles' overridecalculateupdates() so the unresolved values
(i.e. 'cd' or 'dc' rather than 'g', 'r', 'a' and missing). This allows
overridecalculateupdates() to ask the user whether to keep the normal
file or the largefile before the user gets the cd/dc prompt. Whichever
answer the user gives, we make overridecalculateupdates() replace 'cd'
or 'dc' action, saving the user one annoying (and less clear)
question.
Since addremove on the top of a directory tree will recursively handle sub
directories, it should be the same with deep subrepos, once the user has
explicitly asked to process a subrepo. This really only has an effect when a
path that is a subrepo (or is in a subrepo) is given, since -S causes all
subrepos to be processed already. An addremove without a path that crosses into
a subrepo, will still not enter any subrepos, per backward compatibility rules.
Git and svn subrepos are currently not supported. It doesn't look like git or
svn have these commands natively, so that's an area for a git or svn expert.
The recursive addremove operation occurs completely before the first subrepo is
committed. Only hg subrepos support the addremove operation at the moment- svn
and git subrepos will warn and abort the commit.
It looks like a bad path is the only mode of failure for addremove. This
warning is probably useful for the standalone command, but more important for
'commit -A'. That command doesn't currently abort if the addremove fails, but
it will be made to do so prior to adding subrepo support, since not all subrepos
will support addremove. We could just abort here, but it looks like addremove
has always silently ignored bad paths, except for the exit code.
It is easier to reason about certain algorithms in terms of a
file->action mapping than the current action->list-of-files. Bid merge
is already written this way (but with a list of actions per file), and
largefiles' overridecalculateupdates() will also benefit. However,
that requires us to have at most one action per file. That requirement
is currently violated by 'dr' (divergent rename) and 'rd' (rename and
delete) actions, which can exist for the same file as some other
action.
These actions are only used for displaying warnings to the user; they
don't change anything in the working copy or the dirstate. In this
way, they are similar to the 'k' (keep) action. However, they are even
less action-like than 'k' is: 'k' at least describes what to do with
the file ("do nothing"), while 'dr' and 'rd' or only annotations for
files for which there may exist other, "real" actions.
As a first step towards separating these acitons out, stop including
them in the progress output, just like we already exclude the 'k'
action.
So far, git subrepositories were silently ignored for diffs.
This patch adds support for git subrepositories,
with the remark that --include and --exclude are not supported.
If --include or --exclude are used, the subrepo is ignored.
It deserves more than a debug message. Show a note like:
updating mq patch p0.patch to 5:9ecc820b1737
The message could also refer to "qrefresh" instead. Same same.
Show status messages while rebasing, similar to what graft do:
rebasing 12:2647734878ef "fork" (tip)
This gives more context for the user when resolving conflicts.
Globbing the hash made it harder to maintain tests with run-tests -i when it
was so far by the generated test output.
The hashes are stable and we just need to add a (glob).
It was just showing a status message with the internal revision number.
Instead, show a warning like
note: graft of 27:3aaa8b6725f0 "28" created no changes to commit
(message tweaked in-flight by mpm)
Show status messages with first line of commit description and names, like
grafting 12:2647734878ef "fork" (tip)
This gives more context for the user when resolving conflicts.
When merging and the remote has turned a normal file into a largefile
and the user chooses to keep the local largefile, we use the 'r'
action for the remote largefile standin. This is wrong, since that
file does not exist in the parent of the working copy. Use 'k', which
does nothing but debug logging, instead.
When merging and the remote has turned a largefile into a normal file
and the user chooses to keep the local largefile, we use the 'r'
action for the remote normal file. This is wrong, since that file does
not exist in the parent of the working copy. Use 'k', which does
nothing but debug logging, instead.
When set to true, this option will make patchbomb always ask for confirmation
before sending the email. Confirmation is a powerful way to prevent stupid
mistakes when the sending patches.
This should let me get rid of my global alias adding
--confirm to hg email.
I know that some people may get bitten when moving from a machine with confirm
configured to a machine where it is not, but I think it is worth the risk.
This option allows the user to control the default behavior for
including an introduction message. This avoids having to tirelessly
skip the intro for people contributing to Mercurial.
The three possibles values are:
- always,
- auto (default, current behavior),
- never.
I was thinking of ("true", "false", "") (empty value being auto) but I ruled it
out as too confusing.
This new config option reuses the pre-existing 'patchbomb' section.
Most merge action messages don't describe the action itself, they
describe the reason the action was taken. The only exeption is the 'k'
action, for which the message is just "keep" and instead there is a
code comment folling it that says "remote unchanged". Let's move that
comment into the merge action message.
This fixes the previously mentioned issue with 7d5fcea60c78, and undoes its
corresponding test change.
The test change demonstrates the correctness when a file is specified (i.e. the
glob is required on Windows because relative paths use '\' and absolute paths
use '/'). It is admittedly very subtle, but there will be a more robust test in
the addremove -S v3 series.
When the local side has renamed a directory from a/ to b/ and added a
file b/c in it, and the remote side has added a file a/c, we end up
overwriting the local file b/c with the contents of remote file
a/c. Add a check for this case and use the merge ('m') action in this
case instead of the directory rename get ('dg') action.
When the remote side has renamed a directory from a/ to b/ and added a
file b/c in it, and the local side has added a file a/c, we end up
moving a/c to b/c without considering the remote version of b/c. Add a
check for this case and use the merge ('m') action in this case
instead of the directory rename ('dm') action.
The "nothing to merge" case is covered by test-merge-default.t.
The "uncommitted changes" case is covered by test-merge1.t (and
others).
The "merge -f" case is covered by test-merge-force.t.
Changset 281e85651e92 (merge: demonstrate that directory renames can
lose local file content, 2014-12-02) should clearly have added the
reverse version of the test: where the remote side renamed a
directory, added a new file in that directory, and the local directory
added a conflicting file in the source directory. Add such a test now,
and also touch up the ones already added slightly (e.g. 'local' was a
stupid value for content that can be on either side of a merge).
One of the graft tests grafts a changeset that changes a file's
content from 'a' to 'b' onto a branch that has changed the file's
content from 'a', via 'b', and then back to 'a' again. To prepare for
not considering this a file in need of merging, let's use 'c' as the
file's new content to make sure it has to be considered
conflicting.
There's a second similar case further down where an ancestor is
grafted. Make sure that is also considered a conflict.
The whitespace diffopts break lossless transmission, and the format-changing
ones make import harder. We expect parsers to be able to read git-style diffs,
though.
With generate-working-copy-states.py generalized to support
arbitrarily many changesets, we can use it for generating test cases
for merge: use one changeset each for base, remote and local. With the
various working copy states, this is a total of 104 cases.
The first candidate for additional testing is 'hg merge --force'. Even
though the force option is deprecated, it is convenient for testing
because it can be tested without first needing to revert any
changes. Except for the lack of checking for uncommitted changes, it
differs in only a few cases from unforced merge.
The new tests cover all the cases in the existing test-merge-force.t,
except for the unforced merge case, which is covered in several other
files anyway, so nothing remains of the file after this patch.
Add a parameter to generate-working-copy-states.py that indicates
how many changesets are wanted. This number impacts all the
subcommands. The current 'filelist' subcommand becomes 'filelist 2',
the current 'base' and 'parent' subcommands become 'state 2 1' and
'state 2 2' respectively, while 'wc' becomes 'state 2 wc'.
See earlier patch for motivation.
The script can currently generate filenames and contents for exactly
two changesets plus the working copy. For some tests (e.g. of plain
dirstate status), only one changeset is needed, while for others
(e.g. merge), three changesets are needed. Let's prepare for such
tests by generalizing the code for any number of changesets.
When a directory has been renamed on the local branch and a file has
been added in the old location on a remote branch, we move that new
file to the new location. Unfortunately, if there is already a file
there, we overwrite it with the contents from the remote branch. For
untracked local files, we should probably abort, and for tracked local
files, we should merge the contents. To start with, let's add a test
to demonstrate the breakage. Also note that while files merged in from
a remote branch are normally (and unintuitively) reported as modified,
these files are reported as added.
93eca2533d2a and 1deb493773a1 fixed issue4453 with a simple insertplainheader
function that fixed the regression but didn't make the implementation more
stable.
Now we introduce plain header handling similar to how we handle hg patches. The
whole header is scanned for fields to update while determining the best
position for inserting the field if it is missing. It also makes sure there is
an empty line between headers and body.
The fetch extension has been calling cmdutil.bailifchanged() since 70b2d52341c9,
so this is redundant. Add test coverage to prevent regression. It doesn't look
like there is any testing for fetch with largefiles.
This patch series is intended to allow bundle2 push reply part handlers to
make changes to the local repository; it has been developed in parallel with
an extension that allows the server to rebase incoming changesets while applying
them.
This diff adds an experimental config option "bundle2.pushback" which provides
a transaction to the reply unbundler during a push operation. This behavior is
opt-in because of potential security issues: the response can contain any part
type that has a handler defined, allowing the server to make arbitrary changes
to the local repository.
This patch series is intended to allow bundle2 push reply part handlers to
make changes to the local repository; it has been developed in parallel with
an extension that allows the server to rebase incoming changesets while applying
them.
Most pushes already open a transaction in order to sync phase information.
This diff replaces that transaction with one that spans the entire push
operation.
This transaction will be used in a later patch to guard repository changes
made during the reply handler.
For "hg addremove 'glob:*.py'", we print any paths added or removed as
relative to the current directory, but when "hg addremove -I
'glob:*.py'" is used, we use the absolute path (relative from the repo
root). It seems like they should be the same, so change it so we use
relative paths in both cases. Continue to use absolute paths when no
patterns are given.
We can further limit the scope of the 2-way merge case by breaking out
the case where the file was not created from scratch on both sides but
rather renamed in the same way (and is therefore a 3-way merge). This
involves copying some code, but it makes it clearer which case the
"Note:" in the code refers to.
When 'f' is not in 'ma', 'a' will be 'nullid' and all the if/elif
conditions that check whether some one nodeid is equal to 'a' will
fail, and the else-clause will instead apply. We can make that more
explicit by creating a separate 'm' action for the case where 'a' is
'nullid'. While it does mean copying some code, perhaps it makes it a
little clearer which codepaths are possible, and which cases the
"Note:" in the code refers to. It also lets us make the debug action
messages a little more specific.
The double quotes are necessary, otherwise it tries to pipe into a program named
'short'. An '&' could serve as a command separator on Windows instead of ';',
but I don't see any obvious way to swap these depending on the platform. In
this case though, there really wasn't a need for multiple statements.
This fixes a test failure introduced in 131ac757f996 on Windows and OS X, where
the cached largefile wasn't being deleted because the named .cache directory
didn't exist. It only existed on Linux because the test suite sets $HOME to the
directory of the test being run, and Linux uses $HOME/.cache by default.
Most of the other largefiles tests explicitly set this value at the top of their
scripts, but test-largefiles-update.t didn't pick that up when it was created.
Those scripts that do set a value will override this.
We could just set the parameter in the test-largefiles-update.t script, but
there are a few other non obvious tests that exercise largefiles too. These
largefiles end up being cached in the user's real cache, so proper hygiene
dictates that this not be left to each individual test script.
We are about to make bookmarks and phases available for hooks.
Therefore we need a witness for this new availability. We introduce
the new hooks in a distinct changeset to reduce the noise in the ones
with actual changes.
Such file are generated with a .pending prefix. It is up to the reader to
implement the necessary logic for reading pending files.
We add a test to ensure pending files are properly cleaned-up in both success and
error cases.
When test output is processed, if os.altsep is defined (i.e. on Windows),
TTest.globmatch() will cause a warning later on if a line has a glob that isn't
necessary. Unfortunately, the regex checking in check-code.py doesn't have this
context. Therefore we ended up with cases where the test would get flagged with
a warning only on Windows because a glob was present, because check-code.py
would warn if it wasn't. For example, from test-subrepo.t:
$ hg -R issue1852a push `pwd`/issue1852c
pushing to $TESTTMP/issue1852c (glob)
The glob isn't necessary here because the slash is shown as it was provided.
However, check-code mandates one to handle the case where the default path has
backslashes in it.
Break the cycle by checking against a subset of the check-code rules before
flagging the test with a warning, and ignore the superfluous glob if it matches
a rule. This change fixes warnings in test-largefiles-update.t, test-subrepo.t,
test-tag.t, and test-rename-dir-merge.t on Windows.
I really hate that the rules are copy/pasted here (minus the leading two spaces)
because it would be nice to only update the rules once, in a single place. But
I'm not sure how else to do it. I'm open to suggestions. Splitting some of the
rules out of check-code.py seems wrong, but so does moving check-code.py out of
contrib, given that other checking scripts live there.
There are other glob patterns that could be copied over, but this is enough to
make the current tests run on Windows.
Before this patch, a part of "test-push-hook-lock.t" fails unexpectedly on
Windows environment, because semicolon (";") isn't recognized as the command
separator by "cmd.exe". This is fixed the same way as a similar issue in
7c253c23de3b.
test-largefiles-update.t, test-subrepo.t, test-tag.t, and
test-rename-dir-merge.t still warn about no result returned because of
unnecessary globs that test-check-code-hg.t wants, relating to output for
pushing to, pulling from and moving X to Y.
This fixes test-install.t on Windows that broke in 97300cee8fc0 when
shlex.split() was added to the debuginstall command:
@@ -7,8 +7,11 @@
checking installed modules (*mercurial)... (glob)
checking templates (*mercurial?templates)... (glob)
checking commit editor...
+ Can't find editor 'c:\Python27\python.exe -c "(omitted)"' in PATH
+ (specify a commit editor in your configuration file)
checking username...
- no problems detected
+ 1 problems detected, please check your install!
+ [1]
What happens is that shlex.split() on Windows turns this:
c:\Python27\python.exe -c "import sys; sys.exit(0)"
into this:
['c:Python27python.exe', '-c', 'import sys; sys.exit(0)']
While technically a regression, most programs on Windows live in some flavor of
'Program Files', and therefore the environment variable needs to contain quotes
anyway to handle the space. This wasn't handled prior to the shlex() change,
because it tested the whole environment variable to see if it was an executable,
or split on the first space and tested again.
This allows multiple efficient missing ancestor queries against the same set of
bases. In upcoming patches we'll also define ways to grow the set of bases.
The fact that the test output hasn't changed establishes this patch's
correctness.
Previously, any files relative to the root of the repo that match the -I
patterns would be deleted, but the command exited with 1 after printing a
warning:
$ hg remove -S -I 're:.*.txt' .
removing sub1/sub2/folder/test.txt
removing sub1/sub2/test.txt
not removing .: no tracked files
Like 'forget', git and svn subrepos are currently not supported. Unfortunately
the name 'remove' is already used in the subrepo classes, so we break the
convention of naming the subrepo function after the command.
Because pipe-mode server uses stdio as IPC channel, other modules should not
touch stdio directly and use ui instead. However, this strategy is brittle
because several Python functions read and write stdio implicitly.
print 'hello' # should use ui.write()
# => ch = 'h', size = 1701604463 'ello', data = '\n'
This patch adds protection for such mistakes. Both stdio files and low-level
file descriptors are redirected to /dev/null while command server uses them.
As I've been working on complicated extension code it's been handy to
be able to get standard unittest verbose output so I can find crashers
more efficiently.
See previous patch descriptions for the motivation.
The tests reflect the current state of the world -- as we add support we'll see
changes in the test output.
In an upcoming patch we'll enable support as an option to 'hg diff' as well.
The tests reflect the current state of the world -- as we add support we'll see
changes in the test output.
This is a very silly case and not particularly likely to happen in the wild,
but it turns out we can hit it in a couple of places. As we tune the storage
parameters we're likely to hit more such cases.
The affected test cases all have smaller revlogs now.
Before this patch, while "hg convert", largefiles avoids copying
largefiles in the working directory into the store area by combination
of setting "repo._isconverting" in "mercurialsink{before|after}" and
checking it in "copytostoreabsolute".
This avoiding is needed while "hg convert", because converting doesn't
update largefiles in the working directory.
But this implementation is not efficient, because:
- invocation in "markcommitted" can easily ensure updating
largefiles in the working directory
"markcommitted" is invoked only when new revision is committed via
"commit" of "localrepository" (= with files in the working
directory). On the other hand, "commitctx" may be invoked directly
for in-memory committing.
- committing without updating the working directory (e.g. "import
--bypass") also needs this kind of avoiding
For efficiency of this kind of avoiding, this patch does:
- move "copyalltostore" invocation into "markcommitted"
- remove meaningless procedures below:
- hooking "mercurialsink{before|after}" to (un)set "repo._isconverting"
- checking "repo._isconverting" in "copytostoreabsolute"
This patch invokes "copyalltostore" also in "_commitcontext", because
"_commitcontext" expects that largefiles in the working directory are
copied into store area after "commitctx". In this case, the working
directory is used as a kind of temporary area to write largefiles out,
even though converted revisions are committed via "commitctx" (without
updating normal files).
Before this patch, "hg transplant --continue" may record incorrect
standins, because largefiles extension always avoid updating standins
while transplanting, even though largefiles in the working directory
may be modified manually at the 1st commit of "hg transplant --continue".
But, on the other hand, updating standins should be avoided at
subsequent commits for efficiency reason.
To update standins only at the 1st commit of "hg transplant
--continue", this patch uses "automatedcommithook", which updates
standins by "lfutil.updatestandinsbymatch()" only at the 1st commit of
resuming.
Even after this patch, "repo._istransplanting = True" is still needed
to avoid some status report while updating largefiles in
"lfcommands.updatelfiles()".
This is reason why this patch omits not "repo._istransplanting = True"
in "overriderebase" but examination of "getattr(repo,
"_istransplanting", False)" in "updatestandinsbymatch".
At "hg transplant --merge REV", largefiles newly coming from the 2nd
parent (= REV) are marked as "a"(dded) by "patch.patch()", and have to
be marked as "n"(ormal) after commit.
But until changeset 978713c45992, such largefiles were still marked as
"a" unexpectedly even after commit, because no additional entry is
added to filelog of such largefiles and they aren't listed in
"repo[newnode].files()" in this case: "newnode" is one of newly
committed changeset (= result of "repo.commit()").
"updatelfiles" invocation in "overridetransplant" shadows this problem
by forcibly synchronizing lfdirstate to dirstate.
Now, "updatelfiles" invocation in "overridetransplant" is redundant,
because changeset 978713c45992 made "markcommitted" use "ctx.files()"
to get targets of "synclfdirstate" instead of "repo[newnode].files()".
After running "hg forget README && hg addremove", README will still be
reported as removed, while "hg forget README && hg add README" adds it
back so it gets reported as clean. It seems like they should behave
the same. Furthermore, it seems like no files should remain untracked
after 'hg addremove && hg commit' (or 'hg commit -A'). For these
reasons, change the behavior of addremove so it does add forgotten
files back.
The problem is with scmutil._interestingfiles(), which reports the
file as removed, so scmutil.addremove() does not add it. Fix by
teaching _interestingfiles() to report forgotten files separately from
removed files and make addremove() add forgotten files back. However,
do not treat forgotten files as sources for rename detection. Note
that since removed and forgotten files are treated the same before
this change, forgotten files were considered sources for rename
detection.
Also update the other caller, marktouched(), in the same way as
addremove().
I accidentally did 'hg forget .' and tried to undo the operation with
'hg add .'. I expected the files to be reported as either modified or
clean, but they were still reported as removed. It turns out that
forgotten files are only added back if they are listed explicitly, as
shown by the following two invocations. This makes it hard to recover
from the mistake of forgetting a lot of files.
$ hg forget README && hg add README && hg status -A README
C README
$ hg forget README && hg add . && hg status -A README
R README
The problem lies in cmdutil.add(). That method checks that the file
isn't already tracked before adding it, but it does so by checking the
dirstate, which does have an entry for forgotten files (state 'r'). We
should instead be checking whether the file exists in the
workingctx. The workingctx is also what we later call add() on, and
that method takes care of transforming the add() into a normallookup()
on the dirstate.
Since we're changing repo.dirstate into wctx, let's also change
repo.walk into wctx.walk for consistency (repo.walk calls wctx.walk,
so we're simply inlining the call).
The current heuristic for deciding between storing delta and full texts
is based on ratio of (sizeofdeltas)/(sizeoffulltext).
In some cases (for example a manifest for ahuge repo) this approach
can result in extremely long delta chains (~30,000) which are very slow to
read. (In the case of a manifest ~500ms are added to every hg command because of that).
This commit introduces "revlog.maxchainlength" configuration variable that will
limit delta chain length.
The randomness in the discovery protocol made this problem hard to reproduce.
The test mocks random.sample to make sure we hit the problem every time. The
set iteration order also made the output unstable ... but with the issue fixed,
it is stable.
We have tests for the status across from '.^' to the working copy. It
makes sense to have the similar tests for the inter-revision status
between '.^' and '.' and for the dirstate status in the same
place.
The initial commit was there when we had a group of tests that
compared against an empty base, but since those tests no longer exist,
we can drop the empty commit.
It's getting a little hard to read the ~30 calls to 'hg status' with
one per file. Instead, let's use one glob for each expected
status. For example, modified files can be listed with
'glob:content1_*_content[23]-tracked'. That also nicely becomes an
explanation for why each status is expected.
The second group of tests in test-status-rev compare to an empty
revision. The first group of tests that compare to the first commit
should be testing all the same states with the missing_* files, so
drop the second group of tests.
The post-transaction hooks run after the lock release (because hooks may want to
touch the repository), but they must only run if the transaction is successfully
closed.
We use the new 'addpostclose' method on transaction to register a callback
installing this post-lock-release call.
The status for missing_content2_content2-untracked doesn't get
reported at all. Since the file does exist in the working copy, it
should reported as unknown. Document that in the test.
Start using the generate-working-copy-states.py script that's shared
with test-revert.t, instead of creating the states manually in the
test. This adds several states that are currently missing. We will
start checking those states later.
To prepare for using generate-working-copy-states.py for generating
the files and their content, let's start by renaming the files
according to the naming scheme used by that script.
With the recent change in naming of the generated files, it becomes
much easier to generate the files by iterating over all the possible
states than over the state transitions.
Before this patch, "hg rebase --continue" may record incorrect
standins, because largefiles extension always avoid updating standins
while rebasing, even though largefiles in the working directory may be
modified manually at the 1st commit of "hg rebase --continue".
But, on the other hand, updating standins should be avoided at
subsequent commits for efficiency reason.
To update standins only at the 1st commit of "hg rebase --continue",
this patch introduces state-full callable object
"automatedcommithook", which updates standins by
"lfutil.updatestandinsbymatch()" only at the 1st commit of resuming.
Even after this patch, "repo._isrebasing = True" is still needed to
avoid some status report while updating largefiles in
"lfcommands.updatelfiles()".
This is reason why this patch omits not "repo._isrebasing = True" in
"overriderebase" but examination of "getattr(repo, "_isrebasing",
False)" in "updatestandinsbymatch".
This patch removes "--rebase" specific code path for "hg pull" in
"overridepull", because previous patch makes it meaningless: now,
"rebase.rebase" ("orig" invocation in this patch) can
update/commit largefiles safely without "repo._isrebasing = True".
As a side effect of removing "rebase.rebase" invocation in
"overridepull", this patch removes "nothing to rebase ..." message in
"test-largefiles.t", which is shown only when rebase extension is
enabled AFTER largefiles:
before this patch:
1. "dispatch" invokes "pullrebase" of rebase as "hg pull" at
first, because rebase wraps "hg pull" later
2. "pullrebase" invokes "overridepull" of largefiles as "orig",
even though rebase assumes that "orig" is "pull" of commands
3. "overridepull" executes "pull" and "rebase" directly
3.1 "pull" pulls changesets and creates new head "X"
3.2 "rebase" rebases current working parent "Y" on "X"
4. "overridepull" returns to "pullrebase"
5. "pullrebase" tries to rebase, but there is nothing to be done,
because "Y" is already rebased on "X". then, it shows "nothing
to rebase ..."
after this patch:
1. "dispatch" invokes "pullrebase" of rebase as "hg pull"
2. "pullrebase" invokes "overridepull" of largefiles as "orig"
3. "overridepull" executes "pull" as "orig"
4. "overridepull" returns to "pullrebase"
5. revision "Y" is not yet rebased, so "pullrebase" doesn't shows
"nothing to rebase ..."
As another side effect of removing "rebase.rebase" invocation, this
patch fixes issue3861, which occurs only when rebase extension is
enabled BEFORE largefiles:
before this patch:
1. "dispatch" invokes "overridepull" of largefiles at first,
because largefiles wrap "hg pull" later
2. "overridepull" executes "pull" and "rebase" explicitly
2.1 "pull" pulls changesets and creates new head "X"
2.2 "rebase" rebases current working parent, but fails because
no revision is checked out in issue3861 case
3. "overridepull" returns to "dispatch" with exit code 1 returned
from "rebase" at (2.2)
4. "hg pull" terminates with exit code 1 unexpectedly
after this patch:
1. "dispatch" invokes "overridepull" of largefiles at first
2. "overridepull" invokes "pullrebase" of rebase as "orig"
3. "pullrebase" invokes "pull" as "orig"
4. "pullrebase" invokes "rebase", and it fails
5. "pullrebase" returns to "overridepull" with exit code 0
(because "pullrebase" ignores result of "pull" and "rebase")
6. "overridepull" returns to "dispatch" with exit code 0 returned
from "rebase" at (5)
7. "hg pull" terminates with exit code 0
When a file is missing in the 'parent' version and is tracked but
missing in the working directory, which happens by the 'missing' or
'removed' types, and the 'clean' type in the working directory, the
file does not exist in the working directory (unlike it would had the
'deleted' type been used). Thus, the *_missing_missing_tracked are not
actually tracked and they end up testing the same state as
*_missing_missing_untracked. To make them tracked, add a temporary
file, just like we do for the delete case. For simplicity's sake,
let's make sure the gen-revert-cases.py script always puts a file in
the working directory, whether or not it's going to be deleted.
Future patches will change how the output of 'gen-revert-cases.py
filelist' is generated, so now we want the order to depend on just the
filename again.
This is the main patch in a series. See motivation in earlier patch.
In this patch, we actually change the names of the generated
files. For example, the file that is currently called missing_clean
becomes missing_missing_missing-tracked and it's clearer that it
should be tracked. It turns out that since the state was not
previously clear, it ended up testing an untracked state, which was
the same as for missing_clean. We'll fix this in a later patch.
Let's also change the content from (base,parent,wc) to
(content1,content2,content3) to make them all the same length so they
line up when displayed.
The next patch will change the names of the files produced by the
script in test-revert. In order to reduce the size and increase the
clarity of the next patch, make the order produced by the internal
'gen-revert-cases.py filelist' command independent of the filenames.
By putting the file content rather than keys in the 'combination'
list, we restrict the knowledge of 'ctxcontent' and 'wccontent' to the
loop generating the combinations. That will make it easier to replace
the generation code.
The 'wccontent' variable has eight different states, four of them
tracked, and the other four untracked (at least when the file existed
in the parent revision). Among these eight states, 'removed' sticks
out by lacking the 'untracked-' prefix despite resulting in an
untracked state. To make the symmetry clearer, and to prepare for
future patches, rename 'removed' to 'untracked-deleted', which is
exactly what it is.
Note that, unlike 'remove', 'deleted' is configured in
gen-revert-cases.py to have content in the working directory and that
that content is instead expected to be removed in the test script.
However, no changes are needed to the test script, since it already
contains 'hg forget *untracked*' and 'rm *deleted*', which together
have the same effect as 'hg remove'.
See additional motivation in earlier patch.
The tests for removed_deleted and removed_removed test the same state
as removed_clean and removed_untracked-clean, respectively. Drop the
duplicate tests.
See additional motivation in earlier patch.
The tests for added_revert and added_untracked-revert test the same
state as added_deleted and added_removed, respectively. Drop the
duplicate tests.
See additional motivation in earlier patch.
This is the first step in a series that aims to put the state, not the
state transitions, in the filenames of the files generated by the
gen-revert-cases.py script. The possible state of a file in a revision
and in the working copy is only whether it exists and what its content
is (the tests don't care check flags). In the dirstate, the only state
is whether it's tracked or not. With the new naming, the file that is
currently called modified_untracked-clean now becomes
content1_content2_content2-untracked, for example.
By putting these states in the filename, it becomes easier to see that
we're not missing or duplicating any state, and to check that the
state is what we think it is. For example, the file that is currently
called missing_clean becomes missing_missing_missing-tracked and it's
clearer that it should be tracked.
Putting the content in the filename will also make the tests of file
content (e.g. "cat ../content-parent.txt") very obvious.
When we put the state in the filename, the filenames clearly need to
be unique. However, it turns out that some states are currently tested
multiple times. The 'revert' transition in the script means to take
the content from the grandparent. If the parent is the same as the
grandparent, there is no change compared to the parent, which is
exactly what 'clean' means. Avoid testing the same state twice.
In the dropped example, the extension would look for 'vdiff.diffargs' in the
configuration, and not finding it, would run kdiff3 without the configured
options. That's not obvious to a new user who sees a kdiff3 configuration in
the prepackaged mergetools.rc file, and sees that kdiff3 still runs. While it
is conceivable that the user wants a kdiff3 command that runs without the
preconfigured options, it is more likely what they want is this, which uses the
canned options:
[alias]
vdiff = kdiff3
[extdiff]
kdiff3 =
We could mention alias here, but that seems like it belongs elswhere.
There are three ways to configure an extdiff tool:
1) cmd.tool = (/path/to/exe optional)
2) tool = (path/to/exe optional)
3) tool = sometool someargs
Previously, if no executable is specified in the first two forms, the named tool
must be in $PATH, or the invocation fails. Since the [merge-tools] section
already has the path to the diff executable, and/or the registry keys to find
the executable on Windows, reuse that configuration for forms 1 and 2 instead of
failing. We already fallback to [diff-tools] and then [merge-tools] for program
arguments if they aren't specified in the [extdiff] section.
Since this additional lookup only occurs if an executable is not on the $PATH
for the named tool, this is backwards compatible. For now, we assume the user
knows what he is doing if a path is provided.
This change allows a configuration file like this (assuming beyondcompare3 is
configured in merge-tools), instead of hardcoding system specific a path:
[extdiff]
beyondcompare3 =
Because the original dumbhttp.py exited without waiting for listen(), several
tests could fail with "abort: error: Connection refused" if subsequent hg
command is fast enough.
It seems like the last known misbehvior that the comment was referring
to was dealt with in 708ed580cbea (revert: properly back up added
files with local modification, 2014-08-31).
Further digging on this issue show that the limit on the sample size used in
discovery never works for heads. Here is a quote from the code itself:
desiredlen = size - len(always)
if desiredlen <= 0:
# This could be bad if there are very many heads, all unknown to the
# server. We're counting on long request support here.
The long request support never landed and evolution make the "very many heads,
all unknown to the server" case quite common.
We implement a simple and stupid hard limit of sample size for all query. This
should prevent HTTP 414 error with the current state of the code.
History rewriting commands like histedit tend to use temporary
commits. They may schedule hook execution on these temporary commits
for after the lock has been released. But temporary commits are likely
to have been stripped before the lock is released (and the hook run).
Hook executed for missing revisions leads to various crashes.
We disable hooks execution for revision missing in the repo. This
provides a dirty but simple fix to user issues.
On streaming clone, we were priming the local branch cache with the
remote branchmap, without checking which heads were closed.
This fixes an issue introduced in:
changeset: 17740:f8d7aaf86507
user: Tomasz Kleczek <tomasz.kleczek@fb.com>
date: Wed Oct 03 13:19:53 2012 -0700
summary: branchcache: fetch source branchcache during clone (issue3378)
that was exposed in 2.9 by:
changeset: 20192:6c385e85aa05
user: Brodie Rao <brodie@sf.io>
date: Mon Sep 16 01:08:29 2013 -0700
summary: branches: simplify with repo.branchmap().iterbranches()
8a92e6790099 broke bookmarks getting copied during uncompressed clones. Since
most of the pull logic has been moved into exchange.py, lets just call
exchange.pull to fix up the repo with the latest bits after the streaming clone
has bootstrapped the repo. This keeps us from having to duplicate the bookmark
logic.
This was fixed earlier by moving all the symlink bits to a section to
the end of the file, but then it was broken (by the same person) by
adding more tests at the end.
Before this patch, "test-obsolete.t" fails on Windows environment,
because strings corresponded to "tm_wday" (day of the week) field are
incorrect.
On POSIX environment, "gmtime()" returns correct "tm_wday" value even
for negative "time_t" value. On the other hand, it returns incorrect
one on Windows environment. At least, "gmtime()" of the Windows
runtime library bundled with Python 2.7.3 does.
According to f18f840c2b6e introducing original timestamp value '56
120', it shouldn't cause negative "time_t" value.
test-obsolete: remove subminute timezone in test
Obsmarker format "1" does not supports sub minute timezone. So we
change the test to something slightly more sensible.
It replaced "-d '56 12'" by "-d '56 120'".
Before this patch, a part of "test-alias.t" fails unexpectedly on
Windows environment, because "cmd.exe" can't evaluate "$HG_ARGS"
expression in shell alias correctly.
This patch uses "%HG_ARGS%" in shell alias on Windows instead of
"$HG_ARGS" to expand it correctly.
Before this patch, a part of "test-transplant.t" fails unexpectedly on
Windows environment, because semicolon (";") in HGEDITOR isn't
recognized as the command separator by "cmd.exe".
This patch newly introduces "checkeditform-n-cat.sh" script to invoke
"cat" in it safely anywhere.
Before this patch, "hg status --rev REV" doesn't list largefiles up
with "M" mark, even if exec bit of them is changed, because
"lfilesrepo.status" doesn't examine exec bit in such case.
Before this patch, "hg status --rev REV" listed largefiles removed in
the working directory up with "R" mark, even if they aren't managed in
the REV. Normal files aren't listed up in such case.
When "lfilesrepo.status" is invoked for "hg status --rev REV", it
treats files on conditions below as "removed" (to avoid manifest full
scan in "ctx.status" ?):
- marked as "R" in lfdirstate, or
- files managed in the target revision but unknown in the manifest
of the working context (= not including "R" files)
But the former can include files not managed in the target context.
To ignore removal status of files not managed in the target context,
this patch drops files unknown in the target revision from "removed"
list.
The set discovery start by sending a "known" command with all local heads. When
the number of local heads is massive (eg: using hidden changesets) such request
becomes too large. This lead to 414 error over http, aborting the whole
process.
We limit the size of the sample used by the first query to fix this.
The test are impacted because they do test massive number of heads. But they do
not test it over real world http setup.
48062b2d0f30 regressed the behavior of pushing an unchanged bookmark to
a remote. Before that commit, pushing a unchanged bookmark would result
in "exporting bookmark @" being printed. After that commit, we now see
an incorrect message "bookmark %s does not exist on the local or remote
repository!"
This patch fixes the regression introduced by 48062b2d0f30 by having
the bookmark error reporting code filter identical bookmarks and adds
a test for the behavior.
The comment in workingctx.status() says that "calling 'super' subtly
reveresed the contexts", but that is simply not true, so we should not
be swapping added and removed fields.
This addresses the bug described in issue4405: when obsolescence markers are
enabled, amending a commit with a file move can lead to the copy information
being lost.
However, the bug is more general and can be reproduced without obsmarkers as
well, as demonstracted by Pierre-Yves and put into the updated test.
Specifically, graph topology divergences between the filelogs and the changelog
can cause copy information to be lost during amends.
If an exception is raised during a bundle2 part payload generation it is now
recorded in the bundle. If such exception occurs, we capture it, transmit an
abort exception through the bundle, cleanly close the current part payload and
raise it again. This allow to generate valid bundle even in case of exception so
that the consumer does not wait forever for a dead producer. This also allow to
raise the exception during unbundling at the exact point it happened during
bundling make debugging easier.
The transaction backupfiles logic was broken for 'hg recover'. The file format
is XXX\0XXX\0YYY\0YYY\0 but the parser did a couple things wrong. 1) It went one
step beyond the final \0 and tried to read past the end of the array. 2)
array[i:i+1] returns a single item, instead of two items as intended.
Added a test to catch it, which turns out to be the first actual 'hg recover'
test.
The recent optimization of "and" operation relies on the assumption that
the rhs set does not contain invalid revisions. So rev() has to remove
invalid revisions.
This is still faster than using `.filter(lambda r: r == l)`.
revset #0: rev(25)
0) wall 0.026341 comb 0.020000 user 0.020000 sys 0.000000 (best of 113)
1) wall 0.000038 comb 0.000000 user 0.000000 sys 0.000000 (best of 66567)
2) wall 0.000062 comb 0.000000 user 0.000000 sys 0.000000 (best of 43699)
(0: 428fa22fb2d1^, 1: 3.2-rc, 2: this patch)
Before this patch, "test-context.py" fails on Windows environment,
because "diff" output contains unexpected "\r" character.
Opening the target file in text mode causes this automatic end-of-line
conversion.
Changeset d735f8a82023 introduced "showing prompt choice if input is
not a tty but is forced to be interactive" and adjusted expected
output in test-record.t.
But some tests for no-execbit platform (= Windows) were not adjusted
by d735f8a82023.
This causes unexpected failure of test-record on Windows environment.
This patch adds below to prompt lines for such tests:
- prompt choice at the end of prompt line, and
- empty line after prompt line
The problem in commandserver was addressed by 766cfbe766dc, but it is tricky
to reuse ui.nontty option to disable echo back. Instead, this patch introduces
new option to enable echoing of prompt response.
Prompt echoing is changed to be off by default, which should avoid possible
breakage of output parsing in user scripts.
For now, this option is undocumented because it exists for internal use.
Hooks that run after the transaction need to be able to touch the
repository. So we need to run them after the lock release. This is
similar to what the "changegroup" hook is doing in the
`addchangegroup` function.
Hidden changesets are by far the most common error case and is the only one[1]
that can reach the user. We move to a friendlier message with a hint about how
to access the data anyway. We should probably point to a help topic instead but
we do not have such a topic yet.
Example of the new output
abort: hidden revision '4'!
(use --hidden to access hidden revisions)
[1] Actually, filtering from "served" can also reach the user during certain
exchange operations.
This will help user to debug. A more precise message will be issued
for the most common case ("visible" filter) in the next changesets.
example output:
- abort: filtered revision '4'!
+ abort: filtered revision '4' (not in 'visible' subset)!
The hack for using certificate store in addition to the provided CAs resides in
Apple's OpenSSL. Apple's own Pythons will use it, but other custom built
Pythons might use a custom built OpenSSL without that hack and will fail when
exposed to the dummy cacert introduced in ee8b7fe5e119.
There do not seem to be a simple way to check from Python if we are using a
patched OpenSSL or if it is an Apple OpenSSL.
Instead, check if the Python executable resides in /usr/bin/python* or in
/System/Library/Frameworks/Python.framework/ and assume that all Pythons found
there will be native Pythons using the patched OpenSSL.
Custom built Pythons will not get the benefit of using the CAs from the
certificate store.
The 'status --rev' code is not very well tested, which has bitten us
as recently as in issue4321. Let's add some more tests, some of which
uncover bugs. Remove the few existing tests that are now covered in a
more thorough and consistent way.
Because unix-mode server forks child process per connection, client does not
know the pid of the server that will handle requests. The pid is necessary
to interrupt hung process:
1. client connects to socket server
2. server accepts the connection, forks, and tells pid
3. client requests "runcommand pull"
.. hung ..
4. client sends SIGINT to the (forked) server
5. server returns from I/O wait
Note that getsockopt(SO_PEERCRED) of Linux cannot be used because the server
fork()s after accept().
Consider a hypothetical bug in the release function that causes it to raise an
exception. Also consider the bisect command, which saves its state in a finally
clause. Saving the state requires acquiring the wlock.
If we don't unlink the lockfile when the exception is thrown, we'll try to
acquire the wlock again. We're going to try and acquire a lock again while our
old lockfile is on disk. The PID on disk is our own, and of course we're still
running, so we won't take over the lock. Hence we'll be stuck waiting for a
lock that we left behind ourselves.
To avoid this, always unlink the lockfile. This preserves the invariant that
self.held > 0 is equivalent to the lockfile existing on disk.
Bundle2 opens doors to advanced features allowing to reduce load on
mercurial servers, and improve clone experience for users on unstable or
slow networks.
For instance, it could be possible to pre-generate a bundle of a
repository, and give a pointer to it to clients cloning the repository,
followed by another changegroup with the remainder. For significantly
big repositories, this could come as several base bundles with e.g. 10k
changesets, which, combined with checkpoints (not part of this change),
would prevent users with flaky networks from starting over any time
their connection fails.
While the server-side support for those features doesn't exist yet, it
is preferable to have client-side support for this early-on, allowing
experiments on servers only requiring a vanilla client with bundle2
enabled.
The '-y' in 'hg update -y' was once needed to answer questions about
modify/delete conflicts. That is no longer needed, so remove the '-y'
and the comment justifying its use.
We already have a test for 'hg resolve -m' when there is no merge in
progress. Add one for 'hg resolve --all' as well.
Also add tests for both --all and -m when there is a merge without
conflicts in progress. They should both be successful, just as if
there had been conflicts that had been marked resolved. However, that
is currently broken, so mark the tests broken for now. The behavior
will be fixed in a later patch.
We capture FilteredxxxError and issue a FilteredRepoLookupError instead with a
sightly different messsge. The message will likely get more improvement in the
future.
error: filtered revision '4'
We are changing all integers that denote the size of a chunk to read to int32.
There are two main motivations for that.
First, we change everything to the same width (32 bits) to make it possible for
a reasonably agnostic actor to forward a bundle2 without any extra processing.
With this change, this could be achieved by just reading int32s and forwarding
chunks of the size read. A bit a smartness would be logic to detect the end of
stream but nothing too complicated.
Second, we need some capacity to transmit special information during the bundle
processing. For example we would like to be able to raise an exception while a
part is being read if this exception happend while this part was generated.
Having signed integer let us use negative numbers to trigger special events
during the parsing of the bundle.
The format is renamed for B2X to B2Y because this breaks binary
compatibility. The B2X format support is dropped. It was experimental to
allow this kind of things. All elements not directly related to the binary
format remain flagged "b2x" because they are still compatible.
We would like exceptions raised during the generation process to be gracefully
handled on the receiver side. We add a test for it. It shows that we are not
doing it yet.
We need a wider set of hooks to process all the changes that happened during the
pull transaction. We reuse the experimental `b2x-transactionclose` hook set
from server's unbundle for consistency. This hook is experimental and will not
remains as-is forever, but this will open the door for experimentation in 3.2.
Typical use case of 'unix' mode is a background hg daemon.
$ hg serve --cmdserver unix --cwd / -a /tmp/hg-`id -u`.sock
Unlike 'pipe' mode in which parent process keeps stdio channel, 'unix' server
can be detached. So clients can freely connect and disconnect from server,
saving Python start-up time.
It might be better to write "--cmdserver socket -a unix:/sockpath" instead
of "--cmdserver unix -a /sockpath" in case hgweb gets the ability to listen
on unix domain socket.
The source information can, should be applied once when opening the transaction
for the pull. This will lets element processed within a bundle2 be aware of them
and open the door to running a set of hooks when closing this pull transaction.
This is similar to what is done in server's unbundle call.
We store the source and url of the current data into `transaction.hookargs` this
let us inherit it from upper layers that may have created a much wider
transaction. We have to modify bundle2 at the same time to register the source
and url in the transaction. We have to do it in the same patch otherwise, the
`addchangegroup` call would fill these values and the hook calling will crash
because of the duplicated 'source' and 'url' arguments passed to the hook call.
The transaction is now carrying hook-related informations. So we use it to
retrieve the `node` argument. This will also carry around all kinds of other useful
informations (like: "are we in a bundle2 processing")
A bundle2 may contain multiple parts adding changegroups, in which case there
are multiple operation records for changegroups, each with its own return
value. Those multiple return values are aggregated in a single cgresult value
for the whole operation.
As can be seen in the associated test case, the situation with hooks is not
really the best, but without deeper thoughts and changes, we can't do much
better. Hopefully, things will be improved before bundle2 is enabled by default.
In the meanwhile, multiple changegroups is not expected to be in widespread
use, and even less expected to be used for pushes. Also, not many clients
cloning or pulling bundle2 with multiple changesets are not expected to have
changegroup hooks anyways.
addchangegroup creates a runhook function that is used to invoke the
changegroup and incoming hooks, but at the time the function is called,
the contents of hookargs associated with the transaction may have been
modified externally. For instance, bundle2 code affects it with
obsolescence markers and bookmarks info.
It also creates problems when a single transaction is used with multiple
changegroups added (as per an upcoming change), whereby the contents
of hookargs are that of after adding a latter changegroup when invoking
the hook for the first changegroup.
There are currently two different tests using roughly the same code to
create temporary scripts acting as HTTP servers. As there is going to
be at least one more in an upcoming change, factor those out in a
standalone dumbhttp.py script.
This test actually used the obs.py file as part of the test, so we need to fix
up the test a little more than usual to work with the new obsolete option flags.
The obsolete._enabled flag has become a config option. This updates all but one
of the tests to use the minimal number of flags necessary for them to pass. For
most tests this is just 'createmarkers', for a couple tests it's
'allowunstable', and for even fewer it's 'exchange'.
Previously, obstore read the obsolete._enabled flag to determine whether to
allow writes to the obstore. Since obsolete._enabled will be moving into a repo
specific config, we can't read it globally, and therefore must pass the
information into the constructor.
