The only caveat here is that branches must be displayed first due to backwards
compatibility. The order of namespaces is defined to be the 'update' order
which, unfortunately, is not the same as log output order.
It's worth mentioning that the log output is still translated the same as
before since we are formating our strings the same way:
# i18n: column positioning for "hg log"
_("bookmark: %s\n") % bookmark
becomes
tname = _(("%s:" % ns.templatename).ljust(13) + "%s\n") % name
when name == 'bookmark'. The ljust(13) keeps the strings and whitespace equal.
Adding a new namespace is even easier now because the log output code doesn't
need to change. A future programmer would just need to add the string to the
corresponding .po file (which is the same as they would have had to do
previously).
b04f57726c73 changed basefilectx.annotate() to directly instantiate new
filectx's instead of going through self.filectx(), this breaks extensions that
replace the filectx class, and would also break future uses that would need
memfilectx's.
With this patch, we can make it much easier to specify 'only(A,B)' ->
A%B. Similarly, 'only(A)' -> A%.
On Windows, '%' is a semi-reserved symbol in the following way: using non-bash
shells (e.g. cmd.exe but NOT PowerShell, ConEmu, and cmder), %var% is only
expanded when 'var' exists and is surrounded by '%'.
That only leaves batch scripts which could prove to be problematic. I posit
that this isn't a big issue because any developer of batch scripts already
knows that to use '%' one needs to escape it by using a double '%%'.
Alternatives to '%' could be '=' but that might be limiting our future if we
ever decide to use temporary assignments in a revset.
Previous transaction work added callbacks to be called during regular
transaction commit/close. As part of refactoring Mozilla's pushlog
extension (an extension that opens a SQLite database and tries to tie
its transaction semantics to Mercurial's transaction), I discovered that
the new transaction APIs were insufficient to avoid monkeypatching
transaction instance internals. Adding a callback that is called during
transaction abort removes the necessity for monkeypatching and completes
the API.
Now that we have decided on the use of 'name' instead of 'label' we rename this
function accordingly.
The old method 'debuglabelcomplete' has been left as a deprecated command so
that current scripts don't break.
Iterating over all the namespaces is a common operation, naturally, so we add
those methods now. Since we are using a sorted dictionary, this method just
calls the underlying __iter__ or iteritems method.
Currently, we have no way to list all the names in a given namespace. This is
needed for things such as tab completion. Future patches will use this patch
for exactly that purpose.
This further simplifies the status code.
This simplification comes at a slight performance cost for `hg
export`. Before, on mozilla-central:
perfmanifest tip
! wall 0.265977 comb 0.260000 user 0.240000 sys 0.020000 (best of 38)
perftags
! result: 162
! wall 0.007172 comb 0.010000 user 0.000000 sys 0.010000 (best of 403)
perfstatus
! wall 0.422302 comb 0.420000 user 0.260000 sys 0.160000 (best of 24)
hgperf export tip
! wall 0.148706 comb 0.150000 user 0.150000 sys 0.000000 (best of 65)
after, same repo:
perfmanifest tip
! wall 0.267143 comb 0.270000 user 0.250000 sys 0.020000 (best of 37)
perftags
! result: 162
! wall 0.006943 comb 0.010000 user 0.000000 sys 0.010000 (best of 397)
perfstatus
! wall 0.411198 comb 0.410000 user 0.260000 sys 0.150000 (best of 24)
hgperf export tip
! wall 0.173229 comb 0.170000 user 0.170000 sys 0.000000 (best of 55)
The next set of patches introduces a new manifest type implemented
almost entirely in C, and more than makes up for the performance hit
incurred in this change.
We can do a little tiny bit better by enhancing manifest.diff to
optionally include files that are in both sides. This will be done in
a followup patch.
The 'dodiff' variable is initialized to True and may later be set to
either False or "binary". When it's set to False, we skip everything
after that point, so we can simplify by instead continue-ing (the
loop). We can then also drop the 'if dodiff', since it will always be
true.
The conditional-ness is not clear from the name and there is only one
caller, so it's clearer to check on the call site. Moving it also
makes addindexmeta() no longer close on the 'opts' variable.
commitctx already showed notes with filenames but didn't provide any context.
It is just as relevant to know when manifest or changelog is committed.
So, in addition to filenames, also show headlines 'committing files:',
'committing manifest' and 'committing changelog'.
This is kind of similar to the debugbundle command but gives summarized actual
uncompressed number of bytes when creating the bundle. The numbers are as
usable as the bundle format is efficient. Hopefully bundle2 will make it a
better indicator of actual entropy.
This is useful when accepting pull requests to assess whether the repo size
increase seems reasonable for the diff before pushing stuff upstream, It has
helped me catching large files that should have been committed as largefiles
but was committed as regular files in intermediate changesets.
This output doesn't combine well with debug output so we only enable it when
verbose without debug.
We are using full sampling of 'fullsamplesize' in both case. The only
difference is the debug message. So we factorise the sampling code and put the
message in an extra conditional.
This is going to help making changes around the sampling logic. Such changes are
needed to improve discovery performance on highly headed repository.
I want to supplement changelist entries (used by shortlog and changelog
endpoints) with custom metadata from an extension. i.e. I have extra
per-changeset metadata that I wish to make available to templates so it
can be rendered on hgweb.
To facilitate this, I've extracted the logic for creating a changeset
data structure into its own function, where it can be wrapped by
extensions.
Ideally, hgweb would use the same templater as the command line and have
full access to templatekw.keywords. But that's a lot of work. This patch
gets us some of the benefit without all the work.
Many other hgweb commands could benefit from similar refactorings. I'm
going to wait to see how this patch is received before I go crazy on
extracting inline functions.
Before this patch, there is no way to concatenate strings at runtime.
For example, to search for the issue ID "1234" in descriptions against
all of "issue 1234", "issue:1234", issue1234" and "bug(1234)"
patterns, the revset below should be written fully from scratch for
each issue ID.
grep(r"\bissue[ :]?1234\b|\bbug\(1234\)")
This patch introduces new infix operator "##" to concatenate
strings/symbols at runtime. Operator symbol "##" comes from the same
one of C pre-processor. This concatenation allows parametrizing a part
of strings in revset queries.
In the case of example above, the definition of the revset alias using
operator "##" below can search issue ID "1234" in complicated patterns
by "issue(1234)" simply:
issue($1) = grep(r"\bissue[ :]?" ## $1 ## r"\b|\bbug\(" ## $1 ## r"\)")
"##" operator does:
- concatenate not only strings but also symbols into the string
Exact distinction between strings and symbols seems not to be
convenience, because it is tiresome for users (and
"revset.getstring" treats both similarly)
For example of revset alias "issue()", "issue(1234)" is easier
than "issue('1234')".
- have higher priority than any other prefix, infix and postfix
operators (like as "##" of C pre-processor)
This patch (re-)assigns the priority 20 to "##", and 21 to "(",
because priority 19 is already assigned to "-" as prefix "negate".
This adds UI portion of the feature that has resided in mercurial since 2012.
Back then the interface was added together with the code, but was shortly
backed out because it was deemed "not ready". Code, however, stayed.
For the original feature and its implementation, see issue2810 and
3ff83729b63f.
In short, the backed-out interface had two outstanding issues:
1. it was introducing an entirely new term (baseline) and
2. it was present on every changeset's page, even for changesets with 1 parent
(or no parents), which didn't make sense
This patch implements a hopefully better interface because:
1. it uses the usual terms (diff) and
2. it only shows up when there actually are 2 parents.
Now that there is an object for each individual namespace, we use the
templatename property on that object instead of the method on the collection of
namespaces.
Previous patches changed the namespace api to be more of an object-oriented
approach. This patch updates the template function to use said api changes.
Since the namespaces object uses an underlying (sorted) dictionary to store the
namespaces, it makes sense to expose this to naturally gain access to those
namespaces.
Tracked files that are deleted should always be reported as such, no
matter what their state was in earlier revisions. This is encoded in
in two conditions in the loop in basectx._buildstatus() for modified
and added files, but the check is missing for clean files. We should
check for clean files too, but instead of adding the check in a third
place, move it earlier and skip most of the loop body for deleted
files.
When calculating status involving the working copy and a revision
other than the parent of the working copy, the files that are not in
the working context manifest ('mf2' in the basectx._buildstatus())
will be reported as removed (note that deleted files _are_ in the
working context manifest). However, if the file is reported as deleted
in the dirstate, it will get that status too (as shown by failing
tests).
Fix by removing deleted files from the 'removed' list after the main
loop in _buildstatus().
The fix for linkrev pointing to hidden revision was crashing when the file was
missing from head's manifest. We now properly handle this case.
(yes I feel silly)
The location variable fetch from the loop and the one used to actually fetch it
mismatched. We fix the name to ensure file outside of store are cleaned up.
Before this patch, a problematic revset alias aborts execution
immediately, even if it isn't referred in the specified revset.
If old "hg" may be used too (for example, bisecting Mercurial itself),
it is also difficult to write alias definitions using features newly
introduced by newer "hg" into configuration files, because such alias
definitions cause unexpected abortion at parsing revset aliases with
old "hg".
This patch delays showing parse error for the revset alias until it is
actually referred at runtime.
This patch detects referring problematic aliases in "_expandaliases"
by examination of "revsetalias.error", which is initialized with the
error message only when parsing fails.
For usability, this patch also warns about problematic aliases, even
if they aren't referred at runtime. This should help users to know
potential problems in their alias definitions earlier.
If the file revision with a filtered linkrev does not have any
(unfiltered) children, we cannot use it to bound the search for
another introduction. Instead, we have to look at the file revision
used by each head changeset. If one of them uses this file revision, we
know there is another occurrence and we have a starting point. See
inline comments for details.
Adding some kind of permanent reference of all the introductions of a
file revision instead of just the first one would be much better. But
this is more difficult. I hope to take that into account in the next
repository format.
This revset is used by 'hg log FILENAME'. This prevent bugs when used on
a repository with hidden revisions.
Instead of just discarding file revisions whose linkrevs point to filtered
revisions, we put them aside and post-process them trying to find a non-filtered
introduction. See inline documentation for details about how it works.
This only fixes some of the problems. Once again, more will be needed when we can
cannot rely on child revisions of a file to find linkrev-shadowned revisions.
A test is added for 'hg log' catching such cases.
This isn't as bad as the diff seems, it only looks like scary. In this patch,
we use the 'namespace' object instead of accessing keys in a dictionary.
This required the 'templatename' and 'names' method to change their
implementation. Later, we will remove these functions entirely due to a better
api.
Currently, we use a dictionary object to store the namespace properties. This
is python so let's use an object. This will allow us to be more flexible in our
method signatures in the future.
Before this patch, the result of "status()" on "workingcommitctx" may
incorrectly contain files other than ones to be committed, because
"workingctx._dirstatestatus()" returns the result of
"dirstate.status()" directly.
For correct matching, this patch overrides "_dirstatestatus" in
"workingcommitctx" and makes it return matched files only in
"self._status".
This patch uses empty list for "deleted", "unknown" and "ignored" of
status, because status between "changectx"s also makes them empty.
Before this patch, "status()" on "workingcommitctx" with "always
match" object causes breaking "self._status" in
"workingctx._buildstatus()", because "workingctx._buildstatus()"
caches the result of "dirstate.status()" into "self._status" for
efficiency, even though it should be fixed at construction for
committing.
For example, template function "diff()" without any patterns in
"committemplate" implies "status()" on "workingcommitctx" with "always
match" object, via "basectx.diff()" and "patch.diff()".
Then, broken "self._status" causes committing unexpected files.
To avoid breaking already fixed "self._status" at "ctx.status()", this
patch overrides "_buildstatus" in "workingcommitctx".
This patch doesn't write out the result of template function "diff()"
in "committemplate" in "test-commit.t", because matching against files
to be committed still has an issue fixed in subsequent patch.
Before this patch, "workingctx" is also used for the context to be
committed. But "workingctx" works incorrectly in some cases.
For example, even when only some of changed files in the working
directory are committed, "status()" on "workingctx" object for
committing recognizes files not to be committed as changed, too.
As the preparation for fixing these issues, this patch chooses adding
new class "workingcommitctx" for exact context to be committed,
because switching by the flag (like "self._fixedstatus" or so) in some
code paths of "workingctx" is less readable and maintenancable.
Before this patch, "workingctx.status" caches the result of
"dirstate.status" directly into "self._status".
But "dirstate.status" is invoked with False "list*" arguments in
normal "self._status" accessing route, and this makes
"unknown"/"ignored"/"clean" of status empty.
This may cause unexpected result of code paths internally accessing to
them (accessors for external usage are already removed by previous patch).
This patch makes "unknown"/"ignored"/"clean" of cached status empty
for equivalence. Making them empty is executed only when at least one
of "unknown", "ignored" or "clean" has files, for efficiency.
The json filter was previously iterating over keys in an object in an
undefined order. Let's throw a sorted() in there so output is
consistent.
It's somewhat frightening that there are no tests for the json filter.
Subsequent commits will add them, so we pass on the opportunity to add
them here.
The "changeset" template from hgweb is using a lambda in the
"diffsummary" key. In preparation for enabling JSON output from hgweb,
teach the json filter how to call functions.
The annotate logic now use the new 'introrev' method to bootstrap its traversal.
This catches issues from linkrev-shadowing of the changeset introducing the
version of a file in source changeset.
More tests have been added to display pathological cases.
The follow revset (used by `hg log --follow`) now uses the new 'introrev'
method to bootstrap its traversal. This catches issues from linkrev-shadowing of
the changesets introducing the version of a file in source changeset.
A new test has been added to display pathological cases.
Another test is affected because it was meant to test this behavior but actually
failed to do so for half of the output. The output are now similar.
The previous changeset properly fixed the ancestors computation, but we need to
ensure that the initial filectx is also using the right changeset.
When asking for log or annotation from a certain point, the first step is to
define the changeset that introduced the current file version. We cannot just
pick the "starting point" changesets as it may just "use" the file revision,
unchanged.
Currently, we were using 'linkrev' for this purpose, but this exposes us to
unexpected branch-jumping when the revision introducing the starting point
version is itself linkrev-shadowed. So we need to take the topology into
account again. Therefore, we introduce an 'introrev' function, returning the
changeset which introduced the file change in the current changeset.
This function will be used to fix linkrev-related issues when bootstrapping 'hg
log --follow' and 'hg annotate'.
It reuses the '_adjustlinkrev' function, extending it to allow introspection of
the initial changeset too. In the previous usage of the '_adjustlinkrev' the
starting rev was always using a children file revisions, so it could be safely
ignored in the search. In this case, the starting point is using the revision
of the file we are looking, and may be the changeset we are looking for.
Because of the way filenodes are computed, you can have multiple changesets
"introducing" the same file revision. For example, in the changeset graph
below, changeset 2 and 3 both change a file -to- and -from- the same content.
o 3: content = new
|
| o 2: content = new
|/
o 1: content = old
In such cases, the file revision is create once, when 2 is added, and just reused
for 3. So the file change in '3' (from "old" to "new)" has no linkrev pointing
to it). We'll call this situation "linkrev-shadowing". As the linkrev is used for
optimization purposes when walking a file history, the linkrev-shadowing
results in an unexpected jump to another branch during such a walk.. This leads to
multiple bugs with log, annotate and rename detection.
One element to fix such bugs is to ensure that walking the file history sticks on
the same topology as the changeset's history. For this purpose, we extend the
logic in 'basefilectx.parents' so that it always defines the proper changeset
to associate the parent file revision with. This "proper" changeset has to be an
ancestor of the changeset associated with the child file revision.
This logic is performed in the '_adjustlinkrev' function. This function is
given the starting changeset and all the information regarding the parent file
revision. If the linkrev for the file revision is an ancestor of the starting
changeset, the linkrev is valid and will be used. If it is not, we detected a
topological jump caused by linkrev shadowing, we are going to walk the
ancestors of the starting changeset until we find one setting the file to the
revision we are trying to create.
The performance impact appears acceptable:
- We are walking the changelog once for each filelog traversal (as there should
be no overlap between searches),
- changelog traversal itself is fairly cheap, compared to what is likely going
to be perform on the result on the filelog traversal,
- We only touch the manifest for ancestors touching the file, And such
changesets are likely to be the one introducing the file. (except in
pathological cases involving merge),
- We use manifest diff instead of full manifest unpacking to check manifest
content, so it does not involve applying multiple diffs in most case.
- linkrev shadowing is not the common case.
Tests for fixed issues in log, annotate and rename detection have been
added.
But this changeset does not solve all problems. It fixes -ancestry-
computation, but if the linkrev-shadowed changesets is the starting one, we'll
still get things wrong. We'll have to fix the bootstrapping of such operations
in a later changeset. Also, the usage of `hg log FILE` without --follow still
has issues with linkrev pointing to hidden changesets, because it relies on the
`filelog` revset which implement its own traversal logic that is still to be
fixed.
Thanks goes to:
- Matt Mackall: for nudging me in the right direction
- Julien Cristau and Rémi Cardona: for keep telling me linkrev bug were an
evolution show stopper for 3 years.
- Durham Goode: for finding a new linkrev issue every few weeks
- Mads Kiilerich: for that last rename bug who raise this topic over my
anoyance limit.
There are two caching routes for (propertycache-ed) "_status" below in
committablectx:
- invoking "status()":
"dirstate.status()" is invoked, and the result of it is cached
into "_status". In this case, any of "listignored", "listclean"
and "listunknown" may be True.
- accessing "_status" directly before "status()":
Own "status()" is invoked, but all of "listignored", "listclean"
and "listunknown" arguments are False, in this case.
"ignored"/"clean"/"unknown" accessor methods of "committablectx" use
corresponded fields of "_status", but these fields aren't reliable,
because these fields are empty when:
- "_status" method is executed before accessors, or
- "status()" is executed with "list*=False" before accessors
In addition to it, these accessors aren't used in the recent Mercurial
implementation. At least, removing them doesn't cause any test
failures.
Before this patch, "workingctx.status" always replaces "self._status"
by the recent result, even though:
- status isn't calculated against the parent of the working directory, or
- specified "match" isn't "always" one
(status is only visible partially)
If "workingctx" object is shared between some procedures indirectly
referring "ctx._status", this incorrect caching may cause unexpected
result: for example, "ctx._status" is used via "manifest()", "files()"
and so on.
To cache "self._status" correctly at "workingctx.status", this patch
overwrites "self._status" in "workingctx._buildstatus" only when:
- status is calculated against the parent of the working directory, and
- specified "match" is "always" one
This patch can be applied (and effective) only on default branch,
because procedure around "basectx.status" is much different between
stable and default: for example, overwriting "self._status" itself is
executed not in "workingctx._buildstatus" but in
"workingctx._poststatus", on stable branch.
we are going to need this filelog for the linkrev adjustment, so we better
normalise the list and have the filelog in all case.
This is done in a previous changeset to help readability.
This is slightly more controversial than diff, but we hope that HGPLAIN=1
covers all the format-breaking ones.
A possible alternative here that breaks BC is to honor all opts except the
whitespace ones.
We are going to introduce a linkrev-correction phases when computing parents.
It will be more convenient to have the nullid parent filtered out earlier. I
had to make a minimal adjustment to the rename handling logic to keep it
functional. That logic have been documented in the process since it took me
some time to check all the cases out.
The full path is propagated to the original match object since this is often
used directly for printing a file name to the user. This is cleaner than
requiring each caller to join the prefix with the file name prior to calling it,
and will lead to not having to pass the prefix around separately. It is also
consistent with the bad() and abs() methods in terms of the required input. The
uipath() method now inherits this path building property.
There is no visible change in path style for rel() because it ultimately calls
util.pathto(), which returns an os.sep based path. (The previous os.path.join()
was violating the documented usage of util.pathto(), that its third parameter be
'/' separated.) The doctest needed to be normalized to '/' separators to avoid
test differences on Windows, now that a full path is returned for a short
filename.
The test changes are to drop globs that are no longer necessary when printing an
absolute file in a subrepo, as returned from match.uipath(). Previously when
os.path.join() was used to add the prefix, the absolute path to a file in a
subrepo was printed with a mix of '/' and '\'. The absolute path for a file not
in a subrepo, as returned from match.uipath(), is still purely '/' based.
This is a utility to make it easier for subrepos to convert a file name to the
full path rooted at the top repository. It can replace the various path joining
lambdas, and doesn't require the prefix to be passed into the method that wishes
to build such a path.
The name is derived from the following pattern in annotate() and other places:
name = ((pats and rel) or abs)
The pathname separator is not os.sep in part to avoid confusion with variables
named 'abs' or similar that _are_ '/' separated, and in part because some
methods like cmdutils.forget() and maybe cmdutils.add() need to build a '/'
separated path to the root repo. This can replace the manual path building
there.
This patch makes "posix.shellquote" examine the specified string and
quote it only when it may have to be quoted for safety, like as the
previous patch for "windows.shellquote".
In fact, on POSIX environment, quoting itself doesn't cause issues
like issue4463. But (almost) equivalent quoting policy can avoid
examining test result differently on POSIX and Windows (even though
showing command line with "%r" causes such examination in
"test-extdiff.t").
The last hunk for "test-extdiff.t" in this patch isn't needed for the
previous patch for "windows.shellquote", because the code path of it
is executed only "#if execbit" (= avoided on Windows).
Before this patch, "windows.shellquote" (as used as "util.shellquote")
always quotes specified strings with double quotation marks, for
external process invocation.
But some problematic applications can't work correctly, when command
line arguments are quoted: see issue4463 for detail.
On the other hand, quoting itself is needed to specify arguments
containing whitespaces and/or some special characters exactly.
This patch makes "windows.shellquote" examine the specified string and
quote it only when it may have to be quoted for safety.
Previously, revert was only possible if the '--no-backup'
switch was specified.
Now, to support backups, we explicitly go over all modified
files in the subrepo.
This will make it possible to customize the behavior of the join method by
changing the vfs class (e.g. by using the altvfs" class introduced recently).
Note that we could have modified the VFS join methods to acept a set of optional
paths in the same way thta the localrepo join method does. However it seemed
simpler to simply call os.path.join before calling self.vfs.join.
Up until now we compared the "path" and "sharedpath" repository properties to
check if a repository is shared. This was relying an implementation detail of
shared repositories. In order to make it possible to change the way shared
repositories are implemented, we encapsulate this check into its own localrepo
method, called shared.
This new method returns None if the repository is shared, and something else
(for now a string describing the short of share) otherwise.
The reason why I did not call this method "isshared" and made it return a
boolean is that I plan to introduce a new type of shared repository soon.
# NOTES:
This is the first patch in a series whose purpose is to add support for
creating "full repository shares", which are repositories that share everything
with the repository source except their current revision, branch and bookmark.
This series is RFC because I am not very sure of some of the solutions I took.
Comments are welcome!
In the body of the loop in trydiff(), there are conditions like:
addedset or (f in modifiedset and to is None)
The second half of that expression is to account for the fact that
merge-in additions appear as additions. By instead fixing up the sets
of modified and added files to compensate for this fact, we can
simplify the body of the loop. It also fixes one case where the
addedset was checked without the additional check (the "have we
already reported a copy above?" case in the code, also see fixed test
case).
The similar condition with 'removedset' in it seems to have served no
purpose even before this change, so it could have been simplified even
before.
Note that there is a comment saying "ctx2 date may be dynamic". The
comment was introduced in 8ed3d2a60500 (patch: use contexts for diff,
2006-12-25), but it seems like it stopped being dynamic in that very
changeset -- before that changeset, the date seems to have been the
file's mtime, but after the changeset, it seems to be the changeset's
timestamp (current time for workingctx). Since no one seems to have
missed the "dynamicness", let's simplify and extract a date2 for
symmetry with date1.
This only has a noticeable effect on diffs touching A LOT of
files. For example, it takes
hg diff -r FIREFOX_AURORA_30_BASE -r FIREFOX_AURORA_35_BASE
from 1m55.465s to 1m32.354s. That diff has over 50k files.
This moves most reading of filelogs out of manifestmerge, making it
easy for a narrow clone extension to filter out or translate unwanted
actions before any filelogs are read. The only call left is inside of
copies.mergecopies(), which can be overridden separately at a lower
level.
We still have one case of a call to _checkunknownfile() in
manifestmerge(): when force=True and branchmerge=True and the remote
side has a file that the local side doesn't. This combination of
arguments is used by 'hg merge --force', but also by rebase and
unshelve. In this scenario, we try to create the file from the
contents from the remote, but if there is already a local untracked
file in place, we merge it instead.
When a directory was renamed and a new untracked file was added in the
new directory and the remote directory added a file by the same name
in the old directory, the local untracked file gets overwritten, as
demonstrated by the broken test case in test-rename-dir-merge.
Fix by checking for unknown files for 'dg' actions too. Since
_checkunknownfile() currently expects the same filename in both
contexts, we need to add a new parameter for the remote filename to
it.
The 'c' and 'dc' actions include creating a file on disk and we need
to check that no conflicting file exists unless force=True. Move two
of the calls to _checkunknownfile() to a single place at the end of
manifestmerge(). This removes some of the reading of filelogs from the
heart of manifestmerge() and collects it in one place close to where
its output (entries in the 'aborts' list) is used.
Note that this removes the unnecessary call to _checkunknownfile()
when force=True in one of the code paths.
_checkunknownfile() reads the filelog of the remote side's file. For
narrow clones, the filelog will not exist for all files and we need a
way to avoid reading them. While it would be easier for the narrow
extension to just override _checkunknownfile() and make it ignore
files outside the narrow clone, it seems cleaner to have
manifestmerge() not care about filelogs (considering its
name).
In order to move the calls to _checkunknownfile() out, we need to be
able to tell in which cases we should check for unknown files. Let's
start by introducing a new action distinct from 'g' for this
purpose. Specifically, the new action will be just like 'g' except
that it will check that for conflicting unknown files first. For now,
just add the new action type and convert it to 'g'.
The pushkey operation used to be in its own wireprotocol command and (in
practice) always be lock free when running the hook. With bundle2, it happen in
a greater scheme and a hook running locking command would get stuck. We now run
such hooks after the lock release as similar hook do.
Bundle2 test are altered to ensure we are lockfree at hook running time.
This is different from latesttagdistance in that while latesttagdistance is
defined to be the length of the longest path to the latest tag,
changessincelatesttag is the number of changes contained in @ that aren't
contained in the latest tag. So, if 't' is the latest tag in the repository
below:
t
|
v
--o--o----o
\ \
..o..o..@
then latesttagdistance is 2, but changessincelatesttag is 4.
Note that changessincelatesttag is always greater than or equal to the
latesttagdistance -- that's because changessincelatesttag counts all the
changes in the longest path since the latest tag, and possibly others. This is
an important fact that we'll take advantage of in upcoming patches.
This simplifies largefiles' overridecalculateupdates(), which no
longer has to do the conversion it started doing in 478d610ca1b0
(largefiles: rewrite merge code using dictionary with entry per file,
2014-12-09).
To keep this patch small, we'll leave the name 'actionbyfile' in
overrides.py. It will be renamed in the next patch.
By moving the conversion from the file->action dict after
_forgetremoved(), we make that method shorter by removing the need for
the confusing 'xactions' variable.
By moving the conversion from the file->action dict after the bid
merge code, bid merge can be simplified a little.
A few tests are affected by this change. Where we used to iterate over
the actions first in order of the action type ('g', 'r', etc.) [1], we
now iterate in order of filename. This difference affects the order of
debug log statements.
[1] And then in the non-deterministic order of files in the manifest
dictionary (the order returned from manifest.diff()).
In the same vein as 478d610ca1b0 (largefiles: rewrite merge code using
dictionary with entry per file, 2014-12-09), rewrite manifestmerge()
itself as dictionary with the filename as key. This will let us
simplify some of the other code in merge.py and eventually drop the
conversion in the largefiles code.
No difference in speed could be detected (well within the noise level
when run in Mozilla repo).
Monkey patching repoview does not really work and making it really work will
be really hard. So we better have it broken without complexity than broken with
extra complexity.
This patch makes bundrepo retract the phase boundary for new commits to 'draft'
status, which is consistent with the behavior of 'hg unbundle'. The old
behavior was for commits to appear with the same phase as their nearest
ancestor in the base repository.
This affects several classes of operation:
* Inspecting a bundle with the -B flag
* Treating a bundle file as a peer (old: everything public, new: everything draft)
* Incoming command (neither old or new behavior is sensible -- fixed in next patch)
This patch makes bundlerepo use a subclass of phasecache that will allow phase
boundaries to be moved around, but will never write them to the underlying
repository.
Previously these would be considered to be relative to the current working
directory. That behavior is both undocumented and doesn't really make sense.
There are two reasonable options for how to resolve relative paths:
- relative to the repo root
- relative to the config file
Resolving these files relative to the repo root matches existing behavior with
hooks. An earlier discussion about this is available at
http://mercurial.markmail.org/thread/tvu7yhzsiywgkjzl.
Thanks to Isaac Jurado <diptongo@gmail.com> for the initial patchset that
spurred the discussion.
workingctx.ancestors() was not returning the dirstate parents as part of the
result set. The only place this function is used is for copy detection when
committing a file, and that code already checks the parents manually, so this
change has no affect at the moment.
I found it while playing around with changing how copy detection works.
Mercurial backout command makes a commmit by default only when the backed out
revision is the parent of working directory and doesn't commit in any other
case.
The --commit option changes behaviour of backout to make a commit whenever
possible (i.e. there is no unresolved conflicts). This behaviour seems more
intuitive to many use (especially git users migrating to hg).
This patch adds the -B/--bookmarks option to the share command added by the
share extension. All it does for now is create a marker, 'bookmarks.shared',
that will be used by future code to implement the sharing functionality.
Now that we have the machinery of namespaces in-place, we use that instead of
hand-rolling our own template function.
Note, this can only be used for tags because both branches and bookmarks have
special case logic for 'default' and the current bookmark (which is something
outside the namespace api for now).
For any namespace, we generate a template keyword. For example, given a
namespace 'babar', we automatically have the ability to use it in a template:
hg log -r . -T '{babars % "King: {babar}\n"}'
Furthermore, we only generate this keyword for a namespace if one doesn't
already exist. This is necessary for 'branches' and 'bookmarks' since both of
those have concepts of 'current' (something outside the namespace api) and also
allows extensions to override default behavior if desired.
This marks our second feature of the namespace api: automatic template keyword.
This patch adds a method that takes in a namespace and uses the node-to-name
map to output the list of names.
This patch adds a node-to-name map property to the namespace. This is necessary
because we cannot simply invert the name-to-node map because we do not assume
the name-to-node map to be unique (for example, consider named branches: many
nodes have one branch name).
The node-to-name is helpful in log commands where we are already iterating over
a set of nodes and want to display some kind of naming information to the user.
This patch adds the public api for getting the template name of a namespace so
that the next patch can use it to generate a template keyword automatically.
The template name property will be used in upcoming patches to automatically
generate a template keyword. For example, given a namespace called 'babars', we
will automatically generate a template keyword 'babar' such that we can use it
in the following way,
$ hg log -r . -T '{babars % "King: {babar}\n"}'
Before this patch, "memctx._manifest" updates all entries in the
(parent) manifest. But this is inefficiency, because almost all files
may be clean in that context.
On the other hand, just updating entries for changed "files" specified
at construction causes unexpected abortion, when there is at least one
newly removed file (see issue4470 for detail).
To calculate manifest more efficiently, this patch replaces
"pman.iteritems()" for the loop by "self._status.modified" to avoid
updating entries for clean or removed files
Examination of removal is also omitted, because removed files aren't
treated in this loop (= "self[f]" returns not None always).
Before this patch, "memctx._manifest" tries to get (and use normally)
filectx also for newly-removed files, even though "memctx.filectx()"
returns None for such files.
To calculate manifest correctly even with newly-removed files, this
patch does:
- replace "man.iteritems()" for the loop by "self._status.modified"
to avoid accessing itself to newly removed files
this also reduces loop cost for large manifest.
- remove files in "self._status.removed" from the manifest
In this patch, amending is confirmed twice to examine both (1) newly
removed files and (2) ones already removed in amended revision.
Before this patch, "memctx._manifest" calculates the manifest
according to the 1st parent. This causes the disappearance
of newly added files from the manifest.
For example, if newly added files aren't listed up in manifest of
memctx, they aren't listed up in "added" field of "status" returned by
"ctx.status()", and "{diff()}" (= "patch.diff") in "committemplate"
shows nothing for them.
To calculate manifest including newly added files correctly, this
patch puts newly added files (= ones in "self._status.added") into the
manifest.
Some details of changes for "test-commit-amend.t" in this patch:
- "touch foo" is replaced by "echo foo > foo", because newly added
empty file can't be shown in "diff()" output without "diff.git"
configuration
- amending is confirmed twice to examine both (1) newly added files
and (2) ones already added in amended revision
Before this patch, "memctx._status" is initialized by "(files, [], [],
[], [], [], [])" and this causes "memctx.modified" to include not
only modified files but also added and removed ones incorrectly.
This patch adds "_status" method to calculate exact status being
committed according to "files" specified at construction time.
Exact "_status" is useful to share/reuse logic of committablectx.
This patch is also preparation for issues fixed by subsequent patches.
Some details of changes for tests in this patch:
- some filename lines are omitted in "test-convert-svn-encoding.t",
because they are correctly listed up as "removed" files
those lines are written out in "localrepository.commitctx" for
"modified" and "added" files by "ui.note".
- "| fixbundle" filterring in "test-histedit-fold.t" is omitted to
check lines including "added" correctly
"fixbundle" discards all lines including "added".
Encoding whether or not a part is mandatory in the capitalization of the
parttype is unintuitive and error-prone. This sequence of patches separates
these concerns in the API to reduce programmer error and pave the way for
a potential change in how this information is transmitted over the wire.
Since the parttype and mandatory bit are separated in bundle2.unbundlepart
(see previous patch), there is no longer a need to remove the mandatory bit
before working with the parttype.
Encoding whether or not a part is mandatory in the capitalization of the
parttype is unintuitive and error-prone. This sequence of patches separates
these concerns in the API to reduce programmer error and pave the way for
a potential change in how this information is transmitted over the wire.
This patch separates the two pieces of information when reading the part header
so that it's unnecessary to know how they were combined during transmission.
filectxfn returns None for removed files, so we have to check for None
before computing the new file content hash for the manifest.
Includes a test that proves this works, by demonstrating that we can
show the diff of an amended commit in the committemplate.
This method has the same behavior as the 'os.path.split' function, but having
it in vfs will allow handling of tricky encoding situations in the future.
In the same patch, we replace the use of 'os.path.split' in the transaction code.
The vfs.join method only works for absolute paths. We need something
that works for relative paths too when transforming filenames. Since
os.path.join may misbehave in tricky encoding situations, encapsulate
the new join method in our vfs abstraction. The default implementation
remains os.path.join, but this opens the door to other VFSes doing
something more intelligent based on their needs.
In the same go, we replace the usage of 'os.path.join' in transaction code.
This no longer needs to be explicitly passed because the subrepo object tracks
the 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in revert is to emit status and warning messages, and to
check the verbose flag prior to printing the action to be performed on a file.
The local repo's ui was already being used to print a warning message in
wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status()
call. Unlike other methods where the matcher is passed along and narrowed, a
new matcher is created in each repo, and therefore the bad() method already used
the local repo's ui.
This no longer needs to be explicitly passed because the subrepo object tracks
the 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in remove is to emit status and warning messages, and to
check the verbose flag prior to printing files to be removed. The bad() method
on the matcher still uses the root repo's ui, because narrowing the matcher
doesn't change the ui object.
The local repo's ui was already being used to print a warning message in
wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status()
call.
This no longer needs to be explicitly passed because the subrepo object tracks
the 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in forget is to emit status and warning messages, and to
check the verbose flag prior to printing files to be forgotten. The bad()
method on the matcher still uses the root repo's ui, because narrowing the
matcher doesn't change the ui object.
The local repo's ui was already being used to print a warning message in
wctx.forget() and for 'ui.slash' when walking dirstate in the repo.status()
call.
This no longer needs to be explicitly passed because the subrepo object tracks
a 'ui' reference since d4e8aa61370d. See the change to 'archive' for details
about the differences between the output level in the root repo and subrepo 'ui'
object.
The only use for 'ui' in cat is to emit a status message when a subrepo is
missing. The bad() method on the matcher still uses the root repo's ui, because
narrowing the matcher doesn't change the ui object.
The current state of subrepo methods is to pass a 'ui' object to some methods,
which has the effect of overriding the subrepo configuration since it is the
root repo's 'ui' that is passed along as deep as there are subrepos. Other
subrepo method are *not* passed the root 'ui', and instead delegate to their
repo object's 'ui'. Even in the former case where the root 'ui' is available,
some methods are inconsistent in their use of both the root 'ui' and the local
repo's 'ui'. (Consider hg._incoming() uses the root 'ui' for path expansion
and some status messages, but also calls bundlerepo.getremotechanges(), which
eventually calls discovery.findcommonincoming(), which calls
setdiscovery.findcommonheads(), which calls status() on the local repo 'ui'.)
This inconsistency with respect to the configured output level is probably
always hidden, because --verbose, --debug and --quiet, along with their 'ui.xxx'
equivalents in the global and user level hgrc files are propagated from the
parent repo to the subrepo via 'baseui'. The 'ui.xxx' settings in the parent
repo hgrc file are not propagated, but that seems like an unusual thing to set
on a per repo config file. Any 'ui.xxx' options changed by --config are also
not propagated, because they are set on repo.ui by dispatch.py, not repo.baseui.
The goal here is to cleanup the subrepo methods by dropping the 'ui' parameter,
which in turn prevents mixing subtly different 'ui' instances on a given subrepo
level. Some methods use more than just the output level settings in 'ui' (add
for example ends up calling scmutil.checkportabilityalert() with both the root
and local repo's 'ui' at different points). This series just goes for the low
hanging fruit and switches methods that only use the output level.
If we really care about not letting a subrepo config override the root repo's
output level, we can propagate the verbose, debug and quiet settings to the
subrepo in the same way 'ui.commitsubrepos' is in hgsubrepo.__init__.
Archive only uses the 'ui' object to call its progress() method, and gitsubrepo
calls status().
Creation of the subrepo's '_repo' object creates a new 'ui' by combining the
parent repo's 'baseui' and reading in the subrepo's hgrc file. This simply
avoids 'self.ui' and 'self._repo.ui' pointing to different objects, which seems
like a potential source of bugs.
Git and Svn subrepos are unchanged, because they don't have their own ui, and
have always used their parent's for their configuration.
The localrepository class has a 'ui' member, so keeping the names the same will
allow for duck typing with subrepo instances when accessing 'ui'. Changing this
is easier than finding all of the localrepository instance uses and renaming
that to '_ui'.
Instead of just bootstrapping the algorithm with the first revision we
see, allow callers to pass revs that should be displayed first. All
branches are retained until we can display such revision.
Expected usage is to display the current working copy parent first.
The algorithm now works when some revisions are skipped. We now use "first
included ancestors" instead of just "parent" to link changesets with each other.
We are going to add an additional layer of indentation to support non-contiguous
revset. We do it in a pure code movement changeset to help the readability of
the next changeset.
We add an experimental config option to use the topological sorting. I first
tried to hook the 'groupbranchiter' function in the 'sort' revset but this was useless
because graphlog enforces revision number sorting :(
As the goal is to advance on the topological iteration logic, I see this
experimental option as a good way to move forward.
We have to use turn the iterator into a list because the graphlog is apparently
not ready for pure iterator input yet.
This changeset introduces a function to perform topological (one branch after
the other) iteration over a set of changesets. This first version has a lot of
limitations, but the approach should be flexible enough to allow many
improvements in the future. This changeset aims to set the first stone more
than providing a complete solution.
The algorithm does not need to know the whole set of nodes involved
before emitting revision. This makes it a good candidate for usage in place
like `hg log` or graphical tools that need a fast first result time.
Note that the exception-catching from the previous branchtip check is moved up
to catch exceptions from the try block surrounding the namespace lookup.
It turns out that maintaining a reference of any sort (even weak!) to the repo
when constructed doesn't work because we may at some point pass in a repoview
filtered by something other than what the initial repo was.
This marks the first use of abstracting our different types of named objects
(bookmarks, tags, branches, etc.) and upcoming patches will use this to
simplify logic.
