This patch narrows scope of the variable "m" in the function for
revset predicate "contains()", because it is referred only in "else"
code path of "if not matchmod.patkind(pat)" examination.
parse() cannot be called at the same time because a parser object keeps its
states. This is no problem for command-line hg client, but it would cause
strange errors in multi-threaded hgweb.
Creating parser object is not too expensive.
original:
% python -m timeit -s 'from mercurial import revset' 'revset.parse("0::tip")'
100000 loops, best of 3: 11.3 usec per loop
thread-safe:
% python -m timeit -s 'from mercurial import revset' 'revset.parse("0::tip")'
100000 loops, best of 3: 13.1 usec per loop
Some changesets can be wrongly reported as matched by this predicate
due to searching in a string joined with spaces and not individually.
A test case added, which fails without this fix.
Change ancestor to accept 0 or more arguments. The greatest common ancestor of a
single changeset is that changeset. If passed no arguments, the empty list is
returned.
Before this patch, sub expression may return unexpected result, if it
is joined with another expression by "or":
- "^"/parentspec():
"R or R^1" is not equal to "R^1 or R". the former returns only "R".
- "~"/ancestorspec():
"R or R~1" is not equal to "R~1 or R". the former returns only "R".
- ":"/rangeset():
"10 or (10 or 15):" is not equal to "(10 or 15): or 10". the
former returns only 10 and 15 or grater (11 to 14 are not
included).
In "or"-ed expression "A or B", the "subset" passed to evaluation of
"B" doesn't contain revisions gotten from evaluation of "A", for
efficiency.
In the other hand, "stringset()" fails to look corresponding revision
for specified string/symbol up, if "subset" doesn't contain that
revision.
So, predicates looking revisions up indirectly should evaluate sub
expressions of themselves not with passed "subset" but with "entire
revisions in the repository", to prevent "stringset()" from unexpected
failing to look symbols in them up.
But predicates in above example don't so. For example, in the case of
"R or R^1":
1. "R^1" is evaluated with "subset" containing revisions other than
"R", because "R" is already gotten by the former of "or"-ed
expressions
2. "parentspec()" evaluates "R" of "R^1" with such "subset"
3. "stringset()" fails to look "R" up, because "R" is not contained
in "subset"
4. so, evaluation of "R^1" returns no revision
This patch evaluates sub expressions for predicates above with "entire
revisions in the repository".
Now that changelog filtering is in place, it's become evident that
naming the filters according to the set of revs _not_ included in the
filtered changelog is confusing. This is especially evident in the
collaborative branch cache scheme.
This changes the names of the filters to reflect the revs that _are_
included:
hidden -> visible
unserved -> served
mutable -> immutable
impactable -> base
repoview.filteredrevs is renamed to filterrevs, so that callers read a
bit more sensibly, e.g.:
filterrevs('visible') # filter revs according to what's visible
This changesets add a new `divergent()` revset similar to `unstable()` and
`bumped()` one. Introducting this revset allows actuall test of the divergent
detection.
This replaces unnecessary parentrevs() calls with calculating min(parentset).
Even though the min operation is O(size of parentset), since parentrevs is
relatively expensive, this tradeoff almost always works in our favour. In a
repository with over 400,000 changesets, hg perfrevset "children(X)" takes:
Set X Before After
-1 0.51s 0.06s
-1000: 0.55s 0.08s
-10000: 0.56s 0.10s
-100000: 0.60s 0.25s
-100000:-99000 0.55s 0.19s
0:100000 0.60s 0.61s
all() 0.72s 0.74s
The relative performance is similar for Mercurial's own repository -- several
times faster in most cases, slightly slower for revisions close to 0 and
all().
When commiting to a repo with lots of history (>400000 changesets)
checking the results of revset.py:descendants against the subset takes
some time. Since the subset equals the entire changelog, the check
isn't necessary. Avoiding it in that case saves 0.1 seconds off of
a 1.78 second commit. A 6% gain.
We use the length of the subset to determine if it is the entire repo.
There is precedence for this in revset.py:stringset.
bundle() revset expression returns all changes that are present
in the bundle file (no matter whether they are in the repo or not).
Bundle file should be specified via -R option.
The old name was not very good for two reasons:
- caller does not care about "cache",
- set of revision returned may not be obsolete at all.
The new name was suggested by Kevin Bullock.
For changelog level filtering to take effect it need to be used for any
iteration. Some remaining use of `xrange` in revset code is replace by proper
use of `changelog.revs` or direct iteration over changelog.
We don't need to compute the set of all branchpoints. We can just check the
number of children that element of subset have. The extra work did not seems to
had particular performance impact but the code is simpler this way.
The branchpoint() function returns changesets with more than one child.
Eventually I would like to be able to see only branch points and merge
points in a graphical log to see the topology of the repository.
This changeset introduces caches on the `obsstore` that keeps track of sets of
revisions meaningful for obsolescence related logics. For now they are:
- obsolete: changesets used as precursors (and not public),
- extinct: obsolete changesets with osbolete descendants only,
- unstable: non obsolete changesets with obsolete ancestors.
The cache is accessed using the `getobscache(repo, '<set-name>')` function which
builds the cache on demand. The `clearobscaches(repo)` function takes care of
clearing the caches if any.
Caches are cleared when one of these events happens:
- a new marker is added,
- a new changeset is added,
- some changesets are made public,
- some public changesets are demoted to draft or secret.
Declaration of more sets is made easy because we will have to handle at least
two other "troubles" (latecomer and conflicting).
Caches are now used by revset and changectx. It is usually not much more
expensive to compute the whole set than to check the property of a few elements.
The performance boost is welcome in case we apply obsolescence logic on a lot of
revisions. This makes the feature usable!
"extinct" and "unstable" predicates use "obsolete" implementation
internally, but own predicate name should be used in error messages of
them instead of "obsolete".
This predicate is used to find csets that were created because of a graft,
transplant or rebase --keep. An optional revset can be supplied, in which case
the result will be limited to those copies which specified one of the revs as
the source for the command.
hg log -r destination() # csets copied from anywhere
hg log -r destination(branch(default)) # all csets copied from default
hg log -r origin(x) or destination(origin(x)) # all instances of x
This predicate will follow a cset through different types of copies. Given a
repo with a cset 'S' that is grafted to create G(S), which itself is
transplanted to become T(G(S)):
o-S
/
o-o-G(S)
\
o-T(G(S))
hg log -r destination( S ) # { G(S), T(G(S)) }
hg log -r destination( G(S) ) # { T(G(S)) }
The implementation differences between the three different copy commands (see
the origin() predicate) are not intentionally exposed, however if the
transplant was a graft instead:
hg log -r destination( G(S) ) # {}
because the 'extra' field in G(G(S)) is S, not G(S). The implementation cannot
correct this by following sources before G(S) and then select the csets that
reference those sources because the cset provided to the predicate would also
end up selected. If there were more than two copies, sources of the argument
would also get selected.
Note that the convert extension does not currently update the 'extra' map in its
destination csets, and therefore copies made prior to the convert will be
missing from the resulting set.
Instead of the loop over 'subset', the following almost works, but does not
select a transplant of a transplant. That is, 'destination(S)' will only
select T(S).
dests = set([r for r in subset if _getrevsource(repo, r) in args])
This predicate is used to find the original source of csets created by a graft,
transplant or rebase --keep. If a copied cset is itself copied, only the
source of the original copy is selected.
hg log -r origin() # all src csets, anywhere
hg log -r origin(branch(default)) # all srcs of copies on default
By following through different types of copy commands and only selecting the
original cset, the implementation differences between the copy commands are
hidden. (A graft of a graft preserves the original source in its 'extra' map,
while transplant and rebase use the immediate source specified for the
command).
Given a repo with a cset S that is grafted to create G(S), which itself is
grafted to become G(G(S))
o-S
/
o-o-G(S)
\
o-G(G(S))
hg log -r origin( G(S) ) # { S }
hg log -r origin( G(G(S)) ) # { S }, NOT { G(S) }
Even if the last graft were a transplant
hg log -r origin( T(G(S)) ) # { S }
A rebase without --keep essentially strips the source, so providing the cset
that results to this predicate will yield an empty set.
Note that the convert extension does not currently update the 'extra' map in
its destination csets, and therefore copies made prior to the convert will be
unable to find their source.
`extinct` changesets are obsolete changesets with obsolete descendants only. They
are of no interest anymore and can be:
- exclude from exchange
- hidden to the user in most situation
- safely garbage collected
This changeset just allows mercurial to detect them.
The implementation is a bit naive, as for unstable changesets. We better use a
simple revset query and a cache, but simple version comes first.
An unstable changeset is a changeset *not* obsolete but with some obsolete
ancestors.
The current logic to decide if a changeset is unstable is naive and very
inefficient. A better solution is to compute the set of unstable changeset with
a simple revset and to cache the result. But this require cache invalidation
logic. Simpler version goes first.
The new "diff" field lets you use the matching revset keyword to find revisions
that apply the same change as the selected revisions.
The match must be exact (i.e. same additions, same deletions, same modified
lines and same change context, same file renames and copies).
Two revisions matching their diff must also match their files. Thus, to match
the diff much faster we will always check that the 'files' match first, and only
then check that the 'diff' matches as well.
graft, transplant and rebase all embed a different type of source marker in
extra, and each with a different name. The current implementation of each is
such that there will never be more than one of these markers on a node.
Note that the rebase marker can only be resolved if the source is
still present, which excludes the typical rebase usage (without
--keep) from consideration (unless the resulting bundle in
strip-backup is overlayed). There probably isn't any reason to use
rebase --keep as a substitute for transplant or graft at this point,
but maybe there was at one point and there are even a few rebases in
the hg repo, so it may be of historical interest.
This selects changesets added because of repo conversions. For example
hg log -r "converted()" # all csets created by a convertion
hg log -r "converted(rev)" # the cset converted from rev in the src repo
The converted(rev) form is analogous to remote(id), where the remote repo is
the source of the conversion. This can be useful for cross referencing an old
repository into the current one.
The source revision may be the short changeset hash or the full hash from the
source repository. The local identifier isn't useful. An interesting
ramification of this is if a short revision is specified, it may cause more
than one changeset to be selected. (e.g. converted(6) matches changesets with
a convert_revision field of 6e..e and 67..0)
The convert.hg.saverev option must have been specified when converting the hg
source repository for this to work. The other sources automatically embed the
converted marker.
Caching has no performance effect on the revset aliases which triggered
the recent recursive evaluation bug. I wrote it not to feel bad about
expanding several times the same complicated expression.
If the string provided to the 'tag' predicate starts with 're:', the rest
of the string will be treated as a regular expression and matched against
all tags in the repository.
There is a slight backwards-compatibility problem for people who actually
have tags that start with 're:'. As a workaround, these tags can be matched
using a 'literal:' prefix.
If no tags match the pattern, an error is raised. This matches the behaviour
of the previous exact-match code.
There have been quite a few places where we pop elements off the
front of a list. This can turn O(n) algorithms into something more
like O(n**2). Python has provided a deque type that can do this
efficiently since at least 2.4.
As an example of the difference a deque can make, it improves
perfancestors performance on a Linux repo from 0.50 seconds to 0.36.
The alias expansion code it changed from:
1- Get replacement tree
2- Substitute arguments in the replacement tree
3- Expand the replacement tree again
into:
1- Get the replacement tree
2- Expand the replacement tree
3- Expand the arguments
4- Substitute the expanded arguments in the replacement tree
and fixes cases like:
[revsetalias]
level1($1, $2) = $1 or $2
level2($1, $2) = level1($2, $1)
$ hg log -r "level2(level1(1, 2), 3)"
where the original version incorrectly aborted on infinite expansion
error, because it was confusing the expanded aliases with their
arguments.
The current revset alias expansion code works like:
1- Get the replacement tree
2- Substitute the variables in the replacement tree
3- Expand the replacement tree
It makes it easy to substitute alias arguments because the placeholders
are always replaced before the updated replacement tree is expanded
again. Unfortunately, to fix other alias expansion issues, we need to
reorder the sequence and delay the argument substitution. To solve this,
a new "virtual" construct called _aliasarg() is introduced and injected
when parsing the aliases definitions. Only _aliasarg() will be
substituted in the argument expansion phase instead of all regular
matching string. We also check user inputs do not contain unexpected
_aliasarg() instances to avoid argument injections.
The original motivation was changectx.phase() had special logic to
correctly lookup in repo._phaserev, including invalidating it when
necessary. And at other places, repo._phaserev was accessed directly.
This led to the discovery that phases state including _phaseroots,
_phaserev and _dirtyphase was manipulated in localrepository.py,
phases.py, repair.py, etc. phasecache helps encapsulating that.
This patch replaces all phase state in localrepo with phasecache and
adjust related code except for advance/retractboundary() in phases.
These still access to phasecache internals directly. This will be
addressed in a followup.
current description of 'matching()' revset predicate can't be
formatted well on "hg help revset" output.
each descriptions for revset predicates (or something like them) are
split-ed into lines, and spaces on left side of them are stripped
before minirst processing. so, bullet list can't be nested.
this patch just flattens description of 'matching()' predicate to be
formatted well.
The fast path was triggered if the argument was not like "type:value", with
type a known pattern type. This is wrong for several reasons:
- path:value is valid for the fast path
- '*' is interpreted as a glob by default and is not valid for fast path
Fast path detection is now done after the pattern is parsed, and the normalized
path is extracted for direct comparison. All this seems a bit complicated, it
is tempting to drop the fast path completely. Also, the hasfile() revset does
something similar (only check .files()), without a fast path. If the fast path
is really that efficient maybe it should be used there too.
Note that:
$ log 'modifies("set:modified()")'
is different from:
$ log 'modifies("*")'
because of the usual merge ctx.files()/status(ctx.p1(), ctx) differences.
Reported by Steffen Eichenberg <steffen.eichenberg@msg-gillardon.de>
match
This patch sorts the fields that are passed to the matching function so that it
always starts by matching those fields that take less time to match.
Not all fields take the same amount of time to match. I've done several
measurements running the following command:
hg --time log -r "matching(1, field)"
on the mercurial repository, and where 'field' was each one of the fields
accepted by match. In order to avoid the print overhead (which could be
different for different fields, given the different number of matches) I used a
modified version of the matching() function which always returns no matches.
These tests showed that different fields take wildly different amounts of time
to match. Particulary the substate field takes up to 25 seconds to match on my
machine, compared to the 0.3 seconds that takes to match the phase field or the
2 seconds (approx) that takes to match most fields. With this patch, matching
both the phase and the substate of a revision takes the same amount of time as
matching the phase.
The field match order introduced by this patch is as follows:
phase, parents, user, date, branch, summary, files, description, substate
An extra nice thing about this patch is that it makes the match time stable.
Rather than getting all the fields that are being matches from every revision
and then comparing them to those of the target revision, compare each field one
by one and stop the match as soon as there is a match failure.
This can greatly reduce the match time when matching multiple fields.
The impact on match time when matching a single field seems negligible
(according to my measurements).
Apparently the "import x as xy" doesn't manage to update xy in the
current scope's dictionary after load, which causes nodemod.nullrev to do a huge amount of demandload magic in the inner loop.
This solves a similar problem than the previous --follow/--rev patch. This time
we need changelog.ancestors()/descendants() filtering on first parent.
Duplicating the code looked better than introducing keyword arguments. Besides,
the ancestors() version was already implemented in follow() revset.
This keyword can be used to find revisions that "match" one or more fields of a
given set of revisions.
A revision matches another if all the selected fields (description, author,
branch, date, files, phase, parents, substate, user, summary and/or metadata)
match the corresponding values of those fields on the source revision.
By default this keyword looks for revisions that whose metadata match
(description, author and date) making it ideal to look for duplicate revisions.
matching takes 2 arguments (the second being optional):
1.- rev: a revset represeting a _single_ revision (e.g. tip, ., p1(.), etc)
2.- [field(s) to match]: an optional string containing the field or fields
(separated by spaces) to match.
Valid fields are most regular context fields and some special fields:
* regular fields:
- description, author, branch, date, files, phase, parents,
substate, user.
Note that author and user are synonyms.
* special fields: summary, metadata.
- summary: matches the first line of the description.
- metatadata: It is equivalent to matching 'description user date'
(i.e. it matches the main metadata fields).
Examples:
1.- Look for revisions with the same metadata (author, description and date)
as the 11th revision:
hg log -r "matching(11)"
2.- Look for revisions with the same description as the 11th revision:
hg log -r "matching(11, description)"
3.- Look for revisions with the same 'summary' (i.e. same first line on their
description) as the 11th revision:
hg log -r "matching(11, summary)"
4.- Look for revisions with the same author as the current revision:
hg log -r "matching(., author)"
You could use 'user' rather than 'author' to get the same result.
5.- Look for revisions with the same description _AND_ author as the tip of the
repository:
hg log -r "matching(tip, 'author description')"
6.- Look for revisions touching the same files as the parent of the tip of the
repository
hg log -r "matching(p1(tip), files)"
7.- Look for revisions whose subrepos are on the same state as the tip of the
repository or its parent
hg log -r "matching(p1(tip):tip, substate)"
8.- Look for revisions whose author and subrepo states both match those of any
of the revisions on the stable branch:
hg log -r "matching(branch(stable), 'author substate')"
When _followfirst() revset was introduced it seemed to be the sole user of such
an argument, so filectx.ancestors() was duplicated and modified instead. It now
appears this argument could be used when computing the set of files to be
considered when --patch or --stat are passed along with --follow FILE.
This subtlety is not documented yet but:
- pats/--include/--exclude filesets are evaluated against the working directory
- --rev filesets are reevaluated against every revisions
log --graph --follow-first FILE cannot be compared with the regular version
because it never worked: --follow-first is not taken in account in
walkchangerevs() fast path and is explicitely bypassed in FILE case in
walkchangerevs() nested iterate() function.
The filtering logic of match objects cannot be reproduced with the existing
revsets as it operates at changeset files level. A changeset touching "a" and
"b" is matched by "-I a -X b" but not by "file(a) and not file(b)".
To solve this, a new internal "_matchfiles(...)" revset is introduced. It works
like "file(x)" but accepts more than one argument and its arguments are
prefixed with "p:", "i:" and "x:" to be used as patterns, include patterns or
exclude patterns respectively.
The _matchfiles revset is kept private for now:
- There are probably smarter ways to pass the arguments in a user-friendly way
- A "rev:" argument is likely appear at some point to emulate log command
behaviour with regard to filesets: they are evaluated for the parent revision
and applied everywhere instead of being reevaluated for each revision.
current documentation for 'remote()' predicate is wrong about
specification of parameters.
there are 3 patterns:
# of
param: id: remote:
- 0 current branch "defult" remote
- 1 specified "defult" remote
- 2 specified specified
The revset aliases expansion worked like:
expr = "some revset"
for alias in aliases:
expr = alias.process(expr)
where "process" was replacing the alias with its *unexpanded* substitution,
recursively. So it only worked when aliases were applied in proper dependency
order.
This patch rewrites the expansion process so all aliases are expanded
recursively at every tree level, after parent alias rewriting and variable
expansion.
Previously we always included '.', which may not touch a file.
Instead, find the file revision present in '.' and add its linkrev.
This matches the results of 'hg log --follow file'.
