It was introduced at deb42ca4dd93, where spanset.__contains__() did not exist.
Nowadays, we have to pay huge penalty for len(subset).
The following example showed that OR operation could be O(n * m^2)
(n: len(repo), m: number of OR operators, m >= 2) probably because of
filteredset.__len__.
revset #0: 0|1|2|3|4|5|6|7|8|9
0) wall 8.092713 comb 8.090000 user 8.090000 sys 0.000000 (best of 3)
1) wall 0.445354 comb 0.450000 user 0.430000 sys 0.020000 (best of 22)
2) wall 0.000389 comb 0.000000 user 0.000000 sys 0.000000 (best of 7347)
(0: 3.2.4, 1: 3.1.2, 2: this patch)
The matcher variable 'm' in checkstatus() is reset to None on each
call, so the caching of the matcher no longer happens as it was
intended. This seems to be a regression in 6b9fbae54476 (revset: added
lazyset implementation to checkstatus, 2014-01-03).
Fix by moving the cached matcher into the enclosing function so it's
actually cached across calls. This speeds up
hg log -r 'modifies(mercurial/context.py)' >/dev/null
from 7.5s to 4s.
Also see similar fix in 5ff5c5c9e69f (revset: avoid recalculating
filesets, 2014-10-22).
hg log -r 1 ... -r 100 was never returning due to a regression in the
way addset computes __nonzero__. It used 'bool(self._r1 or self._r2)'
which required executing self._r1.__nonzero__ twice (once for the or,
once for the bool). hg log with a lot of -r's happens to build a one
sided addset tree of N length, which ends up being 2^N performance.
This patch fixes it by converting to bool before or'ing.
This problem can be repro'd with something as simple as:
hg log `for x in $(seq 1 50) ; do echo "-r $x "; done`
Adding '1 + 2 + ... + 20' to the revsetbenchmark.txt didn't seem to repro the
problem, so I wasn't able to add a revset benchmark for this issue.
0cc5c10d5dc7 was not the final version of that patch. It was really slow
because `l not in repo.changelog` iterates revisions up to `l`. Instead,
rev() should utilize spanset.__contains__().
revset #0: rev(210000)
0) wall 0.000039 comb 0.000000 user 0.000000 sys 0.000000 (best of 67978)
1) wall 0.002721 comb 0.000000 user 0.000000 sys 0.000000 (best of 1055)
2) wall 0.000059 comb 0.000000 user 0.000000 sys 0.000000 (best of 45599)
(0: 3.2-rc, 1: 0cc5c10d5dc7, 2: this patch)
Note that the benchmark result described in 0cc5c10d5dc7 is wrong because
it is the one of the initial version.
The recent optimization of "and" operation relies on the assumption that
the rhs set does not contain invalid revisions. So rev() has to remove
invalid revisions.
This is still faster than using `.filter(lambda r: r == l)`.
revset #0: rev(25)
0) wall 0.026341 comb 0.020000 user 0.020000 sys 0.000000 (best of 113)
1) wall 0.000038 comb 0.000000 user 0.000000 sys 0.000000 (best of 66567)
2) wall 0.000062 comb 0.000000 user 0.000000 sys 0.000000 (best of 43699)
(0: 428fa22fb2d1^, 1: 3.2-rc, 2: this patch)
The phase retrieval is fast enough to not require caching the result of the
functions.
draft()
0) wall 0.017209 comb 0.020000 user 0.020000 sys 0.000000 (best of 149)
1) wall 0.011654 comb 0.010000 user 0.010000 sys 0.000000 (best of 186)
public()
0) wall 0.018687 comb 0.010000 user 0.010000 sys 0.000000 (best of 128)
1) wall 0.013290 comb 0.010000 user 0.010000 sys 0.000000 (best of 181)
secret()
0) wall 0.017464 comb 0.020000 user 0.020000 sys 0.000000 (best of 127)
1) wall 0.011499 comb 0.000000 user 0.000000 sys 0.000000 (best of 196)
draft() - ::bookmark()
0) wall 0.020099 comb 0.020000 user 0.020000 sys 0.000000 (best of 127)
1) wall 0.014399 comb 0.020000 user 0.020000 sys 0.000000 (best of 169)
Instead of checking all elements of the subset against a single rev, just check
if this rev is in the subset. The old way was inherited from when the subset was
a list.
Non surprise, this provide massive speedup.
id("b7dc31e4baa4")
before) wall 0.008205 comb 0.000000 user 0.000000 sys 0.000000 (best of 302)
after) wall 0.000069 comb 0.000000 user 0.000000 sys 0.000000 (best of 34518)
revset #1: public() and id("b7dc31e4baa4")
before) wall 0.019763 comb 0.020000 user 0.020000 sys 0.000000 (best of 124)
after) wall 0.000101 comb 0.000000 user 0.000000 sys 0.000000 (best of 20130)
The & version is more likely to be optimised.
only(.)
before) wall 0.003216 comb 0.000000 user 0.000000 sys 0.000000 (best of 768)
after) wall 0.001086 comb 0.000000 user 0.000000 sys 0.000000 (best of 2231)
only(default, stable)
before) wall 0.018469 comb 0.020000 user 0.020000 sys 0.000000 (best of 138)
after) wall 0.015888 comb 0.010000 user 0.010000 sys 0.000000 (best of 156)
Except when stated otherwise, the condition used in `smartset.filter` will be
cached. A new argument has been introduced to disable that behavior. We use it
for filters created from `and` and `sub` operations.
This gives massive performance boosts for revsets with expensive conditions.
revset: branch(stable) or branch(default)
before) wall 4.329070 comb 4.320000 user 4.310000 sys 0.010000 (best of 3)
after) wall 2.356451 comb 2.360000 user 2.330000 sys 0.030000 (best of 4)
revset: author(mpm) or author(lmoscovicz)
before) wall 4.434719 comb 4.440000 user 4.440000 sys 0.000000 (best of 3)
after) wall 2.321720 comb 2.320000 user 2.320000 sys 0.000000 (best of 4)
Lazy revset broke the ordering of the `or` revset. We now stop assuming that
two ascending revset are combine into an ascending one.
Behavior in 3.0:
3:4 or 2:5 == [2, 3, 4, 5]
Behavior in 2.9:
3:4 or 2:5 == [3, 4, 2, 5]
We are adding a test for it.
For unclear reason, the performance `or` revset with expensive filter are
getting even worse than they used to be. This is probably caused by extra
uncached containment check or iteration.
revset #9: author(lmoscovicz) or author(mpm)
before) wall 3.487583 comb 3.490000 user 3.490000 sys 0.000000 (best of 3)
after) wall 4.481486 comb 4.480000 user 4.470000 sys 0.010000 (best of 3)
revset #10: author(mpm) or author(lmoscovicz)
before) wall 3.164839 comb 3.170000 user 3.160000 sys 0.010000 (best of 3)
after) wall 4.574965 comb 4.570000 user 4.570000 sys 0.000000 (best of 3)
We use the & operator to combine with subset (since this is more likely to be
optimised than filter) and we enforce the sorting of the result. Without this
enforced sorting, we may result in a different iteration order than the set
_descendent was computed from.
This reverts a bad `test-glog.t` change from 7904906883bd.
Another side effect is that `test-mq.t` shows `qparent::` including `-1` if
`qparent is -1`. This sound like a positive change.
This has good and bad impacts on the benchmarks, here is a good ones:
revset: 0::
before) wall 0.045489 comb 0.040000 user 0.040000 sys 0.000000 (best of 100)
after) wall 0.034330 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
revset: roots((0::) - (0::tip))
before) wall 0.134090 comb 0.140000 user 0.140000 sys 0.000000 (best of 63)
after) wall 0.128346 comb 0.130000 user 0.130000 sys 0.000000 (best of 69)
revset: ::p1(p1(tip))::
before) wall 0.143892 comb 0.140000 user 0.140000 sys 0.000000 (best of 55)
after) wall 0.124502 comb 0.130000 user 0.130000 sys 0.000000 (best of 65)
revset: roots((0:tip)::)
before) wall 0.204966 comb 0.200000 user 0.200000 sys 0.000000 (best of 43)
after) wall 0.184455 comb 0.180000 user 0.180000 sys 0.000000 (best of 47)
Here is a bad one:
revset: (20000::) - (20000)
before) wall 0.009592 comb 0.010000 user 0.010000 sys 0.000000 (best of 222)
after) wall 0.029837 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
The previous implementation was consuming the whole revset when asked for any
sort. The addset class is now doing lazy sorting like all other smarset classes.
This has no significant impact in the benchmark as-is. But this is important
to later change.
