We were manually creating a base with explicit subset testing. We should let
smartset magic happen and optimise that logic if needed.
benchmark show some massive speedup when "parents set" is huge and "subset" is
small.
revset: 42:68 and roots(42:tip)
0) wall 0.011322 comb 0.010000 user 0.010000 sys 0.000000 (best of 161)
1) wall 0.002282 comb 0.010000 user 0.010000 sys 0.000000 (best of 1082)
Minor speedup in simple case (were fullreposet helps)
revset: roots(0::tip)
0) wall 0.095688 comb 0.100000 user 0.100000 sys 0.000000 (best of 85)
1) wall 0.084448 comb 0.080000 user 0.080000 sys 0.000000 (best of 95)
revset: roots((0:tip)::)
0) wall 0.146752 comb 0.140000 user 0.140000 sys 0.000000 (best of 58)
1) wall 0.143538 comb 0.140000 user 0.140000 sys 0.000000 (best of 59)
And small overhead then the "parents set" is fairly complicated (transforming it
into a revset once and for all appears to be faster).
revset: roots((tip~100::) - (tip~100::tip))
0) wall 0.004652 comb 0.010000 user 0.010000 sys 0.000000 (best of 544)
1) wall 0.004878 comb 0.010000 user 0.010000 sys 0.000000 (best of 479)
revset: roots((0::) - (0::tip))
0) wall 0.146587 comb 0.150000 user 0.150000 sys 0.000000 (best of 53)
1) wall 0.157192 comb 0.160000 user 0.160000 sys 0.000000 (best of 53)
revset: first(roots((0::) - (0::tip)))
0) wall 0.152924 comb 0.150000 user 0.150000 sys 0.000000 (best of 57)
1) wall 0.153192 comb 0.160000 user 0.160000 sys 0.000000 (best of 55)
The existing roots(x - y) revset only considered the most recent 100
revisions. This was a good start. But expanding it to the full history
of the repository can dramatically increase execution time and thus
constitutes a useful benchmark.
The histedit command uses a revset like:
(_intlist('1234\x001235')) and merge()
Previously the optimizer gave a weight of 1.5 to the _intlist side (1 for the
function, 0.5 for the string) which caused it to process the merge() side first.
This caused it to evaluate merge against every commit in the repo, which took
2.5 seconds on a large repo.
I changed the weight of _intlist to 0, since it's a trivial calculation, which
makes it process intlist first, which makes merge apply only to the revs in the
list. Which makes the revset take 0.15 seconds now. Cutting off 2.4 seconds off
our histedit performance.
>From the revset benchmark:
revset #25: (_intlist('20000\x0020001')) and merge()
0) obsolete feature not enabled but 54243 markers found!
! wall 0.036767 comb 0.040000 user 0.040000 sys 0.000000 (best of 100)
1) obsolete feature not enabled but 54243 markers found!
! wall 0.000198 comb 0.000000 user 0.000000 sys 0.000000 (best of 9084)
Strip executes a revset like this:
max(parents(_intlist('1234\x001235')) - _intlist('1234\x001235'))
Previously the parents() revset would do 'subset & parents' which iterates over
each item in the subset and checks if it's in parents. subset is usually the
entire repo (a spanset) so this takes a while.
Reversing the parameters to be 'parents & subset' means the operation becomes
O(number of parents) instead of O(size of repo). It also means the result gets
evaluated immediately (since parents isn't a lazy set), but I think this is a
win in most scenarios.
This shaves 0.3 seconds off strip (amend/histedit/rebase/etc) for large repositories.
revset #0: parents(20000)
0) obsolete feature not enabled but 54243 markers found!
! wall 0.006256 comb 0.010000 user 0.010000 sys 0.000000 (best of 289)
1) obsolete feature not enabled but 54243 markers found!
! wall 0.000391 comb 0.000000 user 0.000000 sys 0.000000 (best of 4323)
Previously descendants() would force the provided subset to become a set. In
the case of revsets like '(%ld::) - (%ld)' (as used by histedit) this would
force the '- (%ld)' set to be evaluated, which produced a set containing every
commit in the repo (except %ld). This takes 0.6s on large repos.
This changes descendants to trust the subset to implement __contains__
efficiently, which improves the above revset to 0.16s. Shaving 0.4 seconds off
of histedit.
revset #27: (20000::) - (20000)
0) obsolete feature not enabled but 54243 markers found!
! wall 0.023640 comb 0.020000 user 0.020000 sys 0.000000 (best of 100)
1) obsolete feature not enabled but 54243 markers found!
! wall 0.019589 comb 0.020000 user 0.020000 sys 0.000000 (best of 100)
This commit removes the final revset related perf hotspot from histedit.
Combined with the previous two patches, they shave a little over 3 seconds off
histedit on large repos.
The added revset is used by obsolescence and currently results in
recursion in __contains__ between 2 lazysets. We should have
coverage of this revset.
f5a63a5506d2 regressed performance of baseset.__sub__ by introducing
a lazyset. This patch restores that lost performance by eagerly
evaluating baseset.__sub__ if the other set is a baseset.
revsetbenchmark.py results impacted by this change:
revset #6: roots(0::tip)
0) wall 2.923473 comb 2.920000 user 2.920000 sys 0.000000 (best of 4)
1) wall 0.077614 comb 0.080000 user 0.080000 sys 0.000000 (best of 100)
revset #23: roots((0:tip)::)
0) wall 2.875178 comb 2.880000 user 2.880000 sys 0.000000 (best of 4)
1) wall 0.154519 comb 0.150000 user 0.150000 sys 0.000000 (best of 61)
On the author's machine, this slowdown manifested during evaluation of
'roots(%ln::)' in phases.retractboundary after unbundling the Firefox
repository. Using `time hg unbundle firefox.hg` as a benchmark:
Before: 8:00
After: 4:28
Delta: -3:32
For reference, the subset and cs baseset instances impacted by this
change were of lengths 193634 and 193627, respectively.
Explicit test coverage of roots(%ln::), while similar to the existing
roots(0::tip) benchmark, has been added.
This revset is used in evolve. The new revset lazyness should make it all faster
but in practice it is significantly slower.
Below is a timing for this entry on my Mercurial repo.
2.9.2) ! wall 0.034598 comb 0.040000 user 0.040000 sys 0.000000 (best of 100)
3.0+@) ! wall 0.062268 comb 0.060000 user 0.060000 sys 0.000000 (best of 100)
The ~20 have been taken arbitrary.
This matters as `author(mpm)` have a lot of matches evenly split in the repo,
while `author(lmoscoviz)` have less match (and later). This changes the execution
path of the "or" operator a lot.
Calling a function is super expensive in python. We inline the trivial range
comparison to get back to more sensible performance on common revset operation.
Benchmark result below:
Revision mapping:
0) bced32a3fd6c 2.9.2 release
1) 2ab64f462d81 current @
2) This revision
revset #0: public()
0) wall 0.010890 comb 0.010000 user 0.010000 sys 0.000000 (best of 201)
1) wall 0.012109 comb 0.010000 user 0.010000 sys 0.000000 (best of 199)
2) wall 0.012211 comb 0.020000 user 0.020000 sys 0.000000 (best of 197)
revset #1: :10000 and public()
0) wall 0.007141 comb 0.010000 user 0.010000 sys 0.000000 (best of 361)
1) wall 0.014139 comb 0.010000 user 0.010000 sys 0.000000 (best of 186)
2) wall 0.008334 comb 0.010000 user 0.010000 sys 0.000000 (best of 308)
revset #2: draft()
0) wall 0.009610 comb 0.010000 user 0.010000 sys 0.000000 (best of 279)
1) wall 0.010942 comb 0.010000 user 0.010000 sys 0.000000 (best of 243)
2) wall 0.011036 comb 0.010000 user 0.010000 sys 0.000000 (best of 239)
revset #3: :10000 and draft()
0) wall 0.006852 comb 0.010000 user 0.010000 sys 0.000000 (best of 383)
1) wall 0.014641 comb 0.010000 user 0.010000 sys 0.000000 (best of 183)
2) wall 0.008314 comb 0.010000 user 0.010000 sys 0.000000 (best of 299)
We can see this changeset gains back the regression for `and` operation on
spanset. We are still a bit slowerfor the `public()` and `draft()`. Predicates
not touched by this changeset.
Previously revset._descendants would iterate over the entire subset (which is
often the entire repo) and test if each rev was in the descendants list. This is
really slow on large repos (3+ seconds).
Now we iterate over the descendants and test if they're in the subset.
This affects advancing and retracting the phase boundary (3.5 seconds down to
0.8 seconds, which is even faster than it was in 2.9). Also affects commands
that move the phase boundary (commit and rebase, presumably).
The new revsetbenchmark indicates an improvement from 0.2 to 0.12 seconds. So
future revset changes should be able to notice regressions.
I removed a bad test. It was recently added and tested '1:: and reverse(all())',
which has an amibiguous output direction. Previously it printed in reverse order,
because we iterated over the subset (the reverse part). Now it prints in normal
order because we iterate over the 1:: . Since the revset itself doesn't imply an
order, I removed the test.
All revsets added to benchmark so far are aimed to show an improvement of
performance from laziness. We had more wider version to track impact of laziness
on them.
