Except when stated otherwise, the condition used in `smartset.filter` will be
cached. A new argument has been introduced to disable that behavior. We use it
for filters created from `and` and `sub` operations.
This gives massive performance boosts for revsets with expensive conditions.
revset: branch(stable) or branch(default)
before) wall 4.329070 comb 4.320000 user 4.310000 sys 0.010000 (best of 3)
after) wall 2.356451 comb 2.360000 user 2.330000 sys 0.030000 (best of 4)
revset: author(mpm) or author(lmoscovicz)
before) wall 4.434719 comb 4.440000 user 4.440000 sys 0.000000 (best of 3)
after) wall 2.321720 comb 2.320000 user 2.320000 sys 0.000000 (best of 4)
Lazy revset broke the ordering of the `or` revset. We now stop assuming that
two ascending revset are combine into an ascending one.
Behavior in 3.0:
3:4 or 2:5 == [2, 3, 4, 5]
Behavior in 2.9:
3:4 or 2:5 == [3, 4, 2, 5]
We are adding a test for it.
For unclear reason, the performance `or` revset with expensive filter are
getting even worse than they used to be. This is probably caused by extra
uncached containment check or iteration.
revset #9: author(lmoscovicz) or author(mpm)
before) wall 3.487583 comb 3.490000 user 3.490000 sys 0.000000 (best of 3)
after) wall 4.481486 comb 4.480000 user 4.470000 sys 0.010000 (best of 3)
revset #10: author(mpm) or author(lmoscovicz)
before) wall 3.164839 comb 3.170000 user 3.160000 sys 0.010000 (best of 3)
after) wall 4.574965 comb 4.570000 user 4.570000 sys 0.000000 (best of 3)
We use the & operator to combine with subset (since this is more likely to be
optimised than filter) and we enforce the sorting of the result. Without this
enforced sorting, we may result in a different iteration order than the set
_descendent was computed from.
This reverts a bad `test-glog.t` change from 7904906883bd.
Another side effect is that `test-mq.t` shows `qparent::` including `-1` if
`qparent is -1`. This sound like a positive change.
This has good and bad impacts on the benchmarks, here is a good ones:
revset: 0::
before) wall 0.045489 comb 0.040000 user 0.040000 sys 0.000000 (best of 100)
after) wall 0.034330 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
revset: roots((0::) - (0::tip))
before) wall 0.134090 comb 0.140000 user 0.140000 sys 0.000000 (best of 63)
after) wall 0.128346 comb 0.130000 user 0.130000 sys 0.000000 (best of 69)
revset: ::p1(p1(tip))::
before) wall 0.143892 comb 0.140000 user 0.140000 sys 0.000000 (best of 55)
after) wall 0.124502 comb 0.130000 user 0.130000 sys 0.000000 (best of 65)
revset: roots((0:tip)::)
before) wall 0.204966 comb 0.200000 user 0.200000 sys 0.000000 (best of 43)
after) wall 0.184455 comb 0.180000 user 0.180000 sys 0.000000 (best of 47)
Here is a bad one:
revset: (20000::) - (20000)
before) wall 0.009592 comb 0.010000 user 0.010000 sys 0.000000 (best of 222)
after) wall 0.029837 comb 0.030000 user 0.030000 sys 0.000000 (best of 100)
The previous implementation was consuming the whole revset when asked for any
sort. The addset class is now doing lazy sorting like all other smarset classes.
This has no significant impact in the benchmark as-is. But this is important
to later change.
This add method is enforcing non-laziness, disabling multiple optimisations.
Benchmarks do not spot any significant difference but real usecase may. This
will also be important for further improvements to addset later in this series.
This add method is enforcing non-laziness, disabling multiple optimisations.
Benchmarks do not spot any significant regression but real usecase may. This
even gives some speedup in some cases:
revset #15: min(0::)
before) wall 0.001247 comb 0.000000 user 0.000000 sys 0.000000 (best of 1814)
after) wall 0.000942 comb 0.000000 user 0.000000 sys 0.000000 (best of 2367)
This will also be important for further improvement to addset later in this series.
This add method is enforcing non-laziness, disabling multiple optimisations.
Benchmarks do not spot any significant differences but real usecase may. This
will also be important for further improvements to addset later in this series.
A baseset starts without an explicit order. But as soon as a sort is requested,
we simply register that the baseset has an order and use the ordered version of
the list to behave accordingly.
We will want to properly record the order at creation time in the future. This
would unlock more optimisation and avoid some sorting.
Baseset contains already-computed revisions. It is considered "cheap" to do
operations on an already-computed set. So we add attributes to hold version of
the list in ascending and descending order and use them for `fastasc` and
`fastdesc`. Having distinct lists is important to provide correct iteration in
all cases. Altering a python list will impact an iterator connected to it.
eg: not preserving order at iterator creation time
>>> l = [0, 1]
>>> i = iter(l)
>>> l.reverse()
>>> list(i)
[1, 0]
eg: corrupting in progress iteration
>>> l = [0, 1]
>>> i = iter(l)
>>> i.next()
0
>>> l.reverse()
>>> i.next()
0
The baseset is doing more and more smartset magic and using its list-like
property less and less. So we store the list of revisions in an explicit
attribute and stop inheriting.
This requires reimplementing some basic methods.
In multiple places in the code, we use `someset[0]` or `someset[-1]`. This
works only because the `someset` is usually a baseset. For the same reason we
introduce a `first` and `last` methods to be implemented for all smartset
classes.
A `baseset` has multiple cached results and will get even more in the future.
Making it an object "populated once" like the other smartsets makes it both safer
and simpler. The append method will be removed at some point.
A `baseset` has multiple cached results and will get even more in the future.
Making it an object "populated once" like the other smartsets makes it both safer
and simpler. The append method will be removed at some point.
The expected iteration order may be different than the fast iteration order (eg:
ancestors(42) is expected to be iterated upward but is fast/lazy to compute
downward.
So we explicitly track the iteration order and enforce it if the manual
iteration is requested.
Default expected iteration order of a generator set is ascending because I'm
not aware of any descending revset that need a generatorset. The first to find
such descending revset will have the pleasure to make this configurable.
The utility of this cache is debatable (no visible benchmark impact) and using
generatorset for such purpose makes the code complicated.
We drop it for now. Someone can reintroduce a smart version of it in the future
if it is detected to be relevant.
When all revisions are known, we shortcut most of the class logic to use list
iteration instead. The cost of the sort is expected to be non-significant. The
list creation and sorting could be done lazily in the future. We have to copy
the list to not break existing iterator created before we finished consuming the
generator.
We gain a parameter to inform that the generator is ascending or descending. If
the generator is ordered, it is also used for the `fastasc` or `fastdesc`
version.
The _ascgeneratorset and _descgeneratorset class will be removed soon.
Better revset performance are also achieved with less overlay. There is no good
reason for addset to not be a smartset. We can replace the `_orderedsetmixin`
inheritance since `abstractsmartset` has efficient min and max too.
We have two goals here. First, we would like to restore the former iteration
order we had in 2.9. Second, we want this logic to be reusable for `fastasc`
and `fastdesc` methods.
All smartsets try to be lazy. The purpose of this class is to apply a
filter on another set. So we rename the class (and all its occurences) to
`filteredset`.
Just a bit of extra code makes the lazyset aware of order. This renders
orderedlazyset useless.
At some point, the `subset` will become responsible for this ordering logic. But
we are not there yet because the various objects used as subsets are not good enough.
When the filtered subset has such methods, we can use them. It is implemented
as properties to be able to quickly return None if no corresponding fastasc exists
on the subset.
Instead of having the direction of iteration enforced through the ordering of
`start` and `end` attributes of spanset, we encode the iteration direction in
an explicit attribute and always store start < end. The logic for sort and
reverse has to be updated. The __iter__ is now based on the newly introduced
`fastasc` and `fastdesc` methods.
This will allow other code simplifications in the future.
These two methods are actually silly aliases for `sort()` and
`sort(reverse=True)`. So we get that aliasing at the abstractsmartset level. We
will slowly phase out all the custom implementations and eventually remove any
mentions of it from the code.
Changeset 1440ec8e33c0 switched the order of the operand of the "&" computation
to work around an issue from repo-wide spanset. The need for a workaround has been
alleviated by the introduction of `fullreposet`. So we restore it to normal.
The benchmark shows no significant changes as expected.
We also revert the bogus test change introduced by 1440ec8e33c0. The order is
actually important.
This class documents all methods required by a smartset. This makes it easier
for people to respect the API and ensure we fail loudly when something does
not. It will later also contain common default implementations for multiple
methods, making it easier to have smartset classes with minimal work.
We are about to add a base class for `baseset` with an abstract `__nonzero__`
method. So we need this method to be explicitly defined to avoid issues. The
built-in list object apparently does not have a `__nonzero__` and relies on
`__len__` for this purpose?
Using `x.__contains__(r)` instead of `r in x` does not matter for built-in type
(set) but have a positive impact for all other classes. This will let us drop
some usage of baseset.set() in future patches. This also probably improves some
performance.
Doing manual iteration is expensible. We rely on built in list iteration
whenever possible. The other case has to become a closure we cannot have a both
yield and return in the same function.
This takes advantage of the `fullreposet` smartness with a nice
speedup. It's a similar speedup to `p1()` when a merge is in progress
(the non merge case is already lightning fast anyway.)
This takes advantage of the `fullreposet` smartness and yields a nice
speedup.
revset #0: p1()
0) wall 0.003256 comb 0.010000 user 0.010000 sys 0.000000 (best of 527)
1) wall 0.000066 comb 0.000000 user 0.000000 sys 0.000000 (best of 23224)
This takes advantage of the `fullreposet` smartness and yields a nice
speedup.
revset #0: rev(25)
0) wall 0.005480 comb 0.000000 user 0.000000 sys 0.000000 (best of 305)
1) wall 0.000052 comb 0.000000 user 0.000000 sys 0.000000 (best of 21891)
This takes advantage of the `fullreposet` smartness.
revset #0: origin(tip)
0) wall 0.005353 comb 0.000000 user 0.000000 sys 0.000000 (best of 354)
1) wall 0.003080 comb 0.000000 user 0.000000 sys 0.000000 (best of 446)
This takes advantage of the `fullreposet` smartness.
revset #0: follow(COPYING)
0) wall 0.002446 comb 0.000000 user 0.000000 sys 0.000000 (best of 735)
1) wall 0.000331 comb 0.000000 user 0.000000 sys 0.000000 (best of 5672)
This takes advantage of the `fullreposet` smartness.
revset #0: file(COPYING)
0) wall 3.179066 comb 3.180000 user 3.140000 sys 0.040000 (best of 3)
1) wall 2.723699 comb 2.730000 user 2.690000 sys 0.040000 (best of 4)
This takes advantage of the `fullreposet` smartness.
revset #0: divergent()
0) wall 0.002047 comb 0.000000 user 0.000000 sys 0.000000 (best of 813)
1) wall 0.000052 comb 0.000000 user 0.000000 sys 0.000000 (best of 22757)
This takes advantage of the `fullreposet` smartness.
revset #0: bisect(range)
0) wall 0.014007 comb 0.010000 user 0.010000 sys 0.000000 (best of 115)
1) wall 0.005556 comb 0.010000 user 0.010000 sys 0.000000 (best of 235)
This takes advantage of the `fullreposet` smartness.
revset #0: tip~25
0) wall 0.004800 comb 0.010000 user 0.010000 sys 0.000000 (best of 259)
1) wall 0.002475 comb 0.000000 user 0.000000 sys 0.000000 (best of 717)
This should give us the same benefit as elsewhere. Result is simpler (and
"faster").
Outgoing is dominated by the discovery so no benchmark is provided.
Python lookups are slow, so do all lookup outside of the for loop.
This provide a small but still significant speedup:
revset #0: 0::
0) wall 0.063258 comb 0.060000 user 0.060000 sys 0.000000 (best of 100)
1) wall 0.057776 comb 0.050000 user 0.050000 sys 0.000000 (best of 100)
Attribute lookup is slow in python. So this version is going to be a bit
faster. This does not have a visible impact since the rest of the stack is much
slower but this shaves the yak a few extra nanometers.
Moreover the new version is more readable so it worth doing this change for code
quality purpose.
This optimisation was approved by a core python dev.
"And" operation with something that contains the whole repo should be super
cheap. Check method docstring for details.
This provide massive boost to simple revset that use `subset & xxx`
revset #0: p1(20000)
0) wall 0.002447 comb 0.010000 user 0.010000 sys 0.000000 (best of 767)
1) wall 0.000529 comb 0.000000 user 0.000000 sys 0.000000 (best of 3947)
revset #1: p2(10000)
0) wall 0.002464 comb 0.000000 user 0.000000 sys 0.000000 (best of 913)
1) wall 0.000530 comb 0.000000 user 0.000000 sys 0.000000 (best of 4226)
No other regression spotted.
More performance improvements are expected in the future as more
revset predicate are converted to use `subset & xxx`
The relaxed way `fullreposet` handles "&" operation may cause some trouble for
people comparing smartset from different filter levels. I'm not sure such people
exist and we can improve that aspect in later patches.
We rename the `spanset` class to `_spanset`. `spanset` is now a function that
builds either a `fullreposet` or a `_spanset` according to the argument passed.
At some point, we may force people to explicitly use the `fullreposet`
constructor, but the current approach makes it easier to ensure we use the new
class whenever possible and focus on the benefits of this class.
Every revset evaluation starts from `subset = spanset(repo)` and a lot of
revset predicates build a `spansetrepo` for their internal needs.
`spanset` is a generic class that can handle any situation. As a result a lot
of operation between spanset result in an `orderedlazyset`, a safe object but
suboptimal in may situation.
So we introduce a `fullreposet` class where some of the operation will be
overwritten to produce more interesting results.
The old code relied on the subset contents to get rid of invalid values. We would
like to be able to rely more on the computation in parents() so we filter out
the invalid value.
Both paths are doing similar thing in the end. We refactor the function so that
the `ps` set is commonly used at the end.
This will end excluding `nullrev` from this set in a future patch
The old code relied on the subset contents to get rid of invalid values. We would
like to be able to rely more on the computation in p1() and p2() so we filter out
the invalid value
The baseset class is based on a python list. This means that base.__contains__
was absolutely as crappy as list.__contains__. We now rely on __contains__ from
the underlying set.
This will avoid having to explicitly convert the baseset to a set (using
baseset.set()) whenever one want fast membership test.
Apparently there is already code that forgot to do such conversions since we
observe a massive speedup in some test.
revset #25: roots((0::) - (0::tip))
0) wall 2.079454 comb 2.080000 user 2.080000 sys 0.000000 (best of 5)
1) wall 0.132970 comb 0.130000 user 0.130000 sys 0.000000 (best of 65)
No regression is observed in benchmarks.
This change improve the issue4371 back to acceptable situation (but are still
slower than manual substraction)
We can simply use the `self.isascending` value instead of more complex if/else
clause. This get the code simpler.
Benchmarks show no performances harmed in the process.
Before this patches, empty spanset were seen as neither ascending nor
descending. This is mathematically wrong and create some edges case. We put
`isascending` and `isdescending` back on track so we can use them to simplify
some of the spanset code.
Benchmarks show no performances harmed in the process.
The histedit command uses a revset like:
(_intlist('1234\x001235')) and merge()
Previously the optimizer gave a weight of 1.5 to the _intlist side (1 for the
function, 0.5 for the string) which caused it to process the merge() side first.
This caused it to evaluate merge against every commit in the repo, which took
2.5 seconds on a large repo.
I changed the weight of _intlist to 0, since it's a trivial calculation, which
makes it process intlist first, which makes merge apply only to the revs in the
list. Which makes the revset take 0.15 seconds now. Cutting off 2.4 seconds off
our histedit performance.
>From the revset benchmark:
revset #25: (_intlist('20000\x0020001')) and merge()
0) obsolete feature not enabled but 54243 markers found!
! wall 0.036767 comb 0.040000 user 0.040000 sys 0.000000 (best of 100)
1) obsolete feature not enabled but 54243 markers found!
! wall 0.000198 comb 0.000000 user 0.000000 sys 0.000000 (best of 9084)
Strip executes a revset like this:
max(parents(_intlist('1234\x001235')) - _intlist('1234\x001235'))
Previously the parents() revset would do 'subset & parents' which iterates over
each item in the subset and checks if it's in parents. subset is usually the
entire repo (a spanset) so this takes a while.
Reversing the parameters to be 'parents & subset' means the operation becomes
O(number of parents) instead of O(size of repo). It also means the result gets
evaluated immediately (since parents isn't a lazy set), but I think this is a
win in most scenarios.
This shaves 0.3 seconds off strip (amend/histedit/rebase/etc) for large repositories.
revset #0: parents(20000)
0) obsolete feature not enabled but 54243 markers found!
! wall 0.006256 comb 0.010000 user 0.010000 sys 0.000000 (best of 289)
1) obsolete feature not enabled but 54243 markers found!
! wall 0.000391 comb 0.000000 user 0.000000 sys 0.000000 (best of 4323)
Previously descendants() would force the provided subset to become a set. In
the case of revsets like '(%ld::) - (%ld)' (as used by histedit) this would
force the '- (%ld)' set to be evaluated, which produced a set containing every
commit in the repo (except %ld). This takes 0.6s on large repos.
This changes descendants to trust the subset to implement __contains__
efficiently, which improves the above revset to 0.16s. Shaving 0.4 seconds off
of histedit.
revset #27: (20000::) - (20000)
0) obsolete feature not enabled but 54243 markers found!
! wall 0.023640 comb 0.020000 user 0.020000 sys 0.000000 (best of 100)
1) obsolete feature not enabled but 54243 markers found!
! wall 0.019589 comb 0.020000 user 0.020000 sys 0.000000 (best of 100)
This commit removes the final revset related perf hotspot from histedit.
Combined with the previous two patches, they shave a little over 3 seconds off
histedit on large repos.
f5a63a5506d2 regressed performance of baseset.__sub__ by introducing
a lazyset. This patch restores that lost performance by eagerly
evaluating baseset.__sub__ if the other set is a baseset.
revsetbenchmark.py results impacted by this change:
revset #6: roots(0::tip)
0) wall 2.923473 comb 2.920000 user 2.920000 sys 0.000000 (best of 4)
1) wall 0.077614 comb 0.080000 user 0.080000 sys 0.000000 (best of 100)
revset #23: roots((0:tip)::)
0) wall 2.875178 comb 2.880000 user 2.880000 sys 0.000000 (best of 4)
1) wall 0.154519 comb 0.150000 user 0.150000 sys 0.000000 (best of 61)
On the author's machine, this slowdown manifested during evaluation of
'roots(%ln::)' in phases.retractboundary after unbundling the Firefox
repository. Using `time hg unbundle firefox.hg` as a benchmark:
Before: 8:00
After: 4:28
Delta: -3:32
For reference, the subset and cs baseset instances impacted by this
change were of lengths 193634 and 193627, respectively.
Explicit test coverage of roots(%ln::), while similar to the existing
roots(0::tip) benchmark, has been added.
This previously died in _revdescendants() taking the min() of the first set to
only(), when it was empty. An empty second set already worked. Likewise,
descendants() already handled an empty set.
We use the python syntax for range comparison: `a < x < c`. This is shorter,
more readable and less error prone. This comparison escaped the cleanup make in
166d6dde9310
We get rid of lambda in a bunch of other place. This is equivalent and much
faster. (no new timing as this is the same change as three other changesets)
Spanset are massively used in revset. First because the initial subset itself is
a repo wide spanset. We speed up the __and__ operation by getting rid of a
gratuitous lambda call. A more long terms solution would be to:
1. speed up operation between spansets,
2. have a special smartset for `all` revisions.
In the mean time, this is a very simple fix that buyback some of the performance
regression.
Below is performance benchmark for trival `and` operation between two spansets.
(Run on an unspecified fairly large repository.)
revset tip:0
2.9.2) wall 0.282543 comb 0.280000 user 0.260000 sys 0.020000 (best of 35)
before) wall 0.819181 comb 0.820000 user 0.820000 sys 0.000000 (best of 12)
after) wall 0.645358 comb 0.650000 user 0.650000 sys 0.000000 (best of 16)
Proof of concept implementation of an `all` smartset brings this to 0.10 but it's
too invasive for stable.
Calling a function is super expensive in python. We inline the trivial range
comparison to get back to more sensible performance on common revset operation.
Benchmark result below:
Revision mapping:
0) bced32a3fd6c 2.9.2 release
1) 2ab64f462d81 current @
2) This revision
revset #0: public()
0) wall 0.010890 comb 0.010000 user 0.010000 sys 0.000000 (best of 201)
1) wall 0.012109 comb 0.010000 user 0.010000 sys 0.000000 (best of 199)
2) wall 0.012211 comb 0.020000 user 0.020000 sys 0.000000 (best of 197)
revset #1: :10000 and public()
0) wall 0.007141 comb 0.010000 user 0.010000 sys 0.000000 (best of 361)
1) wall 0.014139 comb 0.010000 user 0.010000 sys 0.000000 (best of 186)
2) wall 0.008334 comb 0.010000 user 0.010000 sys 0.000000 (best of 308)
revset #2: draft()
0) wall 0.009610 comb 0.010000 user 0.010000 sys 0.000000 (best of 279)
1) wall 0.010942 comb 0.010000 user 0.010000 sys 0.000000 (best of 243)
2) wall 0.011036 comb 0.010000 user 0.010000 sys 0.000000 (best of 239)
revset #3: :10000 and draft()
0) wall 0.006852 comb 0.010000 user 0.010000 sys 0.000000 (best of 383)
1) wall 0.014641 comb 0.010000 user 0.010000 sys 0.000000 (best of 183)
2) wall 0.008314 comb 0.010000 user 0.010000 sys 0.000000 (best of 299)
We can see this changeset gains back the regression for `and` operation on
spanset. We are still a bit slowerfor the `public()` and `draft()`. Predicates
not touched by this changeset.
The argument is `x` but the variable tested for filtering is `rev`. `rev`
happens to be a revset methods, ... never part of the filtered revs. This
method is now using `rev` for everything.
For a Mercurial new-comer, the distinction between `contains(x)`,
`file(x)`, and `filelog(x)` in the "revsets" help page may not be
obvious. This commit tries to make things more obvious (text based on
an explanation from Matt in an FB group thread).
Previously we would iterate over every item in the subset, checking if it was in
the provided args. This often meant iterating over every rev in the repo.
Now we iterate over the args provided, checking if they exist in the subset.
On a large repo this brings setting phase boundaries (which use this revset
roots(X:: - X::Y)) down from 0.8 seconds to 0.4 seconds.
The "roots((tip~100::) - (tip~100::tip))" revset in revsetbenchmarks shows it
going from 0.12s to 0.10s, so we should be able to catch regressions here in the
future.
This actually introduces a regression in 'roots(all())' (0.2s to 0.26s) since
we're now using spansets, which are slightly slower to do containment checks on.
I believe this trade off is worth it, since it makes the revset O(number of
args) instead of O(size of repo).
Previously revset._descendants would iterate over the entire subset (which is
often the entire repo) and test if each rev was in the descendants list. This is
really slow on large repos (3+ seconds).
Now we iterate over the descendants and test if they're in the subset.
This affects advancing and retracting the phase boundary (3.5 seconds down to
0.8 seconds, which is even faster than it was in 2.9). Also affects commands
that move the phase boundary (commit and rebase, presumably).
The new revsetbenchmark indicates an improvement from 0.2 to 0.12 seconds. So
future revset changes should be able to notice regressions.
I removed a bad test. It was recently added and tested '1:: and reverse(all())',
which has an amibiguous output direction. Previously it printed in reverse order,
because we iterated over the subset (the reverse part). Now it prints in normal
order because we iterate over the 1:: . Since the revset itself doesn't imply an
order, I removed the test.
min([]) raise a ValueError, we do the same thing in smartset.min() and
smartset.max() for the sake of consistency.
The min/amax test are greatly improved in the process to prevent this familly
of regression
Back in the time where repo.revs(...) returned a list, calling `len(...)` on the
result was quite common. We reinstall this on _addset.
There is absolutely no easy way to test this from the command line. The commands
using this in the evolve extension will eventually land into core.
If two things were iterating over a generatorset at the same time, they could
miss out on the things the other was generating, resulting in incomplete
results. This fixes it by making it possible for two things to iterate at once,
by always checking the _genlist at the beginning of each iteration.
I was only able to repro it with pending changes from my other commits, but they
aren't ready yet. So I'm unable to add a test for now.
_generatorset.__contains__ and __contains__ from child classes were
calling into __iter__ to look for values. Since all
previously-encountered values from the generator were cached and checked
in __contains__ before this iteration, __contains__ was effectively
performing iteration busy work which could lead to an explosion of
redundant work.
This patch changes __contains__ to be more intelligent. Instead of
looking at all values via __iter__, __contains__ will instead go
straight to "new" values from the underlying generator.
On a clone of the Firefox repository with around 200,000 changesets,
this patch decreases the execution time of the revset '::(200067)::'
from ~100s to ~4s on the author's machine. Rebase operations (which use
the aforementioned revset), speed up accordingly.
Formerly an expression like "2.4-rc::" was tokenized as 2.4|-|rc|::.
This allows dashes in symbols iff the whole symbol-like string can be
looked up. Otherwise, it's tokenized as a series of symbols and
operators.
No attempt is made to accept dashed symbols inside larger symbol-like
string, for instance foo-bar or bar-baz inside foo-bar-baz.
This classes have no particular order so they rely on python min() and max()
implementation. This methods will be implemented in every smartset class in
future patches. For other classes there are lazy implementations that can be
made for this methods.
This methods are intended to duck-type baseset, so we will still have _addset
as a private class but now we can return it without wrapping it into an
orderedlazyset or a lazyset.
These were the last methods to add for smartset compatibility.
This method is intended to duck-type baseset, so we will still have _addset as a
private class but we will be able to return it without wrapping it into an
orderedlazyset or a lazyset.
This method is intended to duck-type baseset, so we will still have _addset as a
private class but now will be able to return it without wrapping it into an
orderedlazyset or a lazyset.
This method is intended to duck-type baseset, so we will still have _addset as a
private class but we will be able to return it without wrapping it into an
orderedlazyset or a lazyset.
This methods are intended to duck-type baseset, so we will still have _addset
as a private class but will be able return it without wrapping it into an
orderedlazyset or a lazyset.
This method is intended to duck-type baseset, so we will still have _addset
as a private class but we will be able return it without wrapping it into an
orderedlazyset or a lazyset.
This methods state if the class is sorted in an ascending or descending order
We need this to implement methods based on order on smartset classes in order
to be able to create new objects with a given order.
We cannot just rely on a simple boolean since unordered set are neither
ascending nor descending.
We need this method to duck-type generatorset since this class is not going to
be used outside revset.py and we don't need to duck-type baseset.
This sort method will only do something when the addset is not already sorted
or is not sorted in the way we want it to be.
If the two collections are in ascending order, yield their values in an
ordered way by iterating both at the same time and picking the values to
yield.
When a spanset was being sorted it didn't take into account it's current
state (ascending or descending) and it reversed itself everytime the reverse
parameter was True.
This is not yet used but it will be as soon as the sort revset is changed to
directly use the structures sort method.
Previously the head() revset would iterate over every item in the subset and
check if it was a head. Since the subset is often the entire repo, this was
slow on large repos. Now we iterate over each item in the head list and check if
it's in the subset, which results in much less work.
hg log -r 'head()' on a large repo:
Before: 0.95s
After: 0.28s
Since this class is only going to be used inside revset.py (it does not duck
type baseset) it needs to duck type only a few more methods for the next
patches.
This class is not supposed to be used outside revset.py since it only
wraps content that is used by baseset typed classes.
It only gets created by revset operations or private methods.
This class is not supposed to be used outside revset.py since it only
wraps content that is used by baseset typed classes.
It only gets created by revset operations or private methods.
This class is not supposed to be used outside revset.py since it only
wraps content that is used by baseset typed classes.
It only gets created by revset operations or private methods.
This class are not supposed to be used outside revset.py since it only
wraps content that is used by baseset typed classes.
It only gets created by revset operations or private methods.
Method needed to propagate sort calls amongst lazy structures.
The generated list (stored in the object) is sorted.
If the generated list did not contain all elements from the generator, we
take care of that before sorting the list.
Performance Benchmarking:
$ time hg log -qr "0:: and 0:5"
...
real 0m3.665s
user 0m3.364s
sys 0m0.289s
$ time ./hg log -qr "0:: and 0:5"
...
real 0m0.492s
user 0m0.394s
sys 0m0.097s
This will not improve revsets like "::tip" but will do when that gets
intersected or substracted with another revset.
Performance Benchmarking:
$ time hg log -qr "draft() and ::tip"
...
real 0m3.961s
user 0m3.640s
sys 0m0.313s
$ time ./hg log -qr "draft() and ::tip"
...
real 0m1.080s
user 0m0.987s
sys 0m0.083s
Adds a only() revset that has two forms:
only(<set>) is equivalent to "::<set> - ::(heads() - heads(<set>::))"
only(<include>,<exclude>) is equivalent to "::<include> - ::<exclude>"
On a large repo, this implementation can process/traverse 50,000 revs in 0.7
seconds, versus 4.2 seconds using "::<include> - ::<exclude>".
This is useful for performing histedits on your branch:
hg histedit -r 'first(only(.))'
Or lifting branch foo off of branch bar:
hg rebase -d @ -s 'only(foo, bar)'
Or a variety of other uses.
This method will replace the creation of lazysets inside the revset methods.
Instead, the classes that handle lazy structures will create them based on
their current order.
$ time hg log -qr "first(0:tip or draft())"
...
real 0m1.032s
user 0m0.841s
sys 0m0.179s
$ time ./hg log -qr "first(0:tip or draft())"
...
real 0m0.378s
user 0m0.291s
sys 0m0.085s
Performance Benchmarking:
$ time hg log -qr "first(author(mpm) or branch(default))"
0:3a6a38229d41
real 0m3.875s
user 0m3.818s
sys 0m0.051s
$ time ./hg log -qr "first(author(mpm) or branch(default))"
0:3a6a38229d41
real 0m0.213s
user 0m0.174s
sys 0m0.038s
Instead of using getitem just reverse the revision list and get the first
'lim' elements. With classes like spanset which are easily reversible this
will work faster.
Performance Benchmarking:
$ time hg log -qr "last(all())"
...
real 0m0.569s
user 0m0.447s
sys 0m0.122s
$ time ./hg log -qr "last(all())"
...
real 0m0.215s
user 0m0.150s
sys 0m0.063s
A missing ancestor expression is any expression of the form (::x - ::y) or
equivalent. Such expressions are remarkably common, and so far have involved
multiple walks down the DAG, followed by a set difference operation.
With this patch, such expressions will be transformed into uses of the fast
algorithm at ancestor.missingancestor.
For a repository with over 600,000 revisions, perfrevset for '::tip - ::-10000'
returns:
Before: ! wall 3.999575 comb 4.000000 user 3.910000 sys 0.090000 (best of 3)
After: ! wall 0.132423 comb 0.130000 user 0.130000 sys 0.000000 (best of 75)
