This adds an experimental.bundle-phases config option to include phase
information in bundles. As with the recently added support for
bundling obsmarkers, the support for bundling phases is hidden behind
the config option until we decide to make a bundlespec v3 that
includes phases (and obsmarkers and ...).
We could perhaps use the listkeys format for this, but that's
considered obsolete according to Pierre-Yves. Instead, we introduce a
new "phase-heads" bundle part. The new part contains the phase heads
among the set of bundled revisions. It does not include those in
secret phase; any head in the bundle that is not mentioned in the
phase-heads part is assumed to be secret. As a special case, an empty
phase-heads part thus means that any changesets should be added in
secret phase. (If we ever add a fourth phase, we'll include secret in
the part and we'll add a version number.)
For now, phases are only included by "hg bundle", and not by
e.g. strip and rebase.
Before this patch, checking HG_PENDING in phases.py might cause
unintentional reading unrelated 'phaseroots.pending' in, because it
just examines existence of HG_PENDING environment variable.
This patch uses txnutil.trypending() to check HG_PENDING strictly.
BTW, this patch may cause failure of bisect in the repository of
Mercurial itself, if examination at bisecting assumes that an external
hook can see all pending changes while nested transactions across
repositories.
This invisibility issue will be fixed by subsequent patch, which
allows HG_PENDING to refer multiple repositories.
This is part of a refactoring that moves some phase query optimization from
revset.py to phases.py.
The motivation behind this was chg repo preloading - to make the obsstore
depend on less things (like the revset language). The refactoring also looks
good by itself - phasecache does not expose its private field "_phasesets"
via public methods and revset.py is accessing it in a hacky way.
This patch adds a "getrevset" method, which takes multiple phases and
returns a revset in an best-effort efficient way - for "public" phase, it
returns a lazy generatorset; for "draft" and "secret", it returns efficient
"baseset".
os.environ is a dictionary which has string elements on Python 3. We have
encoding.environ which take care of all these things. This is the first patch
of 5 patch series which tend to replace the occurences of os.environ with
encoding.environ as using os.environ will result in unusual behaviour.
Cached attribute repo._phasecache uses stat of '.hg/phaseroots' file
to examine validity of cached contents. If writing '.hg/phaseroots'
file out keeps ctime, mtime and size of it, change is overlooked, and
old contents cached before change isn't invalidated as expected.
To avoid ambiguity of file stat, this patch writes '.hg/phaseroots'
file out with checkambig=True.
This patch is a part of "Exact Cache Validation Plan":
https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan
The existing retractboundary implementation computed the new boundary by walking
all descendants of all existing roots and computing the new roots. This is
O(commits since first root), which on long repos can be hundreds of thousands of
commits.
The new algorithm only updates roots that are greater than the new root
locations. For common operations like commit on a repo with the earliest root
several hundred thousand commits ago, this makes retractboundary go from
1 second to 0.008 seconds.
I tested it by running the test suite with both implementations and checking
that the root results were always the identical.
There was some discussion on IRC about the safety of this (i.e. what if the new
nodes are already part of the phase, etc). I've looked into it and believe this
patch is safe:
1) The old existing code already filters the input nodes to only contain nodes
that require retracting (i.e. we only make node X a new root if the old phase
is less than the target phase), so there's no chance of us adding a
unnecessary root to the phase (unless the input root is made unnecessary by
another root in the same input, but see point #3).
2) Another way of thinking about this is: the only way the new algorithm would
be different from the old algorithm is if it added a root that is a
descendant of an old root (since the old algorithm would've caught this in
the big "roots(%ln::)". At the beginning of the function, when we filter out
roots that already meet the phase criteria, the *definition* of meeting the
phase criteria is "not being a descendant of an existing root". Therefore,
by definition none of the new roots we are processing are descendants of an
existing root.
3) If two nodes are passed in as input, and one node is an ancestor of the other
(and therefore the later node should not be a root), this is still caught by
the 'roots(%ln::)' revset. So there's no chance of an extra root being
introduced that way either.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
This function is:
- already loading the data in place,
- used once in the code.
So we drop the return value and change the name to make this obvious. We keep
the function public because we'll have to use it in revset.
For some reason (probably rebase issue, leprechaun or badly resolved .rej)
4d2ffbaf1d5d contains only half of the emailed patches and do not fix the bug.
This patch adds the other half and enable the sweet native computation for real.
As expected this provide massive speedup along the board.
revset #0: not public()
plain first
0) 0.011960 0.010523
1) 0.000465 3% 0.000492 4%
revset #1: (tip~1000::) - public()
plain first
0) 0.025700 0.025169
1) 0.002864 11% 0.001899 7%
revset #2: not public() and branch("default")
plain first
0) 0.022842 0.020863
1) 0.011418 49% 0.010948 52%
However, it has a less impact (even bad) on first result time in simple
situation. This comes from the overhead of building the set and filtering it.
This is especially true on my Mercurial repository (used here) where about 1/3
of the changesets are non public and hidden. This could be mitigated by a
caching of the set and a better usage of smartset in '_notpublic'. (But this
won't happen in this patch because the win is massive everywhere else).
revset #0: not public()
last
0) 0.000081
1) 0.000493 x6.1 <-- bad impact
revset #1: (tip~1000::) - public()
last
0) 0.013966
1) 0.002737 19%
revset #2: not public() and branch("default")
last
0) 0.011021
1) 0.011038
The effect mostly disappear when the number of non-public changesets is small
and/or the repo get bigger. Result for Mozilla central:
Mozilla
revset #0: not public()
plain first last
0) 0.092787 0.084094 0.000080
1) 0.000054 0% 0.000083 0% 0.000083
revset #1: (tip~1000::) - public()
plain first last
0) 0.215607 0.183996 0.124962
1) 0.031620 14% 0.006616 3% 0.031168 24%
revset #2: not public() and branch("default")
plain first last
0) 0.092626 0.082687 0.000162
1) 0.000139 0% 0.000165 0% 0.000167
To speed up the computation of draft(), secret(), divergent(), obsolete() and
unstable() we need to have a fast way of getting the list of revisions that
are in draft(), secret() or the union of both: not public().
This patch extends the work on phase computation in C and make the phase
computation code also return a list of set for each non public phase.
Using these sets we can quickly obtain all the revisions of a given phase.
We do not return a set for the public phase as we expect it to be roughly the
size of the repo. Also, it can be computed easily by substracting the entries in the
non public phases from all the revs in the repo.
This change touches every module in which repository.sopener was being used, and
changes it for the equivalent repository.svfs.
It should now be possible to remove localrepo.sopener.
This patch makes bundlerepo use a subclass of phasecache that will allow phase
boundaries to be moved around, but will never write them to the underlying
repository.
This moves the initial root phase assignment to it's own function. Future
patches which make phase calculations lazy will use this function to pre-fill
certain phases which can be deduced from the roots.
The phase cache file is no longer written on lock release, it is now handled by
the transaction (as changesets and obsolescence markers are).
(Hooray)
As we stop relying on the lock to write phase, repos with no existing phase
information will need to wait for a phase move or a strip to happen in order to
get the first write in the `phaseroots` file. This explain the change in
test-inherit-mode.t.
This should not have any side effects but in very obscure cases where
people interact with pre-2.1 and post-2.1 versions of Mercurial on the
same repo while having MQ patches applied but the MQ extension
disabled from time to time. A case unlikely enough to not be worth
preserving the old behavior with awful hacks.
In most case, the file creation logic will be handled by the
transaction itself. The write method has to stay around for the case
where the repository is modified outside a transaction (strip).
We now pass a transaction option to this phase movement function. The
object is currently not used by the function, but it will be in the
future.
All call sites have been updated. Most call sites were already enclosed in a
transaction for a long time. The handful of others have been recently
updated in previous commit.
We now pass a transaction option to this phase movement function. The object
is currently not used by the function, but it will be in the future.
All call sites have been updated. Most call sites were already enclosed in a
transaction for a long time. The handful of others have been recently
updated in previous commit.
The retractboundary function remains to be upgraded.
When aborting a rebase where tip-1 is public, rebase would fail to undo the merge
state. This caused unexpected dirstate parents and also caused unshelve to
become unabortable (since it uses rebase under the hood).
The problem was that rebase uses -2 as a marker rev, and when it checked for
immutableness during the abort, -2 got resolved to the second to last entry in
the phase cache.
Adds a test for the fix. Add exception to phase code to prevent this in the
future.
When revisions are destroyed, the `phaserevs` cache becomes invalid in most case.
This cache hold a `{rev => phase}` mapping and revision number most likely
changed.
Since cc14b89388cb, we filter unknown phases' roots after changesets
destruction. When some roots are filtered the `phaserevs` cache is invalidated.
But not if none root where destroyed.
We now invalidate the cache in all case filtered root or not.
This bug was a bit tricky to reproduce as in most case we either:
* rebase a set a draft changeset including root (phaserev invalidated)
* strip tip-most changesets (no re-numbering of revision)
Note that the invalidation of `phaserevs` are not strictly needed when only
tip-most part of the history have been destroyed. But I do not expect the
overhead to be significant.
Recomputing the filtered revisions at every access to changelog is far too
expensive. This changeset introduce a cache for this information. This cache is
hold by the repository (unfiltered repository) and invalidated when necessary.
This cache is not a protected attribute (leading _) because some logic that
invalidate it is not held by the local repo itself.
We can only use copy clone if the cloned repo do not have any secret changeset.
The current method for that is to run the "secret()" revset on the remote repo.
But with proper filtering of hidden or unserved revision by the remote this
revset won't return any revision even if some exist remotely. This changeset
adds an explicit function to know if a repo have any secret revision or not.
The other option would be to disable filtering for the query but I prefer the
approach above, lighter both regarding code and performance.
