Instead of walking all ancestors to compute the full set we now check membership
lazily. This massively speed.
On a million-ish revision repository, this remove 14 seconds from the push logic,
making the checkheads function disappear from profile.
The obsolete post-processing needs to know the extend of the pushed set. The
way it is implemented is... suboptimal. It build a full set of all nodes in the
pushset and it does so using changectx. We have much better API for this now.
The simplest is to use the existing lazy ancestors computation. That logic uses
revnum and not node (for good reason) so we start with updating the
post-processing code to handle a "allfuturecommon" set containing revision
numbers.
We are about to call 'torev' on node that might be locally missing. In this
case, 'nodemap.revs' will return None (something valid in our usecase) while
'changelog.rev' would raise an exception.
We make this change in a distinct changeset to show it does not impact the
tests.
We are about to make "allfuturecommon" a set of revs instead of a set of nodes.
The function updated in this patch do not needs to know about these details so
we just pass it a 'ispushed(node)' function. This will simplify the next
changeset.
Client has a mechanism for the server to check that nothing changed server side
since the client prepared a push. That check is wide and any head changed on
the server will lead to an aborted push. We introduce a way for the client to
send a less strict checking. That logic will check that no heads impacted by
the push have been affected. If other unrelated heads (including named branches
heads) have been affected, the push will proceed.
This is very helpful for repositories with high developers traffic on different
heads, a common setup.
That behavior is currently controlled by an experimental option. The config
should live in the "server" section but bike-shedding of the name will happen
in the next changesets. Servers advertise this capability through a new bundle2
capability 'checkeads', using the value 'related'.
The 'test-push-race.t' is updated to check that new capabilities on the
documented cases.
The goal is to have the function directly return something meaningful and
useful for the whole pull.
Note: we skip adding post-processing in '_oldheadssummary' because if a client
is too old for branchmap it will be too old for obsolescence too.
Our goal is to be able to perform the post processing directly into the
'_headssummary' function. However before this patch the '_headsummary' function
only had access to repo, remote, outgoing while the '_postprocessobsolete'
function takes a 'pushop' object. Experience shows that having the 'pushop'
object helps extensions so we update '_headssummary' to take a pushop
object as argument.
We extract this function from the loop and gather it with the rest of the
obsolescence specific code. That will help to clarify the move of the whole logic
inside the "heads summary" computation.
The 'discardedheads' return become unused and the relationship between newheads
and newhs can be clarified. Our next goal here is to be able to extract the
_postprocessobsolete call outside of the loop.
We keep returning the 'discardedheads' because we'll start using it again soon
in this series.
Since b5554fb97d46, heads unknown locally no longer get any post processing
from obsolescence markers. We clarify this fact by only feeding the list of
locally known new heads to the function. This simplification of the input will
help moving that post-processing earlier in the function.
The '_headssummary' function is documenting and using list objects in its
return. We now use them in _oldheadssummary too for consistency. This does not
affect any usages of these values.
On repository with many many heads, this is called often enough to show an
overhead to appears in some profile. So we skip the 'changectx' creation to test
the node phases.
If a marker has no parent information, parents field is set to None, which
caused TypeError. I think this shouldn't normally happen, but somehow
I have such marker in my repo.
The previous logic had many shortcoming (eg: looking at the head only, not
handling prune, etc...), the new logic use a more robust approach:
For each head, we check if after the push all changesets exclusive to this heads
will be obsolete. If they are, the branch considered be "replaced".
To check if a changeset will be obsolete, we simply checks:
* the changeset phase
* the existence of a marker relevant to the "pushed set" that affects the
changesets..
This fixes two major issues of the previous algorithm:
* branch partially rewritten (eg: head but not root) are no longer detected as
replaced,
* Prune are now properly handled.
(This implementation was introduction in the evolve extension, version 6.0.0.)
This new algorithm has an extended number of tests, a basic one is provided
in this patch. The others will be introduced in their own changeset for clarity.
In addition, we stop trying to process heads unknown locally, we do not have
enough data to take an informed decision so we should stop pretending we do.
This reflect a test that is now update.
The checkheads function is long and complex, extract that logic in a subfunction
is win in itself. As the comment in the code says, this postprocessing is
currently very basic and either misbehave or fails to detect valid push in many
cases. My deeper motive for this extraction is to be make it easier to provide
extensive testing of this case and strategy to cover them. Final test and logic
will makes it to core once done.
c8fefe878a71 introduced default params for discovery.outgoing(), but it used a
falsy check instead of an explicit check for None. The result is that callers
that passed in an empty list would have that list overridden by the defaults,
which is not the expected behavior.
This was discovered by changes to the test-pushrebase.t test in Facebook's
repository of mercurial extensions.
This argument can be used instead of 'commonheads' to determine the 'outgoing'
set. We remove the outgoingbetween function as its role can now be handled by
'outgoing' itself.
I've thought of using an external function instead of making the constructor
more complicated. However, there is low hanging fruit to improve the current
code flow by storing some side products of the processing of 'missingroots'. So
in my opinion it make senses to add all this to the class.
We are about to introduce a third option to create an outgoing object:
'missingroots'. This argument will be mutually exclusive with 'commonheads' so
we implement some default value handling in preparation.
This will also help use to make more use of outgoing creation around the code
base.
We are to introduce more code constructing such object in the code base. It will
be more convenient to pass a repository object, all current users already
operate at the repository level anyway. More changes to the contructor argument
are coming in later changeset.
changegroup.changegroupsubset() contained somewhat low-level code for
constructing an "outgoing" instance from a list of roots and heads
nodes. It feels like discovery.py is a more appropriate location for
this code.
This code can definitely be optimized, as outgoing.missing will
recompute the set of changesets we've already discovered from
cl.between(). But code shouldn't be refactored during a move, so
I've simply inserted a TODO calling attention to that.
Before 'hg push -B .' on new remote head complained with:
abort: push creates new remote head ...
It was because _nowarnheads was not expanding active bookmark
name, so it didn't add active bookmark "proper" name to no
warn heads list.
Previously, we passed only enough information to _nowarnheads to perform the old
calculation. However, extensions may want to use more information in this
calculation. Now that pushop is extensible, extensions will often want access
to the additional fields they have added to the pushop, so pass that through.
Specifically, the remotenames extension will use this information to allow
pushing new heads that are bookmarked without requiring --force.
Previously, we passed a bunch of parameters to discovery.checkheads, but all
of the arguments can be fetched out of pushop, which may contain a lot more
useful information for extensions now that pushop is extensible.
In addition to taking a step towards getting an unreasonably large function
factored into smaller, more manageable functions, this will allow extensions
such as remotenames have more control over what pushes are allowed or not.
Having the simple and tiny branch of the conditional first help readability. The
"else" that appears after a screen of code is harder to relate to a conditional.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
In repos with many changesets, the computation of allfuturecommon
can take a significant amount of time. Since it's only used if
there's an obsstore, don't compute it otherwise.
Although Python supports `X = Y if COND else Z`, this was only
introduced in Python 2.5. Since we have to support Python 2.4, it was
a very common thing to write instead `X = COND and Y or Z`, which is a
bit obscure at a glance. It requires some intricate knowledge of
Python to understand how to parse these one-liners.
We change instead all of these one-liners to 4-liners. This was
executed with the following perlism:
find -name "*.py" -exec perl -pi -e 's,(\s*)([\.\w]+) = \(?(\S+)\s+and\s+(\S*)\)?\s+or\s+(\S*)$,$1if $3:\n$1 $2 = $4\n$1else:\n$1 $2 = $5,' {} \;
I tweaked the following cases from the automatic Perl output:
prev = (parents and parents[0]) or nullid
port = (use_ssl and 443 or 80)
cwd = (pats and repo.getcwd()) or ''
rename = fctx and webutil.renamelink(fctx) or []
ctx = fctx and fctx or ctx
self.base = (mapfile and os.path.dirname(mapfile)) or ''
I also added some newlines wherever they seemd appropriate for readability
There are probably a few ersatz ternary operators still in the code
somewhere, lurking away from the power of a simple regex.
When a remote is not capable of the `branchmap` wireproto command, we denote
incoming heads with None. This leads to a crash in the code in charge of
displaying the list of unknown remote heads.
We now properly detect this case and display a shorter message in this case.
The reason for this `set([None])` value is now documented.
This situation is not necessarily a problem and do not deserve a warning. It is
just some information that can guide the user in understanding what is going
on.
Making it 'debug' would usually not give the hint when it is relevant so we
make it a 'status' message.
This message frequently caused confusion. "unsynced" is not a well established
user-facing concept in Mercurial and the message was not very specific or
helpful.
Instead, show a message like:
remote has heads on branch 'default' that are not known locally: 6c0482d977a3
This message will also (when relevant) be shown before aborting on "push
creates new remote head".
A similar (but actually very different) message was addressed in cbd5a12a601a.
This note (which actually is a warning) frequently caused confusion.
"unsynced" is not a well established user-facing concept in Mercurial and the
message was not very specific or helpful.
Instead, show a messages like:
remote has heads on branch 'default' that are not known locally: 6c0482d977a3
and show it before aborting on "push creates new remote head". This will also
give more of a hint in the case where the branch has been closed remotely and
'hg heads' thus not would show any new heads after pulling.
A similar (but actually very different) message was addressed in cbd5a12a601a.
Nodemap is not aware of filtering so we need to ask the changelog itself if a
node is known. This is probably a bit slower but such check does not dominated
discovery time. This is necessary if we want to run discovery on filtered repo.
Push is currently allowed to create a new head if there is a remote
bookmark that will be updated to point to the new head. If the
bookmark is not known remotely then push aborts, even if a -B argument
is about to push the bookmark. This change allows push to continue in
this case. This does not require a wireproto force.
