We are changing all integers that denote the size of a chunk to read to int32.
There are two main motivations for that.
First, we change everything to the same width (32 bits) to make it possible for
a reasonably agnostic actor to forward a bundle2 without any extra processing.
With this change, this could be achieved by just reading int32s and forwarding
chunks of the size read. A bit a smartness would be logic to detect the end of
stream but nothing too complicated.
Second, we need some capacity to transmit special information during the bundle
processing. For example we would like to be able to raise an exception while a
part is being read if this exception happend while this part was generated.
Having signed integer let us use negative numbers to trigger special events
during the parsing of the bundle.
The format is renamed for B2X to B2Y because this breaks binary
compatibility. The B2X format support is dropped. It was experimental to
allow this kind of things. All elements not directly related to the binary
format remain flagged "b2x" because they are still compatible.
We need a wider set of hooks to process all the changes that happened during the
pull transaction. We reuse the experimental `b2x-transactionclose` hook set
from server's unbundle for consistency. This hook is experimental and will not
remains as-is forever, but this will open the door for experimentation in 3.2.
The source information can, should be applied once when opening the transaction
for the pull. This will lets element processed within a bundle2 be aware of them
and open the door to running a set of hooks when closing this pull transaction.
This is similar to what is done in server's unbundle call.
We store the source and url of the current data into `transaction.hookargs` this
let us inherit it from upper layers that may have created a much wider
transaction. We have to modify bundle2 at the same time to register the source
and url in the transaction. We have to do it in the same patch otherwise, the
`addchangegroup` call would fill these values and the hook calling will crash
because of the duplicated 'source' and 'url' arguments passed to the hook call.
A bundle2 may contain multiple parts adding changegroups, in which case there
are multiple operation records for changegroups, each with its own return
value. Those multiple return values are aggregated in a single cgresult value
for the whole operation.
As can be seen in the associated test case, the situation with hooks is not
really the best, but without deeper thoughts and changes, we can't do much
better. Hopefully, things will be improved before bundle2 is enabled by default.
In the meanwhile, multiple changegroups is not expected to be in widespread
use, and even less expected to be used for pushes. Also, not many clients
cloning or pulling bundle2 with multiple changesets are not expected to have
changegroup hooks anyways.
The push process uses a `stepsdone` attribute instead of a `todosteps` one (with
the logic swapped). We unify the two process by picking the `stepsdone` version.
I feel like `stepsdone` better fits extensions that would want to extend the push
exchange process.
We apply the same approach as for push and make the discovery extensible. There
is only one user in core right now, but we already know we'll need something
smarter for obsmarkers. In fact the evolve extension could use this to cleanly
extend discovery.
The main motivation for this change is consistency between push and pull.
We mimic what was done for `push` for similar reason. We are about to drop
`localrepo.pull` (for consistency with dropping `localrepo.push` and we better
have an API as extensible as `push` is.
Find explanations about localrepo.push removal in 88d9d4ec499e.
Retrieving bookmarks before obsmarkers will avoid turning some changesets hidden
right before making them visible again if a bookmark keeps them visible.
The discovery phases for bookmarks now use the list of explicitly pushed bookmarks
to do addition, removal and overwriting.
Tests are impacted because this reduces the amount of listkeys calls issued, removes
some duplicated messages and improves the accuracy of some messages.
To gather all the bookmark pushing actions together, we need code performing
those actions to be ready for them. We need to be able to produce different
messages for different actions.
There is no reason for bookmarks to get a special treatment. As a first step we
move the code as is in the `exchange.pull` function. Integration with the rest
of the flow will come later.
Adding bookmarks to pull means that most clone paths are now pulling bookmarks
through pull. We ensure that bookmark-update messages are properly suppressed in
that case.
In test-pull-http.t the 'requesting all changes' message disappear because we
now get the authentication error on the `listkeys`command before such message
is printed.
This part is responsible for adding new bookmarks on the remote. Before that,
it was done on its own in `commands.push`. The export is still not integrated
with the rest of the push process, but at least it now dwells in the right
function.
Returning the pushop object gives access to more information (upcoming bookmark
push result for example). `localrepo.push` currently extracts the `cgresult` for
callers.
We are about to introduce more results-related attributes on pushop (for
bookmarks) so we need a more distinctive name. We now use `cgresult` as
`pulloperation` does.
The primary goal is to make it easier for extensions to alter how bundle2
parts are laid out. They now can use the getbundle2partsgenerator decorator
to add new parts, or directly act on getbundle2partsmapping to wrap existing
part functions.
Note the 'request for bundle10 must include changegroup' error was kept
under the same conditions as before, although the logic changes don't make
it obvious.
Functions like getbundle and classes like unbundle10 really manipulate
changegroups and not bundles. A HG10 bundle is the same as a changegroup
plus a small header, but this is no longer the case for a HG2X bundle,
so it's better to separate the names a bit.
We use the `cg` argument to disable the generation of a changegroup part. This
is useful to pull information when changesets are already in sync (phases,
obsmarkers).
We should only exchange obsolete markers related to the changesets
that are being exchanged. For example, if `A'` is a successor of `A`,
we do not want to push the marker if we are not exchanging
`A'`. Otherwise `A` would disappear without a successor, leading to confusion
for both users and the evolution mechanism.
Therefore we now exchange only the markers relevant to the subset of nodes
involved in the push (the nodes themselves may be already common but were
selected by --rev (or the lack of --rev)).
Note that all selected markers are still exchanged on each push. We do
not have a discovery protocol for markers in core yet. Such discovery
would save us the exchange of markers known on both side.
We can now directly prevent any evolution-related operation from happening by using
an empty set for outgoing markers. So we detect if no transfers should occur and
use an empty set in this case.
We use the `pushkey` part to exchange bookmark updates within the unified
bundle2 push. Note that this only applies on update (moving a bookmark known on both
sides) since bookmark export (creation of a new bookmark on remote) is apparently
done outside of the _push function.
The discovery of necessary bookmark updates is now done within the "discovery
phase". This opens the door to the inclusion of bookmarks in a unified bundle2
push.
We make a temporary variable for the remote bookmark data and we do not expand
all elements from `bookmark.compare` since we are going to use only one.
Before this patch, there was always an attempt to update bookmark even if prior
steps of the push failed. I cannot see a good semantic reason to do so. We
disable this possibility to simplify the push flow with bundle2. Bookmarks will
be included in the bundle and fail with other steps.
Now that ancestors has the same boolean property as a list, we can stop unrolling
the set of ancestors. This should provide a significant speedup to this step as
ancestor objects are smart and lazy.
