updatephases have no use of the 'addednodes' parameter since 44be3dc1fec8.
However caller are still passing it for nothing, remove the parameter and
remove computing of the added nodes in caller.
If we are bundling secret changeset and the bundle will contain phase, we
request the changegroup to be applied as secret.
It will be useful for next patch as we are now sure that secrets changesets
are applied as secret and not applied as draft then forced to secret.
By default unbundled changesets are drafts. We want to reduce the number of
phases changes during unbundling by giving the possibility to the bundle to
indicate the phase of unbundled changesets.
The longer terms goal is to add phase movement tracking in tr.changes and the
'retractboundary' call is making it more complicated than we want.
This is a first basic visible usage of the changes tracking in the transaction.
We adds a new function computing the pre-existing changesets obsoleted by a
transaction and a transaction call back displaying this information.
Example output:
added 1 changesets with 1 changes to 1 files (+1 heads)
3 new obsolescence markers
obsoleted 1 changesets
The goal is to evolve the transaction summary into something bigger, gathering
existing output there and adding new useful one. This patch is a good first step
on this road. The new output is basic but give a user to the content of
tr.changes['obsmarkers'] and give an idea of the new options we haves. I expect
to revisit the message soon.
The caller recording the transaction summary should also be moved into a more
generic location but further refactoring is needed before it can happen.
This is one step towards removing a bunch of "if isinstance(gen,
unbundle20)" by treating bundle1 and bundle2 more similarly.
The name may sounds ironic for a method in the bundle2 module, but I
didn't think it was worth it yet to create a new 'bundle' module that
depends on the 'bundle2' module. Besides, we'll inline the method
again later.
The results only need to be combined if they come from a bundle2. More
importantly, we'll change its argument to a bundleoperation soon, and
then it definitely will no longer belong in changegroup.py.
This adds an experimental.bundle-phases config option to include phase
information in bundles. As with the recently added support for
bundling obsmarkers, the support for bundling phases is hidden behind
the config option until we decide to make a bundlespec v3 that
includes phases (and obsmarkers and ...).
We could perhaps use the listkeys format for this, but that's
considered obsolete according to Pierre-Yves. Instead, we introduce a
new "phase-heads" bundle part. The new part contains the phase heads
among the set of bundled revisions. It does not include those in
secret phase; any head in the bundle that is not mentioned in the
phase-heads part is assumed to be secret. As a special case, an empty
phase-heads part thus means that any changesets should be added in
secret phase. (If we ever add a fourth phase, we'll include secret in
the part and we'll add a version number.)
For now, phases are only included by "hg bundle", and not by
e.g. strip and rebase.
When adding support for bundling and unbundling phases, it will be
useful to have the list of added changesets. To do that, we return the
list from changegroup.apply().
We move the feature to a proper configuration and document it. The config goes
in the 'server' section because it feels like something the server owner would
want to decide. We pick and open field because it seems likely that other
checking levels will emerge in the future. (eg: server like the mozilla-try
server will likely wants a "none" value)
The option name contains 'push' since this affects 'push' only. The option value
'check-related' is preferred over one explicitly containing 'allow' or 'deny'
because the client still have a strong decision power here. Here, the server is
just advising the client on the check mode to use.
Client has a mechanism for the server to check that nothing changed server side
since the client prepared a push. That check is wide and any head changed on
the server will lead to an aborted push. We introduce a way for the client to
send a less strict checking. That logic will check that no heads impacted by
the push have been affected. If other unrelated heads (including named branches
heads) have been affected, the push will proceed.
This is very helpful for repositories with high developers traffic on different
heads, a common setup.
That behavior is currently controlled by an experimental option. The config
should live in the "server" section but bike-shedding of the name will happen
in the next changesets. Servers advertise this capability through a new bundle2
capability 'checkeads', using the value 'related'.
The 'test-push-race.t' is updated to check that new capabilities on the
documented cases.
The "hg bundle" command is a good place to test if the inclusion of obsmarkers
within a bundle is working well (part exists, content is correct etc). So we
add a way to have them included.
Ideally, this would be controlled by a change around bundlespec (bundlespec
"v3" + arguments). However, my main goal is to have obsmarkers included in
bundle created by the 'hg strip' command, not the 'hg bundle' so for now I'm
avoiding the detour through bundlespec rework territory.
Better debug output for obsmarkers in 'debugbundle' will be added in later
changesets. The 'test-obsolete-bundle-strip.t' test will also get updated in a
later changeset to keep the current changeset smaller.
We move it next to similar part building functions. We will need it for the
"writenewbundle" logic. This will allow us to easily include obsmarkers in
on-disk bundle, a necessary step before having `hg strip` also operate on
markers.
(Yes, the bundle2 module was already too large, but there any many
interdependencies between its components so it is non-trivial to split, this is
a quest for another adventure.)
Adding markers to the repository might affect the set of obsolete changesets. So we
most remove the "volatile" set who rely in that data. We add two missing
invalidations after merging markers. This was caught by code change in the evolve
extensions tests.
This issues highlight that the current way to do things is a bit fragile,
however we keep things simple for stable.
Many have seen a "stream ended unexpectedly" error. This message is
raised from changegroup.readexactly() when a read(n) operation fails
to return exactly N bytes.
I believe most occurrences of this error in the wild stem from
the code changed in this patch. Before, if bundle2's part applicator
raised an Exception when processing/applying parts, the exception
handler would attempt to iterate the remaining parts. If I/O
during this iteration failed, it would likely raise the
"stream ended unexpectedly" exception.
The problem with this approach is that if we already encountered
an I/O error iterating the bundle2 data during application, then
any further I/O would almost certainly fail. If the original stream
were closed, changegroup.readexactly() would obtain an empty
string, triggering "stream ended unexpectedly" with "got 0." This is
the error message that users would see. What's worse is that the
original I/O related exception would be lost since a new exception
would be raised. This made debugging the actual I/O failure
effectively impossible.
This patch changes the exception handler for bundle2 application to
ignore errors when seeking the underlying stream. When the underlying
error is I/O related, the seek should fail fast and the original
exception will be re-raised. The output changes in
test-http-bad-server.t demonstrate this.
When the underlying error is not I/O related and the stream can be
seeked, the old behavior is preserved.
These methods are unrelated to unpacking. They are used internally by the
'unbundlepart' class only. So me move them there as private methods.
In the same go, we clarify their internal role in the their docstring.
The unpackermixin is a utility used to implement the bundle2 protocol. It should
not be used when writing part handlers. We update the docstring to clarify this.
There are some non-obvious limitations on the parameters of this method.
Add some documentation where people will likely look to understand how
to use this API.
The current function ('writebundle') is focussing on getting an existing
changegroup to disk. It is no easy ways to includes more part in the generated
bundle2. So we introduce a slightly higher level function that is fed the
'outgoing' object (that defines the bundled spec) and the bundlespec parameters
(to control the changegroup generation and inclusion of other parts).
This is creating the third logic dedicated to create a consistent bundle2 (the
other 2 are the push code and the getbundle code). We should probably reconcile
them at some points but they all takes different types of input. So we need to
introduce an intermediate "object" that each different input could be converted
to. Such unified "bundle2 specification" could be fed to some unified code.
We start by having the `hg bundle` related code on its own to helps defines its
specific needs first. Once the common and specific parts of each logic will be
known we can start unification.
Compression engines allow options to be passed to them to control
behavior. This patch exposes an argument to bundle2.writebundle()
that passes options to the compression engine when writing compressed
bundles. The argument is honored for both bundle1 and bundle2, the
latter requiring a bit of plumbing to pass the value around.
An upcoming patch will change the "alg" argument passed to this
function from None to "UN" when no compression is wanted.
The existing implementation of bundle2 does not set a "Compression"
parameter if no compression is used. In theory, setting
"Compression=UN" should work. But I haven't audited the code to see if
all client versions supporting bundle2 will accept this.
Rather than take the risk, avoid the BC breakage and treat "UN"
the same as None.
Like the recent change for the compressor side, this too is
relatively straightforward. We now store a compression engine
on the instance instead of a low-level decompressor. Again, this
will allow us to easily transition to different compression engine
APIs when they are implemented.
Now that we have a new API to define compression engines, let's put it
to use!
The new code stores a reference to the compression engine instead of
a low-level compressor object. This will allow us to more easily
transition to different APIs on the compression engine interface
once we implement them.
As part of this, we change the registration in bundletypes to use 'UN'
instead of None. Previously, util.compressors had the no-op compressor
registered under both the 'UN' and None keys. Since we're switching to
a new API, I don't see the point in carrying this dual registration
forward.
This is similar to 72dcaa40df76. Not all calls into the compressor
return compressed data, as the compressor may buffer compressed
output internally. It is cheaper to check for empty chunks than to
send empty chunks through the generator.
When generating a gzip-v2 bundle of the mozilla-unified repo, this
change results in 50,093 empty chunks not being sent through the
generator (out of 1,902,996 total input chunks).
