The part generation process was lacking a ui object and could not produce debug
output. It seems valuable to have some debug output on this part too, especially
now that we are planning to be able to hide it in the default --debug output.
The bundling process is very verbose, we would like to be able to hide such
output behind a configuration flag and have it more explicitly referencing
bundle2. The first step is to gather all these messages in a dedicated function.
The bundling process is very verbose, we would like to be able to hide such
output behind a configuration flag and have it more explicitly referencing
bundle2. The first step is to gather all these messages in a dedicated
function.
The same gathering will be later do for debug message issue by unbundling.
The current bundle2 processing was capturing all output. This is nice as it
provide better meta data about what output what, but this was changing two
things:
1) adding a prefix "remote: " to "other" output during local push (issue4613)
2) local and ssh push does not provide real time output anymore (issue4615)
As we are unsure about what form should be used in (1) and how to solve (2) we
disable output capture in this two cases. Output capture can be forced using an
experimental option.
Until this changeset, we were only able to save output if an error happened
during the 'transaction.close()' phase. If the 'processbundle' call raised an
exception, the 'bundleoperation' object was never returned, so the reply bundle
was never accessible and no output could be salvaged. We introduce a quick (but
not very elegant) fix to gain access to any reply created during the processing.
This conclude this output related series. We should hopefully be able client-side to see the
whole server output, in a proper order.
The code is now complex enough that a refactoring of it would make sense on
default.
External hook used to directly write on stdout and stderr. As a result their
output was not captured by the bundle2 processing. This resulted in confusing
out of order output on the client side. We are now capturing hooks output in
this context.
Remote output should be silenced by --quiet. The issue was found while running
`test-largefiles-cache.t` so it will get tested once we switch bundle2 by
default.
The re-handling of output is happening in some 'unbundle' callers. We have to
transmit the output information to this place so we stick it on the exception.
This is the third step in our quest for preserving the server output on error
(issue4594). We want to be able to copy the output part from the aborted reply
into the exception bundle.
This method returns a copy of all 'output' parts added to the bundler.
This is the second step in our quest for preserving the server output on error
(issue4594). We want to be able to copy the output parts from the aborted reply
into the exception bundle.
The function will be used in a later patch.
This is the first step in our quest for preserving the server output on error
(issue4594). We want to be able to copy the output parts from the aborted reply
into the exception bundle.
The function will be used in a later patch.
We want to preserve output even when the unbundling fails (eg: hook output). So
we must make sure that everything we have is flushed into the reply bundle.
(This is related to issue4594)
The obsolescence markers exchange is still experimental. We (developer) need
more information about what is going on. I'm adding an experimental flag to add
display the amount of data exchanged during bundle2 exchanges.
It is finally time to freeze the bundle2 format! To do so we:
- rename HG2Y to HG20,
- drop "b2x:" prefix from all part names,
- rename capability to "bundle2-exp" to "bundle2"
- rename the hook flag from 'bundle2-exp' to 'bundle2'
We now take full advantage of the 'getunbundler' function by using a
'{version -> unbundler-class}' mapping. This map currently contains a single
entry but will make it easy to support more versions from an extension/the
future.
At some point, this map will probably contain bundler-class information too,
in the same fashion the packer map does. However, this is not critically required
right now so it will happen by itself when needed.
The main target is to allow HG2Y support in an extension to ease transition of
companies using the experimental protocol in production (yeah...) But I've no
doubt this will be useful when playing with a future HG21.
To support multiple bundle2 formats, we will need a function returning
the proper unbundler according to the header. We introduce such aa
function and change the usage in the code base. The function will get
smarter in later changesets.
This is somewhat similar to the dispatching we do for 'HG10' and 'HG11'.
The main target is to allow HG2Y support in an extension to ease transition of
companies using the experimental protocol in production (yeah...) But I've no
doubt this will be useful when playing with a future HG21.
This makes it easy to create a new bundler class that inherits from
the core one. This matches the way 'changegroup' packers work.
The main target is to allow HG2Y support in an extension to ease transition of
companies using the experimental protocol in production (yeah...) But I've no
doubt this will be useful when playing with a future HG21.
Bundlerepo uses the compressed() method to determine whether it should write
an uncompressed temporary file. Since we don't support compressed bundle2 files
at the moment, make this method return true.
Replace bare part.read() calls with part.seek(0, 2) since the return value is
being ignored. As this doesn't necessarily require building a string that
contains the rest of the part, the potential exists to reduce the memory
footprint of these operations.
This implements a seek() method for unbundlepart. This allows on-disk bundle2
parts to behave enough like files for bundlerepo to handle them. A future
patch will add support for bundlerepo to read the bundle2 files that are
written when the experimental.strip-bundle2-version config option is used.
This patch adds seek(), tell(), and close() implementations for unpackermixin
which forward to the file descriptor's implementation if possible. A future
patch will use this to make bundle2.unbundlepart seekable, which will in turn
make it usable as a file descriptor for bundlerepo.
The binary format description has always stated that the parttype should be simple,
but it was never really enforced. Recent discussions have convinced me we want to
keep the part type simple and easy to debug. There is enough extensibility in
the rest of the format.
Encoding whether or not a part is mandatory in the capitalization of the
parttype is unintuitive and error-prone. This sequence of patches separates
these concerns in the API to reduce programmer error and pave the way for
a potential change in how this information is transmitted over the wire.
Since the parttype and mandatory bit are separated in bundle2.unbundlepart
(see previous patch), there is no longer a need to remove the mandatory bit
before working with the parttype.
Encoding whether or not a part is mandatory in the capitalization of the
parttype is unintuitive and error-prone. This sequence of patches separates
these concerns in the API to reduce programmer error and pave the way for
a potential change in how this information is transmitted over the wire.
This patch separates the two pieces of information when reading the part header
so that it's unnecessary to know how they were combined during transmission.
This patch series is intended to allow bundle2 push reply part handlers to
make changes to the local repository; it has been developed in parallel with
an extension that allows the server to rebase incoming changesets while applying
them.
This diff adds an experimental config option "bundle2.pushback" which provides
a transaction to the reply unbundler during a push operation. This behavior is
opt-in because of potential security issues: the response can contain any part
type that has a handler defined, allowing the server to make arbitrary changes
to the local repository.
This patch series is intended to allow bundle2 push reply part handlers to
make changes to the local repository; it has been developed in parallel with
an extension that allows the server to rebase incoming changesets while applying
them.
The default transaction getter for processbundle is a private function that
raises an exception; this diff lets calling code pass None as the transaction
getter to explicitly request this default behavior.
The next diff will check a config option to determine whether to provide a
transaction to the reply bundle processor. If one shouldn't be provided, the
code needs a way to specify that the default behavior should be used.
This will let the bundle2 client and server detect what packer they should be using.
This detection part is not done. I expect it to be done with the addition of the
second packer (with generaldelta support).
