We have a new unbundle class and it is now responsible from extracting its own
data. The top level bundler only extracts the header (to detect an end of stream
marker) then leaves everything else to the `unbundlepart` class. The ultimate
goal is to have `unbundlepart` responsible for lazily extracting its payload.
This is mostly code movement.
The coming `unbundlepart` will need the same kind of method than `unbundle20`
for unpacking data from the stream. We extract them into a mixin class before
the creation of `unbundlepart`.
We are going to introduce an `unbundlepart` dedicated to reading bundle. So we
need to rename the one used to create bundle. Even if dedicated to creation, this
is still used for unbundling until we get the new class.
When the `part.data` attribute is an iterator, we assume it is an iterator of
chunks and use it.
We use a chunkbuffer to yield chunks of 4096 bytes.
The tests are updated to use this feature.
We are preparing streaming capability for part. So the generation of payload
chunk will becomes more complex. We extract this part in its own function before
any changes.
This code used to be in `writebundle` only. We needs to make it more broadly
available for bundle2. The "changegroup" bundle2 part has to retrieve the
binary content of changegroup stream. We moved the chunks retrieving code into
the `unbundle10` object directly and the `writebundle` code is now using that.
This split is useful for bundle2 purpose, we want to be able to easily stream
changegroup content in a part.
To keep thing simples, we kept compression out of the new methods. If it make
more sense in the future, compression may get included in this function too.
We now have an official way to return the result of addchangegroup. The tests are
updated to check that the return bundle is properly created. It will be used
when push is bundle2 enabled.
We do not know yet what kind of data future features and extensions will need to
exchange. To handle that, bundle2 allows to send arbitrary content to the
server. As a consequence, we need to be able to reply arbitrary content to the
client. And, we can use bundle2 to transmit those arbitrary data.
When a client will push a bundle2 to the server, the server will reply with a
bundle2 itself.
This changeset installs the first stone of this logic and test it.
For sending response to a pushed bundle, we need to link reply parts to request
part. We introduce a part id for this purpose. This is a 32 bit unique
integer stored in the header.
This patch makes "repair.strip()" treat bundle files via vfs.
This patch also avoids applying "vfs.join()" on the value returned by
"changegroup.writebundle()", to get relative path from "_bundle()".
This patch makes paths below in "_bundle()" relative to ".hg":
- backup directory ("strip-backup"), and
- bundle file under backup directory
"vfs" is passed to "changegroup.writebundle()" to use relative path
directly.
This patch applies "vfs.join()" on the value returned by "_bundle()",
because the caller expect it to return absolute path.
This will be changed by succeeding patch changing the caller side.
Before this patch, filename specified to "changegroup.writebundle()"
should be absolute one.
In some cases, they should be relative to repository root, store and
so on (backup before strip, for example).
This patch adds "vfs" argument to "writebundle()", and makes
"writebundle()" open (and unlink) "filename" via vfs for relative
access, if both filename and vfs are specified.
Before this patch, "localrepository.undofiles()" returns list of
absolute filename of undo files.
This patch makes it return list of tuples "(vfs, relative filename)"
to access undo files via vfs.
This patch also changes "repair.strip()", which is the only user of
"localrepository.undofiles()".
The default http request character set is UTF-8. If the message is not encoded
in UTF-8, such as big5, it cannot be shown correctly. The 'charset' is
overridden by the root document's, such that the user can select the proper
encoding in the browser.
Localrepo now supports the unbundle method of pushing changegroups. We
plan to use the unbundle call for bundle2 so it is important that all
peers supports it. The `peer.unbundle` and `peer.addchangegroup` code
path have small difference so cause some test output changes. None of those
changes seems problematic.
The `exchange` module now contains an `unbundle` function that holds the core
unbundle logic. The wire protocol keeps its own unbundle function. It enforces
wireprotocol-specific logic and then calls the extracted function.
This aims at implementing unbundle for localrepo.
We are going to refactor the unbundle function to have it working on
a local repository too. Having this function extracted will ease the process.
In the case of non-matching heads, the function now directly raises an
exception. The top level of the function is catching it.
When a changegroup is added by a push on a publishing server, we ensure they
are added as public. This is used to enforce publishing on server when the
client is not aware of phases. It also prevents race conditions where a reader
could see the changesets as draft before they get turned public by the client.
Finally, this save rounds trip as the client does not need additional request to
turn them public.
However, this logic was only enforced when the changegroup was from a "push"
source. And "push" is used for local pushes only. Wireprotocol push uses "serve"
as source since Mercurial 1.9. We now enforce this logic for both "push" and
"serve" sources.
One could note that this logic was mainly useful during wireprotocol exchanges.
So this code is finally put into good use, 9 versions after its introduction.
Python uses a C long (32 bits on Windows 64) rather than an ssize_t in
read(), and thus has a 2G size limit. Work around this by falling back
to reading one chunk at a time on overflow. This approximately doubles
our headroom until we run back into the size limit on single reads.
If backout generated no changes to commit, it showed wrong status, "changeset
<target> backs out changeset <target>", and raised TypeError with -v option.
This changes the return code to 1, which is the same as "hg commit" and
"hg rebase".
Before this patch, "contrib/check-code.py" can't detect these
problems, because the regexp pattern to detect "% inside _()" doesn't
suppose the case that format string consists of multiple string
components concatenated implicitly or explicitly,
This patch does below for that regexp pattern to detect "% inside _()"
problems in such case.
- put "+" into separator part ("[ \t\n]") for explicit concatenation
("...." + "...." style)
- enclose "component and separator" part by "(?:....)+" for
concatenation itself ("...." "...." or "...." + "....")
Before this patch, "contrib/check-code.py" can't detect these
problems, because the regexp pattern to detect "% inside _()" doesn't
suppose the case that the format string and "%" aren't placed in the
same line.
This patch replaces "\s" in that regexp pattern with "[ \t\n]" to
detect "% inside _()" problems in such case.
"[\s\n]" can't be used in this purpose, because "\s" is automatically
replaced with "[ \t]" by "_preparepats()" and "\s" in "[]" causes
nested "[]" unexpectedly.
Since changeset a8955c4d9ef5, "reposetup()" of each extensions is
invoked only on repositories enabling corresponded extensions.
This causes that largefiles specific interactions between the
repository enabling largefiles locally and remote (wire) peer fail,
because there is no way to know whether largefiles is enabled on the
remote repository behind the wire peer, and largefiles specific
"wireproto functions" are not given to any wire peers.
To avoid this problem, largefiles should be enabled in wider scope
than each repositories (e.g. user-wide "${HOME}/.hgrc").
This patch introduces "wirepeersetupfuncs" to setup wire peer by
extensions already enabled. Functions registered into
"wirepeersetupfuncs" are invoked for all wire peers.
This patch uses plain list instead of "util.hooks" for
"wirepeersetupfuncs", because the former allows to control order of
function invocation by order of extension enabling: it may be useful
for workaround of problems with combination of enabled extensions
