Summary:
We want to allow blocking full repo streaming clones in certain
repositories (since they can take the lock and take a very long time) unless the
client has explicitly asked for it. The existing stream_out wire protocol has no
way of passing an option, so let's create a new endpoint.
Reviewed By: quark-zju
Differential Revision: D7763717
fbshipit-source-id: eace47143f8fdcc4c6e302b5c26678ccf56ca5d4
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
Recently hg-ssh was changed to block writes via in-memory hook configuration
instead of by passing config hooks, and dispatch.py blocks any invocation of hg
serve --stdio that has options passed. We have infrastructure that sets up read
only serve processes without using hg-ssh, and it was broken by this change.
Let's add a --read-only option to hg serve so non-hg-ssh solutions can still
launch hg in read-only mode. This makes it also work with non-stdio serve
processes as well.
(grafted from 7a0ed9aad8526f689343a5a02aa4a66e5f3be1f2)
(grafted from bf33f750447d8b0dfeae2a311e1d1eb93e19d6a0)
(grafted from 9ada6a6e4ac2a92325592cd58edd9160e17c9e31)
(grafted from 50e676e99c3b7cc929ceaaebbd3c684a8a58f9d8)
(grafted from 01833a49fa4dca204dc0e606f21279530925307c)
(grafted from 301af2e1a42fe912acf90ae9a87ca6a20ce5cd5d)
(grafted from 6ae2eaad3edbfdfb04ac5880a86341e69980529c)
(grafted from fcdedd417b29d28797840fc2393e0ff846fa54c8)
(grafted from ddda3705adfb2ac103f506d694d2b30028dfaca9)
(grafted from 138e3cf3bc394c4ff507341a390e1876c7104042)
(grafted from f8073d595e87086213525dbb642962b84158ee9a)
(grafted from 96bac04dc722030250a53616f0fb55125829f25e)
(grafted from 2aeed48cc3b3324b564343d8515aed1ecec69b14)
(grafted from 4fb2e02a273c868febdae9530b0a07a53a0e92a7)
(grafted from c57b28835f0b880c075d5b8aa99ddb9da54b21b1)
(grafted from 35fd78b021bec96db63e8dd99f98efc3b2342380)
(grafted from 9ac14f96c9a82068f07a709374f359283c206791)
(grafted from 4b64c191a9aacd52ea58ae8ec943605667759398)
(grafted from 1db390a79e32db12dde7a225e26b86ed245f9473)
(grafted from 9a999ed1ce50af8e5fa03dc270488c37304d8c94)
(grafted from b6791f2eb83f176192c9df50c736bc4c54fab5a6)
(grafted from 4b6e87c5be38d9971399e4ad989e4261f283b93a)
(grafted from ae660c075b4af0849d1ff5d36404ef66aeea9933)
(grafted from ff0f3bf0834b38a527654495369cd538ca8744f4)
(grafted from bff8177767c9023295ff93bc520114bb909952a8)
(grafted from 05a833e4071b9da7b447669f6bd8a3f62c1d3c27)
(grafted from be8ab299731fb8295efbe10a014798c7a177d4a0)
(grafted from beec2bc2ebd9eaf7093bba5fca8fef07c669d970)
(grafted from 03d88ba3cd8795d17a99dc1b50ac55e1937d38e1)
(grafted from 92a33bc0d275a96c61553f8bccefcd32f1360931)
(grafted from 3d37833f54e37356f3e32db2ad8fb2ffe4fa14f2)
(grafted from 77fa3393787a9410e14afc26465abb5561253075)
(grafted from 9d908f86cc7986c167cef5cdccaabb565fd2bc04)
(grafted from 0dbc2023c42f72aea6b608e5111725163dcbb64b)
(grafted from 3acf4e9bb718617efaf31abcba583b9b7be2559d)
(grafted from c671696a06e418f5f040427efb3e51fe4c9fa6cd)
(grafted from 309f11e682eb3c6fa497bf767cbcbef3b0dbaf4b)
(grafted from 4f828ef4b70a6a16fe747d5f6393042bff204b5e)
(grafted from 71193e84a71d029dedc744882978285cbe5722e6)
(grafted from 2929832c61c9727bd884f94da5afa29e80334a96)
(grafted from 2ff8a9f1761f82ffa8ebd2a96d86f7de7c712d9c)
(grafted from b438cb1e6cff36e7e197da7669def8a5e528053c)
(grafted from a9ed103481b779af9e160d2b81a9bfe81cc7d173)
(grafted from d139e95d22dc811002dafef1ecaac5dac99825fe)
(grafted from b4e41a9f2c3a6328ada72810407686d11833347c)
(grafted from 2b3826c7b3bdf669b397f1ad31ae106a05d7b05b)
(grafted from 23737fb5e1d6874cf79a1eb841dd1614c0295a1b)
(grafted from 69588396b17d3dafeece8bc9e2101559d871d9fb)
(grafted from b3252a277a53b1657e6bcf31359b413d2becffcc)
(grafted from 12c8e6062d34d4d6cf0b0128084278800a1ed8f7)
(grafted from 8ce5c67748afbe6e82fc3f822e35ddc58cb03694)
(grafted from b0f656426efcf9a70386b1c781507f40ab95af49)
(grafted from 147ae96993dcffbed2f39f31795ef3d60631d43f)
(grafted from 2ecb34a565accf638e6004c59aa5b2d2361f9428)
(grafted from 6bfc53cd4c479b4e789d4086a2c7c2f4045a288f)
(grafted from bf73f92394a079928db6d4b0b3e7aa78448af91a)
(grafted from b69a654342339eb740527fc84af523eb53edeb71)
(grafted from 3f090f44e8a33cde8d5708454a5292d0976269e6)
(grafted from ac4432275d3b750405e53f67b1267579764f4fee)
(grafted from dbfd0bce0eded53dc7d824393f03ddd2f2e693fb)
(grafted from bf7087d072ca6c5d5dac2ddef4c43339d02f6133)
(grafted from b9d63feb8c90f83e74f3e9a89328419c81088082)
(grafted from 718e93a4e545f3e16d09c66f210a567427f1068a)
(grafted from e0ba57c8bf13ccc45b7eaa62d64e03038cd002ad)
(grafted from 5c849011421ad00ef190c2bf15c640656424f681)
(grafted from e833de714167fe6039b42f1cd1890b0470a32ea2)
(grafted from 480872890137130564910a29ed8ef3890810f0c4)
(grafted from 6224dc455a24542cf7d55721fceb14a08e92d391)
(grafted from 24ced5d2b0d6fb837a3994a80ef808e29f62ccc5)
(grafted from 452eb5c8624cc22867fafa692c6c7905e46da27a)
(grafted from cdc9f1b121878c26c986eca2233b5d03ea50ad74)
(grafted from 8b3a45fe3a612fdbee3a1f291f41bfaadfd16a6f)
(grafted from 2a07c0b3cb9785a9f8d5d669b885044e4d4544b1)
(grafted from 56d892df53cfdf3a13f38cd386a437ea59ef0d77)
(grafted from b63c65fad2d28a86a3bc3871d58e45019b11e6a1)
(grafted from 8618bd56f309543d8577000a4310fdf8648f1087)
(grafted from e04c7ddddc5cc40d6347d2336b81d5be2289243e)
(grafted from 5951fe6318d02a9b739f0174f3aecc3d5eead31c)
(grafted from 0f4d380f641a55791ca9eef13cd49da24cf40a7a)
(grafted from d7ecf3376e572d77b670cbe2184370b08d38dcf7)
(grafted from a75534c9e6d7a481303096e44e265593fb5b0b2f)
We already have the ability to customize the ssh command line arguments, let's
add the ability to customize its environment as well.
Example use-case is ssh.exe from Git on Windows. If `HOME` enviroment variable
is present and has some non-empty value, ssh.exe will try to access that
location for some stuff (for example, it seems for resolving `~` in
`.ssh/config`). Git for Windows seems to sometimess set this variable to the
value of `/home/username` which probably works under Git Bash, but does not
work in a native `cmd.exe` or `powershell`. Whatever the root cause, setting
`HOME` to be an empty string heals things. Therefore, some distributors
might want to set `sshenv.HOME=` in the configuration (seems less intrusive
that forcing everyone to tweak their env).
Test Plan:
- rt
Differential Revision: https://phab.mercurial-scm.org/D1683
# skip-blame because this was mechanically rewritten the following script. I
ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t
ones appear to be, and they run without addition failures on both Windows and
Linux.
import argparse
import os
import re
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
globre = re.compile(r'^(.*) \(glob\)(.*)$')
for p in opts.path:
tmp = p + '.tmp'
with open(p, 'rb') as src, open(tmp, 'wb') as dst:
for line in src:
m = globre.match(line)
if not m or '$LOCALIP' in line or '*' in line:
dst.write(line)
continue
if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
dst.write(line)
continue
dst.write(m.group(1) + m.group(2) + '\n')
os.unlink(p)
os.rename(tmp, p)
This makes pull consistent with the part used by push and provide us with a
more compact representation of bookmarks.
In addition, this opens the way for smarter bookmark exchanges (e.g. filtering
by names or only sending the bookmark relevant to the pulled set, etc).
Before updating the actual bookmark update, we can start with updating the way
we check for push race. Checking bookmarks state earlier is useful even if we
still use pushkey. Aborting before the changegroup is added can save a lot of
time.
Adding a possibility to configure error hint to be shown in the case of problems with SSH. Example of such hint can be "Please see http://company/internalwiki/ssh.html".
Test Plan:
- Ran hg pull with broken link and verified the output has no hint by default:
```
pulling from ssh://brokenrepository.com//repo
remote: ssh: Could not resolve hostname brokenrepository.com: Name or service not known
abort: no suitable response from remote hg!
```
- Run hg pull --config ui.ssherrorhint="Please see http://company/internalwiki/ssh.html":
```
pulling from ssh://brokenrepository.com//repo
remote: ssh: Could not resolve hostname brokenrepository.com: Name or service not known
abort: no suitable response from remote hg!
(Please see http://company/internalwiki/ssh.html)
```
Differential Revision: https://phab.mercurial-scm.org/D1431
Upon pull or unbundle, we display a message with the range of new revisions
fetched. This revision range could readily be used after a pull to look out
what's new with 'hg log'. The algorithm takes care of filtering "obsolete"
revisions that might be present in transaction's "changes" but should not be
displayed to the end user.
--uncompressed isn't a very good name and its description in the
help documentation isn't very useful. We refer to this concept as
"stream clones" in a number of places. I think it makes sense to
change the user-facing argument to use the mode --stream. So this
commit does that.
We keep --uncompressed around for backwards compatibility.
While we're here, we overhaul the help docs for streaming clones
to be somewhat useful.
All tests have been updated to reflect the new preferred --stream
argument. A test for backwards compatibility of --uncompressed has
been added.
.. bc::
`hg clone --stream` should now be used instead of --uncompressed.
--uncompressed is marked as deprecated and is an alias for --stream.
There is no schedule for elimination of --uncompressed.
Differential Revision: https://phab.mercurial-scm.org/D864
A new bundle2 capability 'phases' has been added. If 'heads' is part of the
supported value for 'phases', the server supports reading and sending 'phase-
heads' bundle2 part.
Server is now able to process a 'phases' boolean parameter to 'getbundle'. If
'True', a 'phase-heads' bundle2 part will be included in the bundle with phase
information relevant to the whole pulled set. If this method is available the
phases listkey namespace will no longer be listed.
Beside the more efficient encoding of the data, this new method will greatly
improve the phase exchange efficiency for repositories with non-served
changesets (obsolete, secret) since we'll no longer send data about the
filtered heads.
Add a new 'devel.legacy.exchange' config item to allow fallback to the old
'listkey in bundle2' method.
Reminder: the pulled set is not just the changesets bundled by the pull. It
also contains changeset selected by the "pull specification" on the client
side (eg: everything for bare pull). One of the reason why the 'pulled set' is
important is to make sure we can move -common- nodes to public.
When unquoted, MSYS sees the colon between the drive letter and path as a Unix
path separator and unhelpfully splits on it, feeding only the drive letter as
the command. Much chaos ensues.
I vaguely remember trying to get the test runner to use /letter/path/to/exe
syntax the last time this happened, without success. I doubt a check-code rule
would work, since sometimes it is quoted, and sometimes the quotes are escaped.
These are some simple cases. More to come in a future change.
Reviewers: krbullock
Reviewed By: krbullock
Differential Revision: https://phab.mercurial-scm.org/D4
Some shared-ssh installations assume that 'hg serve --stdio' is a safe
command to run for minimally trusted users. Unfortunately, the messy
implementation of argument parsing here meant that trying to access a
repo named '--debugger' would give the user a pdb prompt, thereby
sidestepping any hoped-for sandboxing. Serving repositories over HTTP(S)
is unaffected.
We're not currently hardening any subcommands other than 'serve'. If
your service exposes other commands to users with arbitrary repository
names, it is imperative that you defend against repository names of
'--debugger' and anything starting with '--config'.
The read-only mode of hg-ssh stopped working because it provided its hook
configuration to "hg serve --stdio" via --config parameter. This is banned for
security reasons now. This patch switches it to directly call ui.setconfig().
If your custom hosting infrastructure relies on passing --config to
"hg serve --stdio", you'll need to find a different way to get that configuration
into Mercurial, either by using ui.setconfig() as hg-ssh does in this patch,
or by placing an hgrc file someplace where Mercurial will read it.
mitrandir@fb.com provided some extra fixes for the dispatch code and
for hg-ssh in places that I overlooked.
Previously Abort raised during 'getbundle' call poorly reported (HTTP-500 for
http, some scary messages for ssh). Abort error have been properly reported for
"push" for a long time, there is not reason to be different for 'getbundle'. We
properly catch such error and report them back the best way available. For
bundle, we issue a valid bundle2 reply (as expected by the client) with an
'error:abort' part. With bundle1 we do as best as we can depending of http or
ssh.
The python hooks have access to the hook type information. There is not reason
for external hook to not be aware of it too.
For the record my use case is to make sure a hook script is configured for the
right type.
Hooks related to the transaction are aware of the transaction id. By definition
this txn-id is unique and different for each transaction. As a result it can
never be predicted in test and always needs matching. As a result, touching any
like with this data is annoying. We solve the problem once and for all by
installing an automatic replacement. In test, this will now show as:
TXNID=TXN:$ID$
Previously, the capabilities list was protocol agnostic and we
advertised the same capabilities list to all clients, regardless of
transport protocol.
A few capabilities are specific to HTTP. I see no good reason why we
should advertise them to SSH clients. So this patch limits their
advertisement to HTTP clients.
This patch is BC, but SSH clients shouldn't be using the removed
capabilities so there should be no impact.
There was a similar issue to febfe464a80f. If an in-process hook writes
to stdout, the data may be buffered. In which case, stdout must be flushed
before restoring its file descriptor. Otherwise, remaining data would be sent
over the ssh wire and corrupts the protocol.
Note that this is a different redirection from the one I've just removed.
This patch replaces hardcoded 127.0.0.1 with $LOCALIP in all tests.
Till now, the IPv6 series should make tests pass on common IPv6 systems
where the local device has the address "::1" and the hostname "localhost"
resolves to "::1".
On Windows platform, invoking printenv.py directly via hook is
problematic, because:
- unless binding between *.py suffix and python runtime, application
selector dialog is displayed, and running test is blocked at each
printenv.py invocations
- it isn't safe to assume binding between *.py suffix and python
runtime, because application binding is easily broken
For example, installing IDE (VisualStudio with Python Tools, or
so) often requires binding between source files and IDE itself.
This patch invokes printenv.py via sh -c for test portability. This is
a kind of follow up for 9e4331825bea, which eliminated explicit
"python" for printenv.py. There are already other 'sh -c "printenv.py"'
in *.t files, and this fix should be reasonable.
This changes were confirmed in cases below:
- without any application binding for *.py suffix
- with binding between *.py suffix and VisualStudio
This patch also replaces "echo + redirection" style with "heredoc"
style, because:
- hook command line is parsed by cmd.exe as shell at first, and
- single quotation can't quote arguments on cmd.exe, therefore,
- "printenv.py foobar" should be quoted by double quotation, but
- nested quoting (or tricky escaping) isn't readable
This patch disables delta chains on changelogs. After this patch, new
entries on changelogs - including existing changelogs - will be stored
as the fulltext of that data (likely compressed). No delta computation
will be performed.
An overview of delta chains and data justifying this change follows.
Revlogs try to store entries as a delta against a previous entry (either
a parent revision in the case of generaldelta or the previous physical
revision when not using generaldelta). Most of the time this is the
correct thing to do: it frequently results in less CPU usage and smaller
storage.
Delta chains are most effective when the base revision being deltad
against is similar to the current data. This tends to occur naturally
for manifests and file data, since only small parts of each tend to
change with each revision. Changelogs, however, are a different story.
Changelog entries represent changesets/commits. And unless commits in a
repository are homogonous (same author, changing same files, similar
commit messages, etc), a delta from one entry to the next tends to be
relatively large compared to the size of the entry. This means that
delta chains tend to be short. How short? Here is the full vs delta
revision breakdown on some real world repos:
Repo % Full % Delta Max Length
hg 45.8 54.2 6
mozilla-central 42.4 57.6 8
mozilla-unified 42.5 57.5 17
pypy 46.1 53.9 6
python-zstandard 46.1 53.9 3
(I threw in python-zstandard as an example of a repo that is homogonous.
It contains a small Python project with changes all from the same
author.)
Contrast this with the manifest revlog for these repos, where 99+% of
revisions are deltas and delta chains run into the thousands.
So delta chains aren't as useful on changelogs. But even a short delta
chain may provide benefits. Let's measure that.
Delta chains may require less CPU to read revisions if the CPU time
spent reading smaller deltas is less than the CPU time used to
decompress larger individual entries. We can measure this via
`hg perfrevlog -c -d 1` to iterate a revlog to resolve each revision's
fulltext. Here are the results of that command on a repo using delta
chains in its changelog and on a repo without delta chains:
hg (forward)
! wall 0.407008 comb 0.410000 user 0.410000 sys 0.000000 (best of 25)
! wall 0.390061 comb 0.390000 user 0.390000 sys 0.000000 (best of 26)
hg (reverse)
! wall 0.515221 comb 0.520000 user 0.520000 sys 0.000000 (best of 19)
! wall 0.400018 comb 0.400000 user 0.390000 sys 0.010000 (best of 25)
mozilla-central (forward)
! wall 4.508296 comb 4.490000 user 4.490000 sys 0.000000 (best of 3)
! wall 4.370222 comb 4.370000 user 4.350000 sys 0.020000 (best of 3)
mozilla-central (reverse)
! wall 5.758995 comb 5.760000 user 5.720000 sys 0.040000 (best of 3)
! wall 4.346503 comb 4.340000 user 4.320000 sys 0.020000 (best of 3)
mozilla-unified (forward)
! wall 4.957088 comb 4.950000 user 4.940000 sys 0.010000 (best of 3)
! wall 4.660528 comb 4.650000 user 4.630000 sys 0.020000 (best of 3)
mozilla-unified (reverse)
! wall 6.119827 comb 6.110000 user 6.090000 sys 0.020000 (best of 3)
! wall 4.675136 comb 4.670000 user 4.670000 sys 0.000000 (best of 3)
pypy (forward)
! wall 1.231122 comb 1.240000 user 1.230000 sys 0.010000 (best of 8)
! wall 1.164896 comb 1.160000 user 1.160000 sys 0.000000 (best of 9)
pypy (reverse)
! wall 1.467049 comb 1.460000 user 1.460000 sys 0.000000 (best of 7)
! wall 1.160200 comb 1.170000 user 1.160000 sys 0.010000 (best of 9)
The data clearly shows that it takes less wall and CPU time to resolve
revisions when there are no delta chains in the changelogs, regardless
of the direction of traversal. Furthermore, not using a delta chain
means that fulltext resolution in reverse is as fast as iterating
forward. So not using delta chains on the changelog is a clear CPU win
for reading operations.
An example of a user-visible operation showing this speed-up is revset
evaluation. Here are results for
`hg perfrevset 'author(gps) or author(mpm)'`:
hg
! wall 1.655506 comb 1.660000 user 1.650000 sys 0.010000 (best of 6)
! wall 1.612723 comb 1.610000 user 1.600000 sys 0.010000 (best of 7)
mozilla-central
! wall 17.629826 comb 17.640000 user 17.600000 sys 0.040000 (best of 3)
! wall 17.311033 comb 17.300000 user 17.260000 sys 0.040000 (best of 3)
What about 00changelog.i size?
Repo Delta Chains No Delta Chains
hg 7,033,250 6,976,771
mozilla-central 82,978,748 81,574,623
mozilla-unified 88,112,349 86,702,162
pypy 20,740,699 20,659,741
The data shows that removing delta chains from the changelog makes the
changelog smaller.
Delta chains are also used during changegroup generation. This
operation essentially converts a series of revisions to one large
delta chain. And changegroup generation is smart: if the delta in
the revlog matches what the changegroup is emitting, it will reuse
the delta instead of recalculating it. We can measure the impact
removing changelog delta chains has on changegroup generation via
`hg perfchangegroupchangelog`:
hg
! wall 1.589245 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! wall 1.788060 comb 1.790000 user 1.790000 sys 0.000000 (best of 6)
mozilla-central
! wall 17.382585 comb 17.380000 user 17.340000 sys 0.040000 (best of 3)
! wall 20.161357 comb 20.160000 user 20.120000 sys 0.040000 (best of 3)
mozilla-unified
! wall 18.722839 comb 18.720000 user 18.680000 sys 0.040000 (best of 3)
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)
pypy
! wall 4.828317 comb 4.830000 user 4.820000 sys 0.010000 (best of 3)
! wall 5.415455 comb 5.420000 user 5.410000 sys 0.010000 (best of 3)
The data shows eliminating delta chains makes the changelog part of
changegroup generation slower. This is expected since we now have to
compute deltas for revisions where we could recycle the delta before.
It is worth putting this regression into context of overall changegroup
times. Here is the rough total CPU time spent in changegroup generation
for various repos while using delta chains on the changelog:
Repo CPU Time (s) CPU Time w/ compression
hg 4.50 7.05
mozilla-central 111.1 222.0
pypy 28.68 75.5
Before compression, removing delta chains from the changegroup adds
~4.4% overhead to hg changegroup generation, 1.3% to mozilla-central,
and 2.0% to pypy. When you factor in zlib compression, these percentages
are roughly divided by 2.
While the increased CPU usage for changegroup generation is unfortunate,
I think it is acceptable because the percentage is small, server
operators (those likely impacted most by this) have other mechanisms
to mitigate CPU consumption (namely reducing zlib compression level and
pre-generated clone bundles), and because there is room to optimize this
in the future. For example, we could use the nullid as the base revision,
effectively encoding the full revision for each entry in the changegroup.
When doing this, `hg perfchangegroupchangelog` nearly halves:
mozilla-unified
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)
! wall 11.196461 comb 11.200000 user 11.190000 sys 0.010000 (best of 3)
This looks very promising as a future optimization opportunity.
It's worth that the changes in test-acl.t to the changegroup part size.
This is because revision 6 in the changegroup had a delta chain of
length 2 before and after this patch the base revision is nullrev.
When the base revision is nullrev, cg2packer.deltaparent() hardcodes
the *previous* revision from the changegroup as the delta parent.
This caused the delta in the changegroup to switch base revisions,
the delta to change, and the size to change accordingly. While the
size increased in this case, I think sizes will remain the same
on average, as the delta base for changelog revisions doesn't matter
too much (as this patch shows). So, I don't consider this a regression.
getbundle was requesting the "phase" namespace instead of the "phases"
namespace, which led to the client still requesting the phases
separately after getbundle finished.
In order to give us the freedom to change the changegroup3 format,
let's hide it behind an experimental config. Since it is required by
treemanifests, that will override the cg3 config.
Sometimes a txnclose or changegroup hook wants to iterate through all
the changesets in transaction: in that situation usually the revset
`$HG_NODE:` is used to select the revisions. Unfortunately this revset
sometimes may contain too many changesets because we don't have the
write lock while the hook runs newer changes may be added to
repository in the meantime.
That's why there is a need for extra variable carrying the information about
the last change in the transaction.
I'm not entirely happy with using a trailing / on a "file" entry for
transferring a treemanifest. We've discussed putting some flags on
each file header[0], but I'm unconvinced that's actually any better:
if we were going to add another feature to the cg format we'd still be
doing a version bump anyway to cg4, so I'm inclined to not spend time
coming up with a more sophisticated format until we actually know what
the next feature we want to stuff in a changegroup will be.
Test changes outside test-treemanifest.t are only due to the new CG3
bundlecap showing up in the wire protocol.
Many thanks to adgar@google.com and martinvonz@google.com for helping
me with various odd corners of the changegroup and treemanifest API.
0: It's not hard refactoring, nor is it a lot of work. I'm just
disinclined to do speculative work when it's not clear what the
customer would actually be.
The other invocations aren't quoted, and Windows doesn't like the single quotes:
diff --git a/tests/test-ssh.t b/tests/test-ssh.t
--- a/tests/test-ssh.t
+++ b/tests/test-ssh.t
@@ -520,20 +520,8 @@ remote hook failure is attributed to rem
$ echo "pretxnchangegroup.fail = python:$TESTTMP/failhook:hook" >> remote/.hg/hgrc
$ hg -q --config ui.ssh="python '$TESTDIR/dummyssh'" clone ssh://user@dummy/remote hookout
+ abort: no suitable response from remote hg!
+ [255]
$ cd hookout
+ $TESTTMP.sh: line 264: cd: hookout: No such file or directory
$ touch hookfailure
- $ hg -q commit -A -m 'remote hook failure'
....
Before bundle2, hook output from hook failures was prefixed with
"remote: ". Up to this point with bundle2, the output was converted to
the message to print in an Abort exception. This had 2 implications:
1) It was unclear whether an error message came from the local repo
or the remote
2) The exit code changed from 1 to 255
This patch changes the handling of error:abort bundle2 parts during push
to prefix the error message with "remote: ". This restores the old
behavior.
We still preserve the behavior of raising an Abort during bundle2
application failure. This is a regression from pre-bundle2 because the
exit code changed.
Because we no longer raise an Abort with the remote's message, we needed
to insert a message for the new Abort. So, I invented a new error
message for that. This is another change from pre-bundle2. However, I
like the new error message because it states unambiguously who aborted
the push failed, which I think is important for users so they can decide
what's next.
The added tests don't agree in their output. This demonstrates a
difference in `hg push` behavior between pre-bundle2 and bundle2.
A subsequent patch will attempt to restore some of the pre-bundle2
behavior to bundle2.
Before this patch, repo could be set to None for wrong -R. It's okay for
commands that can reject repo=None, but the command server have a problem
because:
- it accepts repo=None for "unbound" mode
- and it reenters dispatch() where repo object is created for cwd by default
Test outputs are changed because the error is detected earlier. I think new
message is better than ".hg not found".
This is a backout of 93589179c542, and a partial backout of 9b1628b91e74.
Windows won't execute 'dummyssh' directly, presumably because CreateProcess()
doesn't know how to execute a bash script:
$ hg clone -e "dummyssh" ssh://user@dummy/cloned sshclone
remote: 'dummyssh' is not recognized as an internal or external command,
remote: operable program or batch file.
abort: no suitable response from remote hg!
[255]
With the restoration of python as the executable, $TESTDIR needs to be restored
for these invocations, because python won't search $PATH for 'dummyssh':
$ hg clone -e "python dummyssh" ssh://user@dummy/cloned sshclone
remote: python: can't open file 'dummyssh': [Errno 2] No such file or directory
abort: no suitable response from remote hg!
[255]
The 'bundlecaps' argument is expected to be a set, but 'listkeys' is
expected to be a list where ordering matters. We introduce a new 'scsv'
argument type for the 'set' version and move 'csv' to the 'list'
version.
'test-ssh.t' is changed because this introduced an instability in the order we
were producing listkeys parts.
.hgtags fnodes cache entries can be expensive to compute, especially
if there are hundreds of even thousands of them. This patch implements
support for receiving a bundle2 part that contains a mapping of
changeset to .hgtags fnodes.
An upcoming patch will teach the server to send this part, allowing
clients to bypass having to redundantly compute these values.
A number of tests changed due to the client advertising the "hgtagsfnodes"
capability.
Now that we have a bundle1 version of this test, we can move the main
version to bundle2. This lets us handle the ouput change from using
the bundle2 protocol earlier.
