Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.
Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.
Reviewed By: phillco
Differential Revision: D13868981
fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
Summary:
Switch `ui.load` and `ui.readconfig` to use the Rust config parser.
`ui` now no longer depends on `config.config` or `rcutil`.
Pest's error messages are fancier, thus most test changes.
For the fbsource repo, debugshell shows the new code is 10+x faster:
On laptop:
Before:
In [1]: %timeit m.ui.ui.load()
10 loops, best of 3: 27.8 ms per loop
After:
In [1]: %timeit m.ui.ui.load()
100 loops, best of 3: 1.85 ms per loop
On devserver:
Before:
In [1]: %timeit m.ui.ui.load()
100 loops, best of 3: 16.8 ms per loop
After:
In [1]: %timeit m.ui.ui.load()
1000 loops, best of 3: 1.28 ms per loop
Since `ui._rcfg` is no longer copy-on-write, there is concern about `ui.copy()`
performance. It is faster too (on devserver):
Before:
In [1]: %timeit ui.copy()
1000 loops, best of 3: 198 µs per loop
After:
In [1]: %timeit ui.copy()
10000 loops, best of 3: 157 µs per loop
The old `ui.py` was copied to `legacyui.py` and can replace the new `ui.py` if
a config file exists on the system. This provides a way to switch back to the
old config parser in case of emergency.
Reviewed By: mitrandir77
Differential Revision: D8887375
fbshipit-source-id: 2951ca622c77bf41187ad5c5cab3445cda0dc519
Summary:
We're seeing "Permission denied" errors on some of our automation, but
it's difficult to track down where it's coming from. Let's make each message
more descriptive.
Also updates, hgsql to handle the hg-rsh hook, which I noticed while
investigating.
Reviewed By: phillco, farnz
Differential Revision: D8188414
fbshipit-source-id: 5f8c99e8ba896c2636b1a04716125bc6a9df0591
Summary:
We want to allow blocking full repo streaming clones in certain
repositories (since they can take the lock and take a very long time) unless the
client has explicitly asked for it. The existing stream_out wire protocol has no
way of passing an option, so let's create a new endpoint.
Reviewed By: quark-zju
Differential Revision: D7763717
fbshipit-source-id: eace47143f8fdcc4c6e302b5c26678ccf56ca5d4
Summary:
We want to deprecate the bundlev1 format, so let's start by adding a
develwarn. Later diffs will update the tests to not use v1, then remove v1 as a
supported outgoing bundle entirely.
Reviewed By: quark-zju
Differential Revision: D7591166
fbshipit-source-id: 143ad029bfe4d141f91d6d5077342dfa44ad2944
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
Differential Revision: https://phabricator.intern.facebook.com/D4599415
(grafted from 35efb60b54b45e4bf8a281da35377ceabd9a2d0d)
(grafted from c976227665e29759c3bfd62a78b653815999f33c)
(grafted from ce81c51c90c6a7e67377b8368ff056be747ba808)
(grafted from f8ebd7782719ffb9f6c9bbdc740af4d628bc4c38)
(grafted from b683d8dc9677c2e4f24395f2ae849129bd4d6c81)
(grafted from 98300e5ba307c80f257e9d28b3afa7ef30d9a109)
(grafted from 6baa745cb3ea1a59c5e194b28d1db7e6a14da8fa)
(grafted from 0bd1e681e370164b2b58ed6329dcceab8a45ba1a)
(grafted from b88e051e45b3b34eebfe68b579d40e0848dd9164)
(grafted from d7d3ea27b8e834ca6a7ee86d67109f68dff8b627)
(grafted from 07f80547629b090986e880005f17af3470927034)
(grafted from f3ecb29d9b68d2adcf28a1986c070ceb85da8421)
(grafted from 23c77af0834657915d69f06623b8d831fc03dfbc)
(grafted from 6c0111953ffaacad3deb12f81bf05454f4c8bfc1)
(grafted from e6564dd98f0ab63736ba70500d60db0def7025f8)
(grafted from 3228fe1cc79cea64fa4cc674bf71f24123433ec7)
(grafted from 5a8c522c3db09d88bc6d45393c40256139ab4d42)
(grafted from 8d3f1b10a2bcda17dc55cdc97712bdc456acc406)
(grafted from 1f994f75a2a4f57c0f19981f71254e22963eaa04)
(grafted from 9bf91f3db246c40475f57657d82bd3f50e571e84)
(grafted from 2f88c5c88fd55b48fc0c30aa2abd701b2b7b9d7d)
(grafted from fab2b4f5d6e855c657f69dccb08f720bab4a82b1)
(grafted from e2fc694c13c74f7948680d653568e62d7010c4f4)
(grafted from d474fcb2eb58404a689024cc503abe28136c8395)
(grafted from 8b2233600993b96119186ddbfb16232c48247fe1)
(grafted from ba5486b8c838e72c35cb437cbdf0add5b2100c6b)
(grafted from 42699d89bd0da2e204075246eb4c27baa642fc2a)
(grafted from 02d0e17b6b7f938ba58f26d84191ef347bcdbd17)
(grafted from 9f9bba61993924c1c96a5dd15c0c50895ec8e525)
(grafted from c3bc9b3844d852ba7365574d3fbd4aa8e2763aa0)
(grafted from fc5d31337891e9eb218f100b177f8ead2f6353da)
(grafted from baad4585f877267a3264dc4c70e8cd75bfc6688e)
(grafted from 6d46603cea8fc9d04628817ae1e239b38e62ce62)
(grafted from c54d6c0b3b3dae086b8ad186e178a1e137b57cae)
(grafted from 1755e00c824c7d5b30a7daff573c3ab1e2f1fd2c)
(grafted from 81826a9db3f5f9c8e760d2b09afb06d9f74a1609)
(grafted from 73bce05fdb8cb8f8739bd2c90741ac7d422e24a4)
(grafted from c942744744e071464d3a476e57fd74f922fca497)
(grafted from 77b797a46b52249b26da6d30a0c63e381d1af0fe)
(grafted from 9b09bef68b1aa1ef3804aec330e793dcc7ada944)
(grafted from fefea5a4a24f0c8219b4b0e2b0cc70b4dfadfde2)
(grafted from 3428e6ab54999e70b38a5ee62291c8774bae7e4a)
(grafted from dd57d190fa75589d800beed188c5a381678cca3c)
(grafted from 9005ba0d5e6420ed8d048c08fc38a9682e536a26)
(grafted from daeb4bf87b68309c4c3c608853affe2adeb9fcbd)
(grafted from 6ad984397b22666ecebc2ff737b0d52ffb95b9b7)
(grafted from bd1f17920c18d88b6c4dd451f76699f6f665bc32)
(grafted from 31a92a48004ab6a4bfc389f674cacf549703e2b9)
(grafted from 7e80636f1529b8902292980912586f1db8598a83)
(grafted from 55ecf6a9f9bc89fd0330c4684f225e213be18033)
(grafted from 3361f627a0b8cb7666a529494ed80a0e1d7a365c)
(grafted from f51abda49b2d33a936567b786ce785c97e4b9c64)
(grafted from 5c01ed3bf394351e116fcece1fbd9e1b19f57ea4)
(grafted from 9503cdb335a95950235b7871978bcb6cb3e87222)
(grafted from 220d31489a377997e71d5766f9f29b30fd1662e1)
(grafted from 97483c98cf95113e95d03bf69f3307f92ef232a4)
(grafted from b3e9f8f43b4dd76422c20234acf76ec82a9c5f5b)
(grafted from 831d9f1b2940b6944fc1015c57f348a22de7a149)
(grafted from 051e1081a45e85944ea4414174a452e208904371)
(grafted from 9fb7c4c494166dee97bb04826cf71b0fa774cd2b)
(grafted from bfb4cb9c08c582323ceb82eb5cd2bcaeaf70329c)
(grafted from ab548e4f700a65bd24fbbbdb43f971eb5da73be5)
(grafted from 8870c382cfecbcbd4f8cec3ca0cb8759a43cf31c)
(grafted from 8e09e732b846b42beb49866d3785572df89f1e38)
(grafted from 951064f2a2980a00d085fce0d207c0225447b5d5)
(grafted from fce58e29903c2ef95b0481c11d59e5ef4fb56d15)
(grafted from 391f610601eeb542b9aa2bed3f541609772b8599)
(grafted from 681ea78b8ae1c29589a49a5bb74e0f3dff30e1d7)
(grafted from add76a9d94ee5d7b829be52458d1dfb65585c2d3)
(grafted from 475ae856da634e53637131717723fe895be0def6)
(grafted from b22d9651b8a7957c8275301d79f6ba62fc5a38f4)
(grafted from 3bf8c57dfd2fb432f4d499014dd9ab4ced26133d)
(grafted from 47718758ce30722c7a6e4d4dc324bdef20d45163)
(grafted from 59270eeea34cc381933ce50600983819e8a35794)
(grafted from 5397fdfa10d831ba358925e4a32faa23fed18173)
(grafted from de9cc0df3d097d25a8a218010e4a8370373f5068)
(grafted from e7f5a21e9dd2f2a4527943f5dad0d597c0f2b7a4)
(grafted from 5c383ebe8f57ccabf1ede74905a951631c7e32de)
(grafted from bf4ad8ffd761b575a843fe5f8370fb4a5c1a2eb9)
(grafted from 8523ff03bfa8708eefb667d9df64fba58e5a9f8c)
(grafted from 5220db13df5a745d113ebba89185ac42f4f890d1)
(grafted from 8a66c7352d03e545e8cbe80a1f84122624f2035c)
(grafted from 94cf7cdbce472e0694582b18acd6f2fc3f46e839)
(grafted from 29575e1ce294a235a1df9136a6b807b8947a20cd)
(grafted from d23b69b215e4201e90b600797b41ab362e7c7044)
(grafted from 386af1481e70f160f6b3d0e08ba8934372446b68)
(grafted from fc69c78396e9ab15fd961540204db004db933799)
(grafted from 8a8e35573518b72937af626999b885c6f6491683)
(grafted from 9fb083f031e7358fdb73ef6fac5278f841f2b084)
(grafted from aa20fd817539bdd8d15adb340ef714121dd273b2)
(grafted from 2b68df547651d1727b4320f2ffbb8fded3ab49ac)
(grafted from ff3bf5623756d977c88a25d546b34160229177d3)
(grafted from fd5f39d43bac7fa2c8a1e5785e3dc060f539277c)
(grafted from 8871ae64560384cf7693a5c32333b90e6774ba2a)
(grafted from 3a01e81a1a78138af5892129f447bee53fb85a73)
(grafted from ca1bc04c027005d42b4dc5eee52c3874a13790df)
(grafted from c68f7302ce65f055e205ed701490529571aaaa32)
(grafted from c7e264107eb95c8ab01804699775374b43bcbd84)
(grafted from 3748e7898899c994a4b5019b66f705e2c7a8d2eb)
(grafted from fe2d08b837334f9b3b1bb3a18a28f010a02c0340)
(grafted from dd29c6c56f0de76550d2f61671eb05da6a03fbc3)
Recently hg-ssh was changed to block writes via in-memory hook configuration
instead of by passing config hooks, and dispatch.py blocks any invocation of hg
serve --stdio that has options passed. We have infrastructure that sets up read
only serve processes without using hg-ssh, and it was broken by this change.
Let's add a --read-only option to hg serve so non-hg-ssh solutions can still
launch hg in read-only mode. This makes it also work with non-stdio serve
processes as well.
(grafted from 7a0ed9aad8526f689343a5a02aa4a66e5f3be1f2)
(grafted from bf33f750447d8b0dfeae2a311e1d1eb93e19d6a0)
(grafted from 9ada6a6e4ac2a92325592cd58edd9160e17c9e31)
(grafted from 50e676e99c3b7cc929ceaaebbd3c684a8a58f9d8)
(grafted from 01833a49fa4dca204dc0e606f21279530925307c)
(grafted from 301af2e1a42fe912acf90ae9a87ca6a20ce5cd5d)
(grafted from 6ae2eaad3edbfdfb04ac5880a86341e69980529c)
(grafted from fcdedd417b29d28797840fc2393e0ff846fa54c8)
(grafted from ddda3705adfb2ac103f506d694d2b30028dfaca9)
(grafted from 138e3cf3bc394c4ff507341a390e1876c7104042)
(grafted from f8073d595e87086213525dbb642962b84158ee9a)
(grafted from 96bac04dc722030250a53616f0fb55125829f25e)
(grafted from 2aeed48cc3b3324b564343d8515aed1ecec69b14)
(grafted from 4fb2e02a273c868febdae9530b0a07a53a0e92a7)
(grafted from c57b28835f0b880c075d5b8aa99ddb9da54b21b1)
(grafted from 35fd78b021bec96db63e8dd99f98efc3b2342380)
(grafted from 9ac14f96c9a82068f07a709374f359283c206791)
(grafted from 4b64c191a9aacd52ea58ae8ec943605667759398)
(grafted from 1db390a79e32db12dde7a225e26b86ed245f9473)
(grafted from 9a999ed1ce50af8e5fa03dc270488c37304d8c94)
(grafted from b6791f2eb83f176192c9df50c736bc4c54fab5a6)
(grafted from 4b6e87c5be38d9971399e4ad989e4261f283b93a)
(grafted from ae660c075b4af0849d1ff5d36404ef66aeea9933)
(grafted from ff0f3bf0834b38a527654495369cd538ca8744f4)
(grafted from bff8177767c9023295ff93bc520114bb909952a8)
(grafted from 05a833e4071b9da7b447669f6bd8a3f62c1d3c27)
(grafted from be8ab299731fb8295efbe10a014798c7a177d4a0)
(grafted from beec2bc2ebd9eaf7093bba5fca8fef07c669d970)
(grafted from 03d88ba3cd8795d17a99dc1b50ac55e1937d38e1)
(grafted from 92a33bc0d275a96c61553f8bccefcd32f1360931)
(grafted from 3d37833f54e37356f3e32db2ad8fb2ffe4fa14f2)
(grafted from 77fa3393787a9410e14afc26465abb5561253075)
(grafted from 9d908f86cc7986c167cef5cdccaabb565fd2bc04)
(grafted from 0dbc2023c42f72aea6b608e5111725163dcbb64b)
(grafted from 3acf4e9bb718617efaf31abcba583b9b7be2559d)
(grafted from c671696a06e418f5f040427efb3e51fe4c9fa6cd)
(grafted from 309f11e682eb3c6fa497bf767cbcbef3b0dbaf4b)
(grafted from 4f828ef4b70a6a16fe747d5f6393042bff204b5e)
(grafted from 71193e84a71d029dedc744882978285cbe5722e6)
(grafted from 2929832c61c9727bd884f94da5afa29e80334a96)
(grafted from 2ff8a9f1761f82ffa8ebd2a96d86f7de7c712d9c)
(grafted from b438cb1e6cff36e7e197da7669def8a5e528053c)
(grafted from a9ed103481b779af9e160d2b81a9bfe81cc7d173)
(grafted from d139e95d22dc811002dafef1ecaac5dac99825fe)
(grafted from b4e41a9f2c3a6328ada72810407686d11833347c)
(grafted from 2b3826c7b3bdf669b397f1ad31ae106a05d7b05b)
(grafted from 23737fb5e1d6874cf79a1eb841dd1614c0295a1b)
(grafted from 69588396b17d3dafeece8bc9e2101559d871d9fb)
(grafted from b3252a277a53b1657e6bcf31359b413d2becffcc)
(grafted from 12c8e6062d34d4d6cf0b0128084278800a1ed8f7)
(grafted from 8ce5c67748afbe6e82fc3f822e35ddc58cb03694)
(grafted from b0f656426efcf9a70386b1c781507f40ab95af49)
(grafted from 147ae96993dcffbed2f39f31795ef3d60631d43f)
(grafted from 2ecb34a565accf638e6004c59aa5b2d2361f9428)
(grafted from 6bfc53cd4c479b4e789d4086a2c7c2f4045a288f)
(grafted from bf73f92394a079928db6d4b0b3e7aa78448af91a)
(grafted from b69a654342339eb740527fc84af523eb53edeb71)
(grafted from 3f090f44e8a33cde8d5708454a5292d0976269e6)
(grafted from ac4432275d3b750405e53f67b1267579764f4fee)
(grafted from dbfd0bce0eded53dc7d824393f03ddd2f2e693fb)
(grafted from bf7087d072ca6c5d5dac2ddef4c43339d02f6133)
(grafted from b9d63feb8c90f83e74f3e9a89328419c81088082)
(grafted from 718e93a4e545f3e16d09c66f210a567427f1068a)
(grafted from e0ba57c8bf13ccc45b7eaa62d64e03038cd002ad)
(grafted from 5c849011421ad00ef190c2bf15c640656424f681)
(grafted from e833de714167fe6039b42f1cd1890b0470a32ea2)
(grafted from 480872890137130564910a29ed8ef3890810f0c4)
(grafted from 6224dc455a24542cf7d55721fceb14a08e92d391)
(grafted from 24ced5d2b0d6fb837a3994a80ef808e29f62ccc5)
(grafted from 452eb5c8624cc22867fafa692c6c7905e46da27a)
(grafted from cdc9f1b121878c26c986eca2233b5d03ea50ad74)
(grafted from 8b3a45fe3a612fdbee3a1f291f41bfaadfd16a6f)
(grafted from 2a07c0b3cb9785a9f8d5d669b885044e4d4544b1)
(grafted from 56d892df53cfdf3a13f38cd386a437ea59ef0d77)
(grafted from b63c65fad2d28a86a3bc3871d58e45019b11e6a1)
(grafted from 8618bd56f309543d8577000a4310fdf8648f1087)
(grafted from e04c7ddddc5cc40d6347d2336b81d5be2289243e)
(grafted from 5951fe6318d02a9b739f0174f3aecc3d5eead31c)
(grafted from 0f4d380f641a55791ca9eef13cd49da24cf40a7a)
(grafted from d7ecf3376e572d77b670cbe2184370b08d38dcf7)
(grafted from a75534c9e6d7a481303096e44e265593fb5b0b2f)
# skip-blame because this was mechanically rewritten the following script. I
ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t
ones appear to be, and they run without addition failures on both Windows and
Linux.
import argparse
import os
import re
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
globre = re.compile(r'^(.*) \(glob\)(.*)$')
for p in opts.path:
tmp = p + '.tmp'
with open(p, 'rb') as src, open(tmp, 'wb') as dst:
for line in src:
m = globre.match(line)
if not m or '$LOCALIP' in line or '*' in line:
dst.write(line)
continue
if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
dst.write(line)
continue
dst.write(m.group(1) + m.group(2) + '\n')
os.unlink(p)
os.rename(tmp, p)
Before updating the actual bookmark update, we can start with updating the way
we check for push race. Checking bookmarks state earlier is useful even if we
still use pushkey. Aborting before the changegroup is added can save a lot of
time.
Upon pull or unbundle, we display a message with the range of new revisions
fetched. This revision range could readily be used after a pull to look out
what's new with 'hg log'. The algorithm takes care of filtering "obsolete"
revisions that might be present in transaction's "changes" but should not be
displayed to the end user.
--uncompressed isn't a very good name and its description in the
help documentation isn't very useful. We refer to this concept as
"stream clones" in a number of places. I think it makes sense to
change the user-facing argument to use the mode --stream. So this
commit does that.
We keep --uncompressed around for backwards compatibility.
While we're here, we overhaul the help docs for streaming clones
to be somewhat useful.
All tests have been updated to reflect the new preferred --stream
argument. A test for backwards compatibility of --uncompressed has
been added.
.. bc::
`hg clone --stream` should now be used instead of --uncompressed.
--uncompressed is marked as deprecated and is an alias for --stream.
There is no schedule for elimination of --uncompressed.
Differential Revision: https://phab.mercurial-scm.org/D864
A new bundle2 capability 'phases' has been added. If 'heads' is part of the
supported value for 'phases', the server supports reading and sending 'phase-
heads' bundle2 part.
Server is now able to process a 'phases' boolean parameter to 'getbundle'. If
'True', a 'phase-heads' bundle2 part will be included in the bundle with phase
information relevant to the whole pulled set. If this method is available the
phases listkey namespace will no longer be listed.
Beside the more efficient encoding of the data, this new method will greatly
improve the phase exchange efficiency for repositories with non-served
changesets (obsolete, secret) since we'll no longer send data about the
filtered heads.
Add a new 'devel.legacy.exchange' config item to allow fallback to the old
'listkey in bundle2' method.
Reminder: the pulled set is not just the changesets bundled by the pull. It
also contains changeset selected by the "pull specification" on the client
side (eg: everything for bare pull). One of the reason why the 'pulled set' is
important is to make sure we can move -common- nodes to public.
When unquoted, MSYS sees the colon between the drive letter and path as a Unix
path separator and unhelpfully splits on it, feeding only the drive letter as
the command. Much chaos ensues.
I vaguely remember trying to get the test runner to use /letter/path/to/exe
syntax the last time this happened, without success. I doubt a check-code rule
would work, since sometimes it is quoted, and sometimes the quotes are escaped.
These are some simple cases. More to come in a future change.
Reviewers: krbullock
Reviewed By: krbullock
Differential Revision: https://phab.mercurial-scm.org/D4
The python hooks have access to the hook type information. There is not reason
for external hook to not be aware of it too.
For the record my use case is to make sure a hook script is configured for the
right type.
Hooks related to the transaction are aware of the transaction id. By definition
this txn-id is unique and different for each transaction. As a result it can
never be predicted in test and always needs matching. As a result, touching any
like with this data is annoying. We solve the problem once and for all by
installing an automatic replacement. In test, this will now show as:
TXNID=TXN:$ID$
This patch replaces hardcoded 127.0.0.1 with $LOCALIP in all tests.
Till now, the IPv6 series should make tests pass on common IPv6 systems
where the local device has the address "::1" and the hostname "localhost"
resolves to "::1".
Previously Abort raised during 'getbundle' call poorly reported (HTTP-500 for
http, some scary messages for ssh). Abort error have been properly reported for
"push" for a long time, there is not reason to be different for 'getbundle'. We
properly catch such error and report them back the best way available. For
bundle, we issue a valid bundle2 reply (as expected by the client) with an
'error:abort' part. With bundle1 we do as best as we can depending of http or
ssh.
Previously, the capabilities list was protocol agnostic and we
advertised the same capabilities list to all clients, regardless of
transport protocol.
A few capabilities are specific to HTTP. I see no good reason why we
should advertise them to SSH clients. So this patch limits their
advertisement to HTTP clients.
This patch is BC, but SSH clients shouldn't be using the removed
capabilities so there should be no impact.
On Windows platform, invoking printenv.py directly via hook is
problematic, because:
- unless binding between *.py suffix and python runtime, application
selector dialog is displayed, and running test is blocked at each
printenv.py invocations
- it isn't safe to assume binding between *.py suffix and python
runtime, because application binding is easily broken
For example, installing IDE (VisualStudio with Python Tools, or
so) often requires binding between source files and IDE itself.
This patch invokes printenv.py via sh -c for test portability. This is
a kind of follow up for 9e4331825bea, which eliminated explicit
"python" for printenv.py. There are already other 'sh -c "printenv.py"'
in *.t files, and this fix should be reasonable.
This changes were confirmed in cases below:
- without any application binding for *.py suffix
- with binding between *.py suffix and VisualStudio
This patch also replaces "echo + redirection" style with "heredoc"
style, because:
- hook command line is parsed by cmd.exe as shell at first, and
- single quotation can't quote arguments on cmd.exe, therefore,
- "printenv.py foobar" should be quoted by double quotation, but
- nested quoting (or tricky escaping) isn't readable
This patch disables delta chains on changelogs. After this patch, new
entries on changelogs - including existing changelogs - will be stored
as the fulltext of that data (likely compressed). No delta computation
will be performed.
An overview of delta chains and data justifying this change follows.
Revlogs try to store entries as a delta against a previous entry (either
a parent revision in the case of generaldelta or the previous physical
revision when not using generaldelta). Most of the time this is the
correct thing to do: it frequently results in less CPU usage and smaller
storage.
Delta chains are most effective when the base revision being deltad
against is similar to the current data. This tends to occur naturally
for manifests and file data, since only small parts of each tend to
change with each revision. Changelogs, however, are a different story.
Changelog entries represent changesets/commits. And unless commits in a
repository are homogonous (same author, changing same files, similar
commit messages, etc), a delta from one entry to the next tends to be
relatively large compared to the size of the entry. This means that
delta chains tend to be short. How short? Here is the full vs delta
revision breakdown on some real world repos:
Repo % Full % Delta Max Length
hg 45.8 54.2 6
mozilla-central 42.4 57.6 8
mozilla-unified 42.5 57.5 17
pypy 46.1 53.9 6
python-zstandard 46.1 53.9 3
(I threw in python-zstandard as an example of a repo that is homogonous.
It contains a small Python project with changes all from the same
author.)
Contrast this with the manifest revlog for these repos, where 99+% of
revisions are deltas and delta chains run into the thousands.
So delta chains aren't as useful on changelogs. But even a short delta
chain may provide benefits. Let's measure that.
Delta chains may require less CPU to read revisions if the CPU time
spent reading smaller deltas is less than the CPU time used to
decompress larger individual entries. We can measure this via
`hg perfrevlog -c -d 1` to iterate a revlog to resolve each revision's
fulltext. Here are the results of that command on a repo using delta
chains in its changelog and on a repo without delta chains:
hg (forward)
! wall 0.407008 comb 0.410000 user 0.410000 sys 0.000000 (best of 25)
! wall 0.390061 comb 0.390000 user 0.390000 sys 0.000000 (best of 26)
hg (reverse)
! wall 0.515221 comb 0.520000 user 0.520000 sys 0.000000 (best of 19)
! wall 0.400018 comb 0.400000 user 0.390000 sys 0.010000 (best of 25)
mozilla-central (forward)
! wall 4.508296 comb 4.490000 user 4.490000 sys 0.000000 (best of 3)
! wall 4.370222 comb 4.370000 user 4.350000 sys 0.020000 (best of 3)
mozilla-central (reverse)
! wall 5.758995 comb 5.760000 user 5.720000 sys 0.040000 (best of 3)
! wall 4.346503 comb 4.340000 user 4.320000 sys 0.020000 (best of 3)
mozilla-unified (forward)
! wall 4.957088 comb 4.950000 user 4.940000 sys 0.010000 (best of 3)
! wall 4.660528 comb 4.650000 user 4.630000 sys 0.020000 (best of 3)
mozilla-unified (reverse)
! wall 6.119827 comb 6.110000 user 6.090000 sys 0.020000 (best of 3)
! wall 4.675136 comb 4.670000 user 4.670000 sys 0.000000 (best of 3)
pypy (forward)
! wall 1.231122 comb 1.240000 user 1.230000 sys 0.010000 (best of 8)
! wall 1.164896 comb 1.160000 user 1.160000 sys 0.000000 (best of 9)
pypy (reverse)
! wall 1.467049 comb 1.460000 user 1.460000 sys 0.000000 (best of 7)
! wall 1.160200 comb 1.170000 user 1.160000 sys 0.010000 (best of 9)
The data clearly shows that it takes less wall and CPU time to resolve
revisions when there are no delta chains in the changelogs, regardless
of the direction of traversal. Furthermore, not using a delta chain
means that fulltext resolution in reverse is as fast as iterating
forward. So not using delta chains on the changelog is a clear CPU win
for reading operations.
An example of a user-visible operation showing this speed-up is revset
evaluation. Here are results for
`hg perfrevset 'author(gps) or author(mpm)'`:
hg
! wall 1.655506 comb 1.660000 user 1.650000 sys 0.010000 (best of 6)
! wall 1.612723 comb 1.610000 user 1.600000 sys 0.010000 (best of 7)
mozilla-central
! wall 17.629826 comb 17.640000 user 17.600000 sys 0.040000 (best of 3)
! wall 17.311033 comb 17.300000 user 17.260000 sys 0.040000 (best of 3)
What about 00changelog.i size?
Repo Delta Chains No Delta Chains
hg 7,033,250 6,976,771
mozilla-central 82,978,748 81,574,623
mozilla-unified 88,112,349 86,702,162
pypy 20,740,699 20,659,741
The data shows that removing delta chains from the changelog makes the
changelog smaller.
Delta chains are also used during changegroup generation. This
operation essentially converts a series of revisions to one large
delta chain. And changegroup generation is smart: if the delta in
the revlog matches what the changegroup is emitting, it will reuse
the delta instead of recalculating it. We can measure the impact
removing changelog delta chains has on changegroup generation via
`hg perfchangegroupchangelog`:
hg
! wall 1.589245 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! wall 1.788060 comb 1.790000 user 1.790000 sys 0.000000 (best of 6)
mozilla-central
! wall 17.382585 comb 17.380000 user 17.340000 sys 0.040000 (best of 3)
! wall 20.161357 comb 20.160000 user 20.120000 sys 0.040000 (best of 3)
mozilla-unified
! wall 18.722839 comb 18.720000 user 18.680000 sys 0.040000 (best of 3)
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)
pypy
! wall 4.828317 comb 4.830000 user 4.820000 sys 0.010000 (best of 3)
! wall 5.415455 comb 5.420000 user 5.410000 sys 0.010000 (best of 3)
The data shows eliminating delta chains makes the changelog part of
changegroup generation slower. This is expected since we now have to
compute deltas for revisions where we could recycle the delta before.
It is worth putting this regression into context of overall changegroup
times. Here is the rough total CPU time spent in changegroup generation
for various repos while using delta chains on the changelog:
Repo CPU Time (s) CPU Time w/ compression
hg 4.50 7.05
mozilla-central 111.1 222.0
pypy 28.68 75.5
Before compression, removing delta chains from the changegroup adds
~4.4% overhead to hg changegroup generation, 1.3% to mozilla-central,
and 2.0% to pypy. When you factor in zlib compression, these percentages
are roughly divided by 2.
While the increased CPU usage for changegroup generation is unfortunate,
I think it is acceptable because the percentage is small, server
operators (those likely impacted most by this) have other mechanisms
to mitigate CPU consumption (namely reducing zlib compression level and
pre-generated clone bundles), and because there is room to optimize this
in the future. For example, we could use the nullid as the base revision,
effectively encoding the full revision for each entry in the changegroup.
When doing this, `hg perfchangegroupchangelog` nearly halves:
mozilla-unified
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)
! wall 11.196461 comb 11.200000 user 11.190000 sys 0.010000 (best of 3)
This looks very promising as a future optimization opportunity.
It's worth that the changes in test-acl.t to the changegroup part size.
This is because revision 6 in the changegroup had a delta chain of
length 2 before and after this patch the base revision is nullrev.
When the base revision is nullrev, cg2packer.deltaparent() hardcodes
the *previous* revision from the changegroup as the delta parent.
This caused the delta in the changegroup to switch base revisions,
the delta to change, and the size to change accordingly. While the
size increased in this case, I think sizes will remain the same
on average, as the delta base for changelog revisions doesn't matter
too much (as this patch shows). So, I don't consider this a regression.
chg currently does not support hg serve -d. It has a quick path testing if the
command is hg serve -d and fallbacks to hg if so. But the test only works if
"serve" is the first argument since the test wants to avoid false positives
(for example, "-r serve" is different).
This patch reorders "hg server" commands in tests, making them chg friendly.
In order to give us the freedom to change the changegroup3 format,
let's hide it behind an experimental config. Since it is required by
treemanifests, that will override the cg3 config.
Sometimes a txnclose or changegroup hook wants to iterate through all
the changesets in transaction: in that situation usually the revset
`$HG_NODE:` is used to select the revisions. Unfortunately this revset
sometimes may contain too many changesets because we don't have the
write lock while the hook runs newer changes may be added to
repository in the meantime.
That's why there is a need for extra variable carrying the information about
the last change in the transaction.
I'm not entirely happy with using a trailing / on a "file" entry for
transferring a treemanifest. We've discussed putting some flags on
each file header[0], but I'm unconvinced that's actually any better:
if we were going to add another feature to the cg format we'd still be
doing a version bump anyway to cg4, so I'm inclined to not spend time
coming up with a more sophisticated format until we actually know what
the next feature we want to stuff in a changegroup will be.
Test changes outside test-treemanifest.t are only due to the new CG3
bundlecap showing up in the wire protocol.
Many thanks to adgar@google.com and martinvonz@google.com for helping
me with various odd corners of the changegroup and treemanifest API.
0: It's not hard refactoring, nor is it a lot of work. I'm just
disinclined to do speculative work when it's not clear what the
customer would actually be.
The other invocations aren't quoted, and Windows doesn't like the single quotes:
diff --git a/tests/test-ssh.t b/tests/test-ssh.t
--- a/tests/test-ssh.t
+++ b/tests/test-ssh.t
@@ -520,20 +520,8 @@ remote hook failure is attributed to rem
$ echo "pretxnchangegroup.fail = python:$TESTTMP/failhook:hook" >> remote/.hg/hgrc
$ hg -q --config ui.ssh="python '$TESTDIR/dummyssh'" clone ssh://user@dummy/remote hookout
+ abort: no suitable response from remote hg!
+ [255]
$ cd hookout
+ $TESTTMP.sh: line 264: cd: hookout: No such file or directory
$ touch hookfailure
- $ hg -q commit -A -m 'remote hook failure'
....
Before bundle2, hook output from hook failures was prefixed with
"remote: ". Up to this point with bundle2, the output was converted to
the message to print in an Abort exception. This had 2 implications:
1) It was unclear whether an error message came from the local repo
or the remote
2) The exit code changed from 1 to 255
This patch changes the handling of error:abort bundle2 parts during push
to prefix the error message with "remote: ". This restores the old
behavior.
We still preserve the behavior of raising an Abort during bundle2
application failure. This is a regression from pre-bundle2 because the
exit code changed.
Because we no longer raise an Abort with the remote's message, we needed
to insert a message for the new Abort. So, I invented a new error
message for that. This is another change from pre-bundle2. However, I
like the new error message because it states unambiguously who aborted
the push failed, which I think is important for users so they can decide
what's next.
The added tests don't agree in their output. This demonstrates a
difference in `hg push` behavior between pre-bundle2 and bundle2.
A subsequent patch will attempt to restore some of the pre-bundle2
behavior to bundle2.
Before this patch, repo could be set to None for wrong -R. It's okay for
commands that can reject repo=None, but the command server have a problem
because:
- it accepts repo=None for "unbound" mode
- and it reenters dispatch() where repo object is created for cwd by default
Test outputs are changed because the error is detected earlier. I think new
message is better than ".hg not found".
This is a backout of 93589179c542, and a partial backout of 9b1628b91e74.
Windows won't execute 'dummyssh' directly, presumably because CreateProcess()
doesn't know how to execute a bash script:
$ hg clone -e "dummyssh" ssh://user@dummy/cloned sshclone
remote: 'dummyssh' is not recognized as an internal or external command,
remote: operable program or batch file.
abort: no suitable response from remote hg!
[255]
With the restoration of python as the executable, $TESTDIR needs to be restored
for these invocations, because python won't search $PATH for 'dummyssh':
$ hg clone -e "python dummyssh" ssh://user@dummy/cloned sshclone
remote: python: can't open file 'dummyssh': [Errno 2] No such file or directory
abort: no suitable response from remote hg!
[255]
.hgtags fnodes cache entries can be expensive to compute, especially
if there are hundreds of even thousands of them. This patch implements
support for receiving a bundle2 part that contains a mapping of
changeset to .hgtags fnodes.
An upcoming patch will teach the server to send this part, allowing
clients to bypass having to redundantly compute these values.
A number of tests changed due to the client advertising the "hgtagsfnodes"
capability.
We want to keep both code paths tested. The test is a bit too extensive to
simply introduce dual testing in it so we make a copy for each protocol
version.
