(grafted from 9e9ff559440e4733fda49ddeb1b9ce5e7c9660af)
(grafted from b14a9cd166ad6e48284ad887a7a47537ab4b1df4)
(grafted from 749be17f25a14a0a91d9e68e68df77a1b305eb7f)
(grafted from fd3caadd0a3f365551334f346875799964056354)
(grafted from ce1070bbc53c18d0afa9ba6fe000588c4fb3cf33)
(grafted from 834c5948156ceb0be31c40e95293277d9648f423)
(grafted from 24ec6b8d9402b53d38f9763b4706afaca4fb5b20)
(grafted from 0104f253d24f416e8034f302fd2bd57ec2cdbd3e)
(grafted from 893aadeea79010ca5e039a6bfaaca7471f7d9061)
(grafted from dc447bcdf1499da52b43723426aefd4b1cec4c01)
(grafted from 2db9cf55d1566b7273342b711073a57e075b11f3)
This should be effective to reduce the number of chg servers on OS X.
(grafted from 64cc416c50a4c54dfcebdbdcea73ea0a4dcc6b12)
(grafted from 4d35490df4c269132c43e5783d12697735ef1eae)
(grafted from 0f739f622491de9c6f7cfe3d8949843923291d95)
(grafted from ae11921088ae3ec06726719c7037ce7e9fb204a1)
(grafted from c1d14170449337f008d0f09e9c940309021236c8)
(grafted from e5d909b77f32f7d2e36c76cc318876e999d8cffe)
(grafted from 50a780fc7fb2ac422250d4aad426e0d010f5a290)
(grafted from f66d8cc7d1a72ece27e4ec682fa9e2401a50e2d2)
(grafted from a2fae201970502c50a7240a1fd8a597cc83f9db9)
(grafted from a102b2c1deb978bfc69be02730f9685062835691)
(grafted from dd3133eae2ba7268310115196c53135f0ccd3087)
If this feature is enabled, early options are parsed using the global options
table. As the parser stops processing options when non/unknown option is
encountered, it won't mistakenly take an option value as a new early option.
Still "--" can be injected to terminate the parsing (e.g. "hg -R -- log"), I
think it's unlikely to lead to an RCE.
To minimize a risk of this change, new fancyopts.earlygetopt() path is enabled
only when +strictflags is set. Also the strict parser doesn't support '--repo',
a short for '--repository' yet. This limitation will be removed later.
As this feature is backward incompatible, I decided to add a new opt-in
mechanism to HGPLAIN. I'm not pretty sure if this is the right choice, but
I'm thinking of adding +feature/-feature syntax to HGPLAIN. Alternatively,
we could add a new environment variable. Any bikeshedding is welcome.
Note that HGPLAIN=+strictflags doesn't work correctly in chg session since
command arguments are pre-processed in C. This wouldn't be easily fixed.
So we can easily compare it with the corresponding getopt() result.
There's a minor behavior change. Before, "hg --cwd ''" failed with ENOENT.
But with this patch, an empty cwd is silently ignored. "hg -R ''" has always
worked as such, so -R has no BC.
When `$CHGHG` is set, `$HG` is ignored by the chg client. Removing it from
chg's sensitive environment list would avoid starting up servers
unnecessarily when `$CHGHG` is the same while `$HG` is different.
Differential Revision: https://phab.mercurial-scm.org/D1177
The show extension reads `commands.show.aliasprefix` config in its
`extsetup` and that causes issues with chg. This patch adds that config item
to chg confighash to solve the issue.
Test Plan:
`run-tests.py -l --chg test-show.t`
Differential Revision: https://phab.mercurial-scm.org/D1158
The eol extension may mangle the [eol] config section and that means chg is
unable to detect config file change (because it re-applies setconfig
changes).
This makes test-eol.t pass with chg.
Differential Revision: https://phab.mercurial-scm.org/D917
In chg's case, making modules lazily loaded could actually slow down things
since chg pre-imports them. Therefore disable demandimport if chg is being
used.
This is not done by setting `HGDEMANDIMPORT` chg client-side because that
has side-effects on child processes (hooks, etc).
Differential Revision: https://phab.mercurial-scm.org/D351
This is done by a script [2] using RedBaron [1], a tool designed for doing
code refactoring. All "default" values are decided by the script and are
strongly consistent with the existing code.
There are 2 changes done manually to fix tests:
[warn] mercurial/exchange.py: experimental.bundle2-output-capture: default needs manual removal
[warn] mercurial/localrepo.py: experimental.hook-track-tags: default needs manual removal
Since RedBaron is not confident about how to indent things [2].
[1]: https://github.com/PyCQA/redbaron
[2]: https://github.com/PyCQA/redbaron/issues/100
[3]:
#!/usr/bin/env python
# codemod_configitems.py - codemod tool to fill configitems
#
# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import, print_function
import os
import sys
import redbaron
def readpath(path):
with open(path) as f:
return f.read()
def writepath(path, content):
with open(path, 'w') as f:
f.write(content)
_configmethods = {'config', 'configbool', 'configint', 'configbytes',
'configlist', 'configdate'}
def extractstring(rnode):
"""get the string from a RedBaron string or call_argument node"""
while rnode.type != 'string':
rnode = rnode.value
return rnode.value[1:-1] # unquote, "'str'" -> "str"
def uiconfigitems(red):
"""match *.ui.config* pattern, yield (node, method, args, section, name)"""
for node in red.find_all('atomtrailers'):
entry = None
try:
obj = node[-3].value
method = node[-2].value
args = node[-1]
section = args[0].value
name = args[1].value
if (obj in ('ui', 'self') and method in _configmethods
and section.type == 'string' and name.type == 'string'):
entry = (node, method, args, extractstring(section),
extractstring(name))
except Exception:
pass
else:
if entry:
yield entry
def coreconfigitems(red):
"""match coreconfigitem(...) pattern, yield (node, args, section, name)"""
for node in red.find_all('atomtrailers'):
entry = None
try:
args = node[1]
section = args[0].value
name = args[1].value
if (node[0].value == 'coreconfigitem' and section.type == 'string'
and name.type == 'string'):
entry = (node, args, extractstring(section),
extractstring(name))
except Exception:
pass
else:
if entry:
yield entry
def registercoreconfig(cfgred, section, name, defaultrepr):
"""insert coreconfigitem to cfgred AST
section and name are plain string, defaultrepr is a string
"""
# find a place to insert the "coreconfigitem" item
entries = list(coreconfigitems(cfgred))
for node, args, nodesection, nodename in reversed(entries):
if (nodesection, nodename) < (section, name):
# insert after this entry
node.insert_after(
'coreconfigitem(%r, %r,\n'
' default=%s,\n'
')' % (section, name, defaultrepr))
return
def main(argv):
if not argv:
print('Usage: codemod_configitems.py FILES\n'
'For example, FILES could be "{hgext,mercurial}/*/**.py"')
dirname = os.path.dirname
reporoot = dirname(dirname(dirname(os.path.abspath(__file__))))
# register configitems to this destination
cfgpath = os.path.join(reporoot, 'mercurial', 'configitems.py')
cfgred = redbaron.RedBaron(readpath(cfgpath))
# state about what to do
registered = set((s, n) for n, a, s, n in coreconfigitems(cfgred))
toregister = {} # {(section, name): defaultrepr}
coreconfigs = set() # {(section, name)}, whether it's used in core
# first loop: scan all files before taking any action
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
iscore = ('mercurial' in path) and ('hgext' not in path)
red = redbaron.RedBaron(readpath(path))
# find all repo.ui.config* and ui.config* calls, and collect their
# section, name and default value information.
for node, method, args, section, name in uiconfigitems(red):
if section == 'web':
# [web] section has some weirdness, ignore them for now
continue
defaultrepr = None
key = (section, name)
if len(args) == 2:
if key in registered:
continue
if method == 'configlist':
defaultrepr = 'list'
elif method == 'configbool':
defaultrepr = 'False'
else:
defaultrepr = 'None'
elif len(args) >= 3 and (args[2].target is None or
args[2].target.value == 'default'):
# try to understand the "default" value
dnode = args[2].value
if dnode.type == 'name':
if dnode.value in {'None', 'True', 'False'}:
defaultrepr = dnode.value
elif dnode.type == 'string':
defaultrepr = repr(dnode.value[1:-1])
elif dnode.type in ('int', 'float'):
defaultrepr = dnode.value
# inconsistent default
if key in toregister and toregister[key] != defaultrepr:
defaultrepr = None
# interesting to rewrite
if key not in registered:
if defaultrepr is None:
print('[note] %s: %s.%s: unsupported default'
% (path, section, name))
registered.add(key) # skip checking it again
else:
toregister[key] = defaultrepr
if iscore:
coreconfigs.add(key)
# second loop: rewrite files given "toregister" result
for path in argv:
# reconstruct redbaron - trade CPU for memory
red = redbaron.RedBaron(readpath(path))
changed = False
for node, method, args, section, name in uiconfigitems(red):
key = (section, name)
defaultrepr = toregister.get(key)
if defaultrepr is None or key not in coreconfigs:
continue
if len(args) >= 3 and (args[2].target is None or
args[2].target.value == 'default'):
try:
del args[2]
changed = True
except Exception:
# redbaron fails to do the rewrite due to indentation
# see https://github.com/PyCQA/redbaron/issues/100
print('[warn] %s: %s.%s: default needs manual removal'
% (path, section, name))
if key not in registered:
print('registering %s.%s' % (section, name))
registercoreconfig(cfgred, section, name, defaultrepr)
registered.add(key)
if changed:
print('updating %s' % path)
writepath(path, red.dumps())
if toregister:
print('updating configitems.py')
writepath(cfgpath, cfgred.dumps())
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Environment variables like HGUSER, HGEDITOR, HGEDITFROM should not trigger
a new chgserver. This patch uses a whitelist for environ variables starting
with "HG" to reduce the number of servers.
I have went through `grep -o "[\"']HG[A-Z_0-9]*['\"]" -hR . | sort -u` so
the list should be up-to-date.
This enables chg to replace a server socket in an atomic way:
1. bind to a temp address
2. listen
3. rename
Currently 3 happens before 2 so a client may see the socket file but fails
to connect to it.
See the previous commit for why. Marked as API change since osutil.listdir()
seems widely used in third-party extensions.
The win32mbcs extension is updated to wrap both util. and windows. aliases.
When --pager=on is given, dispatch.py spawns a pager before setting up color.
If the pager failed to launch, ui.pageractive was left set to True, so color
configured itself based on 'color.pagermode'. A typical MSYS setting would be
'color.mode=auto, color.pagermode=ansi'. In the failure case, this would print
a warning, disable the pager, and then print the raw ANSI codes to the terminal.
Care needs to be taken, because it appears that leaving ui.pageractive=True was
the only thing that prevented an attempt at running the pager again from inside
the command. This results in a double warning message, so pager is simply
disabled on failure.
The ui config settings didn't need to be moved to fix this, but it seemed like
the right thing to do for consistency.
Since a33895509ccb changed the signature of ui.system(), chgui.system()
should have been updated. This patch factors out the util.system() call
so that chg can override how a shell command is executed.
os.fdopen() does not accepts bytes as its second argument which represent the
mode in which the file is to be opened. This patch makes sure unicodes are
passed in py3 by using pycompat.sysstr().
We have enough bits to switch to the new chg pager code path in runcommand.
So just remove the legacy getpager support.
This is a red-only patch, and will break chg's pager support temporarily.
This patch implements chgui._runpager in a relatively simple way. A more
clean way is to move the core logic of "attachio" to "ui", which will be
done later after chg runs uisetup per request.
This patch adds the "pager" support for the S channel. The pager API allows
running some subcommands, namely attachio, and waiting for the client to be
properly synchronized.
Previously S channel is only used to send system commands. It will also be
used to send pager commands. So add a type parameter.
This breaks older chg clients. But chg and hg should always come from a
single commit and be packed into a single package. Supporting running
inconsistent versions of chg and hg seems to be unnecessarily complicated
with little benefit. So just make the change and assume people won't use
inconsistent chg with hg.
On Wed, 21 Dec 2016 15:39:05 +0000, Jun Wu wrote:
> Actually, patch 1 is unnecessary if we go with the "ui._runpager" approach.
> Maybe someone can drop it without adding too many markers.
Previously, the "system" channel is inside the ui object. In the future, chg
will let dispatch to create a new ui object from scratch, to maximize
compatibility. And chgserver will use a "uisetup" like an extension to wrap
ui.system. To be able to do that cleanly, the system channel needs to be
accessed directly.
Previously, the hash address is just appending "-$HASH" to base address.
This patch makes it truncate the basename address at "." before appending
"-$HASH".
This makes it possible to spawn new servers in a racy situation and the
client could be sure the server it connects is the new server just spawned.
This is a step towards removing the lock.
One of the functionalities of the lock is to make sure the connect will
connect to a server it just created:
1. start server --address foo
2. connect to foo # wish "foo" is the server just started
With this change, the client could do:
1. start server --address foo.tmp$PID
2. connect to foo.tmp$PID # is the server just started
(note: if it is not, it does not affect correctness - linux pid
namespace is not a concern here)
3. rename foo.tmp$PID to foo
Another functionality of the lock is to avoid starting multiple servers with
a same confighash in parallel. But that also prevents starting multiple
servers with different confighashes in parallel.
The environment variables `HG_*` are usually used by hooks. Unlike `HGPLAIN`
etc, they do not actually affect hg's behavior. So do not include them in
confighash.
This would avoid spawning an unbound number of chg server processes if
commit hook calls hg frequently.
It was an extension just because there were several dependency cycles I
needed to address.
I don't add 'chgserver' to extensions._builtin since chgserver is considered
an internal extension so nobody should enable it by their config.
