Commit Graph

404 Commits

Author SHA1 Message Date
Jun Wu
14783221f6 hint: add a simple framework for registering hint messages
Summary:
This allows us to have a unified way to print hint messages at the end of a
command. It would be helpful for feature discovery in general.

Reviewed By: mjpieters

Differential Revision: D7392132

fbshipit-source-id: 8b4e94cc2176266652459ecca3428bd86d95bfe2
2018-04-13 21:51:48 -07:00
Durham Goode
df8173b17b hg: add --configfile global option
Summary:
This adds the ability to specify a config file to be used during the
command. This is useful during clones for letting the clone command use the
given repositories system specified repo-specific hgrc file.

Reviewed By: quark-zju

Differential Revision: D7311576

fbshipit-source-id: a97d8ebada2e0bea27c75a7650df8ede00dc10c6
2018-04-13 21:51:31 -07:00
Jun Wu
b7eb2e64e3 mdiff: use xdiff for diff calculation
Summary:
Let's switch to xdiff for its better diff quality and performance!

The test changes demonstrate xdiff's better diff quality.

Reviewed By: ryanmce

Differential Revision: D7135206

fbshipit-source-id: 1775df6fc0f763df074b4f52779835d6ef0f3a4e
2018-04-13 21:51:21 -07:00
Stanislau Hlebik
584943bdd6 fix show failure if alias is set
Summary: dispatch.lazyaliasentry didn't work with slices i.e. lazyasliasentry[:2] failed. This diff fixes it

Reviewed By: farnz

Differential Revision: D7109779

fbshipit-source-id: c704cd44fea0944ae4be68df36d32df98b7fc09b
2018-04-13 21:51:19 -07:00
Jun Wu
d56ebc52af buck: fix tests with opt build
Summary: D6937883 turns out to be ineffective. Let's fix it in another way.

Reviewed By: DurhamG

Differential Revision: D6940006

fbshipit-source-id: 082d0ddcf464d7056426bde2feab52a7717eb057
2018-04-13 21:51:09 -07:00
Jun Wu
2946a1c198 codemod: use single blank line
Summary: This makes test-check-code cleaner.

Reviewed By: ryanmce

Differential Revision: D6937934

fbshipit-source-id: 8f92bc32f75b9792ac67db77bb3a8756b37fa941
2018-04-13 21:51:08 -07:00
Kostia Balytskyi
662ade9719 hg: make sure blocked times logged during atexit handlers are logged
Summary:
Previosuly, we would miss anything logged in the `ui.atexit`-registered
handler, since the actual logging would happen before those handlers were
called.

Reviewed By: quark-zju

Differential Revision: D6912321

fbshipit-source-id: 77600b7ae535b4da56fef1f92b51998de8e304e2
2018-04-13 21:51:06 -07:00
Jun Wu
f623fe9d3c main: add mercurial.main as an entry point
Summary:
The old mercurial.dispatch entry point works, except it breaks
extensions wrapping `dispatch.*` (ex. `dispatch.runcommand`).

That is because things happen in this order:
- Buck's python wrapper imports `mercurial.dispatch` as mod1.
- hgdemandimport is enabled.
- An extension imports `mercurial.dispatch` and it got mod2.
- mod2 != mod1. So any changes to mod2 won't be used.

Using a separate module (`mercurial.main`) as the entry point
solves the problem.

Reviewed By: DurhamG

Differential Revision: D6879873

fbshipit-source-id: 9779edac64bbeb53dd98dfab1b21575c3a60af01
2018-04-13 21:51:02 -07:00
Jun Wu
d942f5a88e hg: basic support for building hg using buck
Summary:
Adds some basic building blocks to build hg using buck.

Header files are cleaned up, so they are relative to the project root.

Some minor changes to C code are made to remove clang build
warnings.

Rust dependencies, fb-hgext C/Python dependencies (ex. cstore,
mysql-connector), and 3rd-party dependencies like python-lz4
are not built yet. But the built hg binary should be able to run
most tests just fine.

Reviewed By: wez

Differential Revision: D6814686

fbshipit-source-id: 59eefd5a3ad86db2ad1c821ed824c9f1878c93e4
2018-04-13 21:50:58 -07:00
Phil Cohen
72085d2e96 testdir: fix test-extension.t for DEFAULT_EXTENSIONS
Differential Revision: https://phabricator.intern.facebook.com/D6718663
2018-01-14 14:53:52 -08:00
Durham Goode
f13c34a644 serve: move hg-ssh readonly logic into hg serve
Recently hg-ssh was changed to block writes via in-memory hook configuration
instead of by passing config hooks, and dispatch.py blocks any invocation of hg
serve --stdio that has options passed. We have infrastructure that sets up read
only serve processes without using hg-ssh, and it was broken by this change.

Let's add a --read-only option to hg serve so non-hg-ssh solutions can still
launch hg in read-only mode. This makes it also work with non-stdio serve
processes as well.
(grafted from 7a0ed9aad8526f689343a5a02aa4a66e5f3be1f2)
(grafted from bf33f750447d8b0dfeae2a311e1d1eb93e19d6a0)
(grafted from 9ada6a6e4ac2a92325592cd58edd9160e17c9e31)
(grafted from 50e676e99c3b7cc929ceaaebbd3c684a8a58f9d8)
(grafted from 01833a49fa4dca204dc0e606f21279530925307c)
(grafted from 301af2e1a42fe912acf90ae9a87ca6a20ce5cd5d)
(grafted from 6ae2eaad3edbfdfb04ac5880a86341e69980529c)
(grafted from fcdedd417b29d28797840fc2393e0ff846fa54c8)
(grafted from ddda3705adfb2ac103f506d694d2b30028dfaca9)
(grafted from 138e3cf3bc394c4ff507341a390e1876c7104042)
(grafted from f8073d595e87086213525dbb642962b84158ee9a)
(grafted from 96bac04dc722030250a53616f0fb55125829f25e)
(grafted from 2aeed48cc3b3324b564343d8515aed1ecec69b14)
(grafted from 4fb2e02a273c868febdae9530b0a07a53a0e92a7)
(grafted from c57b28835f0b880c075d5b8aa99ddb9da54b21b1)
(grafted from 35fd78b021bec96db63e8dd99f98efc3b2342380)
(grafted from 9ac14f96c9a82068f07a709374f359283c206791)
(grafted from 4b64c191a9aacd52ea58ae8ec943605667759398)
(grafted from 1db390a79e32db12dde7a225e26b86ed245f9473)
(grafted from 9a999ed1ce50af8e5fa03dc270488c37304d8c94)
(grafted from b6791f2eb83f176192c9df50c736bc4c54fab5a6)
(grafted from 4b6e87c5be38d9971399e4ad989e4261f283b93a)
(grafted from ae660c075b4af0849d1ff5d36404ef66aeea9933)
(grafted from ff0f3bf0834b38a527654495369cd538ca8744f4)
(grafted from bff8177767c9023295ff93bc520114bb909952a8)
(grafted from 05a833e4071b9da7b447669f6bd8a3f62c1d3c27)
(grafted from be8ab299731fb8295efbe10a014798c7a177d4a0)
(grafted from beec2bc2ebd9eaf7093bba5fca8fef07c669d970)
(grafted from 03d88ba3cd8795d17a99dc1b50ac55e1937d38e1)
(grafted from 92a33bc0d275a96c61553f8bccefcd32f1360931)
(grafted from 3d37833f54e37356f3e32db2ad8fb2ffe4fa14f2)
(grafted from 77fa3393787a9410e14afc26465abb5561253075)
(grafted from 9d908f86cc7986c167cef5cdccaabb565fd2bc04)
(grafted from 0dbc2023c42f72aea6b608e5111725163dcbb64b)
(grafted from 3acf4e9bb718617efaf31abcba583b9b7be2559d)
(grafted from c671696a06e418f5f040427efb3e51fe4c9fa6cd)
(grafted from 309f11e682eb3c6fa497bf767cbcbef3b0dbaf4b)
(grafted from 4f828ef4b70a6a16fe747d5f6393042bff204b5e)
(grafted from 71193e84a71d029dedc744882978285cbe5722e6)
(grafted from 2929832c61c9727bd884f94da5afa29e80334a96)
(grafted from 2ff8a9f1761f82ffa8ebd2a96d86f7de7c712d9c)
(grafted from b438cb1e6cff36e7e197da7669def8a5e528053c)
(grafted from a9ed103481b779af9e160d2b81a9bfe81cc7d173)
(grafted from d139e95d22dc811002dafef1ecaac5dac99825fe)
(grafted from b4e41a9f2c3a6328ada72810407686d11833347c)
(grafted from 2b3826c7b3bdf669b397f1ad31ae106a05d7b05b)
(grafted from 23737fb5e1d6874cf79a1eb841dd1614c0295a1b)
(grafted from 69588396b17d3dafeece8bc9e2101559d871d9fb)
(grafted from b3252a277a53b1657e6bcf31359b413d2becffcc)
(grafted from 12c8e6062d34d4d6cf0b0128084278800a1ed8f7)
(grafted from 8ce5c67748afbe6e82fc3f822e35ddc58cb03694)
(grafted from b0f656426efcf9a70386b1c781507f40ab95af49)
(grafted from 147ae96993dcffbed2f39f31795ef3d60631d43f)
(grafted from 2ecb34a565accf638e6004c59aa5b2d2361f9428)
(grafted from 6bfc53cd4c479b4e789d4086a2c7c2f4045a288f)
(grafted from bf73f92394a079928db6d4b0b3e7aa78448af91a)
(grafted from b69a654342339eb740527fc84af523eb53edeb71)
(grafted from 3f090f44e8a33cde8d5708454a5292d0976269e6)
(grafted from ac4432275d3b750405e53f67b1267579764f4fee)
(grafted from dbfd0bce0eded53dc7d824393f03ddd2f2e693fb)
(grafted from bf7087d072ca6c5d5dac2ddef4c43339d02f6133)
(grafted from b9d63feb8c90f83e74f3e9a89328419c81088082)
(grafted from 718e93a4e545f3e16d09c66f210a567427f1068a)
(grafted from e0ba57c8bf13ccc45b7eaa62d64e03038cd002ad)
(grafted from 5c849011421ad00ef190c2bf15c640656424f681)
(grafted from e833de714167fe6039b42f1cd1890b0470a32ea2)
(grafted from 480872890137130564910a29ed8ef3890810f0c4)
(grafted from 6224dc455a24542cf7d55721fceb14a08e92d391)
(grafted from 24ced5d2b0d6fb837a3994a80ef808e29f62ccc5)
(grafted from 452eb5c8624cc22867fafa692c6c7905e46da27a)
(grafted from cdc9f1b121878c26c986eca2233b5d03ea50ad74)
(grafted from 8b3a45fe3a612fdbee3a1f291f41bfaadfd16a6f)
(grafted from 2a07c0b3cb9785a9f8d5d669b885044e4d4544b1)
(grafted from 56d892df53cfdf3a13f38cd386a437ea59ef0d77)
(grafted from b63c65fad2d28a86a3bc3871d58e45019b11e6a1)
(grafted from 8618bd56f309543d8577000a4310fdf8648f1087)
(grafted from e04c7ddddc5cc40d6347d2336b81d5be2289243e)
(grafted from 5951fe6318d02a9b739f0174f3aecc3d5eead31c)
(grafted from 0f4d380f641a55791ca9eef13cd49da24cf40a7a)
(grafted from d7ecf3376e572d77b670cbe2184370b08d38dcf7)
(grafted from a75534c9e6d7a481303096e44e265593fb5b0b2f)
2018-01-03 05:35:56 -08:00
Pulkit Goyal
616684d4c9 py3: handle keyword arguments correctly in dispatch.py
Differential Revision: https://phab.mercurial-scm.org/D1630
2017-12-10 04:46:13 +05:30
Yuya Nishihara
86043c7487 dispatch: replace _earlygetopt(strip=True) with new parser
The execution order in cmdalias.__init__() is adjusted to set stripped args
to self.givenargs, which is no longer updated in place.
2017-11-23 23:18:56 +09:00
Yuya Nishihara
c2c866a852 dispatch: replace _earlyreq*() with new fancyopts-based parser 2017-11-23 22:23:59 +09:00
Yuya Nishihara
2e53122de6 dispatch: alias --repo to --repository while parsing early options
This prepares for replacing old _early*opt() functions. My initial attempt
was to extend options table to support 'repository|repo' syntax. It worked,
but seemed too invasive. So I decided to add an optional argument to
fancyopts() instead.

This also changes the nevernegate dict to be keyed by a canonical_name,
not by an option-name for clarity.
2017-11-25 17:03:52 +09:00
Kevin Bullock
c86342f882 merge with stable 2017-12-01 15:21:05 -06:00
Yuya Nishihara
5ff5d9b38c dispatch: add HGPLAIN=+strictflags to restrict early parsing of global options
If this feature is enabled, early options are parsed using the global options
table. As the parser stops processing options when non/unknown option is
encountered, it won't mistakenly take an option value as a new early option.
Still "--" can be injected to terminate the parsing (e.g. "hg -R -- log"), I
think it's unlikely to lead to an RCE.

To minimize a risk of this change, new fancyopts.earlygetopt() path is enabled
only when +strictflags is set. Also the strict parser doesn't support '--repo',
a short for '--repository' yet. This limitation will be removed later.

As this feature is backward incompatible, I decided to add a new opt-in
mechanism to HGPLAIN. I'm not pretty sure if this is the right choice, but
I'm thinking of adding +feature/-feature syntax to HGPLAIN. Alternatively,
we could add a new environment variable. Any bikeshedding is welcome.

Note that HGPLAIN=+strictflags doesn't work correctly in chg session since
command arguments are pre-processed in C. This wouldn't be easily fixed.
2017-11-23 22:17:03 +09:00
Yuya Nishihara
52b06fe73d dispatch: verify result of early command parsing
Before, early options were stripped from args, and because of this, some
kind of parsing errors weren't reported. For example,

  $ hg ci -m -Ra file

would execute "hg ci -m file" in repository "a".

This patch fixes the issue by parsing early options again by real getopt-based
parser, and verifying the results. If the early parsing appears wrong, hg just
aborts. The current error message seems not nice, and should be improved, maybe
in V2 or follow-up.

Note that this isn't a security feature because we can still do anything by
using shell aliases.
2017-11-11 12:40:13 +09:00
Yuya Nishihara
fccba4aa77 dispatch: convert non-list option parsed by _earlygetopt() to string
So we can easily compare it with the corresponding getopt() result.

There's a minor behavior change. Before, "hg --cwd ''" failed with ENOENT.
But with this patch, an empty cwd is silently ignored. "hg -R ''" has always
worked as such, so -R has no BC.
2017-11-11 17:55:15 +09:00
Yuya Nishihara
f9dd555dd4 dispatch: add option to not strip command args parsed by _earlygetopt()
This allows us to parse the original args later by full-blown getopt() in
order to verify the result of the faulty early parsing. Still we need the
'strip=True' behavior for shell aliases.

Note that this series is RFC because it seems to change too much to be
included in stable release.
2017-11-11 16:46:41 +09:00
Yuya Nishihara
333fc1d401 dispatch: fix early parsing of short option with value like -R=foo
Before, -R=foo was parsed as '-R' 'foo', which disagrees with the standard
getopt behavior.
2017-11-14 00:25:59 +09:00
Yuya Nishihara
82ae4401fe dispatch: abort if early boolean options can't be parsed
Perhaps we'll need to restrict the parsing rules of --debugger and --profile,
where this patch will help us know why the --debugger option doesn't work.

I have another series to extend this feature to --config/--cwd/-R, but even
with that, shell aliases can be used to get around the restriction.
2017-11-11 14:02:41 +09:00
Yuya Nishihara
8f01395e02 dispatch: stop parsing of early boolean option at "--" 2017-11-10 22:27:26 +09:00
Yuya Nishihara
01e537d4b9 dispatch: extract stub function to peek boolean command option
We should at least stop parsing at "--". The 'name' argument is passed for
future extension.
2017-11-10 22:22:39 +09:00
Yuya Nishihara
7137d3d976 dispatch: do not drop unpaired argument at _earlygetopt()
Before, "hg log -R" just worked.
2017-11-11 12:09:19 +09:00
Augie Fackler
da28f5be4e merge with stable 2017-11-30 15:48:42 -05:00
Pulkit Goyal
044c1aa613 py3: use pycompat.maplist() instead of map()
Differential Revision: https://phab.mercurial-scm.org/D1558
2017-11-29 08:40:58 +05:30
Pulkit Goyal
134a289843 py3: add b'' to regular expressions which are raw strings
Differential Revision: https://phab.mercurial-scm.org/D1538
2017-11-29 04:47:27 +05:30
Pulkit Goyal
aaa435b086 registrar: add support for storing the type of command in func object
This patch adds support for storing the type of command which is going to run in
the func object. For this it does the following:

1) Add three possible values as attributes to the registrar.command class
2) Add a new argument to registrar.command._doregister function
3) Add a new attribute cmdtype to the func object

The type of command will be helpful in deciding what level of access on hidden
commits it can has.

Differential Revision: https://phab.mercurial-scm.org/D736
2017-09-20 04:47:43 +05:30
Jun Wu
c0f8669061 dispatch: when --pager=no is passed, also disable pager on req.repo.ui
With a future chg change, `req.repo` could be set and currently it is
unaffected by `--pager=on`. This patch makes it so.

This could make one of the test cases in `test-pager.t` pass with future chg
changes.

Differential Revision: https://phab.mercurial-scm.org/D990
2017-10-09 12:42:28 -07:00
Yuya Nishihara
f412c69557 dispatch: move initialization of sys.std* files
I'll add another Python 3 hack.
2017-10-02 07:18:24 +01:00
Yuya Nishihara
5362d4999c py3: work around the scope of exception variable in dispatch.run()
https://stackoverflow.com/questions/29268892/
2017-10-02 06:52:10 +01:00
Alex Gaynor
88c328dbf9 style: never use a space before a colon or comma
Differential Revision: https://phab.mercurial-scm.org/D954
2017-09-29 15:48:34 +00:00
Jun Wu
d22993888a alias: make alias command lazily resolved
With many aliases, resolving them could have some visible overhead. Below is
part of traceprof [1] output of `hg bookmark --hidden`:

  (time unit: ms)
  37  \ addaliases                             dispatch.py:526
  37   | __init__ (60 times)                   dispatch.py:402
  33   | findcmd (108 times)                   cmdutil.py:721
  16   | findpossible (49 times)               cmdutil.py:683

It may get better by optimizing `findcmd` to do a bisect, but we don't
really need to resolve an alias if it's not used, so let's make those
command entries lazy.

After this patch, `addalias` takes less than 1ms.

.. perf:: improved performance when many aliases are defined

[1]: 9aca0dbdbd/hgext3rd/traceprof.pyx

Differential Revision: https://phab.mercurial-scm.org/D805
2017-09-23 13:46:12 -07:00
Jun Wu
95208a99f4 alias: test duplicated definition earlier
This patch moves the old definition checking logic introduced by
df4ba5915933 earlier. So that the test itself does not depend on `aliasdef`.

The check is to avoid wrapping a same alias multiple times. It can be done
by checking the config name and value (`definition` in code), without
constructing a `cmdalias` instance.

This makes the next patch easier to review.

Differential Revision: https://phab.mercurial-scm.org/D804
2017-09-23 13:31:09 -07:00
Yuya Nishihara
a71f259bd2 doctest: bulk-replace string literals with b'' for Python 3
Our code transformer can't rewrite string literals in docstrings, and I
don't want to make the transformer more complex.
2017-09-03 14:32:11 +09:00
Jun Wu
f89434c718 wrapfunction: use functools.partial if possible
Every `extensions.bind` call inserts a frame in traceback:

   ... in closure
     return func(*(args + a), **kw)

which makes traceback noisy.

The Python stdlib has a `functools.partial` which is backed by C code and
does not pollute traceback. However it does not support instancemethod and
sets `args` attribute which could be problematic for alias handling.

This patch makes `wrapfunction` use `functools.partial` if we are wrapping a
function directly exported by a module (so it's impossible to be a class or
instance method), and special handles `wrapfunction` results so alias
handling code could handle `args` just fine.

As an example, `hg rebase -s . -d . --traceback` got 6 lines removed in my
setup:

   File "hg/mercurial/dispatch.py", line 898, in _dispatch
     cmdpats, cmdoptions)
  -File "hg/mercurial/extensions.py", line 333, in closure
  -  return func(*(args + a), **kw)
   File "hg/hgext/journal.py", line 84, in runcommand
     return orig(lui, repo, cmd, fullargs, *args)
  -File "hg/mercurial/extensions.py", line 333, in closure
  -  return func(*(args + a), **kw)
   File "fb-hgext/hgext3rd/fbamend/hiddenoverride.py", line 119, in runcommand
     result = orig(lui, repo, cmd, fullargs, *args)
   File "hg/mercurial/dispatch.py", line 660, in runcommand
     ret = _runcommand(ui, options, cmd, d)
  -File "hg/mercurial/extensions.py", line 333, in closure
  -  return func(*(args + a), **kw)
   File "hg/hgext/pager.py", line 69, in pagecmd
     return orig(ui, options, cmd, cmdfunc)
   ....

Differential Revision: https://phab.mercurial-scm.org/D632
2017-09-05 13:37:36 -07:00
Augie Fackler
e2774d9258 python3: wrap all uses of <exception>.strerror with strtolocal
Our string literals are bytes, and we mostly want to %-format a
strerror into a one of those literals, so this fixes a ton of issues.
2017-08-22 20:03:07 -04:00
FUJIWARA Katsunori
af39ee1c25 ui: enable pager always for explicit --pager=on (issue5580)
Before this patch, explicit --pager=on is unintentionally ignored by
any disabling factor, even if priority of it is less than --pager=on
(e.g. "[ui] paginate = off").
2017-08-01 18:52:52 +09:00
Jun Wu
e47f7dc2fa codemod: register core configitems using a script
This is done by a script [2] using RedBaron [1], a tool designed for doing
code refactoring. All "default" values are decided by the script and are
strongly consistent with the existing code.

There are 2 changes done manually to fix tests:

  [warn] mercurial/exchange.py: experimental.bundle2-output-capture: default needs manual removal
  [warn] mercurial/localrepo.py: experimental.hook-track-tags: default needs manual removal

Since RedBaron is not confident about how to indent things [2].

[1]: https://github.com/PyCQA/redbaron
[2]: https://github.com/PyCQA/redbaron/issues/100
[3]:

#!/usr/bin/env python
# codemod_configitems.py - codemod tool to fill configitems
#
# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import, print_function

import os
import sys

import redbaron

def readpath(path):
    with open(path) as f:
        return f.read()

def writepath(path, content):
    with open(path, 'w') as f:
        f.write(content)

_configmethods = {'config', 'configbool', 'configint', 'configbytes',
                  'configlist', 'configdate'}

def extractstring(rnode):
    """get the string from a RedBaron string or call_argument node"""
    while rnode.type != 'string':
        rnode = rnode.value
    return rnode.value[1:-1]  # unquote, "'str'" -> "str"

def uiconfigitems(red):
    """match *.ui.config* pattern, yield (node, method, args, section, name)"""
    for node in red.find_all('atomtrailers'):
        entry = None
        try:
            obj = node[-3].value
            method = node[-2].value
            args = node[-1]
            section = args[0].value
            name = args[1].value
            if (obj in ('ui', 'self') and method in _configmethods
                and section.type == 'string' and name.type == 'string'):
                entry = (node, method, args, extractstring(section),
                         extractstring(name))
        except Exception:
            pass
        else:
            if entry:
                yield entry

def coreconfigitems(red):
    """match coreconfigitem(...) pattern, yield (node, args, section, name)"""
    for node in red.find_all('atomtrailers'):
        entry = None
        try:
            args = node[1]
            section = args[0].value
            name = args[1].value
            if (node[0].value == 'coreconfigitem' and section.type == 'string'
                and name.type == 'string'):
                entry = (node, args, extractstring(section),
                         extractstring(name))
        except Exception:
            pass
        else:
            if entry:
                yield entry

def registercoreconfig(cfgred, section, name, defaultrepr):
    """insert coreconfigitem to cfgred AST

    section and name are plain string, defaultrepr is a string
    """
    # find a place to insert the "coreconfigitem" item
    entries = list(coreconfigitems(cfgred))
    for node, args, nodesection, nodename in reversed(entries):
        if (nodesection, nodename) < (section, name):
            # insert after this entry
            node.insert_after(
                'coreconfigitem(%r, %r,\n'
                '    default=%s,\n'
                ')' % (section, name, defaultrepr))
            return

def main(argv):
    if not argv:
        print('Usage: codemod_configitems.py FILES\n'
              'For example, FILES could be "{hgext,mercurial}/*/**.py"')
    dirname = os.path.dirname
    reporoot = dirname(dirname(dirname(os.path.abspath(__file__))))

    # register configitems to this destination
    cfgpath = os.path.join(reporoot, 'mercurial', 'configitems.py')
    cfgred = redbaron.RedBaron(readpath(cfgpath))

    # state about what to do
    registered = set((s, n) for n, a, s, n in coreconfigitems(cfgred))
    toregister = {} # {(section, name): defaultrepr}
    coreconfigs = set() # {(section, name)}, whether it's used in core

    # first loop: scan all files before taking any action
    for i, path in enumerate(argv):
        print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
        iscore = ('mercurial' in path) and ('hgext' not in path)
        red = redbaron.RedBaron(readpath(path))
        # find all repo.ui.config* and ui.config* calls, and collect their
        # section, name and default value information.
        for node, method, args, section, name in uiconfigitems(red):
            if section == 'web':
                # [web] section has some weirdness, ignore them for now
                continue
            defaultrepr = None
            key = (section, name)
            if len(args) == 2:
                if key in registered:
                    continue
                if method == 'configlist':
                    defaultrepr = 'list'
                elif method == 'configbool':
                    defaultrepr = 'False'
                else:
                    defaultrepr = 'None'
            elif len(args) >= 3 and (args[2].target is None or
                                     args[2].target.value == 'default'):
                # try to understand the "default" value
                dnode = args[2].value
                if dnode.type == 'name':
                    if dnode.value in {'None', 'True', 'False'}:
                        defaultrepr = dnode.value
                elif dnode.type == 'string':
                    defaultrepr = repr(dnode.value[1:-1])
                elif dnode.type in ('int', 'float'):
                    defaultrepr = dnode.value
            # inconsistent default
            if key in toregister and toregister[key] != defaultrepr:
                defaultrepr = None
            # interesting to rewrite
            if key not in registered:
                if defaultrepr is None:
                    print('[note] %s: %s.%s: unsupported default'
                          % (path, section, name))
                    registered.add(key) # skip checking it again
                else:
                    toregister[key] = defaultrepr
                    if iscore:
                        coreconfigs.add(key)

    # second loop: rewrite files given "toregister" result
    for path in argv:
        # reconstruct redbaron - trade CPU for memory
        red = redbaron.RedBaron(readpath(path))
        changed = False
        for node, method, args, section, name in uiconfigitems(red):
            key = (section, name)
            defaultrepr = toregister.get(key)
            if defaultrepr is None or key not in coreconfigs:
                continue
            if len(args) >= 3 and (args[2].target is None or
                                   args[2].target.value == 'default'):
                try:
                    del args[2]
                    changed = True
                except Exception:
                    # redbaron fails to do the rewrite due to indentation
                    # see https://github.com/PyCQA/redbaron/issues/100
                    print('[warn] %s: %s.%s: default needs manual removal'
                          % (path, section, name))
            if key not in registered:
                print('registering %s.%s' % (section, name))
                registercoreconfig(cfgred, section, name, defaultrepr)
                registered.add(key)
        if changed:
            print('updating %s' % path)
            writepath(path, red.dumps())

    if toregister:
        print('updating configitems.py')
        writepath(cfgpath, cfgred.dumps())

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))
2017-07-14 14:22:40 -07:00
FUJIWARA Katsunori
b05e620ac0 dispatch: remove unused _loaded
Now, there is no user for dispatch._loaded.
2017-06-24 02:39:21 +09:00
FUJIWARA Katsunori
fdb6557cf4 extensions: register functions always at loading extension (issue5601)
Before this patch, functions defined in extensions are registered via
extra loaders only in _dispatch(). Therefore, loading extensions in
other code paths like below omits registration of functions.

  - WSGI service
  - operation across repositories (e.g. subrepo)
  - test-duplicateoptions.py, using extensions.loadall() directly

To register functions always at loading new extension, this patch
moves implementation for extra loading from dispatch._dispatch() to
extensions.loadall().

AFAIK, only commands module causes cyclic dependency between
extensions module, but this patch imports all related modules just
before extra loading in loadall(), in order to centralize them.

This patch makes extensions.py depend on many other modules, even
though extensions.py itself doesn't. It should be avoided if possible,
but I don't have any better idea. Some other places like below aren't
reasonable for extra loading, IMHO.

  - specific function in newly added module:
    existing callers of extensions.loadall() should invoke it, too

  - hg.repository() or so:
    no-repo commands aren't covered by this.

BTW, this patch removes _loaded.add(name) on relocation, because
dispatch._loaded is used only for extraloaders (for similar reason,
"exts" variable is removed, too).
2017-06-24 02:39:20 +09:00
Pierre-Yves David
2fdfd87513 profile: drop maybeprofile
It seems sufficiently simple to use "profile(enabled=X)" to not justify having
a dedicated context manager just to read the config.

(I do not have a too strong opinion about this).
2017-06-09 12:29:29 +01:00
Pierre-Yves David
81c8d3b7b6 profile: support --profile in alias and abbreviated version (--prof)
We now process the "--profile" a second time after alias has been processed and
the command argument fully parsed. If appropriate we enable profiling at that
time.

In these situation, the --profile will cover less than if the full --profile
flag was passed on the command line. This is better than the previous behavior
(flag ignored) and still fullfil multiple valid usecases.
2017-06-09 12:36:07 +01:00
Yuya Nishihara
02355fbadc dispatch: do not close stdout and stderr, just flush() instead
Since 348863ccec7e "util: always force line buffered stdout when stdout is
a tty", we have two file objects attached to the same STDOUT_FILENO. If one
is closed, the underlying file descriptor is also closed, and writing to
the other file object would crash the Python interpreter in a hard way, at
least on Windows.

So, it seems safer to not close the standard streams. This also matches
the behavior of the default sys.stdout/stderr.close(), which never close
the FILE* streams in C layer.

https://hg.python.org/cpython/file/v2.7.13/Python/sysmodule.c#l1401
2017-04-23 00:31:29 +09:00
Augie Fackler
9a722d4382 merge with stable 2017-06-03 16:33:28 -04:00
FUJIWARA Katsunori
6e23c59850 dispatch: setup color before pager for correct console information on windows
Before this patch, "hg CMD --pager on" on Windows shows output
unintentionally decorated with ANSI color escape sequences, if color
mode is "auto". This issue occurs in steps below.

  1. dispatch() invokes ui.pager() at detection of "--pager on"
  2. stdout of hg process is redirected into stdin of pager process
  3. "ui.formatted" = True, because isatty(stdout) is so before (2)
  4. color module is loaded for colorization
  5. color.w32effects = None, because GetConsoleScreenBufferInfo()
     fails on stdout redirected at (2)
  6. "ansi" color mode is chosen, because of "not w32effects"
  7. output is colorized in "ansi" mode because of "ui.formatted" = True

Even if "ansi" color mode is chosen, ordinarily redirected stdout
makes ui.formatted() return False, and colorization is avoided. But in
this issue case, "ui.formatted" = True at (3) forces output to be
colorized.

For correct console information on win32, it is needed to ensure that
color module is loaded before redirection of stdout for pagination.

BTW, if any of enabled extensions has "colortable" attribute, this
issue is avoided even before this patch, because color module is
imported as a part of loading such extension, and extension loading
occurs before setting up pager. For example, mq and keyword have
"colortable".
2017-05-23 03:29:23 +09:00
Augie Fackler
3d0bf6c892 freeze: merge default into stable for 4.2 code freeze 2017-04-18 12:24:34 -04:00
Augie Fackler
b2bd435955 dispatch: protect against malicious 'hg serve --stdio' invocations (sec)
Some shared-ssh installations assume that 'hg serve --stdio' is a safe
command to run for minimally trusted users. Unfortunately, the messy
implementation of argument parsing here meant that trying to access a
repo named '--debugger' would give the user a pdb prompt, thereby
sidestepping any hoped-for sandboxing. Serving repositories over HTTP(S)
is unaffected.

We're not currently hardening any subcommands other than 'serve'. If
your service exposes other commands to users with arbitrary repository
names, it is imperative that you defend against repository names of
'--debugger' and anything starting with '--config'.

The read-only mode of hg-ssh stopped working because it provided its hook
configuration to "hg serve --stdio" via --config parameter. This is banned for
security reasons now. This patch switches it to directly call ui.setconfig().
If your custom hosting infrastructure relies on passing --config to
"hg serve --stdio", you'll need to find a different way to get that configuration
into Mercurial, either by using ui.setconfig() as hg-ssh does in this patch,
or by placing an hgrc file someplace where Mercurial will read it.

mitrandir@fb.com provided some extra fixes for the dispatch code and
for hg-ssh in places that I overlooked.
2017-04-12 11:23:55 -07:00
Yuya Nishihara
649f9bf987 dispatch: ignore further SIGPIPE while handling KeyboardInterrupt
I got the following error by running "hg log" and quitting the pager
immediately. Any output here may trigger another SIGPIPE, so only thing
we can do is to swallow the exception and exit with an error status.

  Traceback (most recent call last):
    File "./hg", line 45, in <module>
      mercurial.dispatch.run()
    File "mercurial/dispatch.py", line 83, in run
      status = (dispatch(req) or 0) & 255
    File "mercurial/dispatch.py", line 167, in dispatch
      req.ui.warn(_("interrupted!\n"))
    File "mercurial/ui.py", line 1224, in warn
      self.write_err(*msg, **opts)
    File "mercurial/ui.py", line 790, in write_err
      self._write_err(*msgs, **opts)
    File "mercurial/ui.py", line 798, in _write_err
      self.ferr.write(a)
    File "mercurial/ui.py", line 129, in _catchterm
      raise error.SignalInterrupt
  mercurial.error.SignalInterrupt

Perhaps this wasn't visible before ee4f321cd621 because the original stderr
handle was restored very late.
2017-04-17 23:53:19 +09:00