Commit Graph

33655 Commits

Author SHA1 Message Date
Gregory Szorc
e98da01acf tests: test behavior of IOError during transactions (issue5658)
ui._write(), ui._write_err(), and ui.flush() all trap IOError and
re-raise as error.StdioError. If a caller doesn't catch StdioError
when writing to stdio, it could bubble all the way to dispatch.

This commit adds tests for I/O failures around various transaction
operations.

The most notable badness is during abort. Here, an uncaught StdioError
will result in incomplete transaction rollback, requiring an
`hg rollback` to recover. This can result in a client "corrupting"
a remote repo via terminated HTTP and SSH socket.
2017-08-14 13:12:40 -07:00
Nathan Goldbaum
0de8be7e62 log: mention ui.logtemplate in the help text 2017-08-16 10:24:49 -05:00
Tristan Seligmann
0428b15b61 hg: avoid relying on errno numbers / descriptions
A few tests hardcode errno numbers and/or descriptions in the output, causing
test failures on platforms where these values are different.

Differential Revision: https://phab.mercurial-scm.org/D362
2017-08-12 14:29:22 +02:00
Tristan Seligmann
0fee28d767 hg: tolerate long vs. int in test-context.py
The file times here can be longs instead of ints on some platforms, which will
cause a test failure due to these printing with an L suffix; instead always
format with %d which will produce the same output in either case.

Differential Revision: https://phab.mercurial-scm.org/D361
2017-08-12 14:24:25 +02:00
Jun Wu
d84c305f2a rebase: only change self.state when collapsing in _finishrebase
When the code executes to _finishrebase, self.state should be populated with
correct destinations and do not need to be written to a node. The code was
introduced by 8dc45c9059, which seems to avoid setting state values to None
but it didn't provide more details.

Differential Revision: https://phab.mercurial-scm.org/D346
2017-08-11 01:34:11 -07:00
Jun Wu
13f8aae09c rebase: choose merge base without unwanted revisions
Previously, when there are 2 merge base candidates, we choose p1 blindly,
which may make the merge result to have "unwanted content". This patch makes
rebase smarter - choose a merge base that does not have "unwanted revs" if
possible. Since we don't really have a good solution when there are
"unwanted revs", abort in that case.

Differential Revision: https://phab.mercurial-scm.org/D340
2017-08-10 22:17:15 -07:00
Jun Wu
3e05e789bc demandimport: disable if chg is being used
In chg's case, making modules lazily loaded could actually slow down things
since chg pre-imports them. Therefore disable demandimport if chg is being
used.

This is not done by setting `HGDEMANDIMPORT` chg client-side because that
has side-effects on child processes (hooks, etc).

Differential Revision: https://phab.mercurial-scm.org/D351
2017-08-16 10:44:06 -07:00
Jun Wu
e60bc9fd17 demandimport: move HGDEMANDIMPORT test to __init__.py
The logic is now shared between demandimport{2,3}.

Differential Revision: https://phab.mercurial-scm.org/D445
2017-08-18 11:08:17 -07:00
Danny Hooper
54e3286e1e log: add a "graphwidth" template variable
Wrapping text in templates for 'hg log --graph' can't be done very well,
because the template doesn't know how wide the graph drawing is. The edge
drawing function needs to know the number of lines in the template output, so
we need to also determine how wide that drawing would be before we call the
edgefn or evaluate the template.

This patch makes edgefn compute the graph width and pass it into the template
so that we can do something like this:

COLUMNS=10 hg log --graph --template "{fill(desc, termwidth - graphwidth)}"
@  a a a a
|  a a a a
|  a a a a
o    a a a
|\   a a a
| |  a a a
| |  a a a

Using extensions to do this would be relatively complicated due to a lack of
hooks in this area of the code.

In the future it may make sense to have a more generic "textwidth" that tells
you how many columns you can expect to fill without causing the terminal to
wrap your output. I'm not sure there are other situations to motivate this yet,
or if it is entirely feasible.

Differential Revision: https://phab.mercurial-scm.org/D360
2017-08-15 10:15:31 -07:00
Boris Feld
76d1768227 obsmarker: fix precnode deprecation
The deprecation message for marker.precnode was wrong. Fix the typo.

Differential Revision: https://phab.mercurial-scm.org/D413
2017-08-16 10:18:57 +02:00
Boris Feld
e0873c6bda obsmarker: precnode was renamed into prednode
Update all calls to formatter.write first arguments to remove references to
precnode and use prednode consistently everywhere.

Differential Revision: https://phab.mercurial-scm.org/D414
2017-08-16 10:26:26 +02:00
Boris Feld
7524f9d434 revset: mark evolution-related revsets as experimental
Differential Revision: https://phab.mercurial-scm.org/D416
2017-08-16 16:48:41 +02:00
Jun Wu
8a1b859f80 push: fix docsstring
Seems the code block misses `::`. This patch makes sure `[push]` and
`pushvars.server = true` are in two lines.

Differential Revision: https://phab.mercurial-scm.org/D411
2017-08-15 17:22:57 -07:00
Yuya Nishihara
59eaa37546 py3: select input or raw_input by pycompat
This seems slightly cleaner.
2017-08-16 13:54:24 +09:00
Yuya Nishihara
9837558150 py3: make encoding.strio() an identity function on Python 2
It's the convention the other encoding.str*() functions follow. To make things
simple, this also drops kwargs from the strio() constructor.
2017-08-16 13:50:11 +09:00
Yuya Nishihara
38282452e0 templatekw: specify plural form of instability
Follows up ebfef9a04f8d.
2017-08-13 14:12:28 +09:00
Yuya Nishihara
79af97cbbf templatekw: rename termwidth() per convention 2017-08-16 13:57:19 +09:00
Jun Wu
7c75b06935 rebase: optimize "source" calculation in adjustdest
The "source" variable is calculated inside a loop but it does not depend on
loop variables. Therefore move it outside the loop.

Differential Revision: https://phab.mercurial-scm.org/D345
2017-08-11 01:08:07 -07:00
Jun Wu
7793d2cdf5 rebase: remove "state >= revtodo" condition
Now the minimal value of state is revtodo, that condition is always true,
therefore removed.

Differential Revision: https://phab.mercurial-scm.org/D344
2017-08-11 01:36:59 -07:00
Jun Wu
0c8152cb4c rebase: remove self.destancestors
The state is no longer used after calculating "self.external", therefore
removed.

Differential Revision: https://phab.mercurial-scm.org/D343
2017-08-11 01:05:47 -07:00
Mike Hommey
634713aa96 setup: fix installing in a mingw environment
The addition, in 851d08ff7a58, of a hack for the MSVC compiler class was
overwriting the original class for the Mingw32CCompiler class, leading to an
error when the HackedMingw32CCompiler is instantiated.

Differential Revision: https://phab.mercurial-scm.org/D329
2017-08-11 10:16:00 +09:00
Kevin Bullock
5e1176676c osx: delay version computation on macOS builds
Before this patch, HGVER would be evaluated at the beginning of the make
execution, and would be unset because build/mercurial/ doesn't exist yet
at that point. Now we compute the version after the `make install` run
has completed.

This is backported to stable from 8626b44516c1, but that revision had an
error in the shell invocation syntax.
2017-08-10 21:25:02 -04:00
Mike Hommey
73f4a16a4f branchmap: revert a81106261d38 for Python 2.7 compatibility
Old versions of python 2.7 don't like that the second argument to
struct.unpack_from is a bytearray, so the change removing the util.buffer
around that argument in branchmap broke running on older versions of python
2.7.

Differential Revision: https://phab.mercurial-scm.org/D330
2017-08-10 20:47:19 -07:00
Augie Fackler
abe39330a6 Added signature for changeset 229937197835 2017-08-10 18:46:55 -04:00
Yuya Nishihara
509744ddfc ssh: unban the use of pipe character in user@host:port string
This vulnerability was fixed by the previous patch and there were more ways
to exploit than using '|shellcmd'. So it doesn't make sense to reject only
pipe character.

Test cases are updated to actually try to exploit the bug. As the SSH bridge
of git/svn subrepos are not managed by our code, the tests for non-hg subrepos
are just removed.

This may be folded into the original patches.
2017-08-07 22:22:28 +09:00
Jun Wu
a0e5a4defb ssh: quote parameters using shellquote (SEC)
This patch uses shellquote to quote ssh parameters more strictly to avoid
shell injection.
2017-08-04 23:54:12 -07:00
Sean Farley
39898f2a8a subrepo: add tests for git rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 14:55:11 -07:00
Sean Farley
da301ac6a0 subrepo: add tests for svn rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 16:44:17 -07:00
Sean Farley
f904aef7aa subrepo: add tests for hg rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 16:04:44 -07:00
Sean Farley
ea5a6b2798 push: add tests for unsafe ssh url (SEC) 2017-07-31 14:40:28 -07:00
Sean Farley
616cd64fcf pull: add tests for unsafe ssh url (SEC) 2017-07-28 16:47:32 -07:00
Sean Farley
ee608897f9 clone: add tests for unsafe ssh url (SEC) 2017-07-28 16:36:36 -07:00
Sean Farley
e199b92002 sshpeer: check for safe ssh url (SEC)
Checking in the sshpeer for a rogue ssh:// urls seems like the right
place to do it (instead of whack-a-mole with pull, clone, push, etc).
2017-08-01 14:40:19 -07:00
Augie Fackler
5f2d0af8a2 ssh: ban any username@host or host that starts with - (SEC)
This paranoia probably isn't required, but it can't hurt either.
2017-08-04 14:00:03 -04:00
Sean Farley
608ad9eb9e util: add utility method to check for bad ssh urls (SEC)
Our use of SSH has an exploit that will parse the first part of an url
blindly as a hostname. Prior to this set of security patches, a url
with '-oProxyCommand' could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' can be abused to execute
arbitrary commands in a similar fashion.

We defend against this by checking ssh:// URLs and looking for a
hostname that starts with a - or contains a |.

When this happens, let's throw a big abort into the user's face so
that they can inspect what's going on.
2017-07-28 16:32:25 -07:00
Yuya Nishihara
ba69ca47d4 pathauditor: disable cache of audited paths by default (issue5628)
The initial attempt was to discard cache when appropriate, but it appears
to be error prone. We had to carefully inspect all places where audit() is
called e.g. without actually updating filesystem, before removing files and
directories, etc.

So, this patch disables the cache of audited paths by default, and enables
it only for the following cases:

 - short-lived auditor objects
 - repo.vfs, repo.svfs, and repo.cachevfs, which are managed directories
   and considered sort of append-only (a file/directory would never be
   replaced with a symlink)

There would be more cacheable vfs objects (e.g. mq.queue.opener), but I
decided not to inspect all of them in this patch. We can make them cached
later.

Benchmark result:

- using old clone of http://selenic.com/repo/linux-2.6/ (38319 files)
- on tmpfs
- run HGRCPATH=/dev/null hg up -q --time tip && hg up -q null
- try 4 times and take the last three results

original:
real 7.480 secs (user 1.140+22.760 sys 0.150+1.690)
real 8.010 secs (user 1.070+22.280 sys 0.170+2.120)
real 7.470 secs (user 1.120+22.390 sys 0.120+1.910)

clearcache (the other series):
real 7.680 secs (user 1.120+23.420 sys 0.140+1.970)
real 7.670 secs (user 1.110+23.620 sys 0.130+1.810)
real 7.740 secs (user 1.090+23.510 sys 0.160+1.940)

enable cache only for vfs and svfs (this series):
real 8.730 secs (user 1.500+25.190 sys 0.260+2.260)
real 8.750 secs (user 1.490+25.170 sys 0.250+2.340)
real 9.010 secs (user 1.680+25.340 sys 0.280+2.540)

remove cache function at all (for reference):
real 9.620 secs (user 1.440+27.120 sys 0.250+2.980)
real 9.420 secs (user 1.400+26.940 sys 0.320+3.130)
real 9.760 secs (user 1.530+27.270 sys 0.250+2.970)
2017-07-26 22:10:15 +09:00
Yuya Nishihara
116bb6f6f9 tests: show cache of audited paths is never invalidated 2017-08-01 21:03:25 +09:00
Augie Fackler
25ec33c0a4 stable: merge heads 2017-08-10 14:23:25 -04:00
Augie Fackler
1003d2c0a9 Added signature for changeset 525f2b18248f 2017-08-10 14:14:49 -04:00
Augie Fackler
fa0eb2f7c3 Added signature for changeset 9ab2705099b9 2017-08-10 13:29:09 -04:00
Yuya Nishihara
62967cdb51 ssh: unban the use of pipe character in user@host:port string
This vulnerability was fixed by the previous patch and there were more ways
to exploit than using '|shellcmd'. So it doesn't make sense to reject only
pipe character.

Test cases are updated to actually try to exploit the bug. As the SSH bridge
of git/svn subrepos are not managed by our code, the tests for non-hg subrepos
are just removed.

This may be folded into the original patches.
2017-08-07 22:22:28 +09:00
Mathias De Maré
cafe7e372c chg: define _GNU_SOURCE to allow CentOS 5 compilation
Without this flag, compilation fails with:
 hgclient.c: In function 'hgc_open':
 hgclient.c:466: error: 'O_DIRECTORY' undeclared (first use in this function)
 hgclient.c:466: error: (Each undeclared identifier is reported only once
 hgclient.c:466: error: for each function it appears in.)

Differential Revision: https://phab.mercurial-scm.org/D260
2017-08-07 13:40:36 +02:00
Mathias De Maré
744ba88ca3 centos5: use vault urls
CentOS 5 has reached end-of-life. A working build for it is still
possible using 'vault.centos.org'.

Differential Revision: https://phab.mercurial-scm.org/D261
2017-08-07 13:38:25 +02:00
Kostia Balytskyi
edfeaa5101 ui: make sure buffer is flushed before waiting for user input (issue5587)
Without this patch on Windows 'hg ci -i' hangs waiting for user input
and "examine changes to 'file'? [Ynesfdaq?]" is never displayed (at least
if the diff is sufficiently small). When Ctrl+C is pressed, this prompt
becomes visible, which suggests that the buffer just wasn't flushed.
I've never seen this happening on Linux, but this looks harmless enough
to not platform-gate it.
2017-08-05 13:19:09 -07:00
Jun Wu
6b28fe8df7 ssh: quote parameters using shellquote (SEC)
This patch uses shellquote to quote ssh parameters more strictly to avoid
shell injection.
2017-08-04 23:54:12 -07:00
Yuya Nishihara
caba95785d util: fix sortdict.update() to call __setitem__() on PyPy (issue5639)
It appears that overriding __setitem__() doesn't work as documented on PyPy.
Let's patch it as before e5e7b1586953.

https://docs.python.org/2/library/collections.html#ordereddict-examples-and-recipes

The issue was ui.configitems() wasn't ordered correctly, so the pull command
was wrapped in different order.
2017-08-02 22:51:19 +09:00
Yuya Nishihara
c94a7a8e9e test-revlog-raw: close file handles explicitly (issue5644) 2017-08-02 22:58:38 +09:00
Yuya Nishihara
7e9483b814 tests: clear __pycache__ for PyPy compatibility (issue5638) (issue5642) 2017-08-02 22:04:02 +09:00
Yuya Nishihara
45be99c80d test-extension: glob module name in error message for PyPy 2017-08-02 22:06:45 +09:00
Yuya Nishihara
4be878f7d2 rebase: sort collapsed revisions in commit message (issue5643)
This fixes the test instability.
2017-08-02 21:52:22 +09:00