Commit Graph

34751 Commits

Author SHA1 Message Date
Yuya Nishihara
caba95785d util: fix sortdict.update() to call __setitem__() on PyPy (issue5639)
It appears that overriding __setitem__() doesn't work as documented on PyPy.
Let's patch it as before e5e7b1586953.

https://docs.python.org/2/library/collections.html#ordereddict-examples-and-recipes

The issue was ui.configitems() wasn't ordered correctly, so the pull command
was wrapped in different order.
2017-08-02 22:51:19 +09:00
Yuya Nishihara
c94a7a8e9e test-revlog-raw: close file handles explicitly (issue5644) 2017-08-02 22:58:38 +09:00
Yuya Nishihara
7e9483b814 tests: clear __pycache__ for PyPy compatibility (issue5638) (issue5642) 2017-08-02 22:04:02 +09:00
Yuya Nishihara
45be99c80d test-extension: glob module name in error message for PyPy 2017-08-02 22:06:45 +09:00
Yuya Nishihara
4be878f7d2 rebase: sort collapsed revisions in commit message (issue5643)
This fixes the test instability.
2017-08-02 21:52:22 +09:00
FUJIWARA Katsunori
cfbdc0a924 i18n: use actual filename, in which function is defined, for hg.pot
Before this patch, source filename for msgid in hg.pot file becomes
incorrect, if a function is defined in file A, but detected in dict in
file B,

For example, almost all debug* commands are defined in
debugcommands.py, but hggettext detects them in "table" of
commands.py. Therefore, docstring fragments of debug* commands are
marked as "defined in commands.py" in hg.pot file.

This is serious problem for translation, because the cost to find out
original location of texts increases very much.
2017-08-02 00:02:11 +09:00
FUJIWARA Katsunori
af39ee1c25 ui: enable pager always for explicit --pager=on (issue5580)
Before this patch, explicit --pager=on is unintentionally ignored by
any disabling factor, even if priority of it is less than --pager=on
(e.g. "[ui] paginate = off").
2017-08-01 18:52:52 +09:00
Sean Farley
d2e211e766 subrepo: add tests for git rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 14:55:11 -07:00
Sean Farley
43f1ebbcd6 subrepo: add tests for svn rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 16:44:17 -07:00
Sean Farley
8604805d42 subrepo: add tests for hg rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 16:04:44 -07:00
Sean Farley
93ca65361f push: add tests for unsafe ssh url (SEC) 2017-07-31 14:40:28 -07:00
FUJIWARA Katsunori
d165bc820a README: make paragraphs before example code end with "::" for reST syntax
After this patch, README as reST text can be rendered into HTML more
appropriately. This is useful for hosting servers, which render README
file as a part of index HTML of its directory (for example,
Bitbucket).
2017-07-31 21:47:53 +09:00
Martin von Zweigbergk
9ce4ef597c commit: don't let failed commit with --addremove update dirstate (issue5645)
Differential Revision: https://phab.mercurial-scm.org/D204
2017-07-31 14:54:57 -07:00
Martin von Zweigbergk
ff40788359 tests: demonstrate that failed "hg ci -A" updates dirstate (issue5645)
Differential Revision: https://phab.mercurial-scm.org/D203
2017-07-31 14:54:08 -07:00
Martin von Zweigbergk
cc478f84d6 tests: fix test-flagprocessor.t flakiness
The test for duplicate flag processors depended on the timestamps
being set in the dirstate to work. If the time between the the
previous failed commit (which would set the timestamp, due to bug
5645) and the attempted commit with the duplicate flag processors was
small enough, it would fail. The failure was caused by a call to
commands.status() early in the commit process. If the dirstate did not
have the timestamp set, it would need to fetch the file content to
compare with. Since two flag processors had been registered, it would
attempted to base64 decode the contents twice, which would of course
fail.

This patch adds a "hg debugrebuilddirstate" to make it deterministic
and also replaces the test case's "hg commit" by simply "hg status",
since that will trigger reading of the contents and thereby use of the
flag processors as noted above.

Differential Revision: https://phab.mercurial-scm.org/D202
2017-07-31 16:32:01 -07:00
Martin von Zweigbergk
d9637ae740 tests: clarify that duplicate flag processors is not an error
The flag processors test for duplicate processors for a single flag
was misleading because the file from the previous test case caused it
to fail (making the "echo 'this should fail' > file" part
irrelevant). Let's remove the leftover from the previous test case to
make it clear that duplicate flag processors results only in a
warning.

Note that duplicate flag processors would have resulted in a failure
(not just a warning) until b319e3173a95 (extensions: catch uisetup and
extsetup failures and don't let them break hg, 2017-06-06). I remember
expressing my concern about ending up with half-loaded extensions. It
would be pretty unfortunate to have double-encoded revlog content
enter a repo, so maybe we should reconsider?

Differential Revision: https://phab.mercurial-scm.org/D201
2017-07-31 16:40:31 -07:00
FUJIWARA Katsunori
ac4ce63a67 tests: avoid unexpected result at invocation of *.py file on Windows
Invocation of "diff tool.py" in test-extdiff.t tests whether
shellquote() is applied on specified command as expected.

But direct invocation of "*.py" file might cause unexpected result on
Windows according to suffix binding.

For example, starting IDE, showing dialog to choose program to be
used, and so on. In such case, running test-extdiff.t is easily timed
out.

This patch uses intermediate *.bat file on Windows, to avoid such
unexpected result. Naming that intermediate file as "diff tool.bat" is
enough to test applying shellquote().
2017-08-01 01:27:32 +09:00
FUJIWARA Katsunori
b9e9255ab9 tests: use pdiff instead of diff for portability
"diff" command might cause redundant message, "No differences
encountered" on Solaris for example. But suppressing option like "-q"
isn't portable, because POSIX specification doesn't define it.

pdiff script was introduced by f4cba8b2e7b4 to stabilize output of
standard diff command on each platforms.
2017-07-31 13:16:34 +09:00
FUJIWARA Katsunori
898fb03733 tests: make pdiff return appropriate exit code at comparison of files
Before this patch, pdiff script returns 0, even if diff is detected.

This issue doesn't cause failure of tests using it, if it is invoked
via extdiff extension, because extdiff itself examines changes between
specified revisions and decides exit code.

BTW, this patch ignores recursive comparison case, because:

  - there is no portable way for current while-read based
    implementation to return 1 at detecting changes

  - it isn't yet needed to replace direct "diff -r" invocation by
    pdiff for portability
2017-07-31 13:10:19 +09:00
Martin von Zweigbergk
255c4290a2 tests: make test-profile.t pass if statprof didn't collect samples
We had the first 3 tests in test-profile.t fail because the output
didn't match. I have not yet confirmed that this was because no
samples were collected, but we shouldn't require samples to be
collected for the test to pass either way.

Differential Revision: https://phab.mercurial-scm.org/D199
2017-07-31 12:43:26 -07:00
Sean Farley
d4c4dd7bc9 pull: add tests for unsafe ssh url (SEC) 2017-07-28 16:47:32 -07:00
Sean Farley
8f38da87d6 clone: add tests for unsafe ssh url (SEC) 2017-07-28 16:36:36 -07:00
Sean Farley
c9ceef77c2 sshpeer: check for safe ssh url (SEC)
Checking in the sshpeer for a rogue ssh:// urls seems like the right
place to do it (instead of whack-a-mole with pull, clone, push, etc).
2017-08-01 14:40:19 -07:00
Augie Fackler
59ac847aed ssh: ban any username@host or host that starts with - (SEC)
This paranoia probably isn't required, but it can't hurt either.
2017-08-04 14:00:03 -04:00
Sean Farley
0468d74a37 util: add utility method to check for bad ssh urls (SEC)
Our use of SSH has an exploit that will parse the first part of an url
blindly as a hostname. Prior to this set of security patches, a url
with '-oProxyCommand' could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' can be abused to execute
arbitrary commands in a similar fashion.

We defend against this by checking ssh:// URLs and looking for a
hostname that starts with a - or contains a |.

When this happens, let's throw a big abort into the user's face so
that they can inspect what's going on.
2017-07-28 16:32:25 -07:00
Boris Feld
5e14d97b2b pypy: fix failing test-devel-warnings.t with Pypy5.6.0
In Pypy 5.6.0, traceback exception classes are not displayed with their full
qualified name.

Instead of displaying mercurial.error.ProgrammingError, Pypy displays
ProgrammingError.

Update the test to support both version.
2017-07-27 10:52:56 +02:00
Boris Feld
36f3e363cc pypy: fix failing test files with Pypy5.6.0
Pypy 5.6.0 saves cached bytecode files in __pycache__ directory, clean them in
tests to fix loading old test extensions code.

Doing so should also helps for Python3.x migration.
2017-07-31 17:43:45 +02:00
Pulkit Goyal
6940437f39 commitextras: make sure keys are not empty
Thanks to Yuya for catching this.
2017-07-30 12:19:46 +05:30
Sean Farley
62295e9d68 make: add new directory 'hgdemandimport' to cleanbutpackages rule
This fixes the build process for ppa (at least in my simple check).
Perhaps we should loop through all directory that aren't 'packages'?
2017-07-29 19:12:48 -07:00
Gregory Szorc
b4b2d140d9 statichttprepo: implement wlock() (issue5613)
statichttprepo inherits from localrepository. In doing so, it
obtains default implementations of various methods, like wlock().

Before this change, tags cache writing would call repo.wlock().
This failed on statichttprepo due to localrepository's wlock()
looking for an instance attribute that doesn't exist on statichttprepo
(statichttprepo doesn't call localrepository.__init__).

We /could/ define missing attributes until the base wlock() works.
However, a statichttprepo is remote and read-only and can't be
locked. The class already has a lock() that short circuits. So
it makes sense to implement a short-circuited wlock() as well. That
is what this patch does.

LockError is expected to be raised when locking fails. The constructor
takes a number of arguments that are local repository centric. Rather
than rework LockError to not require them (which would not be
appropriate for stable), this commit populates dummy values. I don't
believe they'll ever be seen by the user, as lock failures on
static http repos should be limited to well-defined (and tested)
scenarios. We can and should revisit the LockError type to improve
this.
2017-07-29 12:50:56 -07:00
Gregory Szorc
99dd48a10c test: add tests for branches and tags with static http repo (issue5613)
Mercurial currently fails to clone by revision on static http
repos with tags. The added tests demonstrate this.
2017-07-29 12:40:05 -07:00
Pulkit Goyal
f30e209b89 commitextras: make sure keys contains ascii letters, numbers, '_' and '-' only 2017-07-28 07:42:55 +05:30
Yuya Nishihara
b53f592e4f pathauditor: disable cache of audited paths by default (issue5628)
The initial attempt was to discard cache when appropriate, but it appears
to be error prone. We had to carefully inspect all places where audit() is
called e.g. without actually updating filesystem, before removing files and
directories, etc.

So, this patch disables the cache of audited paths by default, and enables
it only for the following cases:

 - short-lived auditor objects
 - repo.vfs, repo.svfs, and repo.cachevfs, which are managed directories
   and considered sort of append-only (a file/directory would never be
   replaced with a symlink)

There would be more cacheable vfs objects (e.g. mq.queue.opener), but I
decided not to inspect all of them in this patch. We can make them cached
later.

Benchmark result:

- using old clone of http://selenic.com/repo/linux-2.6/ (38319 files)
- on tmpfs
- run HGRCPATH=/dev/null hg up -q --time tip && hg up -q null
- try 4 times and take the last three results

original:
real 7.480 secs (user 1.140+22.760 sys 0.150+1.690)
real 8.010 secs (user 1.070+22.280 sys 0.170+2.120)
real 7.470 secs (user 1.120+22.390 sys 0.120+1.910)

clearcache (the other series):
real 7.680 secs (user 1.120+23.420 sys 0.140+1.970)
real 7.670 secs (user 1.110+23.620 sys 0.130+1.810)
real 7.740 secs (user 1.090+23.510 sys 0.160+1.940)

enable cache only for vfs and svfs (this series):
real 8.730 secs (user 1.500+25.190 sys 0.260+2.260)
real 8.750 secs (user 1.490+25.170 sys 0.250+2.340)
real 9.010 secs (user 1.680+25.340 sys 0.280+2.540)

remove cache function at all (for reference):
real 9.620 secs (user 1.440+27.120 sys 0.250+2.980)
real 9.420 secs (user 1.400+26.940 sys 0.320+3.130)
real 9.760 secs (user 1.530+27.270 sys 0.250+2.970)
2017-07-26 22:10:15 +09:00
Yuya Nishihara
9267d58ebc tests: show cache of audited paths is never invalidated 2017-08-01 21:03:25 +09:00
Yuya Nishihara
33738d760a setup: forcibly include cext/pure packages in py2exe (issue5625)
Since these modules are implicitly imported by policy.importmod(), py2exe
can't track them statically. cffi modules are excluded for now because they
wouldn't be useful in frozen (i.e. CPython) environment.
2017-07-20 22:47:40 +09:00
Yuya Nishihara
6a9bb7fcb6 setup: do not select hg executable that prints unexpected warnings
Otherwise the subsequent hg.run() would fail. This factors out the filtering
function so the same rule should apply.
2017-07-20 22:32:37 +09:00
Mathias De Maré
1ebe02c078 docker: pass proxy arguments to docker process
Building in Docker when behind a proxy requires passing the proxy variables.

Differential Revision: https://phab.mercurial-scm.org/D190
2017-07-27 13:44:15 +02:00
Mathias De Maré
30343f1379 makefile: build on CentOS 6 with a Python 2.7
Differential Revision: https://phab.mercurial-scm.org/D189
2017-07-27 14:15:26 +02:00
Augie Fackler
9502ad3b9b osx: wire up genosxversion script
The only version strings that are changed are the ones baked into the
.pkg - hg's self-reported version string doesn't change, so users will
still see our mostly-pip-compatible version strings.

For reference, the part of our versioning setup that's not PEP440
compatible is the RC releases - those should be .rc0 insted of
-rc. It's too late to change that for the 4.3 cycle, so I'll worry
about fixing that during the 4.4 cycle.
2017-07-24 20:39:26 -04:00
Augie Fackler
437f1cb3c5 osx: new script for generating OS X package versions
If you're shipping prerelease or rc packages using Munki, you'll
eventually discover that Munki's version comparison logic is not as
good as pip's. In theory we should be able to fix Munki, but it seems
entirely reasonable to produce version strings that sort reasonably
under these conditions. Since the requried logic not brief, add a new
script and some tests of that logic.

A followup change will wire this into the Makefile.
2017-07-24 20:38:09 -04:00
Augie Fackler
396faf88eb tests: update mac packaging test expectations
Some more files are included these days.
2017-07-24 20:34:08 -04:00
Augie Fackler
484a641a36 tests: add support in run-tests.py for an environment variable to stop color
I don't like the color output, so this gives me a way to easily never
see it while still having pygments on my machine.
2017-07-26 17:56:44 -04:00
Augie Fackler
ced1b0963a setup: fix mistake that prevented Python 3 from being excluded
My intent was to only allow Python 3 if the right environment variable
is set (for when people want to use `pip install .` on hg
locally). This fixes the bug in my previous change. I verified that
`python3.6 run-tests.py` still passes the tests that passed before,
and that all tests pass on 2.7 (including our virtualenv-using
installation test).

Differential Revision: https://phab.mercurial-scm.org/D185
2017-07-26 13:21:34 -04:00
Jun Wu
a3005e76f2 rebase: move bookmark to destination for commits becoming empty (issue5627)
When rebasing a changeset X and that changeset becomes empty, we should move
the bookmark on X to rebase destination.

This is a regression caused by the scmutil.cleanupnodes refactoring for
rebase.

The `adjustdest` function calculates the destination of bookmark movement.
It was back-ported from https://phab.mercurial-scm.org/D21. It might be
slightly more powerful than the minimal requirement to solve this issue.
For example, it's impossible for a merge changeset to become empty while any
of its ancestors does not become empty, but the code could handle that case.
Since the code is reasonably short and clean, and helps the upcoming D21
series, I'd like to check-in `adjustdest` now.

Thanks Martin von Zweigbergk for spotting corner cases (-k and descendant
with bookmarks) in this area!
2017-07-24 23:52:56 -07:00
Yuya Nishihara
c49455b105 setup: drop use of set literal so parsable on Python 2.6
Otherwise friendly error wouldn't be displayed.
2017-07-26 23:39:42 +09:00
Augie Fackler
527e838b00 setup: add extra logic to try and recommend a new pip on bad Python
Modern pip can detect supported Python versions (which we now
declare), and pull down a reasonable release. This trick was suggested
in http://bit.ly/pycon2017-build-bridges, and seems like a good
defensive maneuver so that when we want to move to Python 3 it's
less risky for existing users.

This moves the version-check logic after defining our printf function
so we can print more informative messages.
2017-07-21 10:46:31 -04:00
Augie Fackler
4b55d62a6d setup: explicitly declare supported Python versions
I think we should probably backport this to 4.2 as well, and do one
more release there that explicitly declares 2.6 support. That way
anyone stuck on Python 2.6 will end up getting the right hg if they
use a modern pip to install. Users can still use `python setup.py`
incantations to attempt installing Mercurial on unsupported Pythons,
including 3.5 and 3.6.

A followup change will switch to only doing our own
Python-version-check logic if we're not being installed by a
reasonable pip.
2017-07-21 10:39:52 -04:00
Augie Fackler
7dc97f30ea contrib: remove one test from the python 3 whitelist
5ba3f753c9b1 introduced a use of 'hg revert --no-backup' which is
currently broken. Rather than try and do a fix on the (complicated)
revert code during the freeze, let's just give up on this test under
Python 3 until later. Once we do fix things, the test ratchet script
will re-enable it.
2017-07-24 15:37:57 -04:00
Augie Fackler
4dfc9655ac ui: fix configbytes isinstance check to look for bytes and not str
Fixes configbytes on Python 3.
2017-07-24 13:50:25 -04:00
Augie Fackler
2ebd830d1d patch: update copying of dict keys and values to work on Python 3 2017-07-24 14:42:55 -04:00