Commit Graph

33786 Commits

Author SHA1 Message Date
FUJIWARA Katsunori
b9e9255ab9 tests: use pdiff instead of diff for portability
"diff" command might cause redundant message, "No differences
encountered" on Solaris for example. But suppressing option like "-q"
isn't portable, because POSIX specification doesn't define it.

pdiff script was introduced by f4cba8b2e7b4 to stabilize output of
standard diff command on each platforms.
2017-07-31 13:16:34 +09:00
FUJIWARA Katsunori
898fb03733 tests: make pdiff return appropriate exit code at comparison of files
Before this patch, pdiff script returns 0, even if diff is detected.

This issue doesn't cause failure of tests using it, if it is invoked
via extdiff extension, because extdiff itself examines changes between
specified revisions and decides exit code.

BTW, this patch ignores recursive comparison case, because:

  - there is no portable way for current while-read based
    implementation to return 1 at detecting changes

  - it isn't yet needed to replace direct "diff -r" invocation by
    pdiff for portability
2017-07-31 13:10:19 +09:00
Martin von Zweigbergk
255c4290a2 tests: make test-profile.t pass if statprof didn't collect samples
We had the first 3 tests in test-profile.t fail because the output
didn't match. I have not yet confirmed that this was because no
samples were collected, but we shouldn't require samples to be
collected for the test to pass either way.

Differential Revision: https://phab.mercurial-scm.org/D199
2017-07-31 12:43:26 -07:00
Sean Farley
d4c4dd7bc9 pull: add tests for unsafe ssh url (SEC) 2017-07-28 16:47:32 -07:00
Sean Farley
8f38da87d6 clone: add tests for unsafe ssh url (SEC) 2017-07-28 16:36:36 -07:00
Sean Farley
c9ceef77c2 sshpeer: check for safe ssh url (SEC)
Checking in the sshpeer for a rogue ssh:// urls seems like the right
place to do it (instead of whack-a-mole with pull, clone, push, etc).
2017-08-01 14:40:19 -07:00
Augie Fackler
59ac847aed ssh: ban any username@host or host that starts with - (SEC)
This paranoia probably isn't required, but it can't hurt either.
2017-08-04 14:00:03 -04:00
Sean Farley
0468d74a37 util: add utility method to check for bad ssh urls (SEC)
Our use of SSH has an exploit that will parse the first part of an url
blindly as a hostname. Prior to this set of security patches, a url
with '-oProxyCommand' could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' can be abused to execute
arbitrary commands in a similar fashion.

We defend against this by checking ssh:// URLs and looking for a
hostname that starts with a - or contains a |.

When this happens, let's throw a big abort into the user's face so
that they can inspect what's going on.
2017-07-28 16:32:25 -07:00
Boris Feld
5e14d97b2b pypy: fix failing test-devel-warnings.t with Pypy5.6.0
In Pypy 5.6.0, traceback exception classes are not displayed with their full
qualified name.

Instead of displaying mercurial.error.ProgrammingError, Pypy displays
ProgrammingError.

Update the test to support both version.
2017-07-27 10:52:56 +02:00
Boris Feld
36f3e363cc pypy: fix failing test files with Pypy5.6.0
Pypy 5.6.0 saves cached bytecode files in __pycache__ directory, clean them in
tests to fix loading old test extensions code.

Doing so should also helps for Python3.x migration.
2017-07-31 17:43:45 +02:00
Pulkit Goyal
6940437f39 commitextras: make sure keys are not empty
Thanks to Yuya for catching this.
2017-07-30 12:19:46 +05:30
Sean Farley
62295e9d68 make: add new directory 'hgdemandimport' to cleanbutpackages rule
This fixes the build process for ppa (at least in my simple check).
Perhaps we should loop through all directory that aren't 'packages'?
2017-07-29 19:12:48 -07:00
Gregory Szorc
b4b2d140d9 statichttprepo: implement wlock() (issue5613)
statichttprepo inherits from localrepository. In doing so, it
obtains default implementations of various methods, like wlock().

Before this change, tags cache writing would call repo.wlock().
This failed on statichttprepo due to localrepository's wlock()
looking for an instance attribute that doesn't exist on statichttprepo
(statichttprepo doesn't call localrepository.__init__).

We /could/ define missing attributes until the base wlock() works.
However, a statichttprepo is remote and read-only and can't be
locked. The class already has a lock() that short circuits. So
it makes sense to implement a short-circuited wlock() as well. That
is what this patch does.

LockError is expected to be raised when locking fails. The constructor
takes a number of arguments that are local repository centric. Rather
than rework LockError to not require them (which would not be
appropriate for stable), this commit populates dummy values. I don't
believe they'll ever be seen by the user, as lock failures on
static http repos should be limited to well-defined (and tested)
scenarios. We can and should revisit the LockError type to improve
this.
2017-07-29 12:50:56 -07:00
Gregory Szorc
99dd48a10c test: add tests for branches and tags with static http repo (issue5613)
Mercurial currently fails to clone by revision on static http
repos with tags. The added tests demonstrate this.
2017-07-29 12:40:05 -07:00
Pulkit Goyal
f30e209b89 commitextras: make sure keys contains ascii letters, numbers, '_' and '-' only 2017-07-28 07:42:55 +05:30
Jun Wu
51be9cd3a9 phabricator: convert unicode to binary when writing patches
This is a quick fix to make `hg phabread D189` work.

It seems we might want to replace all `r''` to `u''`, and add more
`encoding.*to*` to be more explicit when interacting with `json` module.

Differential Revision: https://phab.mercurial-scm.org/D192
2017-07-27 12:03:01 -07:00
Yuya Nishihara
4b047b0633 setup: silence warning of unknown option python_requires on distutils
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution
  option: 'python_requires'
2017-07-27 23:15:14 +09:00
Yuya Nishihara
b53f592e4f pathauditor: disable cache of audited paths by default (issue5628)
The initial attempt was to discard cache when appropriate, but it appears
to be error prone. We had to carefully inspect all places where audit() is
called e.g. without actually updating filesystem, before removing files and
directories, etc.

So, this patch disables the cache of audited paths by default, and enables
it only for the following cases:

 - short-lived auditor objects
 - repo.vfs, repo.svfs, and repo.cachevfs, which are managed directories
   and considered sort of append-only (a file/directory would never be
   replaced with a symlink)

There would be more cacheable vfs objects (e.g. mq.queue.opener), but I
decided not to inspect all of them in this patch. We can make them cached
later.

Benchmark result:

- using old clone of http://selenic.com/repo/linux-2.6/ (38319 files)
- on tmpfs
- run HGRCPATH=/dev/null hg up -q --time tip && hg up -q null
- try 4 times and take the last three results

original:
real 7.480 secs (user 1.140+22.760 sys 0.150+1.690)
real 8.010 secs (user 1.070+22.280 sys 0.170+2.120)
real 7.470 secs (user 1.120+22.390 sys 0.120+1.910)

clearcache (the other series):
real 7.680 secs (user 1.120+23.420 sys 0.140+1.970)
real 7.670 secs (user 1.110+23.620 sys 0.130+1.810)
real 7.740 secs (user 1.090+23.510 sys 0.160+1.940)

enable cache only for vfs and svfs (this series):
real 8.730 secs (user 1.500+25.190 sys 0.260+2.260)
real 8.750 secs (user 1.490+25.170 sys 0.250+2.340)
real 9.010 secs (user 1.680+25.340 sys 0.280+2.540)

remove cache function at all (for reference):
real 9.620 secs (user 1.440+27.120 sys 0.250+2.980)
real 9.420 secs (user 1.400+26.940 sys 0.320+3.130)
real 9.760 secs (user 1.530+27.270 sys 0.250+2.970)
2017-07-26 22:10:15 +09:00
Yuya Nishihara
9267d58ebc tests: show cache of audited paths is never invalidated 2017-08-01 21:03:25 +09:00
Yuya Nishihara
33738d760a setup: forcibly include cext/pure packages in py2exe (issue5625)
Since these modules are implicitly imported by policy.importmod(), py2exe
can't track them statically. cffi modules are excluded for now because they
wouldn't be useful in frozen (i.e. CPython) environment.
2017-07-20 22:47:40 +09:00
Yuya Nishihara
6a9bb7fcb6 setup: do not select hg executable that prints unexpected warnings
Otherwise the subsequent hg.run() would fail. This factors out the filtering
function so the same rule should apply.
2017-07-20 22:32:37 +09:00
Mathias De Maré
1ebe02c078 docker: pass proxy arguments to docker process
Building in Docker when behind a proxy requires passing the proxy variables.

Differential Revision: https://phab.mercurial-scm.org/D190
2017-07-27 13:44:15 +02:00
Mathias De Maré
30343f1379 makefile: build on CentOS 6 with a Python 2.7
Differential Revision: https://phab.mercurial-scm.org/D189
2017-07-27 14:15:26 +02:00
Augie Fackler
9502ad3b9b osx: wire up genosxversion script
The only version strings that are changed are the ones baked into the
.pkg - hg's self-reported version string doesn't change, so users will
still see our mostly-pip-compatible version strings.

For reference, the part of our versioning setup that's not PEP440
compatible is the RC releases - those should be .rc0 insted of
-rc. It's too late to change that for the 4.3 cycle, so I'll worry
about fixing that during the 4.4 cycle.
2017-07-24 20:39:26 -04:00
Augie Fackler
437f1cb3c5 osx: new script for generating OS X package versions
If you're shipping prerelease or rc packages using Munki, you'll
eventually discover that Munki's version comparison logic is not as
good as pip's. In theory we should be able to fix Munki, but it seems
entirely reasonable to produce version strings that sort reasonably
under these conditions. Since the requried logic not brief, add a new
script and some tests of that logic.

A followup change will wire this into the Makefile.
2017-07-24 20:38:09 -04:00
Augie Fackler
396faf88eb tests: update mac packaging test expectations
Some more files are included these days.
2017-07-24 20:34:08 -04:00
Augie Fackler
484a641a36 tests: add support in run-tests.py for an environment variable to stop color
I don't like the color output, so this gives me a way to easily never
see it while still having pygments on my machine.
2017-07-26 17:56:44 -04:00
Augie Fackler
ced1b0963a setup: fix mistake that prevented Python 3 from being excluded
My intent was to only allow Python 3 if the right environment variable
is set (for when people want to use `pip install .` on hg
locally). This fixes the bug in my previous change. I verified that
`python3.6 run-tests.py` still passes the tests that passed before,
and that all tests pass on 2.7 (including our virtualenv-using
installation test).

Differential Revision: https://phab.mercurial-scm.org/D185
2017-07-26 13:21:34 -04:00
Jun Wu
a3005e76f2 rebase: move bookmark to destination for commits becoming empty (issue5627)
When rebasing a changeset X and that changeset becomes empty, we should move
the bookmark on X to rebase destination.

This is a regression caused by the scmutil.cleanupnodes refactoring for
rebase.

The `adjustdest` function calculates the destination of bookmark movement.
It was back-ported from https://phab.mercurial-scm.org/D21. It might be
slightly more powerful than the minimal requirement to solve this issue.
For example, it's impossible for a merge changeset to become empty while any
of its ancestors does not become empty, but the code could handle that case.
Since the code is reasonably short and clean, and helps the upcoming D21
series, I'd like to check-in `adjustdest` now.

Thanks Martin von Zweigbergk for spotting corner cases (-k and descendant
with bookmarks) in this area!
2017-07-24 23:52:56 -07:00
Yuya Nishihara
c49455b105 setup: drop use of set literal so parsable on Python 2.6
Otherwise friendly error wouldn't be displayed.
2017-07-26 23:39:42 +09:00
Augie Fackler
527e838b00 setup: add extra logic to try and recommend a new pip on bad Python
Modern pip can detect supported Python versions (which we now
declare), and pull down a reasonable release. This trick was suggested
in http://bit.ly/pycon2017-build-bridges, and seems like a good
defensive maneuver so that when we want to move to Python 3 it's
less risky for existing users.

This moves the version-check logic after defining our printf function
so we can print more informative messages.
2017-07-21 10:46:31 -04:00
Augie Fackler
4b55d62a6d setup: explicitly declare supported Python versions
I think we should probably backport this to 4.2 as well, and do one
more release there that explicitly declares 2.6 support. That way
anyone stuck on Python 2.6 will end up getting the right hg if they
use a modern pip to install. Users can still use `python setup.py`
incantations to attempt installing Mercurial on unsupported Pythons,
including 3.5 and 3.6.

A followup change will switch to only doing our own
Python-version-check logic if we're not being installed by a
reasonable pip.
2017-07-21 10:39:52 -04:00
Augie Fackler
7dc97f30ea contrib: remove one test from the python 3 whitelist
5ba3f753c9b1 introduced a use of 'hg revert --no-backup' which is
currently broken. Rather than try and do a fix on the (complicated)
revert code during the freeze, let's just give up on this test under
Python 3 until later. Once we do fix things, the test ratchet script
will re-enable it.
2017-07-24 15:37:57 -04:00
Augie Fackler
4dfc9655ac ui: fix configbytes isinstance check to look for bytes and not str
Fixes configbytes on Python 3.
2017-07-24 13:50:25 -04:00
Augie Fackler
2ebd830d1d patch: update copying of dict keys and values to work on Python 3 2017-07-24 14:42:55 -04:00
Alex Gaynor
dbc20f0fe6 phabricator: include the suggested arc config in the repo
Test Plan:
I uploaded this revision with `arc diff`

Differential Revision: https://phab.mercurial-scm.org/D171
2017-07-21 14:22:08 +00:00
Martin von Zweigbergk
cf9a57caf9 match: override visitdir() in nevermatcher to return False
When we changed basematcher.visitdir() in 0ca205268beb (match: make
base matcher return True for visitdir, 2017-07-14), we forgot to add
an override in nevermatcher. This led to tests failing in narrowhg.

As Durham pointed out, it's high time to add unit tests for the
matcher, so this patch also adds a first unit test.

Differential Revision: https://phab.mercurial-scm.org/D151
2017-07-19 14:50:50 -07:00
Steve Borho
4981c8f4c5 wix: update list of necessary pyd files 2017-07-21 11:26:51 -05:00
Jun Wu
e2c4f27de2 run-tests: pre instantiate pygments objects
This speeds up run-tests.py diff output by 10x, which affects developer
experience significantly. As demonstrated by the following test:

```
#require pygments

  $ for i in `seq 1 200`; do
  >   echo '  $ echo '$i >> test-a.t
  >   echo '  wrong' >> test-a.t
  > done

  $ cat > walltime.py <<EOF
  > from __future__ import absolute_import, print_function
  > import os, sys, time
  > t1 = time.time()
  > os.system(' '.join(sys.argv[1:]) + ' >/dev/null 2>/dev/null')
  > t2 = time.time()
  > print('%0.2f' % (t2 - t1))
  > EOF

  $ $PYTHON walltime.py $TESTDIR/run-tests.py -l --color=never $TESTTMP/test-a.t
  0.24

Before this patch:

  $ $PYTHON walltime.py $TESTDIR/run-tests.py -l --color=always $TESTTMP/test-a.t
  2.46

After this patch:

  $ $PYTHON walltime.py $TESTDIR/run-tests.py -l --color=always $TESTTMP/test-a.t
  0.25
```
2017-07-21 08:43:39 -07:00
Augie Fackler
bb89feb159 contrib: drop Python 2.6 from manylinux1 wheel script 2017-07-19 13:44:22 -04:00
Augie Fackler
60b506ed63 contrib: make build-linux-wheels.sh executable 2017-07-19 13:45:06 -04:00
Siddharth Agarwal
71d0912db5 test-dirstate-race: hide irrelevant hg status output
See the explanation for more.

Differential Revision: https://phab.mercurial-scm.org/D153
2017-07-20 18:04:21 -07:00
Matt Harbison
771937deed test-commit: stabilize for filesystems without symlink support 2017-07-18 20:34:22 -04:00
Matt Harbison
cdf9eba4e7 test-https: properly conditionalize Windows vs non-Windows output
There's a bug in the (feature !) annotation, and this needs to be resolved
before fixing it.  Once fixed, the glob '*/missing' will match the output
'/missing' on non-Windows platforms, consuming the output line and leaving
nothing for the no-windows required line.

I probably should have made (foo !) match only the 'foo' feature, and be ignored
otherwise instead of optional.  (re) didn't work here, because the parenthesis
need to be escaped too.
2017-07-18 19:49:51 -04:00
Kevin Bullock
0a65c05074 Added signature for changeset c586a74f81b3 2017-07-19 07:58:03 -05:00
Rishabh Madan
923d0c8971 releasenotes: add custom admonitions support for release notes
By default, the extension has default sections like fix, feature, perf etc.. This
patch allow user to add support for custom admonition. In order to add a custom
admonition, one needs to have a .hgreleasenotes file inside the repository. All the
custom directive with name specified under the tag [sections] will be
usable by the extension. One important thing to keep in mind is if there exists any
custom admonitions with same key as default then they will override the default ones.
2017-07-18 23:04:08 +05:30
Ryan McElroy
5a6e1ad8d5 check-config: mention the file and line of the error
I used this to more quickly track down a failing test-check-config.t issue
in another repo. I thought it might be useful more generally, so I'm sending
it out in case others think it's a worthwhile change.
2017-07-18 06:27:36 -07:00
Durham Goode
7ab951502c rebase: add config to move rebase into a single transaction
This was previously landed as 507f16f4aa51 but backed out in a5abaa81fa because
it broke hook mid rebase and caused conflict resolution data loss in the event
of unexpected exceptions. This new version adds the behavior back but behind a
config flag, since the performance improvement is notable in large repositories.

The next patch adds a test covering this config.

The old commit message was:

Previously, rebasing would open several transaction over the course of rebasing
several commits. Opening a transaction can have notable overhead (like copying
the dirstate) which can add up when rebasing many commits.

This patch adds a single large transaction around the actual commit rebase
operation, with a catch for intervention which serializes the current state if
we need to drop back to the terminal for user intervention. Amazingly, almost
all the tests seem to pass.

On large repos with large working copies, this can speed up rebasing 7 commits
by 25%. I'd expect the percentage to be a bit larger for rebasing even more
commits.

There are minor test changes because we're rolling back the entire transaction
during unexpected exceptions instead of just stopping mid-rebase, so there's no
more backup bundle. It also leave an unknown file in the working copy, since our
clean up 'hg update' doesn't delete unknown files.
(grafted from cca36c7f35261b0e31beb226bf361067ef0e06ab)
(grafted from dc497d8705b71503e32e07bd33925c1e42cf9c9a)

Differential Revision: https://phab.mercurial-scm.org/D134
2017-07-18 07:47:28 -07:00
Martin von Zweigbergk
d1b1e4a13b run-tests: remove unnecessary 'with_color' variable
Its value is always the same as that of 'pygmentspresent'.

Differential Revision: https://phab.mercurial-scm.org/D118
2017-07-17 16:32:10 -07:00
Martin von Zweigbergk
63ec7f9083 run-tests: warn if --color=always and no pygments installed
Differential Revision: https://phab.mercurial-scm.org/D117
2017-07-17 16:27:13 -07:00