Commit Graph

33889 Commits

Author SHA1 Message Date
Durham Goode
c94bd0e75c changegroup: remove changegroup dependency from revlog.addgroup
Previously revlog.addgroup would accept a changegroup and a linkmapper and use
it to iterate of the deltas. As part of untangling the revlog-changegroup
interdependency, let's move the changegroup delta iteration logic to it's own
function and pass the simple iterator to the revlog instead.

This will make it easier to introduce non-revlogs stores in the future, without
reinventing any changegroup specific logic.

Differential Revision: https://phab.mercurial-scm.org/D688
2017-09-13 10:43:44 -07:00
Durham Goode
d27ceccf8a revlog: refactor chain variable
Previously the addgroup loop would set chain to be the result of
self._addrevision(node,...). Since _addrevision now always returns the passed in
node, we can drop that behavior and just always set chain = node in the loop.

This will be useful in a future patch where we refactor the cg.deltachunk logic
to another function and therefore chain disappears entirely from this function.

Differential Revision: https://phab.mercurial-scm.org/D699
2017-09-13 10:43:16 -07:00
Mark Thomas
3dfe8fba1c scmutil: don't append .orig to backups in origbackuppath (BC)
When ui.origbackuppath is set, .orig files are stored outside of the working
copy, however they still have a .orig suffix appended to them.  This can cause
unexpected conflicts, particularly when tracked files or directories have .orig
at the end.

This change removes the .orig suffix from files stored in an out-of-tree
origbackuppath.

Test Plan:
Update and run unit tests.

Differential Revision: https://phab.mercurial-scm.org/D679
2017-09-11 17:49:49 +00:00
Phil Cohen
be35a1685b merge: move cwd-missing detection to helper functions
This will exist in two places with defered writes, so we want to avoid
duplication.

Differential Revision: https://phab.mercurial-scm.org/D626
2017-09-12 19:27:01 -07:00
Yuya Nishihara
53d5208ef2 doctest: enable tests by default on Python 3
Still several tests fail mostly because of the string issues, sigh. I'll
fix them one by one.
2017-09-03 15:09:04 +09:00
Yuya Nishihara
b2d9e161ed doctest: normalize b'', u'' and exception output on Python 3
The idea is described in the following page.

https://dirkjan.ochtman.nl/writing/2014/07/06/single-source-python-23-doctests.html

# no-check-commit
2017-08-24 22:33:28 +09:00
Yuya Nishihara
69001a6da3 doctest: coerce dict.keys() to list
Otherwise it would be printed as odict_keys([...]) on Python 3.
2017-09-03 17:33:10 +09:00
Yuya Nishihara
6de6cb029d doctest: upgrade old-style "except" clause 2017-09-03 15:16:01 +09:00
Yuya Nishihara
dcc07e5503 doctest: use print_function and convert bytes to unicode where needed 2017-09-03 14:56:31 +09:00
Yuya Nishihara
21687d1395 doctest: do not embed non-ascii characters in docstring
Since the outer docstring is parsed as a unicode on Python 3, we have to
either double-escape or construct non-ascii string from ascii string.
2017-09-03 15:47:17 +09:00
Yuya Nishihara
c7d6fe4d91 doctest: pass encoding name as system string 2017-09-03 15:42:27 +09:00
Yuya Nishihara
a20b4175e2 doctest: replace str() with bytes() 2017-09-03 14:38:58 +09:00
Yuya Nishihara
0435ef1b07 doctest: replace chr() with pycompat.bytechr() 2017-09-03 14:37:25 +09:00
Yuya Nishihara
b8768dba29 doctest: replace .iteritems() with .items() 2017-09-03 14:35:37 +09:00
Yuya Nishihara
a71f259bd2 doctest: bulk-replace string literals with b'' for Python 3
Our code transformer can't rewrite string literals in docstrings, and I
don't want to make the transformer more complex.
2017-09-03 14:32:11 +09:00
Yuya Nishihara
e116353ac2 debuginstall: do not pass exception object to formatter (issue5676) 2017-09-07 22:36:54 +09:00
Yuya Nishihara
c3d677e9b5 debuginstall: use codecs.lookup() to detect invalid encoding
encoding.fromlocal() never tries to decode an ascii string since 3cb2361c60fc,
and there's no universal non-ascii string which can be decoded as any valid
character set.
2017-09-07 22:27:23 +09:00
Yuya Nishihara
04508d7f1c extensions: fix wrapcommand/function of class instance
5f4c097a17d2 changed _updatewrapper() to copy the __name__ attribute, but
not all callable objects has __name__.

Spotted by loading mq with extdiff.
2017-09-10 23:37:14 +09:00
Durham Goode
792f16d38c changegroup: avoid creating empty changegroup part
Previously this check happened in the changegroup code itself. Since its
refactor, this logic needs to move out to callers that care about it, such as
this one. Otherwise we get empty bundle devel-warnings in certain extensions.

Differential Revision: https://phab.mercurial-scm.org/D690
2017-09-12 09:13:02 -07:00
Durham Goode
6fe0002838 tests: split test-revset.t in half
This test has gotten so large that running it can exceed the normal timeout on
systems under load (like if we're running all the tests in parallel). This patch
splits the test cleanly in half.

Differential Revision: https://phab.mercurial-scm.org/D694
2017-09-12 15:12:27 -07:00
Phil Cohen
b481a15354 merge: flush any deferred writes just before recordupdates()
``recordupdates`` calls into the dirstate which requires the files to be
there, so this is the last possible moment we can flush anything.

Differential Revision: https://phab.mercurial-scm.org/D673
2017-09-11 13:17:43 -07:00
Phil Cohen
0e9cb373ec merge: flush any deferred writes before, and after, running any workers
Since we fork to create workers, any changes they queue up will be lost after
the worker terminates, so the easiest solution is to have each worker flush
the writes they accumulate--we are close to the end of the merge in any case.

To prevent duplicated writes, we also have the master processs flush before
forking.

In an in-memory merge (M2), we'll instead disable the use of workers.

Differential Revision: https://phab.mercurial-scm.org/D628
2017-09-11 13:03:27 -07:00
Phil Cohen
152d58b25d filemerge: flush if using deferred writes when running a merge tool
Since merge tools might read from the filesystem, we need to write out our
deferred writes here.

No-ops if not using deferred writes.

Differential Revision: https://phab.mercurial-scm.org/D627
2017-09-11 13:03:27 -07:00
Phil Cohen
a1eba8f292 merge: pass wctx to premerge, filemerge
In the in-memory merge branch. we'll need to call a function (``flushall``) on
the wctx inside of _xmerge.

This prepares the way so it can be done without hacks like ``fcd.ctx()``.

Differential Revision: https://phab.mercurial-scm.org/D449
2017-09-11 13:03:27 -07:00
Boris Feld
60d92de7ba cmdutil: fix amend when passing a date
Following https://phab.mercurial-scm.org/D636, passing the same date that the
changeset to amend would results in no new commits but the output changed
from:

  $ hg amend -d '0 0'
  nothing changed
  [1]

to:

  $ hg amend -d '0 0'

Restore the old behavior by parsing the date passed as parameter so the
condition "date == old.date()" correctly works in cases both dates are
identical.

Add a test for covering this regression.

This bug was found thanks to Evolve test suite.

Differential Revision: https://phab.mercurial-scm.org/D691
2017-09-11 18:07:29 +02:00
Augie Fackler
85e2672134 merge with stable 2017-09-12 11:18:35 -04:00
Mads Kiilerich
3676051d80 mq: create non-lossy patches, also with custom global diff configuration
Users with custom [diff] configuration most certainly didn't intend it to make
mq lose changes. It could:

 * git is handled perfectly fine.

 * nobinary could make mq leave some files out from the patches.

 * noprefix could make mq itself (and probably also other tools) fail to apply
   patches without the usual a/b prefix.

 * ignorews, ignorewsamount, or ignoreblanklines could create patches with
   missing whitespace that could fail to apply correctly.

Thus, when refreshing patches, use patch.difffeatureopts, optionally with git
as before, but without the config options for whitespace and format changing
that most likely will cause loss or problems.

(patch.diffopts is just patch.difffeatureopts with all options enabled and can
be replaced with that.)
2017-09-11 00:42:24 +02:00
Mads Kiilerich
4cfa527133 mq: test coverage of how [diff] configuration influence can break mq patches 2017-09-11 00:42:22 +02:00
Matt Harbison
19d34e4d32 debugssl: allow a URL to be specified without a local repository
This was the original intent, but I bungled the logic.  Otherwise if there is a
certificate chain issue, the repository can't be cloned in order for there to be
a repo object.  I think I missed this case because I was inside of a Mercurial
clone as I was originally developing and testing this.
2017-08-29 16:38:10 -04:00
Jun Wu
45a4782018 record: fix revert -i for lines without newline (issue5651)
This is a regression caused by 10c1efcbeb1e. Code prior to 10c1efcbeb1e
seems to miss the "\ No newline at end of file" line.

Differential Revision: https://phab.mercurial-scm.org/D528
2017-08-27 13:39:17 -07:00
Martin von Zweigbergk
7cd70adbc1 templatekw: choose {latesttag} by len(changes), not date (issue5659)
As Augie reported in the bug, the current heuristic of choosing the
best tag of a merge commit by taking the one with newest tag (in terms
of tagging date) currently fails in the Mercurial repo itself. Copying
the example from Yuya:

  $ hg glog -T '{node|short} {latesttag}+{latesttagdistance}\n' \
    -r '4.2.3: & (merge() + parents(merge()) + tag())'
  o    cc59efae4cc0 4.2.3+5
  |\
  | o    06f60e88fc3a 4.2.3+4
  | |\
  | | o  c191a9eb0b10 4.3-rc+109
  | | |
  | | ~
  o |  49ada93fdc10 4.3.1+2
  : |
  o |  229937197835 4.3.1+0
  |/
  o    6a83ad94c0f2 4.2.3+3
  |\
  | ~
  o  8e9dcdd1de74 4.2.3+2
  :
  o  525f2b18248f 4.2.3+0
  |
  ~

It seems to me like the best choice is the tag with the smallest
number of changes since it (across all paths, not the longest single
path). So that's what this patch does, even though it's
costly. Best-of-5 timings for Yuya's command above shows a slowdown
from 1.293s to 1.610s. We can optimize it later.

Differential Revision: https://phab.mercurial-scm.org/D447
2017-08-15 23:23:55 -07:00
Martin von Zweigbergk
8bb8bc19bb tests: use graph log in {latesttag} tests
The tests are much easier to read if one does not have to re-read the
setup part all the time to understand the graph shape.

Differential Revision: https://phab.mercurial-scm.org/D446
2017-08-18 12:50:26 -07:00
Gregory Szorc
8e7a19b422 ui: restore behavior to ignore some I/O errors (issue5658)
45345e9870c3 and b30126fa95bc refactored ui methods to no longer
silently swallow some IOError instances. This is arguably the
correct thing to do. However, it had the unfortunate side-effect
of causing StdioError to bubble up to sensitive code like
transaction aborts, leading to an uncaught exceptions and failures
to e.g. roll back a transaction. This could occur when a remote
HTTP or SSH client connection dropped. The new behavior is
resulting in semi-frequent "abandonded transaction" errors on
multiple high-volume repositories at Mozilla.

This commit effectively reverts 45345e9870c3 and b30126fa95bc to
restore the old behavior.

I agree with the principle that I/O errors shouldn't be ignored.
That makes this change... unfortunate. However, our hands are tied
for what to do on stable. I think the proper solution is for the
ui's behavior to be configurable (possibly via a context manager).
During critical sections like transaction rollback and abort, it
should be possible to suppress errors. But this feature would not
be appropriate on stable.
2017-08-15 13:04:31 -07:00
Gregory Szorc
e98da01acf tests: test behavior of IOError during transactions (issue5658)
ui._write(), ui._write_err(), and ui.flush() all trap IOError and
re-raise as error.StdioError. If a caller doesn't catch StdioError
when writing to stdio, it could bubble all the way to dispatch.

This commit adds tests for I/O failures around various transaction
operations.

The most notable badness is during abort. Here, an uncaught StdioError
will result in incomplete transaction rollback, requiring an
`hg rollback` to recover. This can result in a client "corrupting"
a remote repo via terminated HTTP and SSH socket.
2017-08-14 13:12:40 -07:00
Nathan Goldbaum
0de8be7e62 log: mention ui.logtemplate in the help text 2017-08-16 10:24:49 -05:00
Tristan Seligmann
0428b15b61 hg: avoid relying on errno numbers / descriptions
A few tests hardcode errno numbers and/or descriptions in the output, causing
test failures on platforms where these values are different.

Differential Revision: https://phab.mercurial-scm.org/D362
2017-08-12 14:29:22 +02:00
Tristan Seligmann
0fee28d767 hg: tolerate long vs. int in test-context.py
The file times here can be longs instead of ints on some platforms, which will
cause a test failure due to these printing with an L suffix; instead always
format with %d which will produce the same output in either case.

Differential Revision: https://phab.mercurial-scm.org/D361
2017-08-12 14:24:25 +02:00
Mike Hommey
634713aa96 setup: fix installing in a mingw environment
The addition, in 851d08ff7a58, of a hack for the MSVC compiler class was
overwriting the original class for the Mingw32CCompiler class, leading to an
error when the HackedMingw32CCompiler is instantiated.

Differential Revision: https://phab.mercurial-scm.org/D329
2017-08-11 10:16:00 +09:00
Kevin Bullock
5e1176676c osx: delay version computation on macOS builds
Before this patch, HGVER would be evaluated at the beginning of the make
execution, and would be unset because build/mercurial/ doesn't exist yet
at that point. Now we compute the version after the `make install` run
has completed.

This is backported to stable from 8626b44516c1, but that revision had an
error in the shell invocation syntax.
2017-08-10 21:25:02 -04:00
Mike Hommey
73f4a16a4f branchmap: revert a81106261d38 for Python 2.7 compatibility
Old versions of python 2.7 don't like that the second argument to
struct.unpack_from is a bytearray, so the change removing the util.buffer
around that argument in branchmap broke running on older versions of python
2.7.

Differential Revision: https://phab.mercurial-scm.org/D330
2017-08-10 20:47:19 -07:00
Augie Fackler
abe39330a6 Added signature for changeset 229937197835 2017-08-10 18:46:55 -04:00
Yuya Nishihara
509744ddfc ssh: unban the use of pipe character in user@host:port string
This vulnerability was fixed by the previous patch and there were more ways
to exploit than using '|shellcmd'. So it doesn't make sense to reject only
pipe character.

Test cases are updated to actually try to exploit the bug. As the SSH bridge
of git/svn subrepos are not managed by our code, the tests for non-hg subrepos
are just removed.

This may be folded into the original patches.
2017-08-07 22:22:28 +09:00
Jun Wu
a0e5a4defb ssh: quote parameters using shellquote (SEC)
This patch uses shellquote to quote ssh parameters more strictly to avoid
shell injection.
2017-08-04 23:54:12 -07:00
Sean Farley
39898f2a8a subrepo: add tests for git rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 14:55:11 -07:00
Sean Farley
da301ac6a0 subrepo: add tests for svn rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 16:44:17 -07:00
Sean Farley
f904aef7aa subrepo: add tests for hg rogue ssh urls (SEC)
'ssh://' has an exploit that will pass the url blindly to the ssh
command, allowing a malicious person to have a subrepo with
'-oProxyCommand' which could run arbitrary code on a user's machine. In
addition, at least on Windows, a pipe '|' is able to execute arbitrary
commands.

When this happens, let's throw a big abort into the user's face so that
they can inspect what's going on.
2017-07-31 16:04:44 -07:00
Sean Farley
ea5a6b2798 push: add tests for unsafe ssh url (SEC) 2017-07-31 14:40:28 -07:00
Sean Farley
616cd64fcf pull: add tests for unsafe ssh url (SEC) 2017-07-28 16:47:32 -07:00
Sean Farley
ee608897f9 clone: add tests for unsafe ssh url (SEC) 2017-07-28 16:36:36 -07:00
Sean Farley
e199b92002 sshpeer: check for safe ssh url (SEC)
Checking in the sshpeer for a rogue ssh:// urls seems like the right
place to do it (instead of whack-a-mole with pull, clone, push, etc).
2017-08-01 14:40:19 -07:00