Commit Graph

29807 Commits

Author SHA1 Message Date
Jun Wu
033b57c8df logtoprocess: do not leak the ui object in uisetup
logtoprocess.log should use "self" passed in function arguments instead
of the "ui" object from outside the function.
2016-06-30 15:26:11 +01:00
Jun Wu
35f0266cea chgserver: document why we don't merge mtimehash and confighash
People may get confused about chg's mtimehash and confighash design: why two
hashes instead of just one. This patch adds text addressing the concern.
2016-06-29 23:53:20 +01:00
Jun Wu
1dbd891ce1 extensions: move uisetup and extsetup to standalone functions
This is to make them wrap-able. chgserver wants to know if an extension
accesses config or environment variables during uisetup and extsetup and
include them in confighash accordingly.
2016-06-30 10:31:50 +01:00
Matt Mackall
c0d551e8ec merge with stable 2016-07-01 16:02:56 -05:00
Anton Shestakov
f13dffc250 hgweb: add absolute urls for archives in json-summary 2016-07-01 13:54:35 +08:00
Anton Shestakov
43becb126d tests: allow bz2 archives in test-hgweb-json.t
Only testing a specific type because list items seem to be in arbitrary order.
2016-07-01 13:36:59 +08:00
Pulkit Goyal
123aebf3ce keepalive: switch from thread to threading module
The thread module in py3 is renamed to _thread, but we can use
 the high level threading module instead.
2016-07-01 19:17:16 +05:30
Pulkit Goyal
af9d7f66d0 py3: conditionalize httplib import
The httplib library is renamed to http.client in python 3. So the
import is conditionalized and a test is added in check-code to warn
to use util.httplib
2016-06-28 16:01:53 +05:30
Gregory Szorc
6adc4b7bac sslutil: emit warning when no CA certificates loaded
If no CA certificates are loaded, that is almost certainly a/the
reason certificate verification fails when connecting to a server.

The modern ssl module in Python 2.7.9+ provides an API to access
the list of loaded CA certificates. This patch emits a warning
on modern Python when certificate verification fails and there are
no loaded CA certificates.

There is no way to detect the number of loaded CA certificates
unless the modern ssl module is present. Hence the differences
in test output depending on whether modern ssl is available.

It's worth noting that a test which specifies a CA file still
renders this warning. That is because the certificate it is loading
is a x509 client certificate and not a CA certificate. This
test could be updated if anyone is so inclined.
2016-06-29 19:43:27 -07:00
Gregory Szorc
169cc46dab tests: test case where default ca certs not available
I'm not a fan of TLS tests not testing both branches of a possible
configuration. While we have test coverage of the inability to validate
a cert later in this file, I insist that we add this branch so
our testing of security code is extra comprehensive.
2016-06-29 19:49:39 -07:00
Gregory Szorc
9e29784a89 sslutil: don't load default certificates when they aren't relevant
Before, we would call SSLContext.load_default_certs() when
certificate verification wasn't being used. Since
SSLContext.verify_mode == ssl.CERT_NONE, this would ideally
no-op. However, there is a slim chance the loading of system
certs could cause a failure. Furthermore, this behavior
interfered with a future patch that aims to provide a more
helpful error message when we're unable to load CAs.

The lack of test fallout is hopefully a sign that our
security code and tests are in a relatively good state.
2016-06-29 19:38:24 -07:00
Gregory Szorc
9653d9a563 sslutil: display a better error message when CA file loading fails
Before, sslcontext.load_verify_locations() would raise a
ssl.SSLError which would be caught further up the stack and converted
to a urlerror. By that time, we lost track of what actually errored.

Trapping the error here gives users a slightly more actionable error
message.

The behavior between Python <2.7.9 and Python 2.7.9+ differs. This
is because our fake SSLContext class installed on <2.7.9 doesn't
actually do anything during load_verify_locations: it defers actions
until wrap_socket() time. Unfortunately, a number of errors can occur
at wrap_socket() time and we're unable to ascertain what the root
cause is. But that shouldn't stop us from providing better error
messages to people running a modern and secure Python version.
2016-06-29 19:37:38 -07:00
Gregory Szorc
3481b4d0f7 tests: add test for empty CA certs file
smf reported that an environment with no loaded CA certs resulted
in a weird error. I'd like to detect this a bit better so we can
display an actionable error message.

The actual error being globbed over in this patch is "unknown error"
with a ssl.c line number. That isn't useful at all.
2016-06-29 18:15:28 -07:00
Maciej Fijalkowski
8e7a874bdf internals: move the bitmanipulation routines into its own file
This is to allow more flexibility with the C sources -- now the
bitmanipulation routines can be safely imported without importing Python.h
2016-06-06 13:08:13 +02:00
Martijn Pieters
5eddc7a7b3 journal: new experimental extension
Records bookmark locations and shows you where bookmarks were located in the
past.

This is the first in a planned series of locations to be recorded; a future
patch will add working copy (dirstate) tracking, and remote bookmarks will be
supported as well, so the journal storage format should be fairly generic to
support those use-cases.
2016-06-24 16:12:05 +01:00
Augie Fackler
6a644ad048 httpclient: update to 54868ef054d2 of httpplus
As of that revision, httpplus fully supports Python 3, including
mimicing all the subtle behavior changes around headers in Python 3's
http.client.
2016-06-27 11:53:50 -04:00
Yuya Nishihara
ba3291048d revset: check invalid function syntax "func-name"() explicitly
Before the error was caught at func() as an unknown identifier, and the
optimizer failed to detect the syntax error. This patch introduces getsymbol()
helper to ensure that a string is not allowed as a function name.
2016-06-27 20:44:14 +09:00
Yuya Nishihara
5bae87a105 chg: silence warning of unused parameter 'sig' 2016-06-28 22:39:06 +09:00
Anton Shestakov
230d011cdc hgweb: reindent atom/changelogentry.tmpl
It was mixing tabs and spaces, and not in a good way.

Indent style of other atom entries seems to be 1 space per level, so let's
apply it here as well.
2016-06-29 22:48:32 +08:00
Anton Shestakov
e5788a6e8c hgweb: remove unused/nonexistent filelogentry from atom/map
filelog in atom style uses changelogentry to show commits, and
atom/filelogentry.tmpl doesn't even exist.
2016-06-29 21:07:59 +08:00
Anton Shestakov
385d153e4e hgweb: remove unnecessary CDATA markup in atom/branchentry.tmpl
Since content is of type "text" (and is already escaped), using a CDATA section
is not required.

Looks like this was just an artifact of copying things from rss style in
529b23a26574, because other entries in atom style don't use CDATA in such
places.
2016-06-29 20:58:09 +08:00
Anton Shestakov
b11305ab3e hgweb: reindent rss/changelogentry.tmpl
It was mixing tabs and spaces, and not in a good way.

Indent style of other rss entries seems to be 4 spaces per level, so let's
apply it here as well.
2016-06-29 20:19:04 +08:00
Anton Shestakov
71ef725f83 hgweb: remove a couple of superfluous spaces in rss style 2016-06-29 19:40:04 +08:00
Pulkit Goyal
c74034c936 py3: add tests in check-code to load modules from util.py
The conditionalize imports are added in util.py and now we import modules from
there. So adding tests so that someone in future can use that.
2016-06-27 19:10:30 +05:30
Pulkit Goyal
38a359ce5c py3: conditionalize SocketServer import
The SocketServer is renamed to socketserver in python 3
2016-06-27 16:48:54 +05:30
Pulkit Goyal
fdc0861e35 py3: conditionalize xmlrpclib import
The xmlrpclib library is renamed to xmlrpc.client in python 3
2016-06-27 16:37:37 +05:30
Pulkit Goyal
5fcc6a2628 py3: conditionalize the urlparse import
The urlparse library is renamed to urllib.parse in python 3
2016-06-27 16:16:10 +05:30
Pulkit Goyal
175e976a5b py3: update tests/test-check-py3-compat.t
The lower part of the test runs with python 3 and hence remain unchanged.
2016-06-27 15:53:38 +05:30
Jun Wu
dfebfdbf56 chg: send SIGPIPE to server immediately when pager exits (issue5278)
If the user press 'q' to leave the 'less' pager, it is expected to end the
hg process immediately. We currently rely on SIGPIPE for this behavior. But
SIGPIPE won't arrive if we don't write anything (like doing heavy
computation, reading from network etc). If that happens, the user will feel
that the hg process just hangs.

The patch address the issue by adding a SIGCHLD signal handler and sends
SIGPIPE to the server as soon as the pager exits.

This is also an issue with hg's pager implementation.
2016-06-24 15:21:10 +01:00
Jun Wu
9579ecf5f2 chgserver: do not ignore SIGPIPE if pager is used
We rely on SIGPIPE to exit when the pager exits. And Python ignores SIGPIPE
by default. Explicitly set SIGPIPE handler to SIG_DFL (terminate) just like
pager.py.
2016-06-24 17:06:41 +01:00
Martin von Zweigbergk
5dcec73439 debug: make debug{revlog,index,data} --dir not just a flag
The directory argument (for tree manifests) should belong to to the
--dir argument. I had mistakenly made --dir a flag. One effect of this
was that I had meant for "-m" to be optional, but instead it changed
the behavior of --dir, so with "hg debugdata -m --dir dir1 0", the -m
took over and the "dir1" got treated as a revision in the root
manifest log.
2016-06-24 11:12:41 -07:00
Martin von Zweigbergk
a785632cca debugdata: disallow trailing option with -c/-m
Before this change, "hg debugdata -c 0 foo" was allowed.
2016-06-24 11:25:55 -07:00
Yuya Nishihara
7323118189 revset: get rid of redundant error checking from match()
Actually there was no additional error checking. It should be caught by
"not all(specs)".
2016-06-26 17:16:57 +09:00
Martijn Pieters
a8afdea978 share: move magic string to a constant 2016-06-24 10:32:38 +01:00
liscju
6d9c9237d1 bookmarks: allow pushing active bookmark on new remote head (issue5236)
Before 'hg push -B .' on new remote head complained with:
abort: push creates new remote head ...

It was because _nowarnheads was not expanding active bookmark
name, so it didn't add active bookmark "proper" name to no
warn heads list.
2016-05-24 23:36:27 +02:00
Laurent Charignon
94c489ecf6 strip: invalidate phase cache after stripping changeset (issue5235)
When we remove a changeset from the changelog, the phase cache must be
invalidated, otherwise it could refer to changesets that are no longer in the
repo.

To reproduce the failure, I created an extension querying the phase cache after
the strip transaction is over.

To do that, I stripped two commits with a bookmark on one of them to force
another transaction (we open a transaction for moving bookmarks)
after the strip transaction.

Without the fix in this patch, the test leads to a stacktrace showing the issue:
      repair.strip(ui, repo, revs, backup)
    File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/repair.py", line 205, in strip
      tr.close()
    File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/transaction.py", line 44, in _active
      return func(self, *args, **kwds)
    File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/transaction.py", line 490, in close
      self._postclosecallback[cat](self)
    File "$TESTTMP/crashstrip2.py", line 4, in test
      [repo.changelog.node(r) for r in repo.revs("not public()")]
    File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/changelog.py", line 337, in node
      return super(changelog, self).node(rev)
    File "/Users/lcharignon/facebook-hg-rpms/hg-crew/mercurial/revlog.py", line 377, in node
      return self.index[rev][7]
  IndexError: revlog index out of range

The situation was encountered in inhibit (evolve's repo) where we would crash
following the volatile set invalidation submitted by Augie in
cbc52a99d057d11790cf5011e877c6f698bf57bf. Before his patch the issue was masked
as we were not accessing the phasecache after stripping a revision.

This bug uncovered another but in histedit (see explanation in issue5235).
I changed the histedit test accordingly to avoid fixing two things at once.
2016-05-12 06:13:59 -07:00
Mateusz Kwapich
718300b2b4 localrepo: prevent executable-bit only changes from being lost on amend
If you have just executable-bit change and amend it twice it will vanish:

 * After the first amend the commit will have the proper executable bit set
   in manifest but it won't have the the file on the list of files in
   changelog.

 * The second amend will read the wrong list of files from changelog and it
   will copy the manifest entry from parent for this file.

 * Voila! The change is lost.

This change repairs the bug in localrepo causing this and adds a test for it.
2016-05-19 14:35:22 -07:00
FUJIWARA Katsunori
e6eaaeeb67 tests: escape bytes setting MSB in input of grep for portability
GNU grep (2.21-2 or later) assumes that input is encoded in LC_CTYPE,
and input is binary if it contains byte sequence not valid for that
encoding.

For example, if locale is configured as C, a byte setting most
significant bit (MSB) makes such GNU grep show "Binary file <FILENAME>
matches" message instead of matched lines unintentionally.

This behavior is recognized as a bug, and fixed in GNU grep 2.25-1 or
later. But some distributions are shipped with such buggy version
(e.g. Ubuntu xenial, which is used by launchpad buildbot).

    http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19230
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800670
    http://packages.ubuntu.com/xenial/grep

This causes failure of test-commit-interactive.t, which applies grep
on CP932 byte sequence since 4681296e309b.

But, explicit setting LC_CTYPE for CP932 might cause another problem,
because it can't be assumed that all environment running Mercurial
tests allows arbitrary locale setting.

To resolve this issue, this patch escapes bytes setting MSB in input
of grep.

For this purpose:

  - str.encode('string-escape') isn't useful, because it escapes also
    control code (less than 0x20), and makes EOL handling complicated

  - "f --hexdump" isn't useful, because it isn't line-oriented

  - "sed -n" seems reasonable, but "sed" itself sometimes causes
    portability issue, too (e.g. 215a8789129e or 6d02ef568139)

This patch is posted with "stable" flag, because 4681296e309b is on
stable branch.
2016-05-21 02:48:51 +09:00
Matt Mackall
6659cb364b Added signature for changeset 923825600655 2016-05-16 17:21:25 -05:00
FUJIWARA Katsunori
17fa5d0520 help: search section of help topic by translated section name correctly
Before this patch, "hg help topic.section" might show unexpected
section of help topic in some encoding.

It applies str.lower() instead of encoding.lower(str) on translated
message to search section case-insensitively, but some encoding uses
0x41(A) - 0x5a(Z) as the second or later byte of multi-byte character
(for example, ja_JP.cp932), and str.lower() causes unexpected result.

To search section of help topic by translated section name correctly,
this patch replaces str.lower() by encoding.lower(str) for both query
string (in commands.help()) and translated help text (in
minirst.getsections()).
2016-05-13 07:19:59 +09:00
FUJIWARA Katsunori
63611bb308 patch: show lower-ed translated message correctly
Before this patch, patch.filterpatch() shows meaningless translation
of help message for chunk selection in some encoding.

It applies str.lower() instead of encoding.lower(str) on translated
message, but some encoding uses 0x41(A) - 0x5a(Z) as the second or
later byte of multi-byte character (for example, ja_JP.cp932), and
str.lower() causes unexpected result.

To show lower-ed translated message correctly, this patch replaces
str.lower() by encoding.lower(str).
2016-05-13 07:19:59 +09:00
Matt Harbison
8d72832828 help: fix the display for hg help internals.revlogs (issue5227)
It previously aborted saying the help section wasn't found.  Credit to Yuya for
figuring out the fix.
2016-05-08 22:28:09 -04:00
Sean Farley
b611d5489b builddeb: add distroseries to tagged versions
This is needed so that launchpad and friends have a unique version number for
each distroseries (trusty, wily, xenial, etc). It was discovered when trying to
upload 3.8 to launchpad.
2016-05-06 14:09:11 -07:00
Yuya Nishihara
961a93ff5b log: drop hack to fix order of revset (issue5100)
Specify ordered=revset.followorder instead.

This patch effectively backs out 54ac550f0fd6. revs.sort(reverse=True)
is replaced by revs.reverse() because the matcher should no longer reorder
revisions.
2016-05-03 14:24:00 +09:00
Yuya Nishihara
a667f16896 revset: add option to make matcher takes the ordering of the input set
This allows us to evaluate match(subset) as if 'subset & expr', which will
be the complete fix for the issue5100.
2016-05-03 14:18:28 +09:00
Martin von Zweigbergk
e3fc1041dc strip: don't use "full" and "partial" to describe bundles
The partial bundle is not a subset of the full bundle, and the full
bundle is not full in any way that i see. The most obvious
interpretation of "full" I can think of is that it has all commits
back to the null revision, but that is not what the "full" bundle
is. The "full" bundle is simply a backup of what the user asked us to
strip (unless --no-backup). The "partial" bundle contains the
revisions we temporarily stripped because they had higher revision
numbers that some commit that the user asked us to strip.

The "full" bundle is already called "backup" in the code, so let's use
that in user-facing messages too. Let's call the "partial" bundle
"temporary" in the code.
2016-09-19 09:14:35 -07:00
Martin von Zweigbergk
fe544b2a62 strip: clarify that user action is required to recover temp bundle
If strip fails when applying the temporary bundle, the commits in the
temporary bundle have not yet been applied, so the user will almost
definitely want to apply the bundle. We should be more clear to the
user about that than our current "partial bundle stored in...".

Note that we will probably not be able to recover it automatically,
since whatever made it fail (e.g. a hook) will most likely make it
fail again. We need to give control back to the user to fix the
problem before trying again.
2016-09-19 09:14:32 -07:00
Martin von Zweigbergk
c435ce4f96 strip: report both bundle files in case of exception (issue5368)
If strip fails while recovering the temporary bundle (e.g. because a
hook fails), we tell the user only about the backup bundle, not about
the temporary bundle. Since the user did not ask to strip the commits
in the temporary bundle, that's the more important bundle to mention,
so let's do that (and also mention the backup bundle as usual).
2016-09-15 09:45:29 -07:00
Martin von Zweigbergk
19513cd917 strip: simplify some repeated conditions
We check "if saveheads or savebases" in several places to see if we
should or have created a bundle of the changesets to apply after
truncating the revlogs. One of the conditions is actually just "if
saveheads", but since there can't be savebases without saveheads, that
is effectively the same condition. It seems simpler to check only once
and from then on see if we created the file.
2016-09-15 10:18:56 -07:00
Mathias De Maré
109af2a941 config: add template support
V2:
- Limit escaping to plain formatting only
- Use the formatter consistently (no more ui.debug)
- Always include 'name' and 'value'

V3:
- Always convert 'value' to string (this also makes sure we handle functions)
- Keep real debug message as ui.debug for now
- Add additional tests.
  Note: I'm not quite sure about the best approach to handling
  the 'print the full config' case.
  For me, it printed the 'ui.promptecho' key at the end.
  I went with globs there as that at least tests the json display reliably.

Example output:
[
 {
  "name": "ui.username",
  "source": "/home/mathias/.hgrc:2",
  "value": "Mathias De Maré <mathias.demare@gmail.com>"
 }
]
2016-08-29 07:07:15 +02:00