The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
This is necessary to communicate with third-party tools through command-server
channel. This requires SSLContext backported to Python 2.7.9+.
It doesn't look nice to pass ui by sslkwargs, but I think it is the only way
to do without touching various client codes including httpclient (aka http2).
ui is mandatory if certfile is specified, so it has no default value.
BTW, test-check-commit-hg.t complains that ssl_wrap_socket() has foo_bar
naming. Should I bulk-replace it to sslwrapsocket() ?
HTTPSConnection of Python 2.7.9 creates SSLContext in __init__, which involves
a password prompt for decrypting the private key. This means the password was
asked twice, one for unused SSLContext, and next for our ssl function.
Because our httpsconnection replaces connect() method at all, we can simply
drop httplib.HTTPSConnection. Instead, class and instance attributes are copied
from it.
HTTPSConnection of Python 2.7.8 and 2.6.9 seem to have no such problem.
https://hg.python.org/cpython/file/v2.7.9/Lib/httplib.py#l1183
SNI is a common way of sharing servers across multiple domains using separate
SSL certificates. As of Python 2.7.9 SSLContext has been backported from
Python 3. This patch changes sslutil's ssl_wrap_socket to use SSLContext and
take a server hostname as and argument. It also changes the url module to make
use of this argument.
The new code for 2.7.9 achieves it's task by attempting to get the SSLContext
object from the ssl module. If this fails the try/except goes back to what was
there before with the exception that the ssl_wrap_socket functions take a
server_hostname argument that doesn't get used. Assuming the SSLContext
exists, the arguments to wrap_socket at the module level are emulated on the
SSLContext. The SSLContext is initialized with the specified ssl_version. If
certfile is not None load_cert_chain is called with certfile and keyfile.
keyfile being None is not a problem, load_cert_chain will simply expect the
private key to be in the certificate file. verify_mode is set to cert_reqs. If
ca_certs is not None load_verify_locations is called with ca_certs as the
cafile. Finally the wrap_socket method of the SSLContext is called with the
socket and server hostname.
Finally, this fails test-check-commit-hg.t because the "new" function
ssl_wrap_socket has underscores in its names and underscores in its arguments.
All the underscore identifiers are taken from the other functions and as such
can't be changed to match naming conventions.
With Python 2.7.7rc1, "hg pull" through HTTP CONNECT tunnel fails due to the
removal of _set_hostport [1].
...
File "mercurial/url.py", line 372, in https_open
return self.do_open(self._makeconnection, req)
...
File "mercurial/url.py", line 342, in connect
_generic_proxytunnel(self)
File "mercurial/url.py", line 228, in _generic_proxytunnel
self._set_hostport(self.host, self.port)
AttributeError: httpsconnection instance has no attribute '_set_hostport'
self._set_hostport(self.host, self.port) should be noop and can be removed
because:
- _set_hostport() [2] was the function to parse "host:port" string and
set them to self.host and self.port,
- and (self.host, self.port) pair should be valid since connect() is called
prior to _generic_proxytunnel().
[1]: http://hg.python.org/cpython/rev/568041fd8090
[2]: http://hg.python.org/cpython/file/3a1db0d2747e/Lib/httplib.py#l721
When users are using a revset they can get multiple password prompts.
This prompts have no extra information about which password is being requested
so I added the authuri to the prompt to make it recognizable.
As in:
$ hg log -r "outgoing('https://bitbucket.org/mg/test') -
outgoing('https://bitbucket.org/nesneros/test')"
http authorization required
realm: Bitbucket.org HTTP
user: interrupted!
I changed it to describe the url when prompting for password.
As in:
$ hg log -r "outgoing('https://bitbucket.org/mg/test') -
outgoing('https://bitbucket.org/nesneros/test')"
http authorization required for https://bitbucket.org/mg/test
realm: Bitbucket.org HTTP
user: interrupted!
This is an attempt to fix issue 2451 and its duplicates (2599 and 2949, AFAIK).
Its main idea is that it is only necessary to clean the proxy environment
variables *when* http_proxy is set in the config file (since it takes
precedence over the environment variables). Otherwise, hg shouldn't bother with
them, since they will most likely be used to reach the server.
urllib2 never handles URIs with credentials, we have to extract them and store
them in the password manager before handing the stripped URI. Half of the
changes deducing the username from the URI in f7ae45a69fcd were incorrect.
Instead, we retrieve the username from the password manager before passing to
readauthforuri().
test-hgweb-auth.py was passing because the test itself was flawed: it was
passing URIs with credentials to find_password(), which never happens.
The [auth] section was ignored when handling URLs like:
http://user@example.com/foo
Instead, we look in [auth] for an entry matching the URL and supplied user
name. Entries without username can match URL with a username. Prefix length
ties are resolved in favor of entries matching the username. With:
foo.prefix = http://example.org
foo.username = user
foo.password = password
bar.prefix = http://example.org/bar
and the input URL:
http://user@example.org/bar
the 'bar' entry will be selected because of prefix length, therefore prompting
for a password. This behaviour ensure that entries selection is consistent when
looking for credentials or for certificates, and that certificates can be
picked even if their entries do no define usernames while the URL does.
Additionally, entries without a username matched against a username are
returned as if they did have requested username set to avoid prompting again
for a username if the password is not set.
v2: reparse the URL in readauthforuri() to handle HTTP and HTTPS similarly.
v3: allow unset usernames to match URL usernames to pick certificates. Resolve
prefix length ties in favor of entries with usernames.
The new http library is wired in via an extra module
(httpconnection.py), as it requires similar but different plumbing to
connect the library to Mercurial's internals and urllib2. Eventualy we
should be able to remove all of keepalive.py and its associated tangle
in url.py and replace it all with the code in httpconnection.py.
To use the new library, set 'ui.usehttp2' to true. The underlying http
library uses the logging module liberally, so if things break you can
use 'ui.http2debuglevel' to set the log level to INFO or DEBUG to get
that logging information (for example, ui.http2debuglevel=info.)
The introduction of the new URL parsing code has created a startup
time regression. This is mainly due to the use of url.hasscheme() in
the ui class. It ends up importing many libraries that the url module
requires.
This fix helps marginally, but if we can get rid of the urllib import
in the URL parser all together, startup time will go back to normal.
perfstartup time before the URL refactoring (707e4b1e8064):
! wall 0.050692 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
current startup time (9ad1dce9e7f4):
! wall 0.070685 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
after this change:
! wall 0.064667 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
While the URL parser is very forgiving about what characters are
allowed in each component, it's useful to be strict about the scheme
so we don't accidentally interpret local paths with colons as URLs.
This restricts schemes to containing alphanumeric characters, dashes,
pluses, and dots (as specified in RFC 2396).
This replaces util.drop_scheme() with url.localpath(), using url.url for
parsing instead of doing it on its own. The function is moved from
util to url to avoid an import cycle.
hg.localpath() is removed in favor of using url.localpath(). This
provides more consistent behavior between "hg clone" and other
commands.
To preserve backwards compatibility, URLs like bundle://../foo still
refer to ../foo, not /foo.
If a URL contains a scheme, percent-encoded entities are decoded. When
there's no scheme, all characters are left untouched.
Comparison of old and new behaviors:
URL drop_scheme() hg.localpath() url.localpath()
=== ============= ============== ===============
file://foo/foo /foo foo/foo /foo
file://localhost:80/foo /foo localhost:80/foo /foo
file://localhost:/foo /foo localhost:/foo /foo
file://localhost/foo /foo /foo /foo
file:///foo /foo /foo /foo
file://foo (empty string) foo /
file:/foo /foo /foo /foo
file:foo foo foo foo
file:foo%23bar foo%23bar foo%23bar foo#bar
foo%23bar foo%23bar foo%23bar foo%23bar
/foo /foo /foo /foo
Windows-related paths on Windows:
URL drop_scheme() hg.localpath() url.localpath()
=== ============= ============== ===============
file:///C:/foo C:/C:/foo /C:/foo C:/foo
file:///D:/foo C:/D:/foo /D:/foo D:/foo
file://C:/foo C:/foo C:/foo C:/foo
file://D:/foo C:/foo D:/foo D:/foo
file:////foo/bar //foo/bar //foo/bar //foo/bar
//foo/bar //foo/bar //foo/bar //foo/bar
\\foo\bar //foo/bar //foo/bar \\foo\bar
Windows-related paths on other platforms:
file:///C:/foo C:/C:/foo /C:/foo C:/foo
file:///D:/foo C:/D:/foo /D:/foo D:/foo
file://C:/foo C:/foo C:/foo C:/foo
file://D:/foo C:/foo D:/foo D:/foo
file:////foo/bar //foo/bar //foo/bar //foo/bar
//foo/bar //foo/bar //foo/bar //foo/bar
\\foo\bar //foo/bar //foo/bar \\foo\bar
For more information about file:// URL handling, see:
http://www-archive.mozilla.org/quality/networking/testing/filetests.html
Related issues:
- issue1153: File URIs aren't handled correctly in windows
This patch should preserve the fix implemented in
5c92d05b064e. However, it goes a step further and "promotes"
Windows-style drive letters from being interpreted as host names to
being part of the path.
- issue2154: Cannot escape '#' in Mercurial URLs (#1172 in THG)
The fragment is still interpreted as a revision or a branch, even in
paths to bundles. However, when file: is used, percent-encoded
entities are decoded, so file:test%23bundle.hg can refer to
test#bundle.hg ond isk.
Like the previous patch to getauthinfo(), this also makes
username/password parsing more forgiving for SSH URLs.
This also opens up the possibility of allowing non-numeric ports,
since the URL parser has no problem handling them.
Related issues:
- issue851: @ in password in http url
- issue2055: nonnumeric port bug with https protocol
This works around a potential issue in Python 2.4 where cloning a repo
with a URL like http://foo:8080 would cause urllib2 to query on
http://foo:8080?cmd=capabilities instead of
http://foo:8080/?cmd=capabilities.
In the past, this issue has been masked by the fact that
url.getauthinfo() added a trailing slash when it was missing.
This adds a url object that re-implements urlsplit() and
unsplit(). The implementation splits out usernames, passwords, and
ports.
The implementation is based on the behavior specified by RFC
2396[1]. However, it is much more forgiving than the RFC's
specification; it places no specific restrictions on what characters
are allowed in each segment of the URL other than what is necessary to
split the URL into its constituent parts.
[1]: http://www.ietf.org/rfc/rfc2396.txt
Internally, the group name is only used in debug statements, but readauthforuri
can be also used externally to determine which group will be matched for a given
URL.
