sslutil: add devel.disableloaddefaultcerts to disable CA loading

There are various tests for behavior when CA certs aren't loaded.
Previously, we would pass --insecure to disable loading of CA
certs. This has worked up to this point because the error message
for --insecure and no CAs loaded is the same. Upcoming commits will
change the error message for --insecure and will change behavior
when CAs aren't loaded.

This commit introduces the ability to disable loading of CA certs
by setting devel.disableloaddefaultcerts. This allows a testing
backdoor to disable loading of CA certs even if system/default
CA certs are available. The flag is purposefully not exposed to
end-users because there should not be a need for this in the wild:
certificate pinning and --insecure provide workarounds to disable
cert loading/validation.

Tests have been updated to use the new method. The variable used
to disable CA certs has been renamed because the method is not
OS X specific.

mercurial/sslutil.py

@@ -112,6 +112,9 @@ def _hostsettings(ui, hostname):
     Returns a dict of settings relevant to that hostname.
     """
     s = {
+        # Whether we should attempt to load default/available CA certs
+        # if an explicit ``cafile`` is not defined.
+        'allowloaddefaultcerts': True,
         # List of 2-tuple of (hash algorithm, hash).
         'certfingerprints': [],
         # Path to file containing concatenated CA certs. Used by
@@ -156,6 +159,9 @@ def _hostsettings(ui, hostname):
         s['disablecertverification'] = True
         s['verifymode'] = ssl.CERT_NONE
 
+    if ui.configbool('devel', 'disableloaddefaultcerts'):
+        s['allowloaddefaultcerts'] = False
+
     # Try to hook up CA certificate validation unless something above
     # makes it not necessary.
     if s['verifymode'] is None:
@@ -176,7 +182,7 @@ def _hostsettings(ui, hostname):
 
         # Require certificate validation if CA certs are being loaded and
         # verification hasn't been disabled above.
-        if cafile or _canloaddefaultcerts:
+        if cafile or (_canloaddefaultcerts and s['allowloaddefaultcerts']):
             s['verifymode'] = ssl.CERT_REQUIRED
         else:
             # At this point we don't have a fingerprint, aren't being
@@ -243,10 +249,12 @@ def wrapsocket(sock, keyfile, certfile, ui, serverhostname=None):
     if settings['cafile'] is not None:
         sslcontext.load_verify_locations(cafile=settings['cafile'])
         caloaded = True
-    else:
+    elif settings['allowloaddefaultcerts']:
         # This is a no-op on old Python.
         sslcontext.load_default_certs()
-        caloaded = _canloaddefaultcerts
+        caloaded = True
+    else:
+        caloaded = False
 
     sslsocket = sslcontext.wrap_socket(sock, server_hostname=serverhostname)
     # check if wrap_socket failed silently because socket had been

tests/test-https.t

@@ -162,21 +162,20 @@ Test server address cannot be reused
 #endif
   $ cd ..
 
-OS X has a dummy CA cert that enables use of the system CA store when using
-Apple's OpenSSL. This trick do not work with plain OpenSSL.
+Our test cert is not signed by a trusted CA. It should fail to verify if
+we are able to load CA certs.
 
-  $ DISABLEOSXDUMMYCERT=
 #if defaultcacerts
   $ hg clone https://localhost:$HGPORT/ copy-pull
   abort: error: *certificate verify failed* (glob)
   [255]
-
-  $ DISABLEOSXDUMMYCERT="--insecure"
 #endif
 
+  $ DISABLECACERTS="--config devel.disableloaddefaultcerts=true"
+
 clone via pull
 
-  $ hg clone https://localhost:$HGPORT/ copy-pull $DISABLEOSXDUMMYCERT
+  $ hg clone https://localhost:$HGPORT/ copy-pull $DISABLECACERTS
   warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostsecurity or web.cacerts config setting)
   requesting all changes
   adding changesets
@@ -202,7 +201,7 @@ pull without cacert
   $ cd copy-pull
   $ echo '[hooks]' >> .hg/hgrc
   $ echo "changegroup = printenv.py changegroup" >> .hg/hgrc
-  $ hg pull $DISABLEOSXDUMMYCERT
+  $ hg pull $DISABLECACERTS
   pulling from https://localhost:$HGPORT/
   warning: localhost certificate with fingerprint 91:4f:1a:ff:87:24:9c:09:b6:85:9b:88:b1:90:6d:30:75:64:91:ca not verified (check hostsecurity or web.cacerts config setting)
   searching for changes