Troubleshooting "ssl certificate verify failed" error

Question

On Windows Vista SP2 + Python 2.7.10 I can connect to https://www.python.org, but not to https://codereview.appspot.com

The script:

HOST1 = 'https://www.python.org'
HOST2 = 'https://codereview.appspot.com'

import urllib2
print HOST1
urllib2.urlopen(HOST1)
print HOST2
urllib2.urlopen(HOST2)

And the output:

E:\>py test.py
https://www.python.org
https://codereview.appspot.com
Traceback (most recent call last):
  File "test.py", line 9, in <module>
    urllib2.urlopen(HOST2)
  File "C:\Python27\lib\urllib2.py", line 158, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Python27\lib\urllib2.py", line 435, in open
    response = self._open(req, data)
  File "C:\Python27\lib\urllib2.py", line 453, in _open
    '_open', req)
  File "C:\Python27\lib\urllib2.py", line 413, in _call_chain
    result = func(*args)
  File "C:\Python27\lib\urllib2.py", line 1244, in https_open
    context=self._context)
  File "C:\Python27\lib\urllib2.py", line 1201, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

How can I troubleshoot, what exactly is wrong with https://codereview.appspot.com/ ?

Solution

My guess is that it is related to the alternative chain handling in OpenSSL, as described in detail in Python Urllib2 SSL error. Although Python uses the windows CA store to get the trusted root certificates the validation of the trust chain itself is done within OpenSSL.

According to "Python 2.7.10 Released" Python 2.7.10 on Windows includes OpenSSL 1.0.2a but the fixes regarding alternative chains were done in 1.0.2b only (and had to be fixed fast afterwards because they contained a serious security bug).

If you look at the SSLLabs report for codereview.appspot.com you can see that there are multiple trust chains which probably causes the problem. Contrary to that python.org only has a single trust chain.

To work around the problem it might be necessary to use your own root CA store which must contain the certificate for "/C=US/O=Equifax/OU=Equifax Secure Certificate Authority" to verify codereview.appspot.com correctly. The certificate can be found here and you can give it with the cafile parameter to urllib2.urlopen.

Comments

[anatoly techtonik]

Thanks. I see _ssl.pyd with version OpenSSL 1.0.2a 19 Mar 2015 in Python 2.7.10 and I haven't seen any new releases.

[anatoly techtonik]

Seems like I won't be able to recompile it myself. A pity that python.org can not release 2.7.11 or at least an independent update for _ssl.pyd

[Steffen Ullrich]

@techtonik: like I recommended you should be able to use a local CA file. For a good start have a look at the bundle from Mozilla which is available at the format support by python at curl.haxx.se/docs/caextract.html. But I recommend you get the old bundle which includes the RSA 1024 keys as long as you don't have an OpenSSL version which can handle multiple trust path properly.

[anatoly techtonik]

That would fix the problem for me personally, but won't help other users of codereview.appspot.com

[PhistucK]

@SteffenUllrich - can you explain why it does work on Windows 7?

[Steffen Ullrich]

@PhistucK: This is strange but Windows sometimes downloads missing root CA, so one system might differ from the other. If you want to ship the program you can just include the needed root CA with it so you don't depend on what is installed on the system.

[anatoly techtonik]

@PhistucK can you check that OpenSSL version is the same 1.0.2a?

[PhistucK]

Yes, this is the same version (I am not sure how to check it, I just opened the file with Notepad++ and looked for a version and found your stated one).