I am trying to use the following code to download a simple https:// URL in Windows 10:
from urllib.request import urlopen
But that code crashes with:
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)>
This error appears to indicate that Python couldn’t validate the SSL certificate from the xkcd.com domain.
ssl.create_default_context() (used above) tells Python to use the same certificate authorities as elsewhere on Windows 10. In a different application (Edge) I can successfully visit xkcd.com so it appears that the system certificate authorities match the certificate provided by that domain. So I’m not sure why Python can’t validate the domain’s certificate.
How can I alter the above code to use the Windows 10 certificate authorities and not give a certificate validation error?
This is very easy with requests (or even urllib3) instead of urllib, you just need to
pip install requests at the command line (something already installed it in my global Python so I didn’t use a venv). Both handle SSL verification automatically for you. This worked just now in Windows 11.
resp = requests.get('https://www.xkcd.com')
Your code does work for me as it is too (in Python 3.11. Perhaps my security settings are lax e.g. dev mode). Do you have a specific requirement for urllib, or to use a special ssl context?
You can play with SSLContext.load_default_certs() and other methods too, to try and find where they are on your system.
I’m actually using the lower-level
http.client) in my production code, which requires an
SSLContext (just like
So the following would be a more accurate snippet that doesn’t work for me on Windows 10†:
from http.client import HTTPSConnection
conn = HTTPSConnection('xkcd.com:443', context=ssl.create_default_context())
conn.request('GET', '/') # fails with: ssl.SSLCertVerificationError
I haven’t tried the above snippet on Windows 11 yet, but I can whip up a VM this evening to check.
† Specifically I’m running the above code on a fresh Windows 10 in a VM with all OS updates applied and the clock set correctly
Cheers. That version still works for me out of the box.
If you need to run it in a VM anyway, can you run it in Windows Subsystem for Linux, or is this code for regular Windows users?
My Windows 7 system certificates contains the CA cert for xkcd.com but my Windows 10 system certificates do NOT:
xkcd certificate (and related CA certificate)
Windows 7 and 10 system CA certificates
Unsurprisingly, certifi contains the CA cert for xkcd.com:
certifi CA certificates
However I don’t want to rely on certifi since any version of certifi I statically bundle with my app will become out of date after a few years and I want old versions of the app to keep running for a long time.
My app is bundled as a native Windows .exe using py2exe so I don’t think I can use Windows Subsystem for Linux.
I wondered how to check those - thanks for that. The GlobalSign cert is there on my Win 11:
ctx = ssl.create_default_context()
for c in ctx.get_ca_certs()
if 'Global' in repr(c)
Did you miss a Windows update from MS that pushed out extra certs?
What version of
libssl-1_1.dll have you got in the
DLLs folder in your
sys.path? Mine’s 188.8.131.52
Is this a problem with your machine or OS only, at the end of the day? If your users’ machines have recent Windows installations like mine, your code will work for them.
I hope so! This is just my Windows 10 testing VM, initially created a year or two ago.
I verified that my Windows 10 VM thinks it is up to date. But still no xkcd.com certificate:
Windows 10 fully updated
Retry certificate check. Fails.
I’ll try rebuilding my Windows 10 VM from scratch from Microsoft’s official evaluation image…
In the meantime, I’ve finally got a fresh Windows 11 VM provisioned, so I’ll check there too…
libssl-1_1.dll is 1.1.14 for me, although it is for Python 3.9 rather than Python 3.11 (which is what I think you’re using):
By the way, thank you @JamesParrott for helping me debug this far!
The above works for me on a fresh Windows 11 VM and a fresh Windows 10 VM
I guess my takeaway is that it’s important to provide a way for the user to extend the system certificates if they get messed up.
I discovered that various programs inspect environment variables to find additional CA certs:
$SSL_CERT_FILE - OpenSSL
$REQUESTS_CA_BUNDLE - requests
$CURL_CA_BUNDLE - curl
So I’ll alter my app to recognize one/more of those as well.