urllib.py doesn't work with https?

Question

In my python app I try to open a https url, but I get:

 File "C:\Python26\lib\urllib.py", line 215, in open_unknown
    raise IOError, ('url error', 'unknown url type', type)
IOError: [Errno url error] unknown url type: 'https'

my code:

import urllib
def generate_embedded_doc(doc_id):
    url = "https://docs.google.com/document/ub?id=" + doc_id + "&embedded=true"
    src = urllib.urlopen(url).read()
    ...
    return src

Solution 1

urllib and Python 2.6 have SSL support and your code example works fine for me. Probably your Python is built without SSL support? Try to reinstall Python 2.6 (or better, 2.7) and use the original build from python.org.

On Google App Engine, try to use the API directly:

from google.appengine.api import urlfetch

url = "https://www.google.com/"
result = urlfetch.fetch(url)
if result.status_code == 200:
  doSomethingWithResult(result.content)

Solution 2

To have SSL support you need to compile Python with OpenSSL. For example under ubuntu lucid you must install the module libcurl4-openssl-dev then re-build Python.

Solution 3

Try using urllib2 instead.

I had the same issue with urllib on OSX 10.6 using python 2.6.6 from macports. Switching to urllib2 fixed it.

Comments

[Vladimir Keleshev]

I have Python 2.6, but since I'm using GAE it might make some restrictions on that (because it's build for Python 2.5)

[leoluk]

I have extended my answer; do you use Google's SDK? it doesn't supports Python 2.6, but the path in your example is C:\Python26\lib\urllib.py

[Bruno]

urllib, urllib2 and httplib don't verify the server certificate when using HTTPS, which doesn't make the use of HTTPS secure unfortunately. (There are ways around it, but it needs a bit more code.)

[Maverik]

How do you switch to urllib2? I have the same problem on my mac!

[s29]

You need to import urllib2 instead of urllib and rewrite your code to suit. It's similar-ish, but not the same. Check the python docs.

[s29]

Another thing from memory that triggers a similar error, is having another file named "ssl.py" or "ssl.pyc" somewhere on your pythonpath. Rename or delete it.