How do I overcome Python http.client.HTTPResponse objects?

Question

I have tried to get response from a url, from the following code. I am using Python 3.x

from urllib.request import urlopen

url_getallfolders = 'https://qa.wittyparrot.com/alfresco/service/acrowit/userfolderinfo?access_token=TICKET_83361146b0e140f48ba404c3d8457452a92e117f'
x = urlopen(url_getallfolders)
print(x)

I get the following error:

<http.client.HTTPResponse object at 0x030304B0>

I even tried with urllib.urlopen:

x = urllib.urlopen(url_getallfolders)
print(x)

then i get this error:

NameError: name 'urllib' is not defined

Kindly help. Thanks in advance

Solution 1

You did not get an error, you instead got an expected response object. If you wanted to access the data from the response then you need to read from that object, or inspect the headers and status code perhaps.

Reading the response body data is as simple as:

x = urlopen(url_getallfolders)
data = x.read()

From the urllib.request.urlopen() documentation:

For http and https urls, this function returns a http.client.HTTPResponse object which has the following HTTPResponse Objects methods.

where I used the HTTPResponse.read() method above.

Note that the result will be encoded bytes, if you expected text you'll still need to decode that text. The URL you called returns JSON, so you probably want to decode that to Python:

import json

x = urlopen(url_getallfolders)
raw_data = x.read()
encoding = x.info().get_content_charset('utf8')  # JSON default
data = json.loads(raw_data.decode(encoding))

after which you can access keys such as 'error', 'errorList', 'respList' and 'warning'.

Solution 2

If you just want super basic command-line oriented, HTTP-client functionality, like curl or wget (the popular CLI utilities) without any options; where you feed it a URL, and it simply returns plaintext and HTML:

#!/usr/bin/env python3
#-*- coding: utf-8 -*-

from urllib.request import urlopen

with urlopen('https://example.com') as x:
     data = x.read().decode('utf-8')

print(data)

If you want the byte objects, just drop the .decode('utf-8') so it looks like:

#!/usr/bin/env python3
#-*- coding: utf-8 -*-

from urllib.request import urlopen

with urlopen('https://example.com') as x:
     data = x.read()

print(data)

I've tried to reduce it to as few lines as possible. Feel free to define your variables (URLs, etc.) separately.