ValueError: No JSON object could be decoded, but positive <Response [200]>

Question

I'm going over some URL's and I can fetch most of the data I can from an API I'm using. *Imgur API. However when it finds an image that has been posted before but was eventually removed it still shows a positive URL get response (code 200), and when I use

    j1 = json.loads(r_positive.text)

I get this error:

http://imgur.com/gallery/cJPSzbu.json
<Response [200]>
Traceback (most recent call last):
  File "image_poller_multiple.py", line 61, in <module>
    j1 = json.loads(r_positive.text)
  File "/usr/lib/python2.7/json/__init__.py", line 326, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

How can I "fetch" the error inside the j1 variable instead? I'd like to use a conditional structure to solve the problem and avoid my program from crashing. Something like

if j1 == ValueError:
  continue
else:
  do_next_procedures()

Solution 1

You need to use try except instead:

try:
    j1 = json.loads(r_positive.text)
except ValueError:
    # decoding failed
    continue
else:
    do_next_procedures()

See Handling Exceptions in the Python tutorial.

What really happens is that you were redirected for that URL and you got the image page instead. If you are using requests to fetch the JSON, look at the response history instead:

if r_positive.history:
    # more than one request, we were redirected:
    continue
else:
    j1 = r_positive.json()

or you could even disallow redirections:

r = requests.post(url, allow_redirects=False)
if r.status == 200:
    j1 = r.json() 

Solution 2

The URL you listed redirects you to a HTML page. (Use curl to check things like this, he's your friend.)

The HTML page obviously cannot be parsed as JSON.

What you probably need is this:

response = fetch_the_url(url)
if response.status == 200:
  try:
    j1 = json.loads(response.text)
  except ValueError:
    # json can't be parsed
    continue