Parsing JSON in Python: how do I get has_key() working again after a change of format?

Question

I have the following block of Python code:

data = json.loads(line)
if data.has_key('derivedFrom'):
     dFin = data['derivedFrom']
     if dFin.has_key('derivedIds'):

This used to work fine on a block of JSON like this:

"derivedFrom": {"source": "FOO", "model": "BAR", "derivedIds": ["123456"]}

Now the format changed to:

"derivedFrom": "{\"source\": \"FOO.\", \"model\": \"BAR\", \"derivedIds\": [\"123456\"]

And so the last line in the Python block throws the following exception:

'unicode' object has no attribute 'has_key'

Is there a way to preprocess JSON to make has_key work again?

Solution 1

"{\"source\": \"FOO.\", \"model\": ...

Is a JSON object inside a JSON string literal. To get at the inner JSON's properties, you'll have to decode it again.

data = json.loads(line)
if 'derivedFrom' in data:
    dFin = json.loads(data['derivedFrom'])
    if 'derivedIds' in dFin:
        ....

JSON-in-JSON is typically a mistake as there's rarely a need for it - what is producing this output, does it need fixing?

Solution 2

Use:

'derivedIds' in dFin

This works both on dictionaries and on unicode, even though with unicode it could give false positives.

A more robust approach could use Duck Typing:

try:
    dFin = json.loads(data['derivedFrom'])  #assume new format
except TypeError:
    dFin = data['derivedFrom']  #it's already a dict
if 'derivedIds' in dFin: # or dFin.has_key('derivedIds')
    #etc