Default dict keys to avoid KeyError

Question

I'm calling some JSON and parsing relevant data as CSV. I cannot figure out how to fill in the intermediate JSON dict file with default keys, as many are unpopulated. The result is a KeyError as I attempt to parse the content into a CSV.

I'm now receiving a 'NoneType' error for (manufacturer):

import urllib2, json, csv, sys, os, codecs, re

from collections import defaultdict

output = 'bb.csv'

csv_writer = csv.writer(open(output, 'w'))

header = ['sku', 'name', 'description', 'image', 'manufacturer', 'upc', 'department', 'class', 'subclass']

csv_writer.writerow(header)

i=1

while i<101:
    print i

    bb_url = urllib2.Request("http://api.remix.bestbuy.com/v1/products(sku=*)?show=sku,name,description,image,manufacturer,upc,department,class,subclass&format=json&sort=sku.asc&page=" + str(i) + "&pageSize=100&apiKey=*****************")
    bb_json = json.load(urllib2.urlopen(bb_url))

    print bb_json

    for product in bb_json['products']:
        row = []

        row.append(product['sku'])
        if product['name']:
            row.append(str((product['name']).encode('utf-8')))
        else:
            row.append("")
        row.append(str(product.get('description',"")))
        row.append(str(product['image'])+ " ")
        if product['name']:
            row.append(str(product.get('manufacturer',"").encode('utf-8')))
        else:
            row.append("")
        row.append(str(product.get('upc','').encode('utf-8')))
        row.append(str((product['department']).encode('utf-8')))
        row.append(str((product['class']).encode('utf-8')))
        row.append(str((product['subclass']).encode('utf-8')))

        csv_writer.writerow(row)

    i = i+1

Solution 1

You can use your_dict.get(key, "default value") instead of directly referencing a key.

Solution 2

Don't use the "default" argument name. For example, if we want 1.0 as default value,

rank = dict.get(key, 1.0)

For more details: TypeError: get() takes no keyword arguments

Solution 3

If you can't define a default value and want to do something else (or just omit the entry):

if key in dict:
    rank = dict[key]
else:
    # do something or just skip the else block entirely

Solution 4

You could use syntax like this: product.get("your field", "default value")

View more solutions

Comments

[Kenfucious]

Thanks for assisting. That line would be inserted before the for loop, or iteratively within?

[Stack of Pancakes]

Whenever you'd normally access a key. So when you go to get the product's name instead of row.append(product['name']) you would use row.append(product.get('name', 'value if name key doesn't exist'))

[Kenfucious]

Thanks, that seemed to work!! One final point: I'm now getting a nonetype error for one of the fields (manufacturer). The if clause for 'name' works well, but a similar syntax yields the dreaded KeyError once again.

[Yep_It's_Me]

Don't use None for the default value. In your code you're setting it to "" if the value doesn't exist. So try row.append(product.get('name', '')) Which will append the value for name if it exists, or an empty string if it doesn't.

[Noam Manos]

Notice that "dict" shouldn't be the name of your dictionary, since "dict" is a reserved built-in symbol in Python