Parse CSV with Django and csv.DictReader

Question

I have a Django model with many fields (roughly 24), and apparently my users would like to be able to create instances of this object using a spreadsheet upload instead of manually entering all the data into forms.

I've found that using python's built in CSV module should make this fairly simple, but I'm having a hard time working out how exactly to use it.

Let's start out with what I have in terms of code:

def upload_file(request):

  if request.method == "POST":
    form = UploadFileForm(request.POST, request.FILES)
    if form.is_valid():
      handle_files(request.FILES['file'])
      return HttpResponseRedirect('/workflow/')
    else:
      print form.errors
      print request.FILES
      return HttpResponseRedirect('/workflow/upload')
  else:
    form = UploadFileForm()
    return render(request, 'fileform.html', {'formset': form})

This accepts a CSV file as an upload, and hands it off to handle_files to deal with the parsing and the object creation. This is the method I'm having trouble with.

def handle_files(f):
    reader = csv.DictReader(f, delimiter=' ', quotechar='|')
    ... #?

I've been trying to mimic the python docs (http://docs.python.org/2/library/csv.html), but DictReader is very poorly documented. Am I supplying csv.DictReader() the appropriate arguments? If I have, how can I get the information out of reader? I'm going to be supplying users with a template, so I can assume that each column of the CSV file has a predictable data. That is, I know that Column A will have data corresponding to field X, and column B corresponds to Y, etc. How do I go about parsing the data from the reader, and then creating an object with that data?

I'm guessing it will be something like:

for row in reader:
    X=row[1]
    Y=row[2]
    #etc
    my_object = MyObject(x=X, y=Y)
    my_object.save()

Is this way off? Should I be using another type of CSV reader?

Thanks for any help, I know there are a lot of questions in this post.

Solution 1

I've just recently started using Django but I've used the python csv library before. When I use it I just do the following:

import csv
...
reader = csv.reader(f)
for row in reader:
    #do something with each row
f.close()

So you were pretty close. Also, indexing starts at 0, so if you want the first element, use row[0].

There is more information on the csv library here. You only want to use delimiter and other parameters if the format of your file is different.

Solution 2

While creating the csv file, add the header:

people.csv

id  age  height
1   20   62
2   22   74
3   24   68

---

def handle_files(f):
    reader = csv.DictReader(open(f))
    for row in reader:
        id=row['id']
        age=row['age']
        height=row['height']
        my_object = MyObject(id=id, age=age,height=height)
        my_object.save()

Comments

[Matthew Wesly]

The docs say that it will "Create an object which operates like a regular reader but maps the information read into a dict whose keys are given by the optional fieldnames parameter. If the fieldnames parameter is omitted, the values in the first row of the csvfile will be used as the fieldnames." Which sounds like you would be able to use row["Column A"] (or a different key depending on the column name)

[teewuane]

@DavidCain, @srinu j : This results in "list indices must be integers, not str" for me. Before I would get the value like id=row[0] age=[1] , etc... Now I'm trying to "smart" detect the headers since id might not always be the first column. Do I need to do something else here?