importing JSON to mongoDB using pymongo

Question

i am trying to import a JSON file i pull from a URL and send it to mongoDB as is, using the pymongo module.

I have the following code

#!/usr/bin/env python
import sys, urllib2, json, pymongo
from pymongo import MongoClient
myurl = "https://gist.githubusercontent.com/border/775526/raw/b921df18ba00262ab5bba8cadb3c178e1f7748f7/config.json"
response = urllib2.urlopen(myurl)
data = response.read()
connection = MongoClient('mongodb://user:[email protected]:27017/database')
connection.database_names()
db = connection.database
posts = db.posts
post_id = posts.insert_many(data).inserted_id

upon executing this, i get this error raise TypeError("documents must be a non-empty list") TypeError: documents must be a non-empty list

ideally, i want to just be able to pull the json from the url and update the mongoDB as this json file will be updated every week. Thanks

Solution

You need to convert JSON to Python objects, which PyMongo will then convert to BSON for sending to MongoDB. To convert JSON to Python objects use the "bson.json_util" module included with PyMongo:

from bson import json_util
data = json_util.loads(response.read())

The standard Python json.loads() function works, too, but PyMongo's json_util.loads() handles some MongoDB-specific details better.