How do I turn MongoDB query into a JSON?

Question

for p in db.collection.find({"test_set":"abc"}):
    posts.append(p)
thejson = json.dumps({'results':posts})
return  HttpResponse(thejson, mimetype="application/javascript")

In my Django/Python code, I can't return a JSON from a mongo query because of "ObjectID". The error says that "ObjectID" is not serializable.

What do I have to do? A hacky way would be to loop through:

for p in posts:
    p['_id'] = ""

Solution 1

The json module won't work due to things like the ObjectID.

Luckily PyMongo provides json_util which ...

... allow[s] for specialized encoding and decoding of BSON documents into Mongo Extended JSON's Strict mode. This lets you encode / decode BSON documents to JSON even when they use special BSON types.

Solution 2

Here is a simple sample, using pymongo 2.2.1

import os
import sys
import json
import pymongo
from bson import BSON
from bson import json_util

if __name__ == '__main__':
  try:
    connection = pymongo.Connection('mongodb://localhost:27017')
    database = connection['mongotest']
  except:
    print('Error: Unable to Connect')
    connection = None

  if connection is not None:
    database["test"].insert({'name': 'foo'})
    doc = database["test"].find_one({'name': 'foo'})
    return json.dumps(doc, sort_keys=True, indent=4, default=json_util.default)

Solution 3

It's pretty easy to write a custom serializer which copes with the ObjectIds. Django already includes one which handles decimals and dates, so you can extend that:

from django.core.serializers.json import DjangoJSONEncoder
from bson import objectid

class MongoAwareEncoder(DjangoJSONEncoder):
    """JSON encoder class that adds support for Mongo objectids."""
    def default(self, o):
        if isinstance(o, objectid.ObjectId):
            return str(o)
        else:
            return super(MongoAwareEncoder, self).default(o)

Now you can just tell json to use your custom serializer:

thejson = json.dumps({'results':posts}, cls=MongoAwareEncoder)

Solution 4

Something even simpler which works for me on Python 3.6 using motor==1.1 pymongo==3.4.0

from bson.json_util import dumps, loads

for mongo_doc in await cursor.to_list(length=10):
    # mongo_doc is a <class 'dict'> returned from the async mongo driver, in this acse motor / pymongo.
    # result of executing a simple find() query.

    json_string = dumps(mongo_doc)
    # serialize the <class 'dict'> into a <class 'str'> 

    back_to_dict = loads(json_string)
    # to unserialize, thus return the string back to a <class 'dict'> with the original 'ObjectID' type.

View more solutions

Comments

[Justin Jenkins]

api.mongodb.org/python/1.9%2B/installation.html & api.mongodb.org/python/1.9%2B/tutorial.html & >>> import pymongo

[ario]

Need import bson for this to work (just mentioning it because I'm a newb and it took me a long time to figure out)

[Simon Wang]

json.dumps(doc, sort_keys=True, indent=4, default=json_util.default) did resolve my problem too!

[njzk2]

what is the difference with using json_util.dumps(doc)?

[Cleb]

This link is broken.

[Leukonoe]

You should as well import json for this code to work, right?