Convert mongodb return object to dictionary

Question

I'm using the bottle framework together with mongoengine. I have an orders model :

class OrderDetail(Option):
    orderDetailsQty = FloatField()

    def to_dict(self):
        return mongo_to_dict_helper(self)


class Order(Document):
    userName = StringField(required=True)
    orderDate = DateTimeField()
    orderStatus = ListField(EmbeddedDocumentField(Status))
    orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
    orderComments = ListField(EmbeddedDocumentField(Comment))
    isActive = BooleanField()

    def to_dict(self):
        orderObj = mongo_to_dict_helper(self)
        orderDetailList = []
        for orderDetail in orderObj["orderDetails"]:
            orderDetailList.append(orderDetail.__dict__)
        orderObj["OrderDetails"] = orderDetailList
        return (self)

When mongodb is queried I get an object which is then converted in to a dict by using the following function :

def mongo_to_dict_helper(obj):
    return_data = []
    for field_name in obj._fields:
        if field_name in ("id",):
            continue
        data = obj._data[field_name]

        if isinstance(obj._fields[field_name], StringField):
            return_data.append((field_name, str(data)))
        elif isinstance(obj._fields[field_name], FloatField):
            return_data.append((field_name, float(data)))
        elif isinstance(obj._fields[field_name], IntField):
            return_data.append((field_name, int(data)))
        elif isinstance(obj._fields[field_name], ListField):
            return_data.append((field_name, int(data)))
        else:
            # You can define your logic for returning elements
            pass
    return dict(return_data)

I found this function after a long search in the internet. Later found out that this function also fails while defining a member as the ListField(EmbeddedDocumentField(obj)).

I also tried writing a condition for catching the specific case of EmbeddedDocumentField :

elif isinstance(obj._fields[field_name], EmbeddedDocumentField):
    return_data.append(mongo_to_dict_helper(data))

but that didn't do any good either.

Anyone have a workaround for this issue ?

Solution 1

What about just using to_mongo method of an object to convert it to a dict?

object.to_mongo()

Solution 2

Expanding on @alexvassel's and @z0r's answers, calling .to_mongo() converts the object to a SON instance. Once you have it, you can call its .to_dict() method to convert it to a dictionary.

For example... (qset is a queryset that's returned from mongoengine, after e.g. Posts.objects.all()).

sons = [ob.to_mongo() for ob in qset]
for son in sons:
    print str(son.to_dict())

Solution 3

import json
json.loads(yourobj.to_json())

Solution 4

Extending on @alexvassel's answer, to_mongo() method returns SON object, which you can convert to dict by calling its to_dict() method

object.to_mongo().to_dict()

Solution 5

you can custom method to convert object to dict

class Order(Document):
    userName = StringField(required=True)
    orderDate = DateTimeField()
    orderStatus = ListField(EmbeddedDocumentField(Status))
    orderDetails = ListField(EmbeddedDocumentField(OrderDetail))
    orderComments = ListField(EmbeddedDocumentField(Comment))
    isActive = BooleanField()

    def as_dict(self):
        return {
            "user_name": self.userName,
            "order_date": self.orderDate.strftime("%Y-%m-%d %H:%M:%S"),
        }

now you can use obj.as_dict() to dict

orders = Order.objects.all()
datas = [each.as_dict() for each in orders]

View more solutions

Comments

[mahesmohan]

Thats exactly what I was looking for. But still I get an error TypeError that the ObjectId is not serializable.

[mahesmohan]

Pasting a link to to_mongo() documentation will be greatly appreaceated as I cant find one anywhere. Thanks @alexvassel

[alexvassel]

Actually, i did not find this method in the official documentation) So it is only looking into a source)

[alexvassel]

And about your error, try to serialize the str(ObjectId). Actually, i don't know what are you trying to do, so it is just a guess.

[Jonathan Simon Prates]

Is possible dereference all reference fields recursively using to_mongo()?

[holms]

@alexvassel how to use this method?? from this inbox = Inbox.objects.filter(messages__to_users__in=[username]).to_m‌​ongo() i got this 'BaseQuerySet' object has no attribute 'to_mongo'

[z0r]

@holms you can use a comprehension: [ob.to_mongo() for ob in query_set]

[Eduardo Pignatelli]

Is there a from_mongo() as well? Does the to_mongo store metainfo to restore the object?

[Mustapha-Belkacim]

object.to_mongo() returns SON object to get a dictionary you need to call to_dict() method: object.to_mongo().to_dict()

[ken]

so how about orderStatus,orderDetails and orderComments which is a EmbeddedDocumentField ,and in some case is a ReferenceField,in such case,how can it be done??

[joeycheng]

@ken,you can define as_dict() at ReferenceField, and` "order_status":self.orderStatus.as_dict(); or, you can use aggregate,obj.objects.aggregate([{"$lookup":{"localField":"o‌​rderStatusId","forei‌​gnFiled":"_id","form‌​":"orderStatus","as"‌​:"order_status"}}])`

[Yannik Suhre]

This is my most favorite answer in this thread, however, it returns a list of dicts and not a dict directly - even though there is just one result. In order to access one dict you have to access the lists objects like list[i]. Otherwise this is a great solution!

[warvariuc]

to_mongo accepts use_db_field argument, which should be False if you have a field with a custom db_field.