Insert Binary data into Mongo field in pymongo

Question

I'm trying to do something that I feel aught to be pretty trivial, so forgive me if there's some easy solution out there elsewhere.

I'm writing tests for some content indexing and for this I'm trying to insert some binary data (a pdf) into a mongo collection that I have. However, I'm having a good deal of trouble with this. This is the current state of my relevant code

pseudo_file = StringIO()
pdf = pisa.CreatePDF("This is a test", pseudo_file)
test = {"data": pseudo_file}
test.update({"files_id": {"name": "random_asset_name"}, "category": "asset"})
self.chunk_collection.insert(json.dumps(test))

I managed to find an old thread on the Pymongo google group addressing this problem (https://groups.google.com/forum/#!topic/mongodb-user/uBAbY1wdQbs), but I can't seem to find the Binary object that was used to fix that problem and it doesn't seem to be included in Python (I'm using 2.7)

Right now the problem I'm getting is that the StringIO object is not JSON serializable, which is sensible, but pymongo needs a valid utf8 object passed to it. I tried using a base64 encoding of the StringIO.getvalue(), and just directly serializing the same value.

Of course the pdf is not value utf8, so I'm wondering if there's another way to have pymongo recognize that I am sending it a raw binary. Any help is appreciated.

Solution 1

The Google group is actually correct however, sometime after the post on there the binary class was moved to the bson namespace as such you must import it from there.

Good examples exist on the documentation page: http://api.mongodb.org/python/current/api/bson/binary.html

Solution 2

This can be achieved with bson. A full round-trip in the example of pickling/unpickling an object would look like:

import bson

# serialization
collection.insert_one({
    "binary_field": bson.Binary(pickle.dumps(my_object)),
})

# deserialization
record = collection.find_one({ ... })
pickle.loads(record["binary_field"])
# Note that the Binary type can be passed into pickle.loads directly.

It should be noted that the bson package is -- despite being a top-level package -- part of pymongo. According to the pymongo package description:

Do not install the “bson” package from pypi. PyMongo comes with its own bson package; doing “easy_install bson” installs a third-party package that is incompatible with PyMongo.