Why do I get a pymongo.cursor.Cursor when trying to query my mongodb db via pymongo?

Question

I have consumed a bunch of tweets in a mongodb database. I would like to query these tweets using pymongo. For example, I would like to query for screen_name. However, when I try to do this, python does not return a tweet but a message about pymongo.cursor.Cursor. Here is my code:

import sys
import pymongo
from pymongo import Connection
connection = Connection()
db = connection.test
tweets = db.tweets
list(tweets.find())[:1]

I get a JSON, which looks like this:

{u'_id': ObjectId('51c8878fadb68a0b96c6ebf1'),
 u'contributors': None,
 u'coordinates': {u'coordinates': [-75.24692983, 43.06183036],
  u'type': u'Point'},
 u'created_at': u'Mon Jun 24 17:53:19 +0000 2013',
 u'entities': {u'hashtags': [],
  u'symbols': [],
  u'urls': [],
  u'user_mentions': []},
 u'favorite_count': 0,
 u'favorited': False,
 u'filter_level': u'medium',
 u'geo': {u'coordinates': [43.06183036, -75.24692983], u'type': u'Point'},
 u'id': 349223725943623680L,
 u'id_str': u'349223725943623680',
 u'in_reply_to_screen_name': None,
 u'in_reply_to_status_id': None,
 u'in_reply_to_status_id_str': None,
 u'in_reply_to_user_id': None,
 u'in_reply_to_user_id_str': None,
 u'lang': u'en',
 u'place': {u'attributes': {},
  u'bounding_box': {u'coordinates': [[[-79.76259, 40.477399],
     [-79.76259, 45.015865],
     [-71.777491, 45.015865],
     [-71.777491, 40.477399]]],
   u'type': u'Polygon'},
  u'country': u'United States',
  u'country_code': u'US',
  u'full_name': u'New York, US',
  u'id': u'94965b2c45386f87',
  u'name': u'New York',
  u'place_type': u'admin',
  u'url': u'http://api.twitter.com/1/geo/id/94965b2c45386f87.json'},
 u'retweet_count': 0,
 u'retweeted': False,
 u'source': u'<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>',
 u'text': u'Currently having a heat stroke',
 u'truncated': False,
 u'user': {u'contributors_enabled': False,
  u'created_at': u'Fri Oct 28 02:04:05 +0000 2011',
  u'default_profile': False,
  u'default_profile_image': False,
  u'description': u'young and so mischievious',
  u'favourites_count': 1798,
  u'follow_request_sent': None,
  u'followers_count': 368,
  u'following': None,
  u'friends_count': 335,
  u'geo_enabled': True,
  u'id': 399801173,
  u'id_str': u'399801173',
  u'is_translator': False,
  u'lang': u'en',
  u'listed_count': 0,
  u'location': u'Upstate New York',
  u'name': u'Joe Catanzarita',
  u'notifications': None,
  u'profile_background_color': u'D6640D',
  u'profile_background_image_url':           u'http://a0.twimg.com/profile_background_images/702001815/f87508e73bbfab8c8c85ebe10b29fcf6.png',
  u'profile_background_image_url_https':     u'https://si0.twimg.com/profile_background_images/702001815/f87508e73bbfab8c8c85ebe10b29fcf6.png',
  u'profile_background_tile': True,
  u'profile_banner_url': u'https://pbs.twimg.com/profile_banners/399801173/1367200323',
  u'profile_image_url':     u'http://a0.twimg.com/profile_images/378800000012256721/d8b5f801fb331de6ead4aed42dc77a46_normal.jpeg',
  u'profile_image_url_https':   u'https://si0.twimg.com/profile_images/378800000012256721/d8b5f801fb331de6ead4aed42dc77a46_normal.jpeg'    ,
  u'profile_link_color': u'140DE0',
  u'profile_sidebar_border_color': u'FFFFFF',
  u'profile_sidebar_fill_color': u'E0F5A6',
  u'profile_text_color': u'120212',
  u'profile_use_background_image': True,
  u'protected': False,
  u'screen_name': u'JoeCatanzarita',
  u'statuses_count': 6402,
  u'time_zone': u'Quito',
  u'url': None,
  u'utc_offset': -18000,
  u'verified': False}}

However, when I try to query for this screen_name, I get:

tweets.find({"screen_name": "JoeCatanzarita"})
<pymongo.cursor.Cursor at 0x52c02f0>

And when I then try to count the number of tweets which have "screen_name": "name", I get:

tweets.find({"screen_name": "name"}).count()
0

Any idea what I am doing wrong/how I can get pymongo to return the tweets I am looking for?

Thanks!

Solution 1

Ok now I see what's your problem:

If you look carefully into your document you will notice that "screen_name" is inside the subdocument user, so if you want to acess it all you have to do is the following:

tweets.find({"user.screen_name": "JoeCatanzarita"})  #for example.

Whenever you are in a situation where the element you are trying to find is inside a subdocument like in this situation or inside an array always use this syntax.

Solution 2

PyMongo's find() method returns a Cursor. To actually execute the query on the server and retrieve results, iterate the cursor with list or a for loop:

for doc in tweets.find({'screen_name': 'name'}):
    print(doc)

# Or:
docs = list(tweets.find({'screen_name': 'name'}))

If tweets.find({"screen_name": "name"}).count() returns 0, it means no documents match your query.

Edit: now that you've posted an example document, I see you want to query like:

list(tweets.find({'user.screen_name': 'name'}))

... since the screen_name field is embedded in the user sub-document.

Solution 3

I think the problem is that "screen_name" is inside a sub-document if you can provide the document structure I may be able to help you.

Solution 4

I had this same problem with a collection.find() call.

I checked the type of the object and it is python dict. so I took the dict and iterated through it even though there was only one item and she's working like a charm.

myResult = db.find({}, {<!-- blah blah blah for the fields you want -->}).sort({"_id":1}).limit(1)
for item in myResult:
    print item

I know this was ages ago but I spent some time surfing this and couldn't find an easy explanation.

Hope this helps.

View more solutions

Comments

[Sammaye]

find will return a cursor, you must iterate that cursor to get results, as to getting a count of 0 the only thing I can think of is that for that particular query it is 0

[user2161725]

Hi. Thanks for your comment. Any suggestions as to how I could "iterate" the cursor? Or, alternatively, do you have a link to somewhere where this is explained in more detail? Thanks!

[Sammaye]

for item in tweets.find({"screen_name":"name"}) with a new line: print item will iterate a cursor

[user2161725]

Thank you for your answer. Strangely, the command works when I query for item in tweets.find({'lang': 'en}): print(item) but not when I query for item in tweets.find({'screen_name': 'name'}): print(item). Is it possible that the 'screen_name' is in a subdocument (as suggested elsewhere on this page) and if so: how can I get around that so that I can execute my query? Thanks again.

[Sammaye]

You want to query for screen name like: tweets.find({'user.screen_name':'name'})

[A. Jesse Jiryu Davis]

@Sammaye is right. I've edited my answer to reflect that.

[user2161725]

This works! Thanks a lot! Still trying to get my head around document structure and how to read it. If I wanted to query for 'hashtags' -given the document structure above - what would the correct code be: tweets.find({'entities.hashtags': ['tourdefrance']}) ? In fact, say I wanted to find out if any of the documents in my collection contained a hashtag, is there a way that I would be able to query that? Thanks again.

[Sammaye]

The query you have there works if you take out the [] surrounding the value for the field. the dot notation reaches into objects/arrys in the root document: docs.mongodb.org/manual/reference/operator/positional if you look towards the bottom

[user2161725]

Hi, thanks for your answer. I've tried both of the methods that you suggest, but python does not return the tweet for me as otherwise expexted. This is strange, because when I query for the first tweet in the collection, I get a result, but when I query for a specific attribute of that tweet - i.e. 'screen_name': 'name'- I get nothing. Any idea what might be going on here?

[A. Jesse Jiryu Davis]

There clearly is no document in your tweets collection where the screen_name field has 'name' as a value. That's what the count of 0 demonstrates in your example code. If you can show us the document you're looking for, we can help you write a query that will find it. =)

[user2161725]

The document structure is too long for the comments box. Is there a way I can get around that, e.g. direct message you?

[Sammaye]

@user2161725 Just add it to the question and make it a code block

[user2161725]

I've now added the document in its entirety to the existing question.

[Codejunky]

@user2161725 I just saw your comment sorry, if you want yeah we can talk anytime you want but I would rather you post the question and make it a code block as Sammaye mentioned so that everyone can benefit from the answer.

[Freddy]

I just tried this method and it worked for me. My question is (newbie here so bare with me), must I iterate through my collection each time I want to see my results in pymongo? Or is there a way to simply view the results without using a for loop? In my code I tried print mycollection.find() and received <pymongo.cursor.Cursor object at 0x101eaaa90> but when I tried your method that produced the results that I wanted, i.e., the readable document.

[Freddy]

Thank you, this was helpful. I had one item in my document so I was sure I did not have a sub document but kept getting the cursor message. Upon reading this method you provided and putting it to use it definitely did the trick.