elastic search and date range filters

Question

I'm trying to filter an elastic search query by date using the rawes python bindings. If I drop the database, specify that the datetime field is a date_optional_time, insert a few rows into the elastic search database, and then query it using a range filter on the datetime field, I get no results (see below). Any idea what I am doing wrong?

>>> result = es.delete(root_url)
>>> result = es.put(root_url+"_mapping", data={
    "tweet": {
        "properties": {
            "datetime": {
                "type": "date", 
                "format": "date_optional_time"
            },
        }
    },
})
>>> result = es.post(root_url, data={
    "datetime": "2012-12-20 12:00:00",
    "name": "freddy",
    "text": "hello world",
})
>>> result = es.post(root_url, data={
    "datetime": "2012-11-20 12:00:00",
    "name": "julie",
    "text": "welcome to the wonderful world of cooking",
})
>>> result = es.get(root_url+"_search", data={
    "query": {
        "range": {  # expect this to return the one result on 2012-12-20
            "datetime": {
                "gte":"2012-12-01", # do not append "T00:00:00"; too slow!
                "lte":"2012-12-31", # do not append "T23:59:59"; too slow! 
            }
        }
    }
})
>>> pprint.pprint(result)
{u'_shards': {u'failed': 0, u'successful': 5, u'total': 5},
 u'hits': {u'hits': [], u'max_score': None, u'total': 0},
 u'timed_out': False,
 u'took': 4}

Solution 1

The problem is apparently with how I was encoding dates as strings (note the "T" in the strings). This works:

>>> result = es.post(root_url, data={
    "datetime": "2012-12-20T12:00:00",
    "name": "freddy",
    "text": "hello world",
})
>>> result = es.post(root_url, data={
    "datetime": "2012-11-20T12:00:00",
    "name": "julie",
    "text": "welcome to the wonderful world of cooking",
})
>>> result = es.get(root_url+"_search", data={
    "query": {
        "range": {  # expect this to return the one result on 2012-12-20
            "datetime": {
                "gte":"2012-12-01", 
                "lte":"2012-12-31",
            }
        }
    }
})

Solution 2

Could you try to launch your request with post instead of get? I don't know how works pyes but in some other client, the payload is not send when using GET instead of POST.

>>> result = es.post(root_url+"_search", data={
    "query": {
        "range": {  # expect this to return the one result on 2012-12-20
            "datetime": {
                "gte":"2012-12-01", 
                "lte":"2012-12-31",
            }
        }
    }
})

Does it work?

Comments

[dino]

Unfortunately, POST does not work either. This doesn't appear to be a problem with the python interface because it doesn't work with curl either. Thanks for the suggestion though!

[dadoonet]

Oh yes ! I did not notice that T was missing here in the source document. Sorry. When querying, I think you can now use "gte":"2012-12-01","lte":"2012-12-31" notation.

[dino]

good to know. I'll edit my search query accordingly. thanks again for your help!