MongoDB query for document older than 30 seconds

Question

Does anyone have a good approach for a query against a collection for documents that are older than 30 seconds. I'm creating a cleanup worker that marks items as failed after they have been in a specific state for more than 30 seconds.

Not that it matters, but I'm using mongojs for this one.

Every document has a created time associated with it.

Solution 1

We are assuming you have a created_at or similar field in your document that has the time it was inserted or otherwise modified depending on which is important to you.

Rather than iterate over the results you might want to look at the multi option in update to apply your change to all documents that match your query. Setting the time you want to look past should be fairly straightforward

In shell syntax, which should be pretty much the same of the driver:

db.collection.update({
     created_at: {$lt: time },
     state: oldstate 
},
{$set: { state: newstate } }, false, true )

The first false being for upserts which does not make any sense in this usage and the second true marking for multi document update.

If the documents are indeed going to be short lived and you have no other need for them afterwards, then you might consider capped collections. You can have a total size or time to live option for these and the natural insertion order favours processing of queued entries.

Solution 2

If you want to do this using mongo shell:

db.requests.find({created: {$lt: new Date((new Date())-1000*60*60*72)}}).count()

...will find the documents that are older than 72 hours ("now" minus "72*60*60*1000" msecs). 30 seconds would be 1000*30.

Solution 3

You could use something like that:

var d = new Date();
d.setSeconds(d.getSeconds() - 30);

db.mycollection.find({ created_at: { $lt: d } }).forEach(function(err, doc) {} );

Solution 4

The TTL option is also an elegant solution. It's an index that deletes documents automatically after x seconds, see here: https://docs.mongodb.org/manual/core/index-ttl/

Example code would be:

db.yourCollection.createIndex({ created:1 }, { expireAfterSeconds: 30 } )

View more solutions