uses for mongodb ObjectId creation time

Question

The ObjectId used as the default key in mongodb documents has embedded timestamp (calling objectid.generation_time returns a datetime object). So it is possible to use this generation time instead of keeping a separate creation timestamp? How will you be able to sort by creation time or query for the last N items efficiently using this embedded timestamp?

Solution 1

I suppose since MongoDB ObjectId contain a timestamp, you can sort by 'created date' if you will sort by objectId:

items.find.sort( [['_id', -1]] ) // get all items desc by created date.

And if you want last 30 created items you can use following query:

items.find.sort( [['_id', -1]] ).limit(30) // get last 30 createad items 

I am actualy not sure,i just suppose that ordering by _id should work as described above. I'll create some tests later.

Update:

Yes it is so. If you order by _id you will automatically order by _id created date. I've done small test in c#, mb someone interest in it:

  public class Item
  {
    [BsonId]
    public ObjectId Id { get; set; }

    public DateTime CreatedDate { get; set; }

    public int Index { get; set; }
  }



 [TestMethod]
 public void IdSortingTest()
 {
   var server = MongoServer.Create("mongodb://localhost:27020");
   var database = server.GetDatabase("tesdb");

   var collection = database.GetCollection("idSortTest");
   collection.RemoveAll();

   for (int i = 0; i <= 500; i++)
   {
     collection.Insert(new Item() { 
             Id = ObjectId.GenerateNewId(), 
             CreatedDate = DateTime.Now, 
             Index = i });
   }

   var cursor = collection.FindAllAs<Item>();
   cursor.SetSortOrder(SortBy.Descending("_id"));
   var itemsOrderedById = cursor.ToList();

   var cursor2 = collection.FindAllAs<Item>();
   cursor2.SetSortOrder(SortBy.Descending("CreatedDate"));
   var itemsOrderedCreatedDate = cursor.ToList();

   for (int i = 0; i <= 500; i++)
   {
     Assert.AreEqual(itemsOrderedById[i].Index, itemsOrderedCreatedDate[i].Index);
   }
}

Solution 2

Yes, you can use the generation_time of BSON ObjectId for the purposes you want. So,

db.collection.find().sort({ _id : -1 }).limit(10)

will return the last 10 created items. However, since the embedded timestamps have a one second precision, multiple items within any second are stored in the order of their creation.

Solution 3

From: http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-DocumentTimestamps

"sorting on an _id field that stores ObjectId values is roughly equivalent to sorting by creation time, although this relationship is not strict with ObjectId values generated on multiple systems within a single second."

Solution 4

The code to convert a DateTime to its corresponding timestamp with the c# driver is as follows:

    public static ObjectId ToObjectId(this DateTime dateTime)
    {
        var timestamp = (int)(dateTime - BsonConstants.UnixEpoch).TotalSeconds;
        return new ObjectId(timestamp, 0, 0, 0);
    }

More info here: http://www.danharman.net/2011/10/26/mongodb-ninjitsu-using-objectid-as-a-timestamp/

Solution 5

See

http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-DocumentTimestamps

Likely doable however I would always prefer having a dedicated timestamp instead of relying on some such internals like timestamp somehow embedded in some object id.

View more solutions

Comments

[kefeizhou]

Please explain why it's doable, how can you efficiently sort data without a btree index?

[Andreas Jung]

When I say likely doable then I must not explain everything in detail. Please check the given link and you will detect yourself that the ObjectIds are increasing only. And as I said: go with a standard created field. I must not explain everything in depth which I would never do.

[kefeizhou]

sorry but this does not answer the question at all

[Geert-Jan]

I'm pretty sure this is only correct when inserting using one process, bc. Mongo objectids are created using something akin to hilo sequences. More specific: A BSON ObjectID is a 12-byte value consisting of a 4-byte timestamp (seconds since epoch), a 3-byte machine id, a 2-byte process id, and a 3-byte counter

[davetapley]

@Geert-Jan Because the time stamp is the most significant part of an ObjectId, it will sort correctly across inserts from multiple processes (to one second precision, assuming the process clocks are well synchronized). To confirm this, see the implementation of getTimeStamp() uses .slice(0,8). This (0,8) is selecting the most significant four bytes of the ObjectId as the time stamp.

[Geert-Jan]

@dukedave: Sure that would work if 1 sec resolution is enough.

[Zaid Masud]

ObjectId constructor now has an overload that accepts DateTime as timestamp, so your conversion is no longer necessary. Simply do new ObjectId(dateTime, 0, 0, 0);

[phil294]

not that I know much of that, but your suggested solution looks insanely inefficient to me. what do you do when you have 10kk entries? should the $where function applied to each and one of them?

[Chen Dachao]

Sorry I didn't get you