MongoDB in Go (golang) with mgo: How do I update a record, find out if update was successful and get the data in a single atomic operation?

Question

I am using mgo driver for MongoDB under Go.

My application asks for a task (with just a record select in Mongo from a collection called "jobs") and then registers itself as an assignee to complete that task (an update to that same "job" record, setting itself as assignee).

The program will be running on several machines, all talking to the same Mongo. When my program lists the available tasks and then picks one, other instances might have already obtained that assignment, and the current assignment would have failed.

How can I get sure that the record I read and then update does or does not have a certain value (in this case, an assignee) at the time of being updated?

I am trying to get one assignment, no matter which one, so I think I should first select a pending task and try to assign it, keeping it just in the case the updating was successful.

So, my query should be something like:

"From all records on collection 'jobs', update just one that has assignee=null, setting my ID as the assignee. Then, give me that record so I could run the job."

How could I express that with mgo driver for Go?

Solution 1

I hope you saw the comments on the answer you selected, but that approach is incorrect. Doing a select and then update will result in a round trip and two machines and be fetching for the same job before one of them can update the assignee. You need to use the findAndModify method instead: http://www.mongodb.org/display/DOCS/findAndModify+Command

Solution 2

This is an old question, but just in case someone is still watching at home, this is nicely supported via the Query.Apply method. It does run the findAndModify command as indicated in another answer, but it's conveniently hidden behind Go goodness.

The example in the documentation matches pretty much exactly the question here:

change := mgo.Change{
        Update: bson.M{"$inc": bson.M{"n": 1}},
        ReturnNew: true,
}
info, err = col.Find(M{"_id": id}).Apply(change, &doc)
fmt.Println(doc.N)

Solution 3

The MongoDB guys describe a similar scenario in the official documentation: http://www.mongodb.org/display/DOCS/Atomic+Operations

Basically, all you have to do, is to fetch any job with assignee=null. Let's suppose you get the job with the _id=42 back. You can then go ahead and modify the document locally, by setting assignee="worker1.example.com" and call Collection.Update() with the selector {_id=42, assignee=null} and your updated document. If the database is still able to find a document that matches this selector, it will replace the document atomically. Otherwise you will get a ErrNotFound, indicating that another thread has already claimed the task. If that's the case, try again.

Comments

[jorelli]

Performing select followed by an update is not atomic; it performs two round-trips! The findAndModify command will perform this operation atomically. Mongo docs for this command are here: mongodb.org/display/DOCS/findAndModify+Command Mgo docs for doing this in Go are here: go.pkgdoc.org/labix.org/v2/mgo#Query.Apply

[Fabio Suarez]

The select+update doesn't need to be atomic for this particular algorithm. As long as the update itself is atomic (which is basically a CompareAndSwap / CompareExchange operation) everything is fine.

[jorelli]

If two machines select the same job on the first query, one of them will fail on the update. Sure, the end result may always be that the data in the database never becomes inconsistent, but the failure case with using two separate operations causes a lot of unnecessary back and forth. This is... the entire reason the findAndModify command exists.

[Sebastián Grignoli]

Actually, the solution I need is the one tux21b offered. I need to retreve all "options" then select one, then try to assign it to myself. If unsuccessful, I would try with another.

[Steven Luu]

Why do you need to select ALL of the options? Did you say you just need to select on that's not taken (aka assignee == null)?

[Sebastián Grignoli]

You are right. My needs turned out to be different than what I thought when I wrote the question, but your answer fits the question better.

[Gustavo Niemeyer]

The findAndModify command is conveniently supported by mgo via the Query.Apply method. Please see the answer below for details.