How to guarantee azure queue FIFO

Question

I understand that MS Azure Queue service document http://msdn.microsoft.com/en-us/library/windowsazure/dd179363.aspx says first out (FIFO) behavior is not guaranteed.

However, our application is such that ALL the messages have to be read and processed in FIFO order. Could anyone please suggest how to achieve a guaranteed FIFO using Azure Queue Service?

Thank you.

Solution 1

The docs say for Azure Storage queues that:

Messages in Storage queues are typically first-in-first-out, but sometimes they can be out of order; for example, when a message's visibility timeout duration expires (for example, as a result of a client application crashing during processing). When the visibility timeout expires, the message becomes visible again on the queue for another worker to dequeue it. At that point, the newly visible message might be placed in the queue (to be dequeued again) after a message that was originally enqueued after it.

Maybe that is good enough for you? Else use Service bus.

Solution 2

The latest Service Bus release offers reliable messaging queuing: Queues, topics and subscriptions

Solution 3

Adding to @RichBower answer... check out this... Azure Storage Queues vs. Azure Service Bus Queues

MSDN (link retired) http://msdn.microsoft.com/en-us/library/windowsazure/hh767287.aspx

docs.microsoft.com https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-azure-and-service-bus-queues-compared-contrasted

Comments

[Chris Rogers]

Unfortunately, this seems to be answering a different question to what the OP asked. The Azure Service Bus is a different beast to the Azure Queue Service

[CodeMonkey]

This is not the situration described above. Even if you only have one serial producer, messages can still arrive out of order if a single consumer crashes and starts reading while the failed message is not visible.

[Deepak]

The question is about Storage queues I guess

[Michael Freidgeim]

@CodeMonkey, Are you sure that if message is not visible, it is not considered a part of the session? Can you provide any link, that proves the behaviour that you mentioned?

[CodeMonkey]

@MichaelFreidgeim I think we are confusing Storage Account Queues and Azure Service Bus Queues here. Storage Account Queues have no order guarantees. Sessions is a feature of Azure Service Bus.