How do I set a number of retry attempts in RabbitMQ?
Solution 1
There are no such feature like retry attempts in RabbitMQ (as well as in AMQP protocol).
Possible solution to implement retry attempts limit behavior:
Redeliver message if it was not previously redelivered (check
redelivered
parameter onbasic.deliver
method - your library should have some interface for this) and drop it and then catch in dead letter exchange, then process somehow.Each time message cannot be processed publish it again but set or increment/decrement header field, say
x-redelivered-count
(you can chose any name you like, though). To get control over redeliveries in this case you have to check the field you set whether it reaches some limit (top or bottom - 0 is my choise, a-lattl
in ip header from tcp/ip).Store message unique key (say uuid, but you have to set it manually when you publish message) in Redis, memcache or other storage, even in mysql alongside with redeliveries count and then on each redelivery increment/decrement this value until it reach the limit.
(for real geeks) write plugin that will implement such behavior like you want.
The pro of #3 is that redelivered message stay in queue head. This is important if you have long queue or if message order is important for you (note, that redeliveries will break strict messages order, see official docs for details or this question on SO).
P.S.:
There is similar answer in this topic, but in php. Look through it, maybe it helps you a bit (start reading it from words "There are multiple techniques to deal with cycle redeliver problem".
Solution 2
Although this is an old question I think you can now easily do this with the combination of dead letter exchanges and the x-death header array added once a message is dead lettered:
The dead-lettering process adds an array to the header of each dead-lettered message named x-death. This array contains an entry for each dead lettering event, identified by a pair of {queue, reason}. Each such entry is a table that consists of several fields:
queue: the name of the queue the message was in before it was dead-lettered
reason: reason for dead lettering, see below
time: the date and time the message was dead lettered as a 64-bit AMQP 0-9-1 timestamp
exchange - the exchange the message was published to (note that this will be a dead letter exchange if the message is dead lettered multiple times)
routing-keys: the routing keys (including CC keys but excluding BCC ones) the message was published with
count: how many times this message was dead-lettered in this queue for this reason
original-expiration (if the message was dead-letterered due to per-message TTL): the original expiration property of the message. The expiration property is removed from the message on dead-lettering in order to prevent it from expiring again in any queues it is routed to.
Read this great article for more info
Check this diagram:
Solution 3
You should use Quorum queue type. That queue type has a Delivery limit argument to specify the number of retries to deliver a message before deleting it.
Specify next arguments when create your queue:
x-queue-type: quorum
x-delivery-limit: 3 // it means rabbitmq will make 3 attempts to deliver a message before deleting it
Related videos on Youtube
user2689570
Updated on May 11, 2022Comments
-
user2689570 about 2 years
I am using RabbitMQ and I have a queue that holds email messages. My consumer service de-queues messages and attempts to send them. If, for any reason, my consumer cannot send the message, I would like to re-queue the message to send again. I realize I can do a basicNack and set the requeue flag to be true, however, I don't want to requeue the message indefinitely (say, if our email system goes down, I don't want to continuously requeue unsent messages). I would like to define a finite number of times that I can requeue the message to be sent again. I can't set a field on the email message object, however, when I dequeue it and send a nack. The updated field is not present on the message in the queue. Is there any other way in which I can approach this? Thanks in advance.
-
Udi Dahan over 6 yearsThis blog post does a pretty good job explaining why you'd want to use a framework like NServiceBus or MassTransit on top of RabbitMQ in .net to solve these kinds of problems: make-awesome.com/2017/12/sure-you-can-just-use-rabbitmq
-
Datz about 6 years@UdiDahan is there an equivalent to MassTransit/NServiceBus in the Java world?
-
Udi Dahan about 6 years@Datz I think the closest thing would be Apache Caml.
-
Roee Gavirel almost 4 years@Datz - if still relevant, for JAVA see: github.com/spring-cloud/spring-cloud-stream-binder-rabbit
-
-
user2689570 about 10 yearsThanks very much. I think I will go the dead letter exchange route.
-
Víctor Romero over 9 yearsAlso, if you don't mind time limiting the message life time rather that limiting the number of retries, you could always set a TTL.
-
fafl almost 6 yearsWriting a plugin does not seem possible at the moment. There is no behaviour in rabbit that lets you hook into message consumption and modify the message. You can only modify the message header during publishing using the rabbit_channel_interceptor, but this does not help here.
-
Alexander Taylor over 2 years+1 for simplicity, but the downside for large applications would be higher RAM usage while messages sit in the consumer's RAM waiting to be retried (or lower throughput if you cap the number of concurrent messages being processed to prevent excess RAM usage). probably works best if you intend to have shorter retry delays and fewer total attempts.