Azure Service Bus Retry Policy doesn't change the behavior

Question

I'm trying to understand the retry policy on the Azure Service Bus but it's not working the way I would expect. I have the following code that both listens for messages and sends a message to a specific azure queue.

using System;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;

namespace ServiceBusTester
{
    class Program
    {
        static void Main(string[] args)
        {
            var connectionString = "Endpoint=sb://<NamespaceName>.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=<SharedAccessKey>";
            var queueName = "MyTestQueue";

            var retryPolicy = new RetryExponential(TimeSpan.FromSeconds(0), TimeSpan.FromSeconds(30), 15);

            var ns = NamespaceManager.CreateFromConnectionString(connectionString);
            ns.Settings.RetryPolicy = retryPolicy;

            if (!ns.QueueExists(queueName))
                ns.CreateQueue(queueName);

            var mf = MessagingFactory.CreateFromConnectionString(connectionString);
            mf.RetryPolicy = retryPolicy;

            var mr = mf.CreateMessageReceiver(queueName);
            mr.RetryPolicy = retryPolicy;

            var retryCount = 0;

            mr.OnMessage(_ =>
            {
                retryCount++;
                Console.WriteLine($"{retryCount.ToString().PadLeft(4, ' ')} - Message Received: {_.GetBody<string>()}");
                _.Abandon();
            }, new OnMessageOptions() { AutoComplete = true });


            var client = QueueClient.CreateFromConnectionString(connectionString, queueName);
            client.RetryPolicy = retryPolicy;

            var message = new BrokeredMessage("This is a test message!");

            client.Send(message);

            Console.WriteLine("Press any key to exit...");
            Console.ReadKey();
        }
    }
}

Even though I'm specifying that the retry policy should retry 15 times, I'm still seeing it only retry the default 10 times. I've even tried using the NoRetry policy, but it still retries 10 times.

I also verified that the Maximum Delivery Count on the queue was set to an arbitrarily large number, but that didn't change anything:

I'm sure I've gone overboard with the assignment of the retry policy to the numerous different clients / factories, but I'm not sure what's wrong here.

Solution

RetryExponential is intended to be used by the ASB client when there are transient errors that are not bubbled up to your code right away. I.e. an internal retry mechanism built into the client to perform retries on your behalf before exception is raised. If there are no exceptions and your callback Abandons the message explicitly, retry policy is not even utilized here and the message just goes through the normal delivery up to MaxDeliveryCount times (50 in your scenario), after which is DLQed.

Use retry policy to specify to the ASB client how to deal with transient errors prior to giving up, not how many times a message can be dequeued.

Comments

[Matt Ruwe]

Ok, thanks for the clarification. I had changed the queue name in my application before uploading to SO and forgot to change the Max Delivery Count. It's now working as you specified.

[Matt Ruwe]

BTW, where can I find documentation that describes what you stated in your answer?

[Sean Feldman]

This should provide some good info: docs.microsoft.com/en-us/azure/…

[Matt Ruwe]

Thanks - I had read that article before posting here - but missed the fact that the retry was only for network/client level retries.

[buzzripper]

So, is there any way to get this retry behavior happening for transient errors that happen my listener's message handler? If I have a transient error (db rebooting, network hiccup) I would like to retry in 1 sec, 3 sec etc - not instantly, which is what happens when I abandon the message. I'm using topic subscriptions so i can't just throw a copy back into the topic (would cause dupes in other subscriptions). Any ideas?

[Sean Feldman]

RetryPolicy is internal to the SDK and is not intended to retry the customer code. For that, you would need to look into something additional. Polly for example. There are tools such as NServiceBus and MassTransit that allow business code retries. See documentation here.