Topic Exchange vs Direct Exchange in RabbitMQ

Question

We've got an application which will be using RabbitMQ and have several different queues for passing messages between tiers.

Initially, I was planning to use multiple direct exchanges, with one for each message type, but it looks like having a single topic exchange with queues using different routing key bindings will achieve the same thing.

Having a single exchange also seems like it would be a bit easier to maintain, but I was wondering if there is any benefit (if any) of doing it one way over the other?

Option 1, using multiple direct exchanges:

ExchangeA (type: direct)
-QueueA

ExchangeB (type: direct)
-QueueB

ExchangeC (type: direct)
-QueueC

Option 2, using single topic exchange:

Exchange (type: topic)
-QueueA  (receives messages from exchange with routing key of "TypeA")
-QueueB  (receives messages from exchange with routing key of "TypeB")
-QueueC  (receives messages from exchange with routing key of "TypeC")

Solution 1

Assuming both models are being considered to be implemented using one broker running, there's little difference that I can see.

Option 2 seems more common in the real world for solving this kind of routing problem (at least in my anecdotal experience) and it's exactly the challenge that Topic Exchanges exist to solve.

The only difference that you might encounter would relate to routing speed. I don't know for sure if Exchange routing (based always on an exact string match) is faster in RabbitMQ when compared to the routing key technique used in Topic Exchanges (which can include wildcards like # and *). My hunch would be that Exchange discrimination would be faster, but you could experiment for yourself to find out, or try contacting the RabbitMQ team to ask them.

Finally, if you go with option 1 end up with lots of queues then you'll have a proportional proliferation of Exchanges. That sounds like a maintenance headache. If you'll only have a handful of queues then it won't be too much of an issue.

Solution 2

Indeed approach 2 is better as it gives you flexibility to use a single queue for multiple routing keys.

Exchange Topic

QueueA-- binding key = India.Karnataka.*

You can route a message to topic exchange with routing key as India.Karnataka.bangalore,India.Karnataka.Mysore.

All the above messages goes to QueueA.

Direct Exchange

But I did not understand on why are you creating multiple direct exchanges in approach 1. You can have single direct exchange and have multiple queues with each queue binding with a unique key.

QueueA-- binding key = Key1
QueueB-- binding Key = Key2
QueueC-- binding Key = Key3

All key1 messages goes to QueueA.Key2 goes to QueueB ... You can still maintain single direct exchange.

Solution 3

For a single small node with little load there is little difference. Most people go with option two for said reasons above.

When designing the system you should ask your self how might this expand in the future.

How is it going to scale ?
Do I need it to scale ? Do I want to add a high a availability cluster in the future ? Is my routing going to change...

Option 2 provides a lot more flexibility in most cases.

It allows you to do things like attach a new consumer to the Exchange with its own queue and capture any subset or all of your message flow easily. ( these queues could be on other nodes in a cluster or mirrored to n nodes providing failover) A typical use case would be logging all messages using a 4th queue.

If your bottleneck is on the processing side, You can also further subdivide your message topics and perform some amount of prioritization with out having to change your publishers. Example: ToppicA.urgent, processed by dedicated consumer, TopicA.log eventually processed.

The short answer is go with option 2 unless you have very specific performance requirements, If for example you need to process say a sustained rate of more than 50k msg/s you might want to think about option 1 on dedicated nodes, but for normal flows option 2 will be easier to scale and maintain.