Is there any recommended lightweight pubsub service/library?

Question

I'm building a small system that contains many parts and I want to use a message pub/sub service to communicate between parts.

I read about some message queue services like RabbitMQ and ZeroMQ but I feel they are too complicated and feel like it was born for distributed system. All parts of my system will be written in C++/Linux and place on a small Raspberry Pi CPU, so I dont need feature like scalable, cross-platform, other language clients ...

Can you guys give me some advice about services or libraries that fit my need?

Solution 1

It's not that hard to do yourself actually.

First of all you need to define the protocol to be used. It can be very simple; like just a message type field, a payload size field, and the actual payload. The message types you need SUBSCRIBE, UNSUBSCRIBE and PUBLISH. The payload for the SUBSCRIBE and UNSUBSCRIBE messages is the name of a channel to subscribe to/unsubscribe from. The payload for the PUBLISH message is the channel name and the actual data (together with the size of the data of course).

To connect all subscribers you need a central server. All subscribers/publishers needs to connect to this server. The server program keeps a collection of queues, one for each channel. When a subscribe or publish message arrives to the server for a channel that doesn't exist, create a new message queue for this channel. For each channel the server also needs a collection of all clients subscribes to that channel. When a publish message arrives to the server, it's added to the end of the queue for the channel in question. While a channel queue is not empty, send a copy of it to all subscribers for that channel, and when all have received it then the message can be removed from the queue.

The hard part of the server is probably going to be the communication part. The easy part will be all queues and collections, as you can use the C++ standard containers for all of them (e.g. std::queue for the actual queue, std::unordered_map for channels, and std::vector for the collection of connected clients.)

The clients are very simple, all the need to do is being able to send the subscription and publish messages to the server, and receive the publish messages from the server. The hard part will once again be the actual communication part.

---

Postscript:

I've never actually built such a system my self, all of the above was just directly of the top of my head. An experienced programmer shouldn't need more than a couple of hours to implement the basics, maybe a couple of days for an inexperienced one.

For the communication you could use e.g. Boost ASIO, maybe use one threads per channel. And you can use something like Boost property tree to construct/parse JSON or XML messages.

However, all of this is kind of reinventing the wheel, when you could probably start using one of the existing systems like RabbitMQ in a couple of hours, saving you a lot of time (and a lot of bugs!)

Solution 2

As far as lightweight servers go, Redis supports pub/sub commands.

The Redis code itself is extremely tight (only a couple files), it's single-threaded (use an event-loop), and the memory consumption is quite low (compared to other Queing systems I have seen).

Solution 3

I know it is late but may be useful for others. I implemented a basic pub/sub in C++ using boost.

CppPubSub

Usage is very simple. From one end publish your data (generic map) on a channel and other side subscribe for same channel and receive the generic map again.

// you should create a singleton object of NotificationService class, make it accessible throughout your application. 
INotificationService* pNotificationService = new NotificationService();

// Subscribe for the event.
function<NotificationHandler> fnNotificationHandler = bind(&SubscriberClass::NotificationHandlerFunction, this, std::placeholders::_1);
subscriptionToken = pNotificationService->Subscribe("TEST_CHANEL", fnNotificationHandler);

// Publish event
NotificationData _data;
_data["data1"] = "Hello";
_data["data2"] = "World";
pNotificationService->Publish("TEST_CHANEL", _data);

// Unsubscribe event.
pNotificationService->Unsubscribe(subscriptionToken);

Comments

[Lorenzo Dematté]

I second the idea, but only if the "framework" you are going to build is super-simple and needs to be lightweight, otherwise.. do not reinvent the wheel! I built this kind of system for a past project, in .NET, in just a couple of days, everything included. It was fun, and was blazing fast. And since it was very economical when we later needed to scale out, we threw it away and replaced it without looking back. In C++, I would use Boost as well.

[Yoshi]

I really like Redis and already used it as a cache/store service on some private projects. Just curious if it fit on a small CPU like raspberry PI?

[Matthieu M.]

@Yoshi: compared to any of the other queuing system (ActiveMQ, RabbitMQ, HornetQ, ...) it is certainly more likely to fit. I would be more concerned with RAM, but you can always remove the Lua JIT etc... to get a lighter process.

[liuyanghejerry]

It seems redis is only good for text-based messages. What about binary?

[Matthieu M.]

@liuyanghejerry: it is agnostic toward keys and values alike, and treat the content as raw memory; so no issue.

[Kumar Roshan Mehta]

What do you mean by a channel? In client server system, a client always initiate the connection. Do you mean, this connection should never be closed?

[Kumar Roshan Mehta]

The client here is meant as subscriber.

[Kumar Roshan Mehta]

So the (client)subscriber connection to server(publisher is persistent? What happens if the connection fails.