Why use redis in a chat application?

Question

I just recently built a chat, it's working pretty well, but I think I need to hook it up to redis.

From what I understand I need redis for scaling and holding some data if a client refreshes or a server goes down.

A core component of the 1on1 chat is that I store the users, and associate a socket.id to those users

var users = {};
io.sockets.on('connection', function (socket) {

  // store the users & socket.id into objects
  users[socket.handshake.headers.user.username] = socket.id;

});

Now on the client side I can say hey I want to chat with "Jack", as long as that is a valid user then I can pass that data to the server, i.e the user name and message just to jack like so.

var chattingWith = data.nickname; // this is Jack passed from the client side
io.to(users[chattingWith]).emit();

My question is, why should I use redis? What should I store in redis? How should I interact with that data?

I am using an io.adapter

io.adapter(redisIo({ 
  host: 'localhost', 
  port: 6379,
  pubClient: pub,
  subClient: sub
}));

Also reading code from an example app I see when a socket connects they save the socket data into redis like so.

// store stuff in redis
redisClientPublish.sadd('sockets:for:' + userKey + ':at:' + room_id, socket.id, function(err, socketAdded) {
  if(socketAdded) {
    redisClientPublish.sadd('socketio:sockets', socket.id);
    redisClientPublish.sadd('rooms:' + room_id + ':online', userKey, function(err, userAdded) {
      if(userAdded) {
        redisClientPublish.hincrby('rooms:' + room_id + ':info', 'online', 1);
        redisClientPublish.get('users:' + userKey + ':status', function(err, status) {
          io.sockets.in(room_id).emit('new user', {
            nickname: nickname,
            provider: provider,
            status: status || 'available'
          });
        });
      }
    });
  }
});

They use it when entering a room, to get information about the room.

app.get('/:id', utils.restrict, function(req, res) {

  console.log(redisClientPublish);

  utils.getRoomInfo(req, res, redisClientPublish, function(room) {

    console.log('Room Info: ' + room); 

    utils.getUsersInRoom(req, res, redisClientPublish, room, function(users) {

      utils.getPublicRoomsInfo(redisClientPublish, function(rooms) {

        utils.getUserStatus(req.user, redisClientPublish, function(status) {
          utils.enterRoom(req, res, room, users, rooms, status);
        });

      });

    });

  });

});

So again, I am asking because I am kind of confused if I need to store anything inside redis/why I need to, for instance we may have a few hundred thousand users and the node.js server "Jack" and "Mike" are chatting on goes down, it then changes to point to a new node.js instance.

Obviously I want the chat to still remember "Jack's" socket id is "12333" and "Mike's" socket id is "09278" so whenever "Jack" says hey I want to send "Mike/09278" a message the server side socket will direct it properly.

Would storing the username as a key and socket ID as a value be a wise use case for redis, would that socket.id still work?

Solution

Redis is a pretty good choice as a database for a chat as it provides a couple of data structures that are not only very handy for various chat use cases but also processed in a really performant way. It also comes along with a PubSub messaging functionality that allows you to scale your backend by spawning multiple server instances.

Scaling socket.io with the socket.io-redis adapter

When you want to run multiple instances of your server - be it because of one server not being able to handle increasing users any more or for setting up a high availablility cluster - then your server instances must communicate with each other in order to be able to deliver messages between users who are connected to different servers. The socket.io-redis adapter solves this by using the redis PubSub feature as a middleware. This won't help you if you are using only a single server instance (in fact I assume it will be slightly less performant) but as soon as you spawn a second server this will work out just fine without any headaches.

Want to get a feeling and some insight on how it's working? Monitor your dev redis while using it and you'll see the internal socket.io messages that are pushed through redis.

redis-cli
monitor

Use cases and their according redis data types

Save active conversations in a SET

A redis set is a collection of unique strings. I don't think storing socket.io id's would work out well as you can't assume that a user will get the same id on a reconnect. Better store his rooms and rejoin him on connect. You add every chat room (btw. direct messages can be defined as a room with two participiants so the handling is the same in both cases) that a user enters to their room set. On a server restart, a client reconnect or second client instance you can retrieve the whole set and rejoin users to their rooms.

/* note: untested pseudo code just for illustration */
io.sockets.on('connection', function (socket) {
    rooms = await redis.smembers("rooms:userA");
    rooms.foreach (function(room) {
        socket.join(room);
    }

    socket.on('leave', room) {
        socket.leave(room);
        redis.srem("rooms:userA", room);
    } 

    socket.on('join', room) {
        socket.join(room);
        redis.sadd("rooms:userA", room);
    }
}

Save the last 10 messages of a conversation using a redis LIST

A redis list is somewhat of an persistent array of strings. You push a new message into a list and pop the oldest when the list size reaches your threshold. Conveniently the push command returns the size right away.

socket.on('chatmessage', room, message) {
    if (redis.lpush("conversation:userA:userB", "Hello World") > 10) {
        redis.rpop("conversation:userA:userB");
    }
    io.to(room).emit(message);
}

To get the message history use lrange:

msgHistory = redis.lrange("conversation:userA:userB", 0, 10)

Save some basic user details in a HASH

A hash is a key/value collection. Use it to store the online status along with avatar urls or whatever.

io.sockets.on('connection', function (socket) {
    redis.hset("userdata:userA", "status", "online");

    socket.on('disconnect', function () {
        redis.hset("userdata:userA", "status", "offline");
    }
}

Maintain a "recent conversations" list in a SORTED LIST

Sorted sets are similar to SETs but you can assign a score value to every element and retrieve the set ordered by this value. Simply use a timestamp as score whenever there is an interaction between two users and that's it.

 socket.on('chatmessage', room, message) {
      io.to(room).emit(message);
      redis.zadd("conversations:userA", new Date().getTime(), room);
 }

 async function getTheTenLatestConversations() {
     return await redis.zrange("conversations:userA", 0, 10);
 }

References

• socket.io-redis: https://github.com/socketio/socket.io-redis
• redis PubSub docs: https://redis.io/topics/pubsub
• redis data types: https://redis.io/topics/data-types-intro

Comments

[Alexey Shabramov]

Best answer ever! Thank you!

[stuartambient]

I'm developing with socket.io-redis. It does keep track of a user's subscribed rooms. I'm not sure where though. Would I still benefit from using sets to track it ? I could just use the socket.io-chat command for user rooms and send it to a Redis set I suppose?