Firebase chat - removing old messages

Question

I have create a chat with just 1 room, private messages, moderation and everything now and it all works great! While I was testing the chat, I realised that all the messages every typed in the chat got saved and if you had a lot of people using the chat it would very quickly take up quite a lot of space in your Firebase.

To give you an example of what I am looking for, let me show you how I handle private messages:

When John sends a private message to Bob, the message will be added to both John and Bobs list of private messages, like this:

/private/John <-- Message appended here
/private/Bob <-- Message appended here

This is an example of how the firebase would look with 2 message in the chat, and 2 private messages:

{
  "chat" : {
    "516993ddeea4f" : {
      "string" : "Some message here",
      "time" : "Sat 13th - 18:20:29",
      "username" : "test",
    },
    "516993e241d1c" : {
      "string" : "another message here",
      "time" : "Sat 13th - 18:20:34",
      "username" : "Test2",
    }
  },
  "private" : {
    "test" : {
      "516993f1dff57" : {
        "string" : "Test PM reply!",
        "time" : "Sat 13th - 18:20:49",
        "username" : "Test2",
      },
      "516993eb4ec59" : {
        "string" : "Test private message!",
        "time" : "Sat 13th - 18:20:43",
        "username" : "test",
      }
    },
    "Test2" : {
      "516993f26dbe4" : {
        "string" : "Test PM reply!",
        "time" : "Sat 13th - 18:20:50",
        "username" : "Test2",
      },
      "516993eab8a55" : {
        "string" : "Test private message!",
        "time" : "Sat 13th - 18:20:42",
        "username" : "test",
      }
    }
  }
}

and the same goes the other way around. Now if Bob where to disconnect, Bobs list of private messages get removed, but John is still able to see his conversation with Bob because he got a copy of all the messages on his list. If John then disconnect after Bob, the Firebase would be cleaned and their conversation no longer stored.

Is there a way to achieve something like this with the General chat? Pushing a message to all the users who are using the chat does not seem like a good solution (?). Or is it possible to somehow make the Firebase only keep the latest 100 messages for example?

I hope it makes sense!

Kind regards

Ps. Big thanks for the Firebase team for all the help so far, I really appreciate it.

Solution

There are a few different ways to tackle this, some more complicated to implement than others. The simplest solution would be have each user only read the latest 100 messages:

var messagesRef = new Firebase("https://<example>.firebaseio.com/message").limit(100);
messagesRef.on("child_added", function(snap) {
  // render new chat message.
});
messagesRef.on("child_removed", function(snap) {
  // optionally remove this chat message from DOM
  // if you only want last 100 messages displayed.
});

Your messages list will still contain all messages but it won't affect performance since every client is only asking for the last 100 messages. In addition, if you want to clean up old data, it is best to set the priority for each message as the timestamp of when the message was sent. Then, you remove all the messages older than 2 days with:

var timestamp = new Date();
timestamp.setDate(timestamp.getDate()-2);
messagesRef.endAt(timestamp).on("child_added", function(snap) {
  snap.ref().remove();
}); 

Hope this helps!

Comments

[MrE]

Hi Anant, The code works in JS, however is this possible via the REST API? Delete messages based on priority? I will mark this today, as it does do what is required if using JS.

[Anant]

It is possible, but requires multiple steps. First, you need to retrieve the data with priorities - you can do this by adding a GET param "format=export". This will include the prirority of every object in a special ".priority" key (see firebase.com/docs/rest-api.html). Then, you can sort the object by priority locally and then issue DELETE requests for all the objects you want to delete.

[MrE]

I thought this would be the way to do it, just wanted to make sure in case there where an obvious alternative I missed. Thanks, it works perfectly!

[Xi 张熹]

Is there a way to run a server daemon for this on Firebase?

[Luke Irvin]

Is there a way to query the last 100 messages, but do not automatically delete an older message when a newer messages comes through? Basically, query the last 100, add to that, and then allow all users to manually delete their own messages.