Is a good idea to store chat messages in a mongodb collection?

Question

I'm developing a chat app with node.js, redis, socket.io and mongodb. MongoDB comes the last and for persisting the messages.

My question is what would be the best approach for this last step?

I'm afraid a collection with all the messages like

{
    id,
    from,
    to,
    datetime,
    message
}

can get too big too soon, and is going to get very slow for reading purposes, what do you think?

Is there a better approach you already worked with?

Solution 1

I think the db structure is fine, the way you mentioned in your question.

You may assign some unique id for chat between each pair and keep it in each record of chat. Retrieve based on that when you want to show it.

Say 12 is the unique id for chat between A and B, retrieve should be based on 12 when you want to show chat for A and B.

So your db structure can be like:-

{
    id,
    from,
    to,
    datetime,
    message,
    uid
}

Remember, you can optimize your retrieve, if you will give some limit(say 100 at a time) for retrieve. If user is scrolling beyond 100 retrieve more 100 chats. Which will solve lots of retrieve.

When using limit, retrieve based on date created and use sort with find query as well.

Solution 2

In MongoDB, you store your data in the format you will want to read them later.

If what you read from the database is a list of messages filtered on the 'to' field and with a dynamic datetime filter, then this schema is the perfect fit.

Don't forget to add an index on the fields you will be querying on, then it will be reasonable fast to query them, even over millions of records.

If you would, for example, always show a full history of a full day, you would store all messages for a single day in one document. If both types of queries occur a lot, you would even store your messages in both formats.

If storage is an issue, you could also use capped collection, which will automatically delete messages of e.g. over 1 year old.

Solution 3

Just a thought here, are the messages plain text or are you allowed to share images and videos as well ?

If it's the latter then storing all the chats for a single day in one collection might not work out.

Actually if you have images and videos shares allowed then you need to take into account the. 16mb document restriction as well.

Comments

[R01010010]

thanks, precisely the capped thing was something I was pondering and didn't know how to do it, thanks for the info

[R01010010]

it is going to be only text for now, but that's a nice point to have in account for the future

[jitendra rajput]

I think no need to store uid. Because mongoDB provide _id it is always unique

[Ishwar Chandra Tiwari]

@jitendrarajput uid will be used to retrieve data easily otherwise you will face some difficulty in fetching messages between two users

[Ishwar Chandra Tiwari]

@Shrabanee what should i do when i am not storing uid and i will have to fetch chat between two users? i think it is possibly with mongo aggression

[alimfazeli]

is it actually a good idea to store media in the DB? I don't think this should happen ever