MongoDB Structure for message app

Question

I am breaking my mind up thinking about a good document structure for handling a message app.

I basically need three (or four) types of objects:

1. The user (username, email, password, etc.)
2. The contacts list (containing different contacts or contacts groups)
3. The conversation (a conversation is a collection of messages between some persons)
4. The message (contains the message body, some timestamp and the creator.)

My idea was to embed the contacts into the user document and to embed the messages in a conversation document:

1. User

{
    username: 'dev.puS',
    usernameCanonical: 'dev.pus', // used for unique constraints
    email: '[email protected],
    emailCanonical: '[email protected],
    salt: 'some hash',
    password: 'hash with salt',
    logs: { last_login: 12.06.2008, last_password_reset: 04.03.2007 },
    state: { online: true, available: false },
    contacts: [ user_id1, user_id2, user_id3 ]
}

2. Conversation

{
    members: [ user_id1, user_id2 ],
    messages: [
        { author: user_2, body: 'Hi what's up' },
        { author: user_1, body: 'Nothing out here :(' },
        { author: user_2, body: 'Whanna ask some question on stackoverflow' },
        { author: user_1, body: 'Okay, lets go' }
    ]
}

What do you think about this schema?

I think it would be better to keep them seperated (so each document for it's own) because each document has different update frequency. But I really don't have any experience about it so it would be good to hear some advices :)

Regards

Solution 1

I see that this question is old, but for anyone interested, a similar question was asked and one answer looks viable https://stackoverflow.com/a/30830429/132610

Conversation : {
 id: 123,
 members: [ user_id1, user_id2 ]
}
Message { conversationId: 123, author: user_2, body: 'Hi what's up' }
Message { conversationId: 123, author: user_1, body: 'Whanna ask some question on stackoverflow' }

Update #1

1) Scalability: MongoDB scales well with very large collection. Billions of messages per collection. There is a technique called sharding that can allow you to split larger collection to multiple nodes.

2) Reading. Since MongoDB has indexing mechanisms, reads are comparable to any fine-tuned database engine. So reading will not be an issue. Especially, when a conversation(group|room) has fewer participants, for example two people messaging each other.

Solution 2

Your question is really one of schema design. I suggest taking a look at this page on MongoDB schema design to get a sense of the choices and trade-offs: http://www.mongodb.org/display/DOCS/Schema+Design

In addition, you should probably review the links in the 'See Also' section of that document. I especially recommend the video presentations.

Finally, you should probably take a look at this document for a discussion of the three possible schemas for a messaging/commenting database, including the trade-offs for each design: http://docs.mongodb.org/manual/use-cases/storing-comments/

Solution 3

Please find my suggestion:

    Person : {
        person_id: '123',
        last_login: 12.06.2008,
        online: true
    }

Conversation : {
 conversation_id: append the greater person_id to the lower person_id, // person_1_id =123 and person_2_id =124 then its 123124

messages: [ 
        { message_id: 1, 
          message_text : 'Hi what's up',
          sender_id : 123,
          receiver_id: 124,
          timestamp : 12344567891
        },
        { message_id: 2, 
          message_text : 'fine',
          sender_id : 124,
          receiver_id: 123,
          timestamp : 12344567891
        }
       ]
}

Comments

[Inzamam Malik]

I have a confusion in my mind, you (and everyone else) said one collection for Conversations and another Collection for Messages. let say we have 1 million users in the messaging and they are talking to each other, Messages table may reach billions of billion documents, does mongodb have capability to manage such big collection of documents, and what about search response time? let say we search for last 100 messages a single user in billions of billion messages how much time it will take to come back?

[Moe kanan]

@InzamamMalik, I have the same exact question!! Were you able to find the "best practice" answer?

[Giovanne Afonso]

Everytime you query this object you would get all the messages? what about a conversation with 1Mi messages? Wouldn't be better to have a separated collection for messages? Or maybe create a cronjob to move older messages to a new collection with older messages to paginate

[Tejinder]

Why you added comment object? Also, are there some recommendations if we want to do big group chats too?

[Sep GH]

There is a limit on size of an array in mongoDb and I guess it was 4MB if I'm correct. So this is really a bad design. Also creates too much complexity querying whats inside the array.

[Vahid Alimohamadi]

Here is a confusion for me. In private conversation time, We need to find "Is there any private conversation there which members are this two people?" if yes, we should load history of that conversation first ( e.g. 10 latest messages ) then send message to that conversation. So every time wee need to search all private conversations which user subscribed, first. But if a side was deleted that conversation (unsubscribed) then we can't subscribe him/her to it again. Is there a comprehensive analysis on this topic?