ServiceStack.Net Redis: Storing Related Objects vs. Related Object Ids

Question

My team has decided to work with Redis via the ServiceStack.net Redis Client as an underlying repository for a new high-volume website we're working on. I'm not really sure where to look for documentation for this question (either for general Redis docs or specific ServiceStack.Net docs or both) - is there actually a definitive source for documentation on how to implement a Redis via ServiceStack.Net that includes all you need to know about both Redis concepts and ServiceStack.Net concepts, or do we need to integrate documentation from both aspects separately to get the full picture?.

I'm just grappling with how exactly to store related objects in our model's object graph. Here's a simple scenario that I want to work with:

There are two objects in the system: User and Feed. In RDBMS terms these two objects have a one-to-many relationship, that is, a User has a collection of Feed objects and a feed can only belong to one User. Feeds will always be accessed from Redis via their user but occasionally we'll want to get access to the user via a feed instance.

So the question I have is whether we should be storing the related objects as properties or should we store the Id values of the related objects? To illustrate:

Approach A:

public class User
{
    public User()
    {
        Feeds = new List<Feed>();
    }

    public int Id { get; set; }

    public List<Feed> Feeds { get; set; }

    // Other properties
}

public class Feed
{
    public long Id { get; set; }

    public User User { get; set; }
}

Approach B:

public class User
{
    public User()
    {
        FeedIds = new List<long>();
    }

    public long Id { get; set; }

    public List<long> FeedIds { get; set; } 

    public List<Feed> GetFeeds()
    {
        return repository.GetFeeds( FeedIds );
    }
}

public class Feed
{
    public long Id { get; set; }

    public long UserId { get; set; }

    public User GetUser()
    {
        return repository.GetUser( UserId );
    }
}

Which of the above approaches will work best? I've seen both approaches used in various examples but I get the impression that some of the examples I've seen may not be best-practice.

A few simple related questions:

• If I make a change to an object will it automatically be reflected in Redis or will it require a save? I'm assuming the latter, but need to be absolutely clear.
• If I (can) use Approach A, will an update to User object X be reflected throughout the entire object graph wherever it is referenced or will it be necessary to save changes across the graph?
• Is there a problem with storing an object via it's interface (i.e. use IList<Feed> as opposed to List<Feed>?

Sorry if these questions are a little basic - until 2 weeks ago I'd never even heard of Redis - let alone ServiceStack - (nor had anyone in my team) so we're really starting from scratch here...

Solution

Rather than re-hash a lot of other documentation that's out there in the wild, I'll list a couple around for some background info around Redis + ServiceStack's Redis Client:

• What to think about when designing a NoSQL Redis application
• Designing a NoSQL Database using Redis
• General Overview of Redis and .NET
• Schemaless versioning and Data Migrations with C# Redis Client

There is no magic - Redis is a blank canvas

First I want to point out that using Redis as a data store just provides a blank canvas and doesn't have any concept of related entities by itself. i.e. it just provides access to distributed comp-sci data structures. How relationships get stored is ultimately up to the client driver (i.e. ServiceStack C# Redis Client) or the app developer, by using Redis's primitive data structure operations. Since all the major data structures are implemented in Redis, you basically have complete freedom on how you want to structure and store your data.

Think how you would structure relationships in code

So the best way to think about how to store stuff in Redis, is to completely disregard about how data is stored in an RDBMS table and think about how it is stored in your code, i.e. using the built-in C# collection classes in memory - which Redis mirrors in behavior with their server-side data-structures.

Despite not having a concept of related entities, Redis's built-in Set and SortedSet data structures provide the ideal way to store indexes. E.g. Redis's Set collection only stores a max of 1 occurrence of an element. This means you can safely add items/keys/ids to it and not care if the item exists already as the end result will be the same had you called it 1 or 100 times - i.e. it's idempotent, and ultimately only 1 element remains stored in the Set. So a common use-case is when storing an object graph (aggregate root) is to store the Child Entity Ids (aka Foreign Keys) into a Set every time you save the model.

Visualizing your data

For a good visualization of how Entities are stored in Redis I recommend installing the Redis Admin UI which works well with ServiceStack's C# Redis Client as it uses the key naming convention below to provide a nice hierarchical view, grouping your typed entities together (despite all keys existing in the same global keyspace).

To view and edit an Entity, click on the Edit link to see and modify the selected entity's internal JSON representation. Hopefully you'll be able to make better decisions about how to design your models once you can see how they're stored.

How POCO / Entities are stored

The C# Redis Client works with any POCOs that have a single primary key - which by default is expected to be Id (though this convention overridable with ModelConfig). Essentially POCOs gets stored into Redis as serialized JSON with both the typeof(Poco).Name and the Id used to form a unique key for that instance. E.g:

urn:Poco:{Id} => '{"Id":1,"Foo":"Bar"}'

POCOs in the C# Client are conventionally serialized using ServiceStack's fast Json Serializer where only properties with public getters are serialized (and public setters to get de-serialized back).

Defaults are overrideable with [DataMember] attrs but not recommended since it uglifies your POCOs.

Entities are blobbed

So knowing that POCOs in Redis are just blobbed, you only want to keep non-aggregate root data on your POCOs as public properties (unless you purposely want to store redundant data). A good convention is to use methods to fetch the related data (since it wont get serialized) but also tells your app which methods make remote calls to read data.

So the question on whether the Feed should get stored with the User is whether or not it's non-aggregate root data, i.e. whether or not you want to access the users feeds outside the context of the user? If no, then leave the List<Feed> Feeds property on the User type.

Maintaining Custom Indexes

If however you would like to keep all feeds accessible independently, i.e. with redisFeeds.GetById(1) then you will want to store it outside of the user and maintain an index linking the 2 entities.

As you've noticed there are many ways to store relationships between entities and how you do so is largely a matter of preference. For the child entity in a parent>child relationship you would always want to store the ParentId with the child entity. For the Parent you can either choose to store a collection of ChildIds with the model and then do a single fetch for all child entities to re-hydrate the model.

Another way is to maintain the index outside of the parent dto in its own Set for each parent instance. Some good examples of this is in the C# Source code of the Redis StackOverflow demo where the relationship of Users > Questions and Users > Answers is stored in:

idx:user>q:{UserId} => [{QuestionId1},{QuestionId2},etc]
idx:user>a:{UserId} => [{AnswerId1},{AnswerId2},etc]

Although the C# RedisClient does include support for a default Parent/Child convention via its TParent.StoreRelatedEntities(), TParent.GetRelatedEntities<TChild>() and TParent.DeleteRelatedEntities() APIs where an index is maintained behind the scene that looks like:

ref:Question/Answer:{QuestionId} => [{answerIds},..]

Effectively these are just some of your possible options, where there are many different ways to achieve the same end and in which you also have the freedom to roll your own.

NoSQL's schema-less, loose-typing freedoms should be embraced and you shouldn't be worried about trying to follow a rigid, pre-defined structure you might be familiar with when using an RDBMS.

In conclusion, there's no real right way to store data in Redis, e.g. The C# Redis Client makes some assumptions in order to provide a high-level API around POCOs and it blobs the POCOs in Redis's binary-safe string values - though there are other clients will prefer to store an entities properties in Redis Hashes (Dictionaries) instead. Both will work.

Comments

[Zac Seth]

Thanks for the reply @mythz - I haven't read this yet (just starting now) but from what I've seen of your other SO answers relating to ServiceStack I'm sure it will be an excellent answer. Love it when OS devs get involved with helping their users out.

[Zac Seth]

Yes, that was very much an excellent answer. I really appreciate the time you've taken to give me this very comprehensive answer.