Why use your application-level cache if database already provides caching?

Question

Modern database provide caching support. Most of the ORM frameworks cache retrieved data too. Why this duplication is necessary?

Solution 1

Because to get the data from the database's cache, you still have to:

1. Generate the SQL from the ORM's "native" query format
2. Do a network round-trip to the database server
3. Parse the SQL
4. Fetch the data from the cache
5. Serialise the data to the database's over-the-wire format
6. Deserialize the data into the database client library's format
7. Convert the database client librarie's format into language-level objects (i.e. a collection of whatevers)

By caching at the application level, you don't have to do any of that. Typically, it's a simple lookup of an in-memory hashtable. Sometimes (if caching with memcache) there's still a network round-trip, but all of the other stuff no longer happens.

Solution 2

Scaling read-write transactions using a strongly consistent cache

Scaling read-only transactions can be done fairly easily by adding more Replica nodes.

However, that does not work for the Primary node since that can be only scaled vertically:

And that's where a cache comes into play. For read-write database transactions that need to be executed on the Primary node, the cache can help you reduce the query load by directing it to a strongly consistent cache, like the Hibernate second-level cache:

Using a distributed cache

Storing an application-level cache in the memory of the application is problematic for several reasons.

First, the application memory is limited, so the volume of data that can be cached is limited as well.

Second, when traffic increases and we want to start new application nodes to handle the extra traffic, the new nodes would start with a cold cache, making the problem even worse as they incur a spike in database load until the cache is populated with data:

To address this issue, it's better to have the cache running as a distributed system, like Redis. This way, the amount of cached data is not limited by the memory size on a single node since sharding can be used to split the data among multiple nodes.

And, when a new application node is added by the auto-scaler, the new node will load data from the same distributed cache. Hence, there's no cold cache issue anymore.

Solution 3

Here are a couple of reasons why you may want this:

• An application caches just what it needs so you should get a better cache hit ratio
• Accessing a local cache will probably be a couple of orders of magnitude faster than accessing the database due to network latency - even with a fast network

Solution 4

Even if a database engine caches data, indexes, or query result sets, it still takes a round-trip to the database for your application to benefit from that cache.

An ORM framework runs in the same space as your application. So there's no round-trip. It's just a memory access, which is generally a lot faster.

The framework can also decide to keep data in cache as long as it needs it. The database may decide to expire cached data at unpredictable times, when other concurrent clients make requests that utilize the cache.

Your application-side ORM framework may also cache data in a form that the database can't return. E.g. in the form of a collection of java objects instead of a stream of raw data. If you rely on database caching, your ORM has to repeat that transformation into objects, which adds to overhead and decreases the benefit of the cache.

Solution 5

A lot of good answers here. I'll add one other point: I know my access pattern, the database doesn't.

Depending on what I'm doing, I know that if the data ends up stale, that's not really a problem. The DB doesn't, and would have to reload the cache with the new data.

I know that I'll come back to a piece of data a few times over the next while, so it's important to keep around. The DB has to guess at what to keep in the cache, it's doesn't have the information I do. So if I fetch it from the DB over and over, it may not be in cache if the server is busy. I could get a cache miss. With my cache, I can be sure I get a hit. This is especially true on data that is non-trivial to get (i.e. a few joins, some group functions) as opposed to just a single row. Getting a row with the primary key of 7 is easy for the DB, but if it has to do some real work, the cost of the cache miss is much higher.

View more solutions

Comments

[chickeninabiscuit]

Some relevant links: docs.jboss.org/hibernate/stable/core/reference/en/html/… and also: javalobby.org/java/forums/t48846.html

[XDS]

The points made on application-level-caching are valid concerns but not insurmountable obstables that cannot possibly be overcome. In the 'go' language there is google's 'group-cache' which (github.com/golang/groupcache) which deals with some of these concerns pretty elegantly. Just my 2c to help folks create a mindmap on what's currently available/achievable in terms of caching options.