Expire a view-cache in Django?

Question

The @cache_page decorator is awesome. But for my blog I would like to keep a page in cache until someone comments on a post. This sounds like a great idea as people rarely comment so keeping the pages in memcached while nobody comments would be great. I'm thinking that someone must have had this problem before? And this is different than caching per url.

So a solution I'm thinking of is:

@cache_page( 60 * 15, "blog" );
def blog( request ) ...

And then I'd keep a list of all cache keys used for the blog view and then have way of expire the "blog" cache space. But I'm not super experienced with Django so I'm wondering if someone knows a better way of doing this?

Solution 1

This solution works for django versions before 1.7

Here's a solution I wrote to do just what you're talking about on some of my own projects:

def expire_view_cache(view_name, args=[], namespace=None, key_prefix=None):
    """
    This function allows you to invalidate any view-level cache. 
        view_name: view function you wish to invalidate or it's named url pattern
        args: any arguments passed to the view function
        namepace: optioal, if an application namespace is needed
        key prefix: for the @cache_page decorator for the function (if any)
    """
    from django.core.urlresolvers import reverse
    from django.http import HttpRequest
    from django.utils.cache import get_cache_key
    from django.core.cache import cache
    # create a fake request object
    request = HttpRequest()
    # Loookup the request path:
    if namespace:
        view_name = namespace + ":" + view_name
    request.path = reverse(view_name, args=args)
    # get cache key, expire if the cached item exists:
    key = get_cache_key(request, key_prefix=key_prefix)
    if key:
        if cache.get(key):
            # Delete the cache entry.  
            #
            # Note that there is a possible race condition here, as another 
            # process / thread may have refreshed the cache between
            # the call to cache.get() above, and the cache.set(key, None) 
            # below.  This may lead to unexpected performance problems under 
            # severe load.
            cache.set(key, None, 0)
        return True
    return False

Django keys these caches of the view request, so what this does is creates a fake request object for the cached view, uses that to fetch the cache key, then expires it.

To use it in the way you're talking about, try something like:

from django.db.models.signals import post_save
from blog.models import Entry

def invalidate_blog_index(sender, **kwargs):
    expire_view_cache("blog")

post_save.connect(invalidate_portfolio_index, sender=Entry)

So basically, when ever a blog Entry object is saved, invalidate_blog_index is called and the cached view is expired. NB: haven't tested this extensively, but it's worked fine for me so far.

Solution 2

The cache_page decorator will use CacheMiddleware in the end which will generate a cache key based on the request (look at django.utils.cache.get_cache_key) and the key_prefix ("blog" in your case). Note that "blog" is only a prefix, not the whole cache key.

You can get notified via django's post_save signal when a comment is saved, then you can try to build the cache key for the appropriate page(s) and finally say cache.delete(key).

However this requires the cache_key, which is constructed with the request for the previously cached view. This request object is not available when a comment is saved. You could construct the cache key without the proper request object, but this construction happens in a function marked as private (_generate_cache_header_key), so you are not supposed to use this function directly. However, you could build an object that has a path attribute that is the same as for the original cached view and Django wouldn't notice, but I don't recommend that.

The cache_page decorator abstracts caching quite a bit for you and makes it hard to delete a certain cache object directly. You could make up your own keys and handle them in the same way, but this requires some more programming and is not as abstract as the cache_page decorator.

You will also have to delete multiple cache objects when your comments are displayed in multiple views (i.e. index page with comment counts and individual blog entry pages).

To sum up: Django does time based expiration of cache keys for you, but custom deletion of cache keys at the right time is more tricky.

Solution 3

I wrote Django-groupcache for this kind of situations (you can download the code here). In your case, you could write:

from groupcache.decorators import cache_tagged_page

@cache_tagged_page("blog", 60 * 15)
def blog(request):
    ...

From there, you could simply do later on:

from groupcache.utils import uncache_from_tag

# Uncache all view responses tagged as "blog"
uncache_from_tag("blog") 

Have a look at cache_page_against_model() as well: it's slightly more involved, but it will allow you to uncache responses automatically based on model entity changes.

Solution 4

With the latest version of Django(>=2.0) what you are looking for is very easy to implement:

from django.utils.cache import learn_cache_key
from django.core.cache import cache
from django.views.decorators.cache import cache_page

keys = set()

@cache_page( 60 * 15, "blog" );
def blog( request ):
    response = render(request, 'template')
    keys.add(learn_cache_key(request, response)
    return response

def invalidate_cache()
    cache.delete_many(keys)

You can register the invalidate_cache as a callback when someone updates a post in the blog via a pre_save signal.

Solution 5

This won't work on django 1.7; as you can see here https://docs.djangoproject.com/en/dev/releases/1.7/#cache-keys-are-now-generated-from-the-request-s-absolute-url the new cache keys are generated with the full URL, so a path-only fake request won't work. You must setup properly request host value.

fake_meta = {'HTTP_HOST':'myhost',}
request.META = fake_meta

If you have multiple domains working with the same views, you should cycle them in the HTTP_HOST, get proper key and do the clean for each one.

View more solutions