Using nginx to serve content directly out of a redis cache

Question

I am using nginx to pass requests to a Node app. The app basically acts as a remote cache for html (checks to see if what the user is requesting is in the redis db, if it is just show that, if not grab it and store it in the redis cache and serve it up.)

I was curious if there was anyway to bypass hitting the Node app by having nginx serve up the content directly from redis? I have been fooling around with the http_redis module but I can't really get it to work.

A simple example would be: http://mywebsite.com/a where nginx would serve the content up in the 'a' key or pass it on to the node app if the key did not exist. Is this even possible?

Solution 1

Maybe something more difficult to setup than Webdis but you can do that directly in the nginx daemon with some extra modules like redis2-nginx-module. You will have to recompile nginx.

There is some good examples of configuration on the home page.

For instance :

# GET /get?key=some_key
location /get {
    set_unescape_uri $key $arg_key;  # this requires ngx_set_misc
    redis2_query get $key;
    redis2_pass foo.com:6379;
}

Of course, with a little more nginx configuration, you can get another URL pattern.

Note that for this example, you will have to compile ngx_set_misc module too.

Solution 2

I know this is an old thread but still, this may be useful for some. I tried the same approach as you having nginx serve from Redis directly without hitting using HttpRedis2Module in nginx. I were happy when I got it working because it was some hassle with it, but when I did some stress-tests I'm afraid that it gave really bad results.

It actually was a bit faster and much more stable to serve using nginx->php->mongodb than just using nginx->redis with the module.

Solution 3

You should be able to get something by setting up Nginx as a reverse proxy for Webdis.

The way you use Webdis is that you put the whole command in the URL, so to GET the key a you request /GET/a. This means that if everything you want to serve is available using GET you can do something like this in Nginx:

location / {
  rewrite ^(.*)$ /GET/$1 break;
  proxy_pass http://127.0.0.1:7379/;
}

(I'm writing the config off the top of my head here, the syntax might be slightly off).

However, the Webdis project is very young so there's no telling how well it will work, and the responses are JSON documents with some extra fluff that you probably don't want to return.

Comments

[Ryan S]

hmmm, I'll check it out. I don't know I whole lot about nginx so I didn't even know if what I was asking was possible. I guess for your method I would just have to see if serving it from Node or from Webdis would be faster. It seems that nginx is able to server directly from memcache and the http_redis module seems modeled after it.

[Goran Jurić]

Did you setup a keepalive for your redis pool in nginx? Last time I was testing this keepalive was a difference between slover than php -> redis and much faster...

[ColinM]

I am not surprised that PHP+MongoDb is faster than just Redis in benchmarks because Redis is a single process with no multi-threading meaning it handles all requests serially with one process no matter how many CPU cores you have. So while it is very fast, it is quite possible that PHP+MongoDb is faster with high concurrency since they have the ability to fully utilize all CPU cores by tackling many requests in parallel. Also MongoDb is just blazing fast.

[rrauenza]

If you're also using this with pub/subs you'll want to turn proxy_buffering off and the proxy_read_timeout to an appropriately long value.