Reverse Proxy Server Cloud Architecture (AWS + nginx)

Question

Currently I cache responses using nginx + fastcgi. I'm in the process of moving my app to AWS. I'm architecting a horizontally scaled approach, but trying to determine where best to put nginx + fastcgi cache.

I found a few solutions from my own research, both with significant drawbacks.

• Use AWS Elastic Load Balancer to balance a cluster of web apps. Each web app runs it's own nginx and web server stack. Drawback: the cache on each server needs to be warmed. Worst case 1/n the hit rate.
• Put nginx + fastcgi in front of the AWS ELB so the cache is centralized for maximum hits. Drawback: single point of failure - if nginx dies traffic doesn't pass thru to the ELB. Added nginx instance.

I'd appreciate any suggestions of common architectures that overcome the drawbacks outlined above.

Solution 1

I've set up this stack many times on AWS. Option #1 has always worked well for me and is the one I normally choose because it's the simplest. I'm curious to know how much traffic you're dealing with where the less-than-ideal initial cache hits are an issue? I'm serving a few million pageviews a month on a pair of m1.small instances and they're barely scratched.

Other thoughts:

• Nginx can use memcached as a caching store. Plug it into an ElasticCache cluster. This will give you a very fast, centralized datastore for multiple instances to connect to for cache contents. (I've never done this, but it looks doable. Please post results if you do attempt it.)

• The SPoF problem mentioned in Option #2 can be mitigated by setting up an autoscaling group that contains a single instance. If it dies, it should come back online in a matter of a couple of minutes. You will need to fiddle with a boot script that automatically grabs the Elastic IP though.

• Keep in mind that putting anything in front of a ELB will require that you use VPC. This is generally good practice anyhow, but it can have a steep learning curve.

• CloudWatch Alarm Actions were just announced today. You could do some cool stuff with this to help minimize the SPoF issue.

Solution 2

Looks like you are thinking about the same architecture what I am planning to implement.

I have thought of both the scenarios and the same approach, and then settling up to this solution.

I will be using the 2nd option, setting up Caching solution in front of Load Balancer. I am going to handle the Single point of Failure problem with adding one more node having standby and using something like HAProxy to keep checking the health of the machine. If first Caching machine went down, then HAProxy will automatically move the IP and move the traffic to the other Caching Standby Machine and hence the Single point of failure will be handled.

Also, in my scenario, I will be using something like Varnish, instead of Nginx+php, which I will suggest you too, incase your application is not dependent on Nginx. Varnish will have it's own built in Load Balancer, hence you will skip the AWS Load Balancer too.

Hope this will help you building a robust and scalable infrastructure.

Comments

[Jason McCreary]

@ceejayoz, true. That eliminates the single point of failure drawback, but combines the other two drawbacks.

[ceejayoz]

Not necessarily. You can likely have fewer servers in the first tier, meaning fewer caches to be warmed up, and you've eliminated the SPOF that's the drawback of the first.

[Jason McCreary]

Thanks for sharing. Nginx has a built in load balancer as well. But using these instead of ELB does not offer auto-scaling, among other things.

[Napster_X]

Oh yes, that's right. My architecture is kind of fixed, so never thought about auto-scaling. Thanx for sharing that thought.

[Jason McCreary]

Thanks. I'll look into memcached as a shared caching layer for nGinx to mitigate the low hit ratio. The SPoF is a large concern given the role of this box (single entry). Minutes of downtime while another instance boots is unacceptable.

[jamieb]

If mins of downtime isn't workable, you can run two identical instances, each in a separate AZ, and use a floating EIP. See this design pattern.