Why does Unicorn need to be deployed together with Nginx?

Question

I would like to know the difference between Nginx and Unicorn. As far as I understand, Nginx is a web server while Unicorn is a Ruby HTTP server.

Since both Nginx and Unicorn can handle HTTP requests, what is the need to use the combination of Nginx and Unicorn for RoR applications?

Solution 1

Nginx

Unicorn

Refer to unicorn on github for more information.

Solution 2

Nginx is a pure web server that's intended for serving up static content and/or redirecting the request to another socket to handle the request.

Unicorn is a Rack web server and only intended to host a 'Rack App' which is usually generating dynamic content. Rack apps can also serve up static content but it's less efficient than most other traditional web servers.

Most RoR setups use a combination of both traditional web servers and Rack servers to apply the best of both of their capabilities. Nginx is incredibly fast at request redirection through proxy balancing and serving up static content. Unicorn is quite capable of processing HTTP headers and balancing inbound requests to Ruby for processing.

Solution 3

This answer is complementary to the other ones and explains why Unicorn needs nginx in front of it.

TL;DR The reason that Unicorn is usually deployed together with a reverse proxy like nginx is because its creators deliberately designed it so, making a tradeoff for simplicity.

First of all, there's nothing keeping you from deploying Unicorn without a reverse proxy. However, that wouldn't be a very good idea; let's see why.

Unicorn follows the Unix philosophy which is to do one thing and do it well, and that is to serve fast, low-latency clients (we'll see what this means later on). The fact that Unicorn is designed for fast, low-latency clients also implies that it's not very good with slow, high-latency clients, which is indeed true. This is one of Unicorn's weak points and it's where a reverse proxy comes into play: it sits in front of Unicorn and takes care of those slow clients (we'll see how later on).

Fortunately, such a reverse proxy already exists and is called nginx.

The decision to handle only fast clients, greatly simplifies the design of Unicorn and allows a much simpler and smaller codebase, at the cost of some added complexity on the deployment department (ie. you have to deploy nginx too in addition to Unicorn).

An alternative decision could be designing Unicorn in such a way that it wouldn't need a reverse proxy. However, this means that it would have to implement extra functionality to do all the things that now nginx does, resulting in a more complex codebase and more engineering efforts.

Instead its creators made the decision to leverage existing software that is battle-tested and very well designed and to avoid wasting time and energy on problems already solved by other software.

But let's get technical and answer your question:

Why does Unicorn needs to be deployed together with nginx?

Here are some of the key reasons:

Unicorn uses blocking I/O for clients

Relying on a reverse proxy means that Unicorn doesn't need to use non-blocking I/O. Instead it can use blocking I/O which is inherently simpler and easier for the programmer to follow.

Also as the DESIGN document states:

[Using blocking I/O] allows a simpler code path to be followed within the Ruby interpreter and fewer syscalls.

However, this also has some consequences:

Key point #1: Unicorn is not efficient with slow clients

(For simplicity's sake, we assume a setup with 1 Unicorn worker)

Since blocking I/O is used, a Unicorn worker can only serve one client at a time, so a slow client (ie. one with a slow connection) would effectively keep the worker busy for a longer time (than a fast client would do). In the meantime, the other clients would just wait until the worker is free again (ie. requests would pile up in the queue).

To get around this issue, a reverse proxy is deployed in front of Unicorn, that fully buffers incoming requests and the application responses, and then sends each of them at once (aka spoon-feeds them) to Unicorn and the clients, respectively. In that regard, you could say that the reverse proxy "shields" Unicorn from slow network clients.

Fortunately Nginx is a great candidate for this role, since it is designed to handle thousands of hundreds concurrent clients efficiently.

It's of crucial importance that the reverse proxy should be within the same local network as Unicorn (typically in the same physical machine communicating w/ Unicorn via a Unix domain socket), so that the network latency is kept to a minimum.

So such a proxy effectively plays the role of a fast client that Unicorn is designed to serve in the first place, since it proxies requests to Unicorn fast and keeps the workers busy for the shortest possible amount of time (compared to how much time a client with a slow connection would do).

Key point #2: Unicorn does not support HTTP/1.1 keep-alive

Since Unicorn uses blocking I/O, it also means that it can't support the HTTP/1.1 keep-alive feature, since the persistent connections of slow clients would quickly occupy all the available Unicorn workers.

Therefore to leverage HTTP keep-alive, guess what: a reverse proxy is used.

nginx on the other hand, can handle thousands of concurrent connections using just a few threads. Therefore, it doesn't have the concurrency limits a server like Unicorn has (which essentially limited to the amount of worker processes), which means it can handle persistent connections just fine. More of how this actually works can be found here.

That's why nginx accepts keep-alive connections from the clients and proxies them to Unicorn over plain connections via typically a Unix socket.

Point #3: Unicorn is not very good at serving static files

Again, serving static files is a thing that Unicorn can do but is not designed to do efficiently.

On the otherhand, reverse proxies like nginx though are much more better at it (ie. sendfile(2) & caching).

More

There are other points which are outlined in the PHILOSOPHY document (see "Improved Performance Through Reverse Proxying").

See also some of nginx's basic features.

We see that by leveraging existing software (ie. nginx) and following the Unix philosophy of "doing one thing and do it well", Unicorn is able to follow a simpler design and implementation while maintaining to be efficient at serving Rack apps (eg. your Rails app).

For more information refer to Unicorn's philosophy and design documents which explain in more detail the choices behind Unicorn's design and why nginx is considered a good reverse-proxy for Unicorn.

Solution 4

Nginx can be used to serve slow clients on a unicorn server as slow clients would choke the unicorn server. Nginx is used as some sort of proxy buffering all requests and responses to slow clients.

See http://unicorn.bogomips.org/

View more solutions

Comments

[loganathan]

Pratik, What my question is unicorn server can serve both static and dynamic process, then why we are using NGinx or Apache those can process the only static contents, combinely with the passenger or unicorn or mod_php ?

[Pratik]

@loganathan, Both Apache and Nginx are much faster at serving static content than ruby or any of the application servers. They also know how to handle caching and are good at allowing concurrent file downloads while still taking in traffic and passing it to the application servers.

[BraveNewCurrency]

Also, if you have large amounts of data coming + going, nginx will buffer it from (and spoon feed to) the client. Without nginx, one of your unicorns will be tied up during uploads/downloads.

[gorn]

This does not answer the question why nginx is needed at all. It just puts it in both pictures without any comment. Nick's answer is much better.

[Joshua Grosso Reinstate CMs]

I agree with @gorn. This meant nothing to me, for example.

[Agis]

Agree with @gom. Not even the linked Github blog post explains why is nginx needed. See my answer for a detailed explanation: stackoverflow.com/questions/8739871/…

[omikes]

Before reading the other answers I wanted to agree that this answer provides no information whatsoever. The best I can make of this diagram is that unicorn used Nginx as a model, rewrote it, and changed some vocabulary words around. Does it really do the same thing?

[omikes]

But why not just use Nginx I think he was asking, not 'why does unicorn require nginx'. What does Unicorn provide that Nginx does not, for example?

[Agis]

@oMiKeY If that is the case, I believe the other answers adress this question fairly good. I still think my answer provides useful information for anyone trying to understand the combination of nginx & unicorn.

[mbigras]

In the bottom picture, is the yellow box supposed to be labeled "unicorn" instead of "nginx"?

[Agis]

@oMiKeY FYI you can "Flag" this answer as "Not an answer" if you think it doesn't help you.

[omikes]

The flagging system on this website is a joke. Useless or stolen answers are no longer moderated when the person giving the answer is well respected in the community. Judging by this persons number of gold medals, flagging this answer would be pointless. Critiquing it in the comments is the last form of criticism allowed on this website, besides downvoting for which I would receive a penalty.

[Magne]

This is pure gold. Thank you!