Handling multiple parallel HTTP requests in Node.js

Question

I know that Node is non-blocking, but I just realized that the default behaviour of http.listen(8000) means that all HTTP requests are handled one-at-a-time. I know I shouldn't have been surprised at this (it's how ports work), but it does make me seriously wonder how to write my code so that I can handle multiple, parallel HTTP requests.

So what's the best way to write a server so that it doesn't hog port 80 and long-running responses don't result in long request queues?

To illustrate the problem, try running the code below and loading it up in two browser tabs at the same time.

var http = require('http');
http.createServer(function (req, res) {
    res.setHeader('Content-Type', 'text/html; charset=utf-8');
    res.write("<p>" + new Date().toString() + ": starting response");
    setTimeout(function () {
        res.write("<p>" + new Date().toString() + ": completing response and closing connection</p>");
        res.end();
    }, 4000);
}).listen(8080);

Solution 1

You are misunderstanding how node works. The above code can accept TCP connections from hundreds or thousands of clients, read the HTTP requests, and then wait the 4000 ms timeout you have baked in there, and then send the responses. Each client will get a response in about 4000 + a small number of milliseconds. During that setTimeout (and during any I/O operation) node can continue processing. This includes accepting additional TCP connections. I tested your code and the browsers each get a response in 4s. The second one does NOT take 8s, if that is how you think it works.

I ran curl -s localhost:8080 in 4 tabs as quickly as I can via the keyboard and the seconds in the timestamps are:

1. 54 to 58
2. 54 to 58
3. 55 to 59
4. 56 to 00

There's no issue here, although I can understand how you might think there is one. Node would be totally broken if it worked as your post suggested.

Here's another way to verify:

for i in 1 2 3 4 5 6 7 8 9 10; do curl -s localhost:8080 &;done                                                                                                                                                                       

Solution 2

Your code can accept multiple connections because the job is done in callback function of the setTimeout call.

But if you instead of setTimeout do a heavy job... then it is true that node.js will not accept other multiple connections! SetTimeout accidentally frees the process so the node.js can accept other jobs and you code is executed in other "thread".

I don't know which is the correct way to implement this. But this is how it seems to work.

Solution 3

Browser blocks the other same requests. If you call it from different browsers then this will work parallelly.

Comments

[Paul]

blog.mixu.net/2011/02/01/understanding-the-node-js-event-loo‌​p

[Andrew]

You are correct. It's Chrome that's doing it. For me at least, the second request won't be issued until the first one is returned.

[Peter Lyons]

Even chrome should normally allow up to 6 concurrent connections per origin.

[Andrew]

I know. That's why I was surprised and assumed it was Node.

[edi9999]

Do you know why Chrome couldn't send the 2 requests at the same time ?

[Peter Lyons]

Frankly I don't believe the OP's claim that chrome is serializing the request as per my above comment. The full interactions between a browser and a server are complex. But without code to repetitive the issue i cant comment further.

[harshes53]

@PeterLyons: Please bare with my stupid question. How did the server handles multiple users logged in their frontend apps ?? I mean how does it instantiates per user? Do you have any article/example I can look and understand??