Nginx - Allow requests without a Host header

Question

When Nginx receives a request that's missing the Host header, it rejects it with a 400 response. As it should.

Is there any way around this?

There is a piece of hardware that needs to be able to make REST calls to my Nginx web server, but this device is not sending a Host header. There is nothing I can do about this, I have no control over the inner workings of this device.

This will be the only device communicating with my web server that lacks a Host header, and it will always be connected to the same location. The server is using name-based virtual hosts.

I've tried rebuilding my server using the headers-more-nginx module since I believe it can add headers to requests before they are processed. I added the following line to the server{} block for this virtual host:

more_set_input_headers  "Host: device.myserver.com";

But requests are still being rejected with a 400.

Edit:

I forgot to mention that these devices are currently able to make requests to a lighttpd 1.4.28 web server. I'm trying to get them working on Nginx. I can't find anything special in the lighttpd config files that should be allowing this to work, it seems like lighttpd just doesn't require this header.

Edit 2:

Results from tcpdump (I X'd out the stuff that I shouldn't put online):

POST http://XXX.XXX.XXX.com/index/get-next-command HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Connection: keep-alive
Content-Length: 62

user=XXX&pass=XXX&v=0103HTTP/1.1 200 OK
X-Powered-By: PHP/5.3.10-1ubuntu3.21
Set-Cookie: PHPSESSID=XXX; path=/
Set-Cookie: username=XXX; path=/
Set-Cookie: password=XXX; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-type: text/html
Transfer-Encoding: chunked
Date: Thu, 03 Dec 2015 19:37:56 GMT
Server: lighttpd/1.4.28

46

!]HdEU
{"XXX":"XXX","XXX":"XXX","parameters":[]}
0

Solution

From here: http://nginx.org/en/docs/http/server_names.html

Miscellaneous names

There are some server names that are treated specially.

If it is required to process requests without the “Host” header field in a server block which is not the default, an empty name should be specified:

server {
    listen       80;
    server_name  example.org  www.example.org  "";
    ...
}

If no server_name is defined in a server block then nginx uses the empty name as the server name.

So adding "" to your server_name seems to do what you want.

Comments

[Nikita Kipriyanov]

That's because HTTP/1.1 request without Host header is invalid (by RFC2616, and recent rfc update hasn't changed things). Send a request to Nginx without HTTP version specification (i.e. GET / crlf crlf instead of GET / HTTP/1.1 crlf crlf ) and it will not answer with 400 (and even will not allow you to specify any request headers). Or use HTTP/1.0 (GET / HTTP/1.0 crlf ...), where Host header was optional, and it again will answer you something other than 400.

[pete]

Lighttpd might be your only option then, as nginx is following the RFC. Short of rewriting and compiling i'll bet there is no option

[Chris.B]

We're upgrading to Nginx because we now need to support WebSockets (which lighttpd doesn't). I guess at this point it's probably easier to try and get lighttpd to work with WebSockets than to get Nginx to work without a Host header. Thanks!

[Michael Hampton]

Are you 100% sure you can't do anything about the device? Like having its programmers taken out back and shot?

[Chris.B]

Ha I've thought about it. Unfortunately there are currently thousands of them already in the field, so even if he can change it I still need to support the existing devices, which can't be remote-updated

[Jaime Hablutzel]

Not precisely a clean solution but maybe you could set a reverse proxy that tolerates the missing Host header in front of nginx.

[Chris.B]

I tried that before but not while I was using the more_set_input_headers command. I'll give them a try together

[pete]

In this case it doesn't work, as Nikita pointed out in a comment above, because nginx is expecting at minimum a blank "Host: " field to match the blank field. When none is seen nginx rejects it.

[Chris.B]

That's the behavior I've been seeing. So I guess it's rejecting the request before it even gets to the point where it tries to inject the header using more_set_input_headers?

[Fredi]

Indeed, he is right, it's a protocol violation. Can you sniff a request made from the device out of curiosity? tcpdump -nnpi any -s0 -w /tmp/sniff.pcap host DEVICE_IP, then you can open it with wireshark -> left clieck a packet and select "Follow TCP Stream" and update your question in case

[Chris.B]

I did earlier, and the device is only sending three headers: Content-Length, Content-Type, and I believe Connection. I'm in a meeting right now but can get the actual tcp dump data this afternoon

[Michael Hampton]

This only works when the user agent specifically requests HTTP/1.0. When it requests HTTP/1.1 and omits the Host: header, nginx (correctly) returns 400.

[Fredi]

@Chris.B, the important part is the GET, if it includes HTTP/1.0 or not. EDIT, did a quick test, with "GET / HTTP/1.1", apache and varnish tolerate the missing Host:, nginx not. Take your time.

[Chris.B]

@Michael & Fredi Good to know, thanks. I'll check that this afternoon and update my question with that info

[Chris.B]

@Fredi OK I've posted the tcpdump results