Nginx serving content from the wrong "virtual host" while accessing with https

Question

I have a server running both Nginx and Apache in a proxy setup, Nginx serves the static content and Apache the dynamic content which works really well.

This setup is currently hosting two versions of the same site, lets call them production.com and staging.com.

I've just finished setting up the production.com site using SSL which also works very well, but discovered that if I were to browse to staging.com using SSL as well, I'd get served the content of production.com's web root, which obviously is wrong.

I was told to use a default handler for both SSL and non-SSL, which would eliminate this behavior, but that's where I'm having trouble.

Right now I have this configuration included in nginx.conf

default_80.conf

server {
    listen 80;
    server_name "";
    return 444;
}

default_443.conf

server {
    listen 443 default_server ssl;
    server_name "";
    return 444;
}

staging.com.conf

server {

    listen 80;
    server_name staging.com;
    access_log /var/log/nginx/staging.com.log;

    # static content folders
    location ^~ /(images|css|js) {
            root /var/www/staging.com/current;
            access_log /var/log/nginx/staging.com.static.log;
    }

    # static content files
    location ~* \.(js|css|rdf|xml|ico|txt|jpg|gif|png|jpeg)$ {
            root /var/www/staging.com/current;
            access_log /var/log/nginx/staging.com.static.log;
    }

    # proxy the rest to apache
    location / {

        proxy_pass         http://127.0.0.1:8080/;
        proxy_redirect     off;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

        client_max_body_size       10m;
        client_body_buffer_size    128k;

        proxy_connect_timeout      90;
        proxy_send_timeout         90;
        proxy_read_timeout         90;

        proxy_buffer_size          4k;
        proxy_buffers              4 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;
    }
}

production.com.conf

server {

    listen 80;
    server_name production.com;
    rewrite ^       https://$server_name$request_uri? permanent;
}

server {

    listen 443 ssl;
    server_name production.com;
    access_log /var/log/nginx/production.com.log;

    ssl_certificate /etc/httpd/conf.d/SSL/ev.crt;
    ssl_certificate_key /etc/httpd/conf.d/SSL/server.key;
    keepalive_timeout 60;

    # static content folders
    location ^~ /(images|css|js) {
            root /var/www/production.com/current;
            access_log /var/log/nginx/production.com.static.log;
    }

    # static content files
    location ~* \.(js|css|rdf|xml|ico|txt|jpg|gif|png|jpeg)$ {
            root /var/www/production.com/current;
            access_log /var/log/nginx/production.com.static.log;
    }

    # proxy the rest to apache
    location / {

        # proxy settings
        proxy_pass         http://127.0.0.1:8080/;
        proxy_redirect     off;

        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;

        client_max_body_size       10m;
        client_body_buffer_size    128k;

        proxy_connect_timeout      90;
        proxy_send_timeout         90;
        proxy_read_timeout         90;

        proxy_buffer_size          4k;
        proxy_buffers              4 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;
    }

}

This settings kills all kind of SSL access to any of the two sites, and if I remove the "default_server" directive from default_443.conf instead it works for both sites.

So the question beeing, how do I turn off SSL access (https://staging.com returns 444) for staging.com and enables it on production.com?

Best Regards Lars

Solution

First, confirm that your version of Nginx supports SNI in case you're using one of those weird distros (you should see TLS SNI support enabled at the top):

nginx -V

I have posted the setup below, here are the results on my box (/var/www/production/index.html contains PRODUCTION and /var/www/staging/index.html, STAGING)

http://192.168.56.101 connection reset (444)
https://192.168.56.101 connection reset (444)
http://staging.example.com STAGING
https://staging.example.com redirection to http
http://production.example.com redirection to https
https://production.example.com PRODUCTION

For reference, I used the stable version of nginx from the debian repositories (0.7.67), but I have a very similar setup on 1.0.something that works almost exactly the same. If you can't get it to work, let us know your exact version please.

In your case, you'll probably want to change both defaults to default_server. You might also want to make the rewrite permanent, and maybe change it to a return 301 if your nginx version allows it.

---

/etc/nginx/sites-enabled/default

server {
    listen 80 default;
    return 444;
}

server {
    listen 443 default;
    ssl on;
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    return 444;
}

/etc/nginx/sites-enabled/production

server {
    listen   80; ## listen for ipv4
    server_name production.example.com;
    rewrite ^ https://production.example.com$request_uri?;
}

server {
    listen  443;
    server_name production.example.com;
    ssl on;
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    keepalive_timeout 60;

    location / {
            proxy_pass      http://127.0.0.1:81;
            proxy_set_header        Host    $host;
            proxy_set_header        X-Real-IP       $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

/etc/nginx/sites-enabled/staging

server {
    listen  80;
    server_name staging.example.com;
    keepalive_timeout 60;

    location / {
            proxy_pass      http://127.0.0.1:81;
            proxy_set_header        Host    $host;
            proxy_set_header        X-Real-IP       $remote_addr;
            proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

server {
    listen   443; ## listen for ipv4
    server_name staging.example.com;
    ssl on;
    ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem;
    ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;
    keepalive_timeout 60;

    rewrite ^(.*) http://staging.example.com$1;
}

/etc/apache2/sites-enabled/production

<VirtualHost *:81>
    ServerAdmin webmaster@localhost
    ServerAlias production.example.com

    DocumentRoot /var/www/production
    <Directory />
            Options FollowSymLinks
            AllowOverride None
    </Directory>
    <Directory /var/www/production>
            Options Indexes FollowSymLinks MultiViews
            AllowOverride None
            Order allow,deny
            allow from all
    </Directory>

    ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
    <Directory "/usr/lib/cgi-bin">
            AllowOverride None
            Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
            Order allow,deny
            Allow from all
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

/etc/apache2/sites-enabled/staging

<VirtualHost *:81>
    ServerAdmin webmaster@localhost
    ServerAlias staging.example.com

    DocumentRoot /var/www/staging
    <Directory />
            Options FollowSymLinks
            AllowOverride None
    </Directory>
    <Directory /var/www/staging>
            Options Indexes FollowSymLinks MultiViews
            AllowOverride None
            Order allow,deny
            allow from all
    </Directory>

    ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
    <Directory "/usr/lib/cgi-bin">
            AllowOverride None
            Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
            Order allow,deny
            Allow from all
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

/etc/apache2/ports.conf

NameVirtualHost *:81
Listen 81

Comments

[gparent]

Are you testing this with Windows XP and IE? It won't work.

[Lars]

Not really, no. Windows 7 and Chrome

[gparent]

Okay, just making sure you weren't testing on a platform that probably would've never worked.

[gparent]

This is simply wrong. I cannot see the problem above, but it is definitely possible to use vhosts with SSL nowadays.

[SiXoS]

If you do SNI yes, which not all browsers are capable of. Verify my claims before you down vote sir, please.

[gparent]

And if you look at the comments, he's not using a browser from 1800. Edit: I did verify your claims by asking the OP!

[Lars]

Yeah I was just told this as well actually, but how come it still continues to serve content when the server_name is wrong? A walkthrough would be: 1) Browser sends SSL request to server 2) Server responds and negotiates SSL due to settings in production.com server block 3) Browser then sends host header which DOES NOT match server_name in production.com server block 4) Server STILL sends back a proper response That's my conundrum.

[SiXoS]

gparted: in order to use SNI in nginx both nginx and openssl must support it. Do we know that this is the case?

[gparent]

No, and you don't ask it in your question either. You just state that it is beyond his control. Do we know that he doesn't have root on his own box? Please let's get off this stupid argument and onto the question.

[gparent]

If you edit yourself to be clearer, I'll remove the downvote, I'm not an asshole. But right now you're not providing any information as to why this won't work, you don't refer to SNI at all, you don't mention anything about OpenSSL or Nginx's compile options, so I think it's a bad and misleading answer.

[Lars]

Please, read my comment and think for a while, I think the problem should be understandable.

[gparent]

I answered with more info above, but as for why nginx would serve anything when the Host doesn't match: by default nginx will serve the first server block it finds, regardless of whether it has server_name clause or not, assuming no default_server is defined in another server block.

[Lars]

That's some strange design. To be honest, it sounds truly ridiculous.

[plasmid87]

This snippet: server { listen 443 default; ssl on; ssl_certificate /etc/ssl/certs/ssl-cert-snakeoil.pem; ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key; return 444; } ... is amazing at reducing load caused by excessive spider/botting, thank you! +1

[maxmitch]

adding the default to the listen did the trick!!