Nginx ip_hash does not load-balance connections to meteor backend

Question

I cannot get nginx to load balance internal connections with ip_hash enabled. I need sticky sessions as I use meteor in the backend with sockets but all requests always hit the same backend.

The nginx access log file shows the following IP addresses:

192.168.0.20 - - [xx/xxx/2017:xx:xx:xx +xxxx] "GET /favicon.ico HTTP/1.1"  404 5 "http://xxxx.lokal/" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
192.168.0.11 - - [xx/xxx/2017:xx:xx:xx +xxxx] "GET /sockjs/602/dpkl6lfe/websocket HTTP/1.1" 101 55045 "-" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
192.168.0.208 - - [xx/xxx/2017:xx:xx:xx +xxxx] "GET /sockjs/031/cx1kml79/websocket HTTP/1.1" 101 1146677 "-" "Mozilla/5.0 (iPad; CPU OS 10_2_1 like Mac OS X) AppleWebKit/602.4.6 (KHTML, like Gecko) Version/10.0 Mobile/14D27 Safari/602.1"

Is it because they are all coming from the same 192.168.0.* subnet? If so, how can I change that behaviour?

Here is my config file:

user  www;
worker_processes  4;
error_log  /var/log/nginx/error.log;

events {
 worker_connections  1024;
}

http {
    map $http_upgrade $connection_upgrade {
      default upgrade;
      ''      close;
    }

    upstream demo {
      ip_hash;
      server 127.0.0.1:5000;
      server 127.0.0.1:5001;
    }

    include       mime.types;
    default_type  application/octet-stream;
    access_log  /var/log/nginx/access.log;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    gzip on;
    gzip_disable "msie6";

    server_tokens off; # for security-by-obscurity: stop displaying nginx version

     server {
          listen       80;
              server_name xxxx.lokal;

          location / {
            proxy_pass http://demo;
            proxy_redirect      off;
            proxy_set_header    Host              $host;
            proxy_set_header    X-Real-IP         $remote_addr;
            proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
            proxy_set_header    X-Forwarded-Proto $scheme;
            proxy_http_version 1.1;
            proxy_set_header    Upgrade           $http_upgrade; # allow websockets
            proxy_set_header    Connection        "upgrade"; 
            proxy_buffering     off;
            proxy_connect_timeout 43200000;
            proxy_read_timeout    43200000;
            proxy_send_timeout    43200000;

            if ($uri != '/') {
                expires 30d;
            }
    }
}

Solution

Yes, this happens because ip_hash uses only the first three octets of the IP address for selecting the backend node.

You can use hash $remote_addr; directive to make nginx use the complete remote IP address for hash. Downside of this is that if a node goes down, all hash mappings change, and sessions will be lost.

More information on upstream selection methods can be found at nginx upstream module documentation.

Comments

[user263367]

thanks replacing ip_hash for hash $remote_addr did the trick

[ninjaneer]

does hash $remote_addr evenly distribute clients among servers or is it premapped?

[Tero Kilkanen]

It cannot do exact even distribution by definition, since hash value is alwys the same for a particular IP address. However, the has is calculated in a way that the distribution is practically even.