Sticky sessions with Nginx proxy

Question

I have an application that's running on two different AWS instances and I'd like to enable "sticky" or "persistent" sessions based on IP so that I can take advantage of web socket technologies in a particular way.

I have two different setups that both involve using ip_hash to enable these sticky sessions.

In the first setup, the app processes are running on the same instance as the Nginx config. This is working, the sessions are persistent as expected.

upstream my_app {
    ip_hash;
    # local servers
    server 127.0.0.1:3001 weight=100 max_fails=5 fail_timeout=300;
    server 127.0.0.1:3002 weight=100 max_fails=5 fail_timeout=300;
    keepalive 8;
}

In the second setup, I'm pointing to external instances and trying to achieve the same effect. This set up is not working. In other words, the sessions are still being load balanced.

upstream my_app {
    ip_hash;
    # external servers
    server 111.11.11.11:3001 weight=100 max_fails=5 fail_timeout=300;
    server 222.22.22.22:3002 weight=100 max_fails=5 fail_timeout=300;
    keepalive 8;
}

Am I using ip_hash correctly? How can I enable "sticky" ip-based sessions for external servers?

Solution 1

My server was behind AWS load balancing, so I needed to pass the correct headers to upstream so it would always reflect the client IP. The following configuration fixed my issue (see the commented line):

upstream my_app {
    ip_hash;
    server 111.11.11.11:3001 weight=100 max_fails=5 fail_timeout=300;
    server 222.22.22.22:3002 weight=100 max_fails=5 fail_timeout=300;
    keepalive 8;
}

server {
    server_name my-app.com;

    location / {
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";

        proxy_set_header X-Real_IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;

        # This is necessary to pass the correct IP to be hashed
        real_ip_header X-Real-IP;

        proxy_pass http://my_app/;
        proxy_redirect off;
    }
}

Solution 2

According to Nginx documentation Sticky session support is only available for their expensive Plus version. I've been researching alternatives and the closer I've been is this old fork that is not compatible with Nginx 1.5+ https://github.com/lusis/nginx-sticky-module

I have also tried to build a LUA module but there are no API hooks for peer selection, just for enumeration and blocking.

Nginx Plus Load Balancing

Update

I've found another great module, see https://bitbucket.org/nginx-goodies/nginx-sticky-module-ng/src

Comments

[James Shewey]

Are you able to link to this documentation?