Nginx real client IP to TCP stream backend

16,428

Solution 1

To my knowledge, there are only two solutions: proxy_bind and proxy_protocol.

proxy_bind

As you quoted from the documentation, the work processes will need superuser privileges. Obviously. this not a best practice. Additionally, it might cause connection issues back to the remote clients.

Assuming the Nginx is running as the nginx user, run this command to give it permissions.

usermod -aG sudo nginx

proxy_protocol

This solution requires the upsteam destination (e.g. the backend app) accepts PROXY protocol.

stream {
    server {
        listen         3333;
        proxy_pass     127.0.0.1:2222;
        proxy_protocol on;
    }
}

The above solutions assume the Nginx server is the entry point to the network. If there is a edge device (e.g. load balancer), it is very likely it is changing the source IP. In this case, you will need to enable proxy protocol on the edge device and enable proxy_protocol listener in the server block. I haven't tested it, but something like this should work.

stream {
    server {
        listen            3333 proxy_protocol;
        proxy_pass        127.0.0.1:2222;
        proxy_protocol    on;
        set_real_ip_from  $proxy_protocol_addr;
    }
}

Solution 2

My setup:

  1. Nginx stream module as Tcp proxy for ssh and https
  2. Nginx http module to serve my content

My requirement: Having the real client ip inside access.log (instead of 127.0.0.1 from stream module) and also for geoblocking

My solution:

  1. Activate proxy_protocol (to add the additional information in the requests)
  2. add "proxy_procotol" in http listen directive (for the http server to accept)
  3. add extra proxy between stream module and ssh to remove proxy_protocol for ssh being able to read
  4. replace "$remote_addr" with "$proxy_protocol_addr" in access.log

For geoblocking you also have to use proxy_protocol_addr, but i'll omit describing this here to keep it short.

nginx.conf:

...
stream {

    upstream ssh {
        server 127.0.0.1:2222;
    }

    upstream https {
        server 127.0.0.1:444;
    }

    map $ssl_preread_protocol $upstream {
        default ssh;
        "TLSv1.2" https;
        "TLSv1.3" https;
        "TLSv1.1" https;
        "TLSv1.0" https;
    }

    server {
        listen 443;
        proxy_pass $upstream;
        proxy_protocol on;
        ssl_preread on;
    }

    server {
        listen 2222 proxy_protocol;
        proxy_pass 192.168.2.76:22;
    }
}

http {
    log_format  main  '$proxy_protocol_addr - $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '"$http_user_agent" "$http_x_forwarded_for"';

    ...
    server {
        listen 444 ssl proxy_protocol;
    ... 
    }
}

Solution 3

"set_real_ip_from" directive should be the address/CIDR of the edge device

stream {
    server {
        listen            3333 proxy_protocol;
        proxy_pass        127.0.0.1:2222;
        proxy_protocol    on;
        set_real_ip_from  10.0.0.0/8;
    }
}
Share:
16,428

Related videos on Youtube

J_z
Author by

J_z

Yesterday all my troubles seemed so far away. Now it looks as though they're here to stay. Oh, I believe in yesterday. Suddenly I'm not half the man I used to be. There's a shadow hanging over me. Oh, yesterday came suddenly. Why she had to go, I don't know, she wouldn't say. I said something wrong, now I long for yesterday. Yesterday love was such an easy game to play. Now I need a place to hide away. Oh, I believe in yesterday.

Updated on October 12, 2022

Comments

  • J_z
    J_z about 1 year

    I'm trying to use Nginx as proxy for my TCP daemon, to make Nginx be SSL/TLS frontend, as well as load control.

    My backend app needs real client IP, and this is a problem.

    stream {
        server {
            listen     3333;
            proxy_pass 127.0.0.1:2222;
        }
    }
    

    I found one solution in docs:

    proxy_bind $remote_addr transparent;
    

    But it's too complicated:

    "In order for this parameter to work, it is necessary to run nginx worker processes with the superuser privileges and configure kernel routing table to intercept network traffic from the proxied server."

    Is there any other way to pass $remote_addr to backend?

    I tried modify client message body using sub_filter, send http headers, etc. But all of this can work only with http context, not with stream unfortunately.

  • Mike G.
    Mike G. almost 3 years
    I'm getting "set_real_ip_from" directive is not allowed here in /etc/nginx/nginx.conf
  • Roc King
    Roc King almost 3 years
    @MikeG. install stream_realip_module
  • Mike G.
    Mike G. almost 3 years
    Can you please point me to the doc how to do it in Ubuntu 20? I struggled for many hours without any success :(
  • asmoljo
    asmoljo about 2 years
    Hi, I tried this but it is not allowed. I get an error: nginx: [emerg] "set_real_ip_from" directive is not allowed here in /etc/nginx/nginx.conf