How to add X-Forwarded-for header in reverse proxy with SSL passthrough

Question

Setup as follows:

• proxy server (RHEL8 apache 2.4) in DMZ
  • contains multiple vhosts
  • each vhost acts as a reverse proxy to a web server in the LAN
  • connections from the proxy to the backend web server are secured via SSL
• backend server (RHEL8 apache 2.4) in LAN

Problem to solve Currently the apache access log of the backend server shows the IP of the proxy instead of the originating client IP. I want to ensure the client IP (who is connecting to the proxy) to be logged in the access log of the backend apache server.

Numerous howto's on the web (e.g. How to get X-Forwarded-For IP addresses in Apache Web Server) propose to use RemoteIPHeader X-Forwarded-For.

However it seems that this only works when the proxy connection to the backend uses HTTP. (discussion on linode forum.)

The proxy server has both the public and private keys of the SSL certifcate.

How can I configure the proxy server to add the X-Forwarded-for header while keeping the SSL connection to the backend?

Virtual host configuration of the proxy server (with IP 192.168.1.2):

<VirtualHost *:443>
  ServerName service.example.com

  SSLEngine on
  SSLCipherSuite AES256+EECDH:AES256+EDH
  SSLProtocol All -SSLv2 -SSLv3
  SSLHonorCipherOrder On
  SSLCompression off
  SSLCertificateFile /etc/pki/tls/certs/service_example_com.crt
  SSLCertificateKeyFile /etc/pki/tls/private/service_example_com.key
  SSLCertificateChainFile /etc/pki/tls/certs/CA.crt

  SSLProxyEngine on
  ProxyPass / https://backend-1.anubex.com/
  ProxyPassReverse / https://backend-1.anubex.com/
  RemoteIPHeader X-Forwarded-For
  RemoteIPInternalProxy 192.168.1.2

</VirtualHost>

Solution

What you show is not SSL passthrough, but SSL termination at the reverse proxy and from there another HTTPS connection to the final server. According to the documentation this will automatically add the X-Forwarded-Header to the new request to the final server, so no special configurations for this need to be done at the reverse proxy.

To actually log this header with %a at the final server (Apache too) one has to interpret this header set by the proxy accordingly. For this mod_remoteip can be used and RemoteIPHeader should be set X-Forwarded-For - as you (wrongly) did in the configuration for the reverse proxy.

Comments

[Steffen Ullrich]

This is impossible with SSL pass through - i.e. passing the original SSL traffic though. The HTTP headers are in the encrypted part then and thus non can be added or changed. But given that you have cert and key of the server, why do you want to use SSL pass through in the first place? Why not terminate SSL, add the header and then make another SSL connection to the final server? This would just be a normal SSL terminating reverse proxy, only that the upstream is accessed with a https:// URL too.

[Bram]

@Steffen-Ullrich can you elaborate a bit please? I added the virtual host of the proxy server to my question to ensure I've explained my setup correctly. I can't find proper documentation explaining the various proxy setups and I'm not sure if what I've set up is a pass through set up or not.

[Steffen Ullrich]

The configuration you've posted looks good. What is not working with this? And no, it is not SSL pass through what you have configured but you are terminating the SSL connection at the reverse proxy and create another SSL connection from the proxy - where the header should be added. Also, based on this documentation the header should be added even if not explicitly configured. And RemoteIPHeader does not set the header but just uses it.

[Bram]

Thanks for reviewing this. I am still getting the IP address of the proxy server in the access log on the backend server. (192.168.1.2 in this example) instead of the IP address of the client connecting to the proxy. I have updated the LogFormat to use %a instead of %h which should log the value of the X-forwarded-for header.

[Steffen Ullrich]

%a will only log the X-Forwarded-For if mod_remoteip is accordingly configured in the backend. It does not help to set RemoteIPHeader in the reverse proxy, it must be applied in the backend. My guess that your problem is less to set the header in the proxy but to read the header in the backend.

[Bram]

Thanks @SteffenUllrich! If you post this as an answer I'll accept it. The RemoteIPHeader and RemoteIPInternalProxy settings need to be specified in the virtual host of the backend rather than the proxy.