Implementing TCP Sticky Sessions With HAProxy to Handle SSL Pass-through Traffic

Question

How can we implement session stickiness in HAProxy when SSL must terminate on the backend servers? We need the stickiness because backends cannot share sessions.

This is my original configuration:

# SSL passthrough
listen https_handler
    bind 1.2.3.4:443
    mode tcp
    balance leastconn
    stick match src
    stick-table type ip size 200k expire 30m
    server s1 1.1.1.1:443
    server s2 1.1.1.2:443

# haproxy logs (not sticking)
10.x.x.2:xxxxx [17/Dec/2014:19:29:41.396] fe BACKEND_Website/s1 37/0/1/3/41 200 8364
10.x.x.2:xxxxx [17/Dec/2014:19:29:41.456] fe BACKEND_Website/s1 36/0/1/1/39 200 9082
10.x.x.2:xxxxx [17/Dec/2014:19:29:41.456] fe BACKEND_Website/s2 35/0/1/3/39 200 2529
10.x.x.2:xxxxx [17/Dec/2014:19:29:41.545] fe BACKEND_Website/s1 35/0/0/3/38 200 1460
10.x.x.2:xxxxx [17/Dec/2014:19:29:41.501] fe BACKEND_Website/s2 36/0/1/1/109 200 376
10.x.x.2:xxxxx [17/Dec/2014:19:29:41.545] fe BACKEND_Website/s1 36/0/1/1/74 200 2298
10.x.x.2:xxxxx [17/Dec/2014:19:29:41.604] fe BACKEND_Website/s1 35/0/1/2/38 200 5542

The config below is my attempt to read the src:

This results in a 502 Bad Gateway error. I assume, it is because the traffic is already decrypted by the time it reaches the backend.

# terminate SSL at HAProxy 
listen https_handler
    bind 1.2.3.4:443 ssl crt /etc/ssl/certs/certs.pem
    mode tcp
    balance leastconn
    stick match src
    stick-table type ip size 200k expire 30m
    server s1 1.1.1.1:443
    server s2 1.1.1.2:443

Notice that I plugged the cert to the binding. This is for HAProxy to be able to read the src and setup the stick-table. (Not sure if this is correct.) And at this point, the traffic is already decrypted.

I think the problem lies when this decrypted traffic is passed to the backend servers which expects encrypted traffic...

I have seen these suggestions:

1. Terminate SSL at HAProxy 1.5 - not possible in my case. SSL need to be handled by the backend servers.
2. Use SSL Session ID to maintain stickiness. - I'm skeptical to try this out because I don't quite understand it yet. And it seems to be using a modified (?) version of haproxy.
3. Use send-proxy directive & X-Forward-Proto header. - but realized this also needs an HTTP-only backend.

Would appreciate any advice.

Solution

The easiest solution is to use balance source, but if many clients come from the same IP, it may not be very fair on your backend servers.

See http://blog.haproxy.com/2013/04/22/client-ip-persistence-or-source-ip-hash-load-balancing/ for more discussion on methods to accomplish this.

Comments

[Felix Frank]

Would it be viable to make HAproxy re-encrypt the HTTP traffic towards the backend servers?

[Ianthe the Duke of Nukem]

@felix-frank, This is what I'm not sure of if it's possible. I originally use just a pass-through setup. Now, I'm trying to implement sticky sessions with that.

[kasperd]

Keep in mind that sticky sessions should only be used as a performance improvement. If a user is moved to a different backend in the middle of a session, it is ok if they experience slower responses for the first few requests after being moved, but you need to ensure that they still do receive correct responses.

[Ianthe the Duke of Nukem]

I am aware of that. My problem is stickiness doesn't work when I use HTTPS both on front and backends (My backend server needs to handle the SSL). I'll add more details to the question.

[Felix Frank]

@Ianthe balance source cannot really not work. Unless the client is using TOR, they cannot conceivably end up on a backend that is different from their originial pick.

[Ianthe the Duke of Nukem]

your second example was my original setup but for some reason it's not sticking. I assume it's because traffic is encrypted. Therefore, I tried decrypting it (like your https_handler) but the problem with that is the backend expects encrypted traffic and gives a Bad Gateway error.

[kasperd]

@FelixFrank Some providers have deployed CGN with a pool of IP addresses. In such a setup different TCP connections from the same client may be assigned different IP addresses by the CGN.