How to enable websockets on AWS Cloudfront

Question

I have an Akka HTTP server running on AWS EC2 Autoscale cluster. This EC2 auto scale cluster has an ELB Application Load balancer in front. In addition to the ELB, we have a cloud front distribution that is set to serve static files.

We're facing an issue where all the websocket connection requests from browsers to the backend fail with HTTP 400 Expected UpgradeToWebsocket header error.

Upon further investigation, we found that the clients are able to connect directly to the load balancer but any connection request via cloudfront fail. Eventually I came across this page on AWS Cloudfront documentation which says that cloudfront strips out any 'upgrade' headers which might be the reason clients are unable to connect.

To work around this issue, I enabled all "header forwarding" option (which disables caching), but it still didn't work. Moreover, I couldn't find any option to selective disable cloudfront caching or bypass cloudfront altogether for certain URLs.

How do I workaround this issue and ensure that websockets work through cloudfront? Or is this just not supported?

Solution 1

CloudFront is not the right solution for web sockets as it is optimized for caching of static web pages, whereas web sockets are mostly dynamic. ELB on the other hand does support both HTTP web sockets (ws://) and Secure web sockets (wss://), AND it's possible to configure it to handle all the SSL hand shake. however you need to configure it with TCP settings in order to keep the HTTP/HTTPS connection open while the server is transmitting. Here's how it's done:

1. Click "Create load balancer" in the EC2 load balancers tab
2. Select "Classic Load Balancer". You need that in order to do a simple TCP
3. Define the source and destination protocols (Choose TCP for plain web sockets) :

4. If you're doing doing secure web sockets you need to choose a certificate, like this:

5. Configure health checks, add instances and press "Create". Define the CNAME and you're all set.

Note that if you select "HTTP" or "HTTPS" as the source protocol, the load balancer will at some point throw a 408 error code (timeout), since it's not designed to keep the connection open too long. That's the reason we chose TCP.

Solution 2

Update

CloudFront announced support for Websockets on 2018-11-20.

CloudFront supports WebSocket connections globally with no required additional configuration. All CloudFront distributions have built-in WebSocket protocol support, as long as the client and server also both support the protocol.

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-working-with.websockets.html

The client is responsible for re-establishing any websocket connection that is lost.

CloudFront does not currently support web sockets.

Certain headers are stripped from requests even if you try to configure CloudFront to forward them. These are indicated in the table on the page you mentioned by "CloudFront removes the header" and Caching Based on Header Values Is Supported = "No".

From the AWS Forums:

Rest assured that the right people are aware of this feature request.

— Richard@AWS (2015-06-06)

https://forums.aws.amazon.com/thread.jspa?messageID=723375

Comments

[xenoterracide]

updated but it didn't provide an answer how to do it after it was supported.

[Michael - sqlbot]

@xenoterracide there's really nothing to say, nothing to do, nothing to configure. Websockets are automatically enabled on all CloudFront distributions.