ActionCable - Failed to upgrade to WebSocket in production

Question

ActionCable doesn't work in production. Works well in development, but not in production.

Running Nginx with Puma on Ubuntu 14.04. I have checked that redis-server is up and running.

Rails -v 5.0.0.1

production.log:

INFO -- : Started GET "/cable/"[non-WebSocket] for 178.213.184.193 at 2016-11-25 14:55:39 +0100
ERROR -- : Failed to upgrade to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: close, HTTP_UPGRADE: )
INFO -- : Finished "/cable/"[non-WebSocket] for 178.213.184.193 at 2016-11-25 14:55:39 +0100

Request from client:

GET ws://mityakoval.com/cable HTTP/1.1
Host: mityakoval.com
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://mityakoval.com
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36
Accept-Encoding: gzip, deflate, sdch
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4,uk;q=0.2,nb;q=0.2
Cookie: _vaktdagboka_session=******
Sec-WebSocket-Key: *******
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sec-WebSocket-Protocol: actioncable-v1-json, actioncable-unsupported

Response:

HTTP/1.1 404 Not Found
Server: nginx/1.4.6 (Ubuntu)
Date: Fri, 25 Nov 2016 13:52:21 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: no-cache
X-Request-Id: d6374238-69ef-476e-8fc5-e2f8bbb663de
X-Runtime: 0.002500

nginx.conf:

upstream puma {
  server unix:///home/mityakoval/apps/vaktdagboka/shared/tmp/sockets/vaktdagboka-puma.sock;
}

server {
  listen 80 default_server deferred;
  # server_name example.com;

  root /home/mityakoval/apps/vaktdagboka/current/public;
  access_log /home/mityakoval/apps/vaktdagboka/current/log/nginx.access.log;
  error_log /home/mityakoval/apps/vaktdagboka/current/log/nginx.error.log info;

  location ^~ /assets/ {
    gzip_static on;
    expires max;
    add_header Cache-Control public;
  }

  try_files $uri/index.html $uri @puma;
  location @puma {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;

    proxy_pass http://puma;
  }

  location /cable {
    proxy_pass http://puma;
    proxy_http_version 1.1;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
  }

  error_page 500 502 503 504 /500.html;
  client_max_body_size 10M;
  keepalive_timeout 10;
}

cable.yml:

redis: &redis
  adapter: redis
  url: redis://127.0.0.1:6379

production: *redis

development:
  adapter: async

test:
  adapter: async

in production.rb:

config.action_cable.allowed_request_origins = ["http://mityakoval.com"]

in routes.rb:

mount ActionCable.server, at: '/cable'

UPDATE:

Don't forget to restart nginx :) That was the problem for me.

Solution 1

You should change the value of proxy_pass property from http://puma to http://puma/cable.

Therefore, the correct location section for the /cable will be:

location /cable {
  proxy_pass http://puma/cable;
  proxy_http_version 1.1;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $http_host;
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "Upgrade";
}

Solution 2

While other posts have correctly posted solutions I thought I'd post a bit more info on how to identify where the problem is/where to fix it for other nginx noobs.

You will know your nginx config needs the proxy_set_header Upgrade at the path you're mounting action cable if the rails error contains HTTP_UPGRADE:. (Meaning nothing is passed to HTTP_UPGRADE). After fixing the problem my logs are showing HTTP_UPGRADE: websocket

• Gotchya 1: As mentioned by the op, make sure you restart nginx after making a change(I was incorrectly doing this).

• Gotchya 2: Also look for include statements in the nginx config as your config could be split across multiple files. The location /cable { section should be inside of server { which in my case was missing because it was in a different config file from an includes statement which I didn't notice for a while.

• Similar error but different problem: Your rails logs will contain an additional error in the logs right before the one the OP mentioned saying the origin is not allowed, that is when your rails config needs to be updated as another answer mentions updating config.action_cable.allowed_request_origins.

The logging is subject to change with rails but hopefully this helps clarify where the problem and a few gotchya's I encountered as someone who knows nothing about nginx.

Solution 3

Super late to this conversation, however, for anyone who is facing the same error message using Rails5, Action Cable, etc. & DEVISE you simply solve it like suggested here. It all comes down to the web socket server not having a session, hence the error message.

app/channels/application_cable/connection.rb

module ApplicationCable
  class Connection < ActionCable::Connection::Base
    identified_by :current_user

    def connect
      self.current_user = find_verified_user
      logger.add_tags 'ActionCable', current_user.name
    end

    protected
      def find_verified_user
        verified_user = User.find_by(id: cookies.signed['user.id'])
        if verified_user && cookies.signed['user.expires_at'] > Time.now
          verified_user
        else
          reject_unauthorized_connection
        end
      end
  end
end

app/config/initializers/warden_hooks.rb

Warden::Manager.after_set_user do |user,auth,opts|
  scope = opts[:scope]
  auth.cookies.signed["#{scope}.id"] = user.id
  auth.cookies.signed["#{scope}.expires_at"] = 30.minutes.from_now
end
Warden::Manager.before_logout do |user, auth, opts|
  scope = opts[:scope]
  auth.cookies.signed["#{scope}.id"] = nil
  auth.cookies.signed["#{scope}.expires_at"] = nil
end

Solution was developed by Greg Molnar

Solution 4

The resolution that needs a NGINX configuration changes to accept this action cable request.

location / {
  proxy_set_header Upgrade $http_upgrade;
  proxy_set_header Connection "upgrade";
}

Add the above lines to your location block in the nginx site configuration, then restart nginx.

Solution 5

My solution was to add these lines to my production.rb file:

  config.action_cable.url = 'ws://your_site.com/your_action_cable'
  config.action_cable.allowed_request_origins = [ 'http://your_site.com' ]

View more solutions

Comments

[Developer]

did you found any solutions?

[mityakoval]

@RameshKumarThiyagarajan did you restart nginx?

[Dave Collins]

Just a heads up to others: If your nginx is setup as a multi-site server, your configuration file will probably be /etc/nginx/sites-enabled/sitename instead of /etc/nginx/sites-available/default

[Krupa Suthar]

How did you resolve this?

[mityakoval]

@KrupaSuthar sorry for the late response. In my case I only had to restart nginx.

[mityakoval]

Thank you for your answer. Where do I get this key for redis?

[mityakoval]

Ah, okay, I got it. I don't use RedisToGo. Redis server is running on my production server. And I guess it should be ws://... instead of wss://... since I'm not using SSL.

[desheikh]

You can just use env['warden'].user to retrieve the user object :)

[Georg Keferböck]

True that. @desheikh

[Micah]

The cookie doesn't seem to set when the client is Safari. Any ideas?

[opensource-developer]

@Micah, i am facing a similar issue on safari, did you manage to find more information about it?

[Иван Бишевац]

That was exactly my problem too.

[Shoaib]

Thanks in my case there was many server { block, I have placed it one which is listing at port 443 and it start working.