Resque is not picking up Redis configuration settings

Question

I am having unexpected and significant problems trying to get a Rails app, running under Unicorn, to connect to a password-protected Redis server.

Using bundle exec rails c production on the command line, I can issue commands through Resque.redis. However, it seems that my configuration is being lost when it's forked under Unicorn.

Using a non-password-protected Redis server Just Works. However, I intend to run workers on other servers than where the Redis server lives, so I need this to be password protected.

I have also had success in using a password protected (using the same technique) but using Passenger rather than Unicorn.

I have the following setup:

# config/resque.yml

development: localhost:6379
test: localhost:6379
production: redis://user:[email protected]:6379

.

# config/initializers/redis.rb

rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'development'

$resque_config = YAML.load_file(rails_root + '/config/resque.yml')
uri = URI.parse($resque_config[rails_env])
Resque.redis = Redis.new(host: uri.host, port: uri.port, password: uri.password)

.

# unicorn.rb bootup file

preload_app true

before_fork do |server, worker|
  Redis.current.quit
end

after_fork do |server, worker|
  Redis.current.quit
end

.

Solution 1

UPDATED Totally different idea based on @lmarlow's comment to a resque issue.

I bet it breaks wherever you have Redis ~>3 (I mean the ruby client version, not the server version).

As of this writing, Resque needs Redis ~>2 but doesn't specify that in its gemspec. Therefore you have to help it out by adding this to your Gemfile:

gem 'redis', '~>2' # until a new version of resque comes out
gem 'resque'

Also, make sure that bundler is being used everywhere. Otherwise, if your system has a new version of the Redis gem, it will get used and Resque will fail like before.

Finally, a cosmetic note... you could simplify the config to:

# config/initializers/redis.rb
$resque_redis_url = uris_per_environment[rails_env] # note no URI.parse
Resque.redis = $resque_redis_url

and then

# unicorn.rb bootup file
after_fork do |server, worker|
  Resque.redis = $resque_redis_url
end

Solution 2

Ok, for the sake of other people who might be googling this problem, I've solved this for myself at least

Basic problem is calling Redis.new other places in the code ,e.g. in your geocoder setup or unicorn config file.

just make sure that every time you call initialize Redis you pass in the appropriate values e.g. something like

REDIS = Redis.connect(:url =>  ENV['REDISTOGO_URL'])

everywhere and you should never have

Redis.new 

as it will default to localhost and the default port

Solution 3

What worked for me was the unicorn config here: https://stackoverflow.com/a/14636024/18706

before_fork do |server, worker|
  if defined?(Resque)
    Resque.redis.quit
    Rails.logger.info("Disconnected from Redis")
  end
end

after_fork do |server, worker|
  if defined?(Resque)
    Resque.redis = REDIS_WORKER
    Rails.logger.info("Connected to Redis")
  end
end

Solution 4

This was helpful to me:

source: https://github.com/redis/redis-rb/blob/master/examples/unicorn/unicorn.rb

require "redis"

worker_processes 3

# If you set the connection to Redis *before* forking,
# you will cause forks to share a file descriptor.
#
# This causes a concurrency problem by which one fork
# can read or write to the socket while others are
# performing other operations.
#
# Most likely you'll be getting ProtocolError exceptions
# mentioning a wrong initial byte in the reply.
#
# Thus we need to connect to Redis after forking the
# worker processes.

after_fork do |server, worker|
  Redis.current.quit
end

View more solutions