Programmatically get the number of jobs in a Resque queue

Question

I am interested in setting up a monitoring service that will page me whenever there are too many jobs in the Resque queue (I have about 6 queues, I'll have different numbers for each queue). I also want to setup a very similar monitoring service that will alert me when I exceed a certain amount of failed jobs in my queue.

My question is, there is a lot of keys and confusion that I see affiliated with Resque on my redis server. I don't necessarily see a straight forward way to get a count of jobs per queue or the number of failed jobs. Is there currently a trivial way to grab this data from redis?

Solution 1

yes it's quite easy, given you're using the Resque gem:

require 'resque'

Resque.info 

will return a hash

e.g/ =>

{
      :pending => 54338,
      :processed => 12772,
      :queues => 2,
      :workers => 0,
      :working => 0,
      :failed => 8761,
      :servers => [
      [0] "redis://192.168.1.10:6379/0"
    ],
    :environment => "development"
}

So to get the failed job count, simply use:

Resque.info[:failed]

which would give => 8761 #in my example

To get the queues use:

Resque.queues

this returns a array

e.g./ =>

[
    [0] "superQ",
    [1] "anotherQ"
]

You may then find the number of jobs per queue:

Resque.size(queue_name)

e.g/ Resque.size("superQ") or Resque.size(Resque.queues[0]) .....

Solution 2

Here is a bash script which will monitor the total number of jobs queued and the number of failed jobs.

while :
do 
  let sum=0
  let errors=$(redis-cli llen resque:failed)
  for s in $(redis-cli keys resque:queue:*)
  do 
    let sum=$sum+$(redis-cli llen $s)
  done
  echo $sum jobs queued, with $errors errors
  sleep 1 # sleep 1 second, probably want to increase this
done

This is for Resque 1.X, 2.0 might have different key names.

Solution 3

There is also a method Resque.queue_sizes That returns a hash of the queue name and size

Resque.queue_sizes => {"default"=>0, "slow"=>0}