rqworker timeout

Question

I am using django-rq to handle some long-running tasks on my django site. These tasks trip the 180 second timeout of the (I assume) rqworker:

JobTimeoutException: Job exceeded maximum timeout value (180 seconds).

How can I increase this timeout value? I've tried adding --timeout 360 to the rqworker command but this isn't recognized.

In my python code, my long-running job is called via

        django_rq.enqueue(
            populate_trends,
            self,
        )

and have tried

        django_rq.enqueue_call(
            func=populate_trends,
            args=(self,),
            timeout=3600,
        )

which I noticed in the rq docs but django-rq has no such method it seems.

Update

For now I forked django-rq and added a placeholder fix to increase the timeout. Probably need to work with the project to get a longer term solution. I've started an issue there to discuss.

Solution 1

This seems to be the right way to approach the problem.

queue = django_rq.get_queue('default')
queue.enqueue(populate_trends, args=(self,), timeout=500) 

If you need to pass kwargs,

queue = django_rq.get_queue('default')
queue.enqueue(populate_trends, args=(self,), kwargs={'x': 1,}, timeout=500) 

Thanks to the selwin at the django-rq project for the help.

Solution 2

An update: You can pass the timeout parameter as keyword argument to the @job decorator of django-rq. Notice that you have to pass the queue name argument first.

@job("default", timeout=600)
def long_running_task():
    ...

Solution 3

Use job_timeout:

queue.enqueue(worker_func, *args, **kwargs, job_timeout=200)

---

As it has been pointed out by Stephen Blair the timeout parameter doesn't work for newer versions of rq:

timeout argument on queue.enqueue() has been deprecated in favor of job_timeout.

https://github.com/rq/rq/blob/master/CHANGES.md#10-2019-04-06

Where job_timeout has just the same effect as timeout had.

Solution 4

For clarity for people using functions with arguments, You can pass the timeout as below

 string_to_print = "Hello World"

 queue = django_rq.get_queue('default')
 queue.enqueue(print_input, to_print=string_to_print, timeout=600)      

for a function taking arguments like below

 @job
 def print_input(to_print):
     print "The input supplied is ", to_print

View more solutions

Comments

[Stephen Blair]

If you're using RQ 1.0 (2019-04-06) or newer use job_timeout instead of timeout! RQ Changelog