Stop the thread until the celery task finishes

Question

I have a django webserver, and a form in which the user enters information. Everytime the form information changes I update the model in my database, and at a certain point when something validates I will create a long running task in celery to get my results even before the user clicked next.

I am using Django Celery with RabbitMQ as broker, and my question is what is the most appropriate way of IN CASE the task is still not finished to just lock the response thread in django until the task is state.SUCCESSFUL I tried using the AsyncResult.get method for that, but it just locks the thread for a real long time and then gives me the result. IE It's not instant, does anyone have an idea how to solve this?

Solution 1

You can just wait until the result is ready().

from time import sleep
result = some_task.apply_async(args=myargs)
while not result.ready():
    sleep(0.5)
result_output = result.get()

It appears there is also a wait(), so you could just use that. The following should is basically doing the same thing as the code above.

result = some_task.apply_async(args=myargs)
result_output = result.wait(timeout=None, interval=0.5)

Solution 2

one way to accomplish that would be to have the results waiting in redis, and get them using a blocking pop operation using some unique value like session id, note its timeout capability.

Comments

[Bojan Jovanovic]

is it necessary to have a sleep loop ?

[Bojan Jovanovic]

This seems like a nice solution, but it would require to add redis just for the sake of pop-ing tasks, it could be a overkill. But i'll take it into consideration

[Bojan Jovanovic]

perhaps if i've already had redis as a broker that would be a neater option

[user2471801]

sleep provides a wait interval, you don't want to be pounding on the ready() check every available cycle.

[Bojan Jovanovic]

Ofcourse, what i wanted to ask is if there is a other solution thats not relying on the 'is it done yet' method

[Bojan Jovanovic]

But given the fact that they implemented interval which does just that i wreckin that is it done yet is the only solution

[user2471801]

hmmm... so you're looking for something like a callback? You could perhaps give the task a callback to the task... or maybe chain tasks. docs.celeryproject.org/en/latest/userguide/canvas.html#chain‌​s

[Bojan Jovanovic]

Yeah there is that solution, but callbacks won't halt my response thread. Therefore I will use your solution as a answer to my question, because there is no alternative way of solving this other than "is it ready yet" way.

[Obeyed]

newer versions of celery also have a simpler delay() function, so if you don't want to add extra options when calling the task, there is no need for the apply_async(). Simply some_task.delay(myargs). source: docs.celeryproject.org/en/latest/userguide/calling.html#exam‌​ple

[sattva_venu]

wait and get points to same function object so both will call same function. Celery get() method is blocking method.