Celery, run task once at a specified time

Question

How can I run a celery task at a given time, but only once?

I read the documentation and couldn't find any example of this.

Solution 1

If you insist on using Celery

To run a task at a specified time, in Celery you would normally use a periodic task, which conventionally is a recurring task.

However, you may create a periodic task with a very specific schedule and condition that happens only once so effectively it runs only once.

Unfortunately we can only specify so much, e.g. we can specify hour, minute, day_of_month and month_of_year but we can't specify year

However with that, your task would run at most 1 time for per year, so below are some workarounds:

Unschedule it after it is ran

It should be relatively easy to unschedule it once it is ran (you have 1 year to do so!)

Use a "DONE" flag when the task completes

With a flag written somewhere (disk or DB), you can first check if the task has run before or not, i.e. if done: exit

Exit if not proper year or you want to be safe, just add code into the task that checks the year, e.g. if year != 2017: exit.

Simple cron/Os level scheduler works too

You may also skip Celery altogether and use some OS level facility like cron for UNIX-like systems, more on that here.

The general idea remains the same.

Solution 2

You can use parameter eta when calling the task. Example:

from datetime import datetime, timedelta

@app.task()
def hello(self):
    return 'hello world'


tomorrow = datetime.utcnow() + timedelta(days=1)
hello.apply_async(eta=tomorrow)

Documentation: http://docs.celeryproject.org/en/latest/userguide/calling.html#eta-and-countdown

Alternatively, when you want to call hello multiple times and to be sure it is executed only one at the time, you can use locking - more about it in the documentation: http://docs.celeryproject.org/en/latest/tutorials/task-cookbook.html#ensuring-a-task-is-only-executed-one-at-a-time

Solution 3

There is a boolean field called one_off in PeriodicTask. Set it to true for running celery tasks only once at the given time.

Example(following code is the dynamic task creation part in views.py):

def test():
    schedule, created = CrontabSchedule.objects.get_or_create(hour=20, minute=6, day_of_month=1, month_of_year=9)
    task = PeriodicTask.objects.create(crontab = schedule, name = 'somename', task = 'core.tasks.methodnameintasks', one_off=True)

However, the task remains enabled until the celery beat is updated. A new task creation or update in the existing tasks will solve this problem.

Comments

[richardabced]

Ok i'll do: periodic task runs every 1min, task looks up on redis for if it has been ran, if not, executes task over celery