run code after transaction commit in Django

Question

Is there any way to run some code after transaction commit in Django?

I need to send some messages to a rabbitmq server for offline processing, but the message gets to the consumer before the Django transaction is commited.

My message is sent in the post_save signal of the model. What I'm looking for is a similar mechanism, using signals or something else, that would execute code after the commit (and do nothing if the transaction fails).

I haven't found any generic way of doing it in Django. Do you have any ideas?

Solution 1

django-transaction-hooks solves this problem for Django < 1.9, and the functionality is built into Django 1.9+:

from django.db import transaction

def do_something():
    pass  # send a mail, invalidate a cache, fire off a Celery task, etc.

transaction.on_commit(do_something)

Solution 2

Hope this may help someone using Django 1.9 or later. Since 1.9 on_commit is available.

So basically you would be doing it like this:

from django.db import transaction

transaction.on_commit(
    lambda: send_msg_to_rabbitmqp(param1, param2, ...)
)

If you wish to keep post_save, you can still use on_commit:

@receiver(pre_save, sender=MyModel)
def my_handler(sender, instance, created, **kwargs):
    transaction.on_commit(
        lambda: send_msg_to_rabbitmqp(instance.id)
    )

Solution 3

I have implemented transaction signals (post_commit and post_rollback) by monkey patching django: http://gist.github.com/247844

Solution 4

One possibility would be to subclass the transaction middleware so that it sends a custom signal on commit. Your code could listen for that signal, rather than post_save.

Solution 5

Have a look at django-celery-transactions for a solution to this.

I've recently finished splitting-out and refactoring the underlying signals code code into a stand-alone app django-db-signals.

View more solutions

Comments

[Grégoire Cachet]

Thanks, I think that I will go with some middleware (I will probably add another middleware and not subclass the transaction middleware). I have a concern about signals though. Are they thread safe? If another thread throws a signal, may the current thread catch it?

[Grégoire Cachet]

I still have an issue with middlewares : if the application is running from a management command, it won't execute my callbacks.

[Carl Meyer]

I don't think your thread question really makes sense. Signals don't do anything special with regard to threads. A signal sent in one thread will call receivers in that same thread only. Built-in signal objects are module-global, however, so a signal handler registered for post_save in one thread is registered in all threads. (I think it might be possible to have signal objects in your own code that are not global, haven't looked at it carefully).

[Grégoire Cachet]

I have to work on some other parts of my app before, but I will follow this path. When it's ready, I'm going to drop an e-mail to django-developers and add a bug report with the patch.

[r3m0t]

If django-transaction-hooks requires too high a Django version, you can use django-db-signals.

[styrofoam fly]

Where should I put this code If I want the hooks to be executed every time on every transaction?

[Greg Schmit]

@styrofoamfly In the save() method of the model is where I put it. You can also register a post_save signal that calls transaction.on_commit and I think it get registered early enough to be called for that save transaction.

[Addison Klinke]

Could you elaborate on why we need the lambda syntax in order to pass parameters to the on_commit callable? Can functools.partial accomplish this in order to make it more obvious that we're calling a function with a full-definition elsewhere as opposed to a true, in-line lambda?

[Igor Kramaric]

lambda is needed solely to turn send_msg_to_rabbitmqp(instance.id) into callable