Django admin inline: select_related

Question

Using Django 1.8 on Python 3.4.1 with models:

class Product(models.Model):
    name = models.CharField(max_length=255)
    # some more fields here

    def __str__(self):
        return self.name


class PricedProduct(models.Model):
    product = models.ForeignKey(Product, related_name='prices')
    # some more fields here

    def __str__(self):
        return str(self.product)

class Coming(models.Model):
    # some unimportant fields here


class ComingProducts(models.Model):
    coming = models.ForeignKey(Coming)
    priced_product = models.ForeignKey(PricedProduct)
    # more unimportant fields

and the following admin.py:

class ComingProductsInline(ForeignKeyCacheMixin, admin.TabularInline):
    model = ComingProducts


class ComingAdmin(admin.ModelAdmin):
    inlines = [ComingProductsInline]

Of course, i have a problem with multiply queries to database: i have a query for each item in list and a query for each line. So, having 100 items i get 100 ^ 2 queries. I've solved the problem with queries for each line with Caching queryset choices for ModelChoiceField or ModelMultipleChoiceField in a Django form But i still having problem with str method. I've tried the following:

1) adding prefetch_related to ComingAdmin:

def get_queryset(self, request):
    return super(ComingAdmin, self).get_queryset(request). \
    prefetch_related('products__product')

2) adding select_related to ComingProductInline:

def get_queryset(self, request):
    return super(ComingProductsInline, self).get_queryset(request). \
    select_related('priced_product__product')

3) Defining custom form for inline and adding select_related to field queryset:

 class ComingProductsInline(ForeignKeyCacheMixin, admin.TabularInline):
     model = ComingProducts
     form = ComingProductsAdminForm

 class ComingProductsAdminForm(ModelForm):
     def __init__(self, *args, **kwargs):
              super(ComingProductsAdminForm, self).__init__(args, kwargs)
              self.fields['priced_product'].queryset = PricedProduct.objects.all(). \
              select_related('product')

     class Meta:
         model = ComingProducts
         fields = '__all__'

4) Defining a custom formset:

 class ComingProductsInline(ForeignKeyCacheMixin, admin.TabularInline):
     model = ComingProducts
     formset = MyInlineFormset

 class MyInlineFormset(BaseInlineFormSet):
     def __init__(self, data=None, files=None, instance=None,
             save_as_new=False, prefix=None, queryset=None, **kwargs):
        super(MyInlineFormset, self).__init__(data, files, instance,
                                          save_as_new, prefix, queryset, **kwargs)
        self.queryset = ComingProducts.objects.all(). \
        prefetch_related('priced_product__product')

5) Different combinations for previous 4 methods

And nothing helps: each call of str for PricedProduct makes Django to perform a query for Product table. All of these methods were mentioned on stackoverflow, but they treated ModelAdmin, and do not help with Inline. What do i miss?

Solution 1

The formset solution does work for me, but with a slightly different approach:

class MyInlineFormset(BaseInlineFormSet):
    def __init__(self, *args, **kwargs):
        super(MyInlineFormset, self).__init__(*args, **kwargs)
        self.queryset = self.queryset.prefetch_related('priced_product__product')

The BaseInlineFormSet class filters the queryset for you, and you need to take that filtered queryset and add the prefetch. With your formset implementation (the all() queryset) you get unrelated ComingProduct objects and it probably takes much too long to render. When it's the filtered queryset it renders very quickly.

Solution 2

You will find this approach very useful:

project/admin.py

from django.contrib import admin
from django.contrib.admin.options import BaseModelAdmin
from django.db.models.constants import LOOKUP_SEP


class AdminBaseWithSelectRelated(BaseModelAdmin):
    """
    Admin Base using list_select_related for get_queryset related fields
    """
    list_select_related = []

    def get_queryset(self, request):
        return super(AdminBaseWithSelectRelated, self).get_queryset(request).select_related(*self.list_select_related)

    def form_apply_select_related(self, form):
        for related_field in self.list_select_related:
            splitted = related_field.split(LOOKUP_SEP)

            if len(splitted) > 1:
                field = splitted[0]
                related = LOOKUP_SEP.join(splitted[1:])
                form.base_fields[field].queryset = form.base_fields[field].queryset.select_related(related)


class AdminInlineWithSelectRelated(admin.TabularInline, AdminBaseWithSelectRelated):
    """
    Admin Inline using list_select_related for get_queryset and get_formset related fields
    """

    def get_formset(self, request, obj=None, **kwargs):
        formset = super(AdminInlineWithSelectRelated, self).get_formset(request, obj, **kwargs)

        self.form_apply_select_related(formset.form)

        return formset


class AdminWithSelectRelated(admin.ModelAdmin, AdminBaseWithSelectRelated):
    """
    Admin using list_select_related for get_queryset and get_form related fields
    """

    def get_form(self, request, obj=None, **kwargs):
        form = super(AdminWithSelectRelated, self).get_form(request, obj, **kwargs)

        self.form_apply_select_related(form)

        return form


class FilterWithSelectRelated(admin.RelatedFieldListFilter):
    list_select_related = []

    def field_choices(self, field, request, model_admin):
        return [
            (getattr(x, field.remote_field.get_related_field().attname), str(x))
            for x in self.get_queryset(field)
        ]

    def get_queryset(self, field):
        return field.remote_field.model._default_manager.select_related(*self.list_select_related)

app/admin.py

from django.contrib import admin

from project.admin import AdminWithSelectRelated, AdminInlineWithSelectRelated, FilterWithSelectRelated
from .models import FormaPago, Comprobante, ItemServicio, ItemBazar


class ItemServicioInlineAdmin(AdminInlineWithSelectRelated):
    model = ItemServicio

    list_select_related = (
        'alumno_servicio__alumno__estudiante__profile',
        'alumno_servicio__servicio__grado',
        'comprobante__forma_pago',
    )


class ItemBazarInlineAdmin(AdminInlineWithSelectRelated):
    model = ItemBazar

    list_select_related = (
        'alumno_item__alumno__estudiante__profile',
        'alumno_item__item__anio_lectivo',
        'comprobante__forma_pago',
    )


class ComprobanteAdmin(AdminWithSelectRelated):
    list_display = ('__str__', 'total', 'estado', 'fecha_generado', 'forma_pago', 'tipo', )
    list_filter = ('estado', 'forma_pago', )

    list_select_related = ('forma_pago', )
    inlines = (ItemServicioInlineAdmin, ItemBazarInlineAdmin, )


class AlumnoFilter(FilterWithSelectRelated):
    list_select_related = ('estudiante__profile', )


class ItemServicioAdmin(AdminWithSelectRelated):
    list_display = ('nombre', 'alumno', 'monto_pagado', 'comprobante', )
    list_filter = (
        'alumno_servicio__alumno__seccion__grado',
        ('alumno_servicio__alumno', AlumnoFilter),
    )

    list_select_related = (
        'comprobante__forma_pago',
        'alumno_servicio__alumno__estudiante__profile',
        'alumno_servicio__alumno__seccion__grado',
        'alumno_servicio__servicio__grado',
    )


class ItemBazarAdmin(AdminWithSelectRelated):
    list_display = ('nombre', 'alumno', 'monto_pagado', 'comprobante', )
    list_filter = (
        'alumno_item__alumno__seccion__grado',
        ('alumno_item__alumno', AlumnoFilter),
    )

    list_select_related = (
        'comprobante__forma_pago',
        'alumno_item__alumno__estudiante__profile',
        'alumno_item__alumno__seccion__grado',
        'alumno_item__item__anio_lectivo',
    )


admin.site.register(FormaPago)
admin.site.register(Comprobante, ComprobanteAdmin)
admin.site.register(ItemServicio, ItemServicioAdmin)
admin.site.register(ItemBazar, ItemBazarAdmin)

All I have to do is define the select_related fields, and the Custom AdminWithSelectRelated, AdminInlineWithSelectRelated, and FilterWithSelectRelated make use of them for Changelists, Changeforms, and even inline Formsets.

Works like a charm.

Solution 3

Inspired from @helpse answer you can also do the following if you just want to override the queryset for a single admin inline:

class ComingProductsInline(admin.TabularInline):
    model = ComingProducts

    def get_formset(self, request, obj=None, **kwargs):
        formset = super(ComingProductsInline, self).get_formset(request, obj, **kwargs)
        queryset = formset.form.base_fields["priced_product"].queryset
        queryset = queryset.select_related("product")
        formset.form.base_fields["priced_product"].queryset = queryset
        return formset

It might be sufficient for most of the cases.

Solution 4

I'm working currently on a similar problem. What I have found is documented at this thread: Translatable Manytomany fields in admin generate many queries

One important observation I did is that my solution works only for Django 1.7x and not for 1.8. Exactly same code, with d1.7 I have order of 10^1 queries, and with new installation of d1.8 I have 10^4.

View more solutions

Comments

[Aitvaras]

Oh. this is a life saver. I had a model A, which had many inlines of model B. B had three m2m, and two fk relationships. moreover. model translation was used. Also, Grappelli with its autocomplete helped.

[noamk]

How so? It doesn't look like Django's relevant code has changed

[5parkp1ug]

@ramusus: tried formfield_for_foreignkey & its works with django 2.1

[bdoubleu]

Works with Django 2.2

[Matthijs Kooijman]

It seems this is supported by django out of the box now, so you can add list_select_related to your Admin classes without using something like AdminBaseWithSelectRelated. See docs.djangoproject.com/en/2.2/ref/contrib/admin/…

[helpse]

Hi @MatthijsKooijman. select_related was working for AdminClass since a lot of time. My solution works for Inlines and Filters.

[Matthijs Kooijman]

Ah. I was thinking that your solution was also for normal admins, since you derive from BaseAdmin. I also assumed that the built-in selectRelated support actually works for inlines as well, since I think that derives from BaseAdmin as well? Haven't tested yet, though.