skip to content

In Django REST Framework, why does a SerializerMethodField often bring back N+1 queries even after the view prefetches, and how do you fix it?

level: middleimportance: should knowfreq 46%

answer

  1. a method runs once per object
  2. a new question skips the cache
  3. filter() and count() on a relation
  4. annotate, to_attr, or Python

basics

~20 s

A SerializerMethodField calls get_<field>(obj) once per object, and any ORM query built there runs per row. A filter() on a prefetched relation ignores the prefetch cache. Fix it with annotate() or Prefetch(to_attr=...) in get_queryset(), then read the attribute.

solid answer

~30 s

`SerializerMethodField` calls `get_<field_name>(obj)` on the serializer for every object rendered, and DRF cannot see what that method does. If it builds a query — `obj.tags.filter(featured=True)`, `obj.comments.count()` on an unprefetched relation, `Comment.objects.filter(article=obj).exists()` — that query runs once per row. A prefetch does not save you when the method asks a *different* question: `filter()` on a prefetched manager is a new queryset and hits the database. Move the work into the view's `get_queryset()`: `annotate(comment_count=Count('comments'))` and read `obj.comment_count`; `Prefetch('tags', queryset=Tag.objects.filter(featured=True), to_attr='featured_tags')` and read the list; or iterate `obj.tags.all()` and filter in Python. Often the method field disappears: an `IntegerField(read_only=True)` renders the annotation.

code

python · 32 lines
python
from django.db.models import Count, Prefetch
from rest_framework import serializers, viewsets

from blog.models import Article, Tag


class ArticleSerializer(serializers.ModelSerializer):
    comment_count = serializers.IntegerField(read_only=True)  # from annotate()
    featured_tags = serializers.SerializerMethodField()

    class Meta:
        model = Article
        fields = ["id", "title", "comment_count", "featured_tags"]

    def get_featured_tags(self, obj):
        # reads the list Prefetch(to_attr=...) stored: no query
        return [tag.name for tag in obj.featured_tag_list]


class ArticleViewSet(viewsets.ReadOnlyModelViewSet):
    serializer_class = ArticleSerializer

    def get_queryset(self):
        return Article.objects.annotate(
            comment_count=Count("comments")
        ).prefetch_related(
            Prefetch(
                "tags",
                queryset=Tag.objects.filter(featured=True),
                to_attr="featured_tag_list",
            )
        )

go deeper

for a junior

Know that a SerializerMethodField's get_ method runs once for every object, so any database call inside it repeats per row.

for a middle

Explain why filter() or order_by() on a prefetched relation ignores the cache, why count() does not, and how annotate() and Prefetch(to_attr=...) move the work into get_queryset().

for a senior

Review method fields as query code: push aggregates and per-user flags into annotations, and keep a query-count test on list endpoints so a new method field cannot slip in an N+1.

for a principal

Set a team norm that computed API fields come from the queryset, not from serializer methods, so query cost stays visible where the queryset is built.

## What `SerializerMethodField` does In Django REST Framework (DRF), `SerializerMethodField` is a read-only field whose value comes from a method on the serializer. When it is bound, it defaults `method_name` to `get_<field_name>`; at render time its `to_representation()` looks that method up on the parent serializer and calls it with the object being serialized. For a list of 50 articles, `get_comment_count(article)` runs 50 times. That is fine for pure Python. It is a trap when the method touches the ORM, because every query inside it becomes **one query per row**, and nothing in the serializer declaration tells the view's author that a loading plan is needed. ## Why a prefetch in the view may not help The view can prefetch relations (`prefetch_related("tags")`), and the related manager then serves `obj.tags.all()` from the **prefetch cache**. But the cache answers only that exact question. The Django docs warn that chained methods which imply a different database query ignore previously cached results. So: | Inside `get_<field>(obj)` | With `prefetch_related("tags")` | Queries per row | |---|---|---| | `[t.name for t in obj.tags.all()]` | served from the cache | 0 | | `obj.tags.count()` | cached queryset, `len()` used | 0 | | `obj.tags.filter(featured=True)` | new queryset, cache ignored | 1 | | `obj.tags.order_by("name")` | new queryset, cache ignored | 1 | | `obj.comments.count()` (comments not prefetched) | no cache | 1 | | `Comment.objects.filter(article=obj).exists()` | unrelated query | 1 | The `count()` row surprises people in both directions: on a prefetched relation, `QuerySet.count()` returns the length of the cached results; without a prefetch it runs `SELECT COUNT(*)` for every row. ## The fixes, in order of preference 1. **Aggregate in SQL with `annotate()`.** `Article.objects.annotate(comment_count=Count("comments"))` computes the number in the main query. Render it with `comment_count = serializers.IntegerField(read_only=True)`, no method needed. 2. **Prefetch the filtered set with `Prefetch(..., to_attr=...)`.** `Prefetch("tags", queryset=Tag.objects.filter(featured=True), to_attr="featured_tags")` runs one query for all rows and stores a plain list on each article. The method (or a nested serializer with `source="featured_tags"`) reads that list. 3. **Filter in Python over the prefetched set.** `[t for t in obj.tags.all() if t.featured]` — fine for small relations. 4. **Move the logic to the model or queryset** so the view that builds the queryset owns it, and the serializer only reads a value. ## Other costs hiding in method fields - **Serializers built inside the method.** `return AuthorSerializer(obj.author).data` creates a new serializer, with freshly copied fields, for every row, and drops the parent's context unless you pass `context=self.context`. Declare a nested field instead. - **Chained attribute access.** `obj.author.team.name` loads author and team per row unless `select_related("author__team")` is in the queryset. - **Request-dependent flags.** "Has the current user liked this article?" is a per-row `exists()` unless you annotate it with an `Exists()` subquery in `get_queryset()`, where `self.request.user` is available. ## A worked count Take a page of 50 articles whose serializer has `get_comment_count()` calling `obj.comments.count()` and `get_featured_tags()` calling `obj.tags.filter(featured=True)`, with `prefetch_related("tags")` already in the view: 1. 1 query for the page of articles and 1 for the tag prefetch; 2. 50 `COUNT(*)` queries for comments; 3. 50 tag queries, because `filter()` bypasses the prefetch that is already paid for. That is 102 queries, and the tag prefetch is wasted. With `annotate(comment_count=Count("comments"))` and `Prefetch("tags", queryset=Tag.objects.filter(featured=True), to_attr="featured_tag_list")` it becomes 2 queries — the count moves into the article query and the filtered tags arrive in one batch. ## How to spot it in review - Read every `get_<field>` method for `.filter(`, `.exclude(`, `.count(`, `.exists(`, `.objects.` and dotted relation access. - Treat adding a `SerializerMethodField` as a change to the view's queryset, and ask for a query-count check on the list endpoint.

  • Why does obj.tags.count() not query when tags were prefetched, while obj.tags.filter(featured=True).count() does?
    With a prefetch, the related manager returns the cached queryset, and `QuerySet.count()` returns the length of an already-populated result cache. `filter()` builds a new queryset with no cache, so its `count()` runs `SELECT COUNT(*)` for that row.
  • How would you render 'has the current user bookmarked this article' without a query per row?
    Annotate it in `get_queryset()`, where the request is available: `annotate(is_bookmarked=Exists(Bookmark.objects.filter(article=OuterRef('pk'), user=self.request.user)))`, then render it with a `BooleanField(read_only=True)`. For anonymous users, skip the annotation or annotate a constant `False` value instead.

A clerk is handed a folder of printouts for each order every morning (the prefetch). As long as he reads the printout, he never calls the warehouse. The moment he needs only the fragile items, he phones the warehouse for each order, because the printout does not answer that exact question. Printing a fragile-items sheet in the morning (Prefetch with to_attr) stops the calls.

saying these in an interview costs you the question

  • Once the view calls prefetch_related('tags'), every query on obj.tags is free.
  • DRF batches the calls made inside SerializerMethodField methods.
  • obj.tags.count() always runs a COUNT query, even when tags are prefetched.
  • Returning AuthorSerializer(obj.author).data from a method field costs nothing extra.
  • SerializerMethodField is evaluated once per page, not once per object.