skip to content

When do you pick collections.defaultdict over dict.setdefault?

level: middleimportance: should knowfreq 45%

answer

  1. A call site versus the whole object
  2. Arguments are evaluated before the call
  3. Lazy factory versus eager default value
  4. You cannot retrofit an existing dict
  5. Convert with dict() at the boundary

basics

~20 s

Pick defaultdict when every missing key deserves the same freshly built default and the mapping is used that way throughout. Pick dict.setdefault for occasional insertion into an ordinary dict, or when a stray missing key elsewhere must still raise KeyError.

solid answer

~50 s

Two mechanical differences drive the choice. First, **evaluation timing**: `d.setdefault(k, [])` builds the default argument on *every* call, hit or miss, because Python evaluates arguments before the call — wasteful when the default is expensive and wrong when building it has side effects. `defaultdict` calls `default_factory` only on a miss. Second, **scope of the change**: `defaultdict` alters the semantics of every subscript on that mapping for its whole lifetime, so a typo'd key silently yields an empty default instead of raising; `setdefault` leaves the container an ordinary `dict`, so lookups elsewhere still raise `KeyError`. There is also a practical constraint — `defaultdict` must be chosen when the mapping is created, so a function handed a plain `dict` can only use `setdefault`. If the values are integer tallies you also want to rank or combine, neither is the answer: that is `collections.Counter`'s job.

code

python · 14 lines
python
from collections import defaultdict

def new_batch():
    print("factory called")
    return []

lazy = defaultdict(new_batch)
lazy["run-1"].append(4.1)
lazy["run-1"].append(3.8)                  # factory not called again

plain = {}
plain.setdefault("run-1", new_batch()).append(4.1)
plain.setdefault("run-1", new_batch()).append(3.8)   # argument built anyway
print(plain)

go deeper

for a junior

Know that both remove the same boilerplate and that setdefault works on any dict you are handed while defaultdict must be chosen when the mapping is built. Being able to write either idiom correctly is enough here.

for a middle

Explain that setdefault's second argument is evaluated on every call because arguments are evaluated before the call, while the factory runs only on a miss. Note that defaultdict changes the object's semantics, not just one call site.

for a senior

Show that you scope the leniency: accumulate permissively, then freeze the factory or hand out a plain dict so later code cannot silently manufacture entries. Be able to say when Counter or a plain dict beats both.

for a principal

Set the house convention for accumulators and enforce it at module boundaries, so the question of which flavour a caller holds never arises and no mapping with surprising read semantics escapes into shared code.

## The same idiom, two shapes Both exist to remove the `if k not in d: d[k] = []` preamble, and on a first read the two lines look interchangeable: ```python groups[k].append(v) # groups is a defaultdict(list) plain.setdefault(k, []).append(v) # plain is an ordinary dict ``` They are not interchangeable, and the differences are worth being precise about in an interview. ## Difference 1: when the default is built `setdefault` is an ordinary method call, so Python evaluates `[]` **before** calling it — every single iteration, whether or not the key is present. On the hot path of a loop over a 6,800-row batch that is 6,800 empty lists allocated and immediately discarded on the hits. With `[]` the cost is small and the garbage collector absorbs it; with a heavier default it is not small, and if the default is produced by a function with side effects — opening a file, taking a sequence number, logging — the side effect fires on hits too, which is a correctness bug rather than a performance one. `defaultdict` inverts this: the factory is a callable stored on the mapping and invoked only from `__missing__`, so exactly one call per key that was actually absent. ## Difference 2: how far the change reaches `setdefault` is a local decision at one call site. The mapping is still a plain `dict`, so every other lookup in the program keeps ordinary semantics: a mistyped key raises `KeyError` and the mistake surfaces immediately. `defaultdict` is a decision about the object. From construction to disposal, *every* subscript on it fills in missing keys — including subscripts in code written months later by someone who did not know which flavour they were handed. That is exactly what you want in an accumulator and exactly what you do not want in a lookup table, where a missing key means somebody asked for something that does not exist. The useful middle ground is scoping the leniency in time rather than in space: accumulate into a `defaultdict`, then either assign `d.default_factory = None` or hand out `dict(d)` when the loading phase ends, so the permissive behaviour never outlives the phase that needed it. ## Difference 3: who gets to choose `defaultdict` has to be chosen at construction. A function that receives an already-built plain `dict` cannot retrofit the behaviour in place; the closest it can do is copy — `defaultdict(list, incoming)` builds a new mapping with the items and a factory attached — which costs a copy and gives back an object the caller does not share. `setdefault` works on whatever mapping arrives, which makes it the pragmatic choice in helper functions and in code operating on a caller's data structure. ## Difference 4: what the result *is* A `defaultdict` keeps its factory in its `repr`, and code that stringifies or logs it produces `defaultdict(<class 'list'>, {...})` rather than a familiar dict literal. Equality is unaffected — it compares items only — but serializers accept it silently as a `dict` subclass, so any keys manufactured by a stray lookup ship out as if they were real data. `setdefault` leaves you holding something that is a plain `dict` all the way down. ## When neither is right If every value is an integer tally, `collections.Counter` is the better answer: it is purpose-built for counting, it returns zero for absent keys *without* inserting them, and it supports ranking and arithmetic between tallies that you would otherwise hand-roll on top of `defaultdict(int)`. If the default depends on the key rather than being the same for all keys, neither fits, because `default_factory` is called with no arguments and `setdefault`'s default is a value you must already have computed. That case calls for a mapping class of your own. And if the mapping is a configuration or lookup table where absence is a defect, use a plain `dict` with no cleverness at all and let `KeyError` do its job. ## The short answer to give "`defaultdict` when the whole mapping is an accumulator with one uniform default; `setdefault` when it is one call site on an ordinary `dict`, or when the default is cheap and I want misses elsewhere to keep raising. And I convert the accumulator to a plain `dict` before it leaves the function."

  • Can you convert between an accumulating defaultdict and a plain dict in either direction?
    Yes, by copying. `defaultdict(list, plain)` builds a new mapping holding the same items with a factory attached, and `dict(dd)` builds a plain `dict` holding the same items with the factory dropped. Neither converts in place, so a caller still holding the original sees no change. Assigning `dd.default_factory = None` is the in-place way to disable the behaviour without copying.
  • Why is `d.setdefault(k, []).append(v)` sometimes called a performance smell?
    Because the empty list is constructed on every iteration, including the ones where the key already exists and the list is thrown away immediately. With a trivial default the waste is minor, but the same shape with an expensive default — or one whose construction has side effects — turns a small allocation cost into a real bug. `defaultdict` builds only on a genuine miss.
  • When would you keep the plain dict and the explicit membership check instead of either shortcut?
    When the missing-key branch does more than build an empty container — logging the new key, emitting a metric, applying validation, or choosing a default that depends on the key. Once the branch has real logic, spelling out `if k not in d:` is clearer than hiding it in a factory, and it keeps the mapping's ordinary lookup semantics intact.

saying these in an interview costs you the question

  • Thinks setdefault's default is evaluated only on a miss
  • Says defaultdict can be applied to an existing dict in place
  • Uses defaultdict where a typo'd key should raise
  • Claims dict(dd) preserves the default_factory
  • Reaches for defaultdict(int) when ranking tallies is the real need

context

resolving links…