skip to content

The collections Module

The specialised containers the standard library ships so you stop hand-rolling them: Counter, defaultdict, deque, namedtuple, ChainMap. Knowing them is the fluency signal interviewers look for.

part ofPythonoverview, primer and where to startread it →
on this pageshow

questions

19

What does collections.Counter return when you look up a key that was never counted?

level: juniorimportance: must knowfreq 70%

answer

  1. A dict subclass with one read hook
  2. Absent key, no exception
  3. Reading never grows the mapping
  4. Zero count still occupies a slot
  5. The hook is named __missing__

basics

~20 s

Zero. Counter's missing hook returns 0 for any absent key without storing it, so the key stays out of the mapping, out of len and out of iteration. Only assigning to it or counting it creates an entry.

solid answer

~40 s

`collections.Counter` is a `dict` subclass whose `__missing__` hook returns `0` instead of raising `KeyError`, and — this is the part people miss — the lookup is **read-only**: the key is not inserted, so `len()`, membership tests and iteration are unchanged. That is what makes `c[word] += 1` safe with no guard: the read yields 0, the assignment then creates the key. It is the opposite of `collections.defaultdict`, where a plain read runs the factory and inserts. A count of `0` is therefore not the same as absence — a key driven down to zero by `Counter.subtract()` stays in the mapping, still shows up in `len()` and `most_common()`, and only disappears from `Counter.elements()` or after an explicit unary `+` cleanup.

code

pycon · 10 lines
pycon
>>> from collections import Counter
>>> c = Counter("banana")
>>> c
Counter({'a': 3, 'n': 2, 'b': 1})
>>> c["z"]
0
>>> "z" in c
False
>>> len(c)
3

go deeper

for a junior

Recall the one-line answer: the lookup gives 0 and raises nothing. Be ready to show the increment loop that relies on it, and to say plainly that reading a key does not add it.

for a middle

Explain the mechanism by name: Counter subclasses dict and implements __missing__, which is consulted only on a failed lookup and returns a value without storing it. Contrast it with defaultdict, which inserts.

for a senior

Show the trap you have actually hit: zero counts are still keys, so len(), items() and most_common() still see them after subtraction. Say how you strip them and when the distinction matters in real code.

for a principal

Frame the choice of mapping as an API decision: a defaulted read that mutates (defaultdict) versus one that does not (Counter) changes what callers can safely do concurrently and how the structure grows. Have a house rule for which you reach for.

## What Counter actually is `collections.Counter` is not a separate data structure; it is a subclass of `dict` whose values are meant to be integer counts. Everything a dict does — insertion-ordered iteration, `keys()`, `values()`, `items()`, membership tests, `len()` — it does too. The one behaviour it changes on read is what happens for a key the mapping does not contain. ## The `__missing__` hook `dict.__getitem__` has a documented escape hatch: when the key is absent, and only then, it calls the subclass's `__missing__(key)` method if one exists, and returns whatever that returns. `Counter.__missing__` is a one-liner that returns `0`. So: ```python from collections import Counter c = Counter("banana") # Counter({'a': 3, 'n': 2, 'b': 1}) c["z"] # 0 -- no KeyError "z" in c # False len(c) # 3 ``` Two facts fall out of that. First, no `KeyError`: you never need a `try`/`except` or an `if key in c` guard around a read. Second, and this is the fact interviews probe, **the lookup does not insert**. `__missing__` returns a value; it does not touch the underlying dict. After a million lookups of keys you never counted, the Counter is still exactly as large as it was. ## Why that makes the counting idiom safe The canonical accumulation loop works without initialisation: ```python counts = Counter() for word in text.split(): counts[word] += 1 ``` `counts[word] += 1` expands to `counts[word] = counts[word] + 1`. The read on the right returns 0 for a first-time word; the **assignment** on the left is what creates the key. Reading is passive, writing is what mutates. (In practice you would write `Counter(text.split())` and let the constructor do the loop in C, but the augmented-assignment form is what you use when the increment is conditional or weighted.) ## The contrast that gets asked next Three mappings, three behaviours for the same read of an absent key: - `dict`: raises `KeyError`. Use `dict.get(key, 0)` for a defaulted read. - `collections.defaultdict(int)`: calls the factory, **inserts** the new key, returns `0`. A read alone grows the mapping — which is why looping `for k in list(d)` over a defaultdict while probing it can surprise you. - `Counter`: returns `0`, inserts nothing. So a Counter is the right choice when you want defaulted reads without the mapping quietly accumulating every key you ever asked about; a defaultdict is the right choice when the insert is the point (grouping into lists, for instance). ## Zero is not absence The subtle half of this question is the mirror image: a key whose count has reached `0` is still present. ```python c = Counter(a=1) c["a"] -= 1 c # Counter({'a': 0}) "a" in c # True len(c) # 1 ``` Nothing prunes it. It will appear in `items()`, in `most_common()` (at the bottom), and it counts toward `len()`. `Counter.elements()` skips it, because that method only expands counts greater than zero, and applying unary `+` to a Counter returns a new Counter with only the positive counts kept — the documented way to clean up after `Counter.subtract()`. That asymmetry is a real source of bugs: `if c[key]` (falsy for both 0 and absent) and `if key in c` (true only for present) answer different questions, and code that mixes them drifts. ## Deletion is forgiving too `Counter` also overrides `__delitem__` so that deleting a key it does not hold is a no-op rather than a `KeyError`. That is a deliberate convenience for the same reason as `__missing__`: counting code should not have to check first. ## A related read that is not defaulted `Counter.get(key)` is the inherited `dict.get`: it returns `None` for an absent key, not 0, because `get` never consults `__missing__` — only `__getitem__` does. So `c.get(k)` and `c[k]` differ on exactly the keys this question is about, and `c.get(k, 0)` is needed to match subscripting. The same applies to `dict.setdefault`, `dict.pop` and the `in` test: none of them route through `__missing__`. Knowing which operations are hooked and which are not is the difference between reciting the behaviour and understanding it. ## What to say in an interview Name `__missing__` as the mechanism, state clearly that the read returns 0 **without inserting**, and volunteer the two neighbouring facts: that this is why `c[k] += 1` needs no initialisation, and that a zero count still occupies a slot until you strip it. If you can also say what `defaultdict` does differently on the same read, you have covered the whole question.

  • Does `del c[k]` raise KeyError on a Counter when the key was never counted?
    No. `Counter` overrides `__delitem__` so that deleting an absent key is a silent no-op, unlike a plain `dict`, which raises `KeyError`. It is the same design instinct as `__missing__`: counting code should not need a guard around either the read or the delete.
  • How does `collections.defaultdict(int)` differ on exactly the same lookup?
    A defaultdict calls its `default_factory`, stores the result under that key and returns it, so the mere read grows the mapping and changes `len()`. A Counter returns 0 and inserts nothing. If you are probing keys you do not intend to keep, that difference is the deciding one.
  • Why does `Counter.fromkeys()` not work like `dict.fromkeys()`?
    It is deliberately disabled: `Counter.fromkeys()` raises `NotImplementedError`, because the inherited meaning (every key mapped to one shared default) is ambiguous for counts. Build one with `Counter(iterable)` instead, which counts occurrences, or pass a mapping of explicit counts.

It answers 'how many of these do you have?' with 'none' rather than throwing you out of the shop — but saying 'none' does not put an empty shelf label up.

saying these in an interview costs you the question

  • Says a missing key raises KeyError on a Counter
  • Thinks reading a missing key inserts it with count 0
  • Assumes a count of 0 means the key is gone
  • Guards every increment with an 'if key in c' check
  • Believes Counter and defaultdict behave identically on reads
  • Says del on an uncounted key raises KeyError

context

open as a page

What does collections.defaultdict(list) do that a plain dict does not?

level: juniorimportance: must knowfreq 70%

basics

~10 s

collections.defaultdict(list) calls its factory when a looked-up key is missing, stores the fresh empty list under that key and returns it, instead of raising KeyError. Grouping becomes a single append with no membership check.

open as a page

Why is `list.pop(0)` O(n) while `collections.deque.popleft()` is O(1)?

level: juniorimportance: must knowfreq 70%

basics

~20 s

A list keeps its items in one contiguous array, so removing index 0 shifts every remaining element down one slot. A deque is a doubly linked chain of small blocks, so popleft just unlinks the front item.

open as a page

What does collections.namedtuple give you that a plain tuple does not?

level: juniorimportance: must knowfreq 62%

basics

~10 s

collections.namedtuple is a class factory: it returns a new class that subclasses tuple and gives every position a name. You can read bid.cpm_cents as well as bid[1], and the repr prints field names.

open as a page

What does collections.OrderedDict still offer now that dict preserves insertion order?

level: middleimportance: must knowfreq 58%

basics

~20 s

Three things dict lacks: move_to_end(key, last=) to reposition a key in O(1), popitem(last=False) to pop the oldest entry, and order-sensitive equality between two OrderedDicts. Since Python 3.7 plain dict keeps insertion order, so ordering alone is no reason.

open as a page

How do Counter's +, -, & and | operators differ from Counter.update() and Counter.subtract()?

level: middleimportance: must knowfreq 60%

basics

~20 s

The four operators are multiset algebra: they build a new Counter and discard any count that is not positive. Counter.update() and Counter.subtract() mutate in place, add or subtract counts key by key, and keep zeros and negatives.

open as a page

Why does a collections.defaultdict grow when you only read a missing key?

level: middleimportance: must knowfreq 60%

basics

~20 s

Subscripting a defaultdict is not a pure read. A miss triggers missing, which calls default_factory and stores the new value under the key before returning it, so the mapping grows. Use .get() or an in test when a lookup must not insert.

open as a page

How do you change a field value on a collections.namedtuple instance?

level: middleimportance: must knowfreq 55%

basics

~20 s

You do not change it in place — namedtuple fields are read-only. Call the instance's _replace() method with keyword arguments; it returns a new instance with those fields swapped and the rest copied, leaving the original untouched.

open as a page

When do you pick collections.defaultdict over dict.setdefault?

level: middleimportance: should knowfreq 45%

basics

~20 s

Pick defaultdict when every missing key deserves the same freshly built default and the mapping is used that way throughout. Pick dict.setdefault for occasional insertion into an ordinary dict, or when a stray missing key elsewhere must still raise KeyError.

open as a page

What happens when you append to a `collections.deque` created with maxlen?

level: middleimportance: should knowfreq 50%

basics

~20 s

Once the deque is full, every push silently discards an item from the opposite end. append drops the leftmost item, appendleft drops the rightmost, and no exception is raised. maxlen is fixed at construction and read-only afterwards.

open as a page

When would you choose typing.NamedTuple over a dataclass for a record?

level: middleimportance: should knowfreq 52%

basics

~20 s

Choose typing.NamedTuple when the record is a small immutable value that should keep tuple behaviour — unpacking, indexing, hashing, sorting. Choose a dataclass when you need mutation, validation, inheritance, or a record that must not act like a sequence.

open as a page

A shared collections.ChainMap holds an invoice renderer's config — why do per-invoice overrides leak into later invoices?

level: seniorimportance: should knowfreq 30%

basics

~20 s

Every ChainMap write lands in its first mapping, and a shared instance shares that mapping, so one invoice's override is still there for the next. Give each invoice its own layer with new_child(), which returns a new ChainMap.

open as a page

Why can Counter.most_common(20) reorder tied segments between runs of a translation-memory updater?

level: seniorimportance: should knowfreq 45%

basics

~20 s

Counter.most_common sorts by count descending and breaks ties by the order elements were first encountered. Rebuild the Counter from files in a different order and equally frequent segments swap places. Sort with an explicit tiebreak key if the ranking must be reproducible.

open as a page

Why is indexing the middle of a `collections.deque` O(n) when `d[0]` is O(1)?

level: seniorimportance: should knowfreq 35%

basics

~20 s

A deque is a doubly linked chain of fixed-size blocks, not one array, so there is no formula from index to address. Reading an index means walking blocks from whichever end is nearer, which is worst at the middle.

open as a page

How does collections.ChainMap resolve a lookup when several of its mappings hold the same key?

level: juniorimportance: nice to knowfreq 24%

basics

~20 s

It searches its mappings left to right and returns the first hit, so the front mapping shadows the ones behind it. Nothing is copied or merged: the underlying dicts stay live and later edits show through.

open as a page

How do Counter.elements() and Counter.total() treat zero and negative counts?

level: middleimportance: nice to knowfreq 25%

basics

~10 s

Counter.elements() expands only counts greater than zero, so zero and negative entries yield nothing. Counter.total() sums every count as written, negatives included. After a subtraction the two can disagree completely.

open as a page

Why does `collections.deque.extendleft()` reverse the iterable you pass it?

level: middleimportance: nice to knowfreq 25%

basics

~20 s

Because extendleft is defined as a series of appendleft calls. Each item is pushed in front of the one before it, so the last item pushed ends up leftmost and the result reads in the opposite order to the input.

open as a page

How do you nest collections.defaultdict, and what breaks when you do?

level: seniorimportance: nice to knowfreq 25%

basics

~20 s

Nest by making the factory build the inner mapping, usually defaultdict(lambda: defaultdict(list)). The common breakage is that a lambda factory cannot be pickled, so the structure will not cross a process boundary or into a cache; a module-level named function fixes it.

open as a page

When does a namedtuple's tuple compatibility become a production hazard?

level: seniorimportance: nice to knowfreq 30%

basics

~20 s

A namedtuple instance really is a tuple, so unrelated code treats it as a sequence: two different record types with equal values compare equal, JSON encodes it as an array, and percent-formatting unpacks it as arguments.

open as a page