skip to content

Why can heapq.heappush raise TypeError when two entry tuples tie on priority?

level: middleimportance: must knowfreq 55%

answer

  1. Comparison does not stop at the priority
  2. It only fails when two priorities collide
  3. A third field nobody ever ties on
  4. The counter also buys push-order fairness
  5. itertools has the right generator

basics

~20 s

Tuples compare element by element. When two priorities are equal, Python moves on to the next field and compares the payloads themselves; if those are dicts or plain objects with no ordering, the comparison raises TypeError.

solid answer

~50 s

`heapq` orders elements with `<`, and a tuple's `<` is lexicographic: it compares field 0, and only if those are equal does it look at field 1. So `(2, payload_a) < (2, payload_b)` falls through to comparing the payloads, and if they are `dict`s, sets, or ordinary objects without `__lt__` you get `TypeError: '<' not supported between instances of ...`. The vicious part is that it fires **only on a tie**, so it survives unit tests and blows up in production. The standard fix is a three-field entry — `(priority, next(counter), payload)` with `counter = itertools.count()`. The counter is unique and strictly increasing, so comparison always stops at field 1, the payload is never compared, and equal priorities come out in push order. `@dataclass(order=True)` with `field(compare=False)` on the payload does the same job with names instead of positions.

code

python · 8 lines
python
import heapq

heap = []
heapq.heappush(heap, (2, {"id": "b"}))
try:
    heapq.heappush(heap, (2, {"id": "a"}))  # tie falls through to the dicts
except TypeError as exc:
    print(exc)

go deeper

for a junior

Know that pushing plain (priority, payload) tuples is fine only while priorities never repeat, and that the standard entry has a third field between them. Recognise the '<' not supported TypeError on sight.

for a middle

Explain lexicographic tuple comparison, why the failure appears only on a tie, and how itertools.count as field 1 both prevents the comparison and delivers FIFO order within a priority band.

for a senior

Treat this as a latent-defect question: it survives tests with distinct priorities and fails hours into a long run. Argue for encoding every ordering field explicitly rather than letting the payload decide, and know the lazy-deletion recipe for cancellations.

for a principal

Own the entry-format decision across a codebase: positional tuple versus ordered dataclass, what determinism the pipeline needs for reproducible runs, and whether entry construction should be behind one helper so no caller can invent its own tuple shape.

### heapq only knows `<` The module has no `key=` parameter and no comparator hook. Every ordering decision it makes is a single `a < b` between two elements of the list, delegated to those objects' own rich-comparison methods. That is why the universal idiom for a priority queue in Python is to push *tuples* whose first field is the priority: tuple comparison does the right thing for free. But tuple comparison does more than most people picture. `tuple.__lt__` walks the fields left to right, and the moment it finds a pair that is unequal it returns that result. If field 0 of both tuples compares equal, it moves to field 1 and compares **those**. So the payload you attached purely as cargo becomes part of the ordering the first time two priorities collide. ```python (1, {"id": "a"}) < (2, {"id": "b"}) # False, decided at field 0 (2, {"id": "a"}) < (2, {"id": "b"}) # TypeError, falls through to the dicts ``` Dicts define `==` but not `<`. Sets define `<` as the subset relation, which is a partial order — arguably worse, because it does not raise; it just yields an ordering that is neither total nor meaningful. Ordinary user classes without `__lt__` raise. Every one of these is a bug that only appears when two priorities happen to be equal. ### Why this is a production-shaped failure Consider a nightly genome-annotation run that queues per-region jobs by priority and executes them from a heap. In a small test fixture, priorities are distinct, the comparison never reaches field 1, and everything passes. On the real corpus, two regions eventually score identically — and the six-hour run dies with a `TypeError` from deep inside `heapq.heappush`, hours in, with a traceback that names the module rather than your data. It is a latent bug measured in wasted machine-hours. ### The counter idiom The stdlib's own priority-queue notes prescribe a monotonically increasing tiebreak field: ```python import heapq from itertools import count tiebreak = count() queue = [] heapq.heappush(queue, (priority, next(tiebreak), payload)) ``` `itertools.count()` hands out `0, 1, 2, ...`. Because every entry gets a distinct integer, no two tuples are ever equal through field 1, so tuple comparison stops there and the payload is never touched — it can be a dict, an un-orderable object, `None`, anything. As a bonus you get a guarantee the heap itself does not offer: among equal priorities, the entry pushed first has the smaller counter and therefore pops first. That is FIFO within a priority band, and it is deterministic, which matters for reproducible pipeline runs. Two things people reach for instead, and why they are worse. A random float as the tiebreak makes ordering nondeterministic between runs and can, in principle, collide. `id(payload)` is unique but arbitrary, address-reuse-prone, and produces an order nobody can reason about. ### When the payload should participate Sometimes you *want* a secondary ordering — same priority, then earliest deadline. Then put that field in the tuple explicitly, before the counter: `(priority, deadline, next(tiebreak), payload)`. Making the order visible in the tuple is the whole point; relying on an accidental fall-through to the payload is what got you the `TypeError`. If positional tuples are getting unwieldy, a dataclass says the same thing with names: ```python from dataclasses import dataclass, field from typing import Any @dataclass(order=True) class Entry: priority: int sequence: int payload: Any = field(compare=False) ``` `order=True` generates `__lt__` from the fields in declaration order, and `compare=False` excludes `payload` from it, which is exactly the invariant the counter idiom enforces by construction. ### Removing an entry The related question interviewers often tack on: you cannot delete from the middle of a heap. The stdlib recipe keeps a dict from key to entry, mutates the entry's payload slot to a removed-sentinel instead of deleting it, and skips sentinels when popping — lazy deletion. Note that this only works because the entry is a mutable object reachable from outside the list; a tuple entry cannot be edited in place, so the recipe uses a list as the entry.

  • Why does the counter also give you FIFO order among equal priorities?
    `itertools.count()` yields strictly increasing integers, so among entries sharing a priority the one pushed earlier carries the smaller counter and wins the comparison at field 1. The heap itself promises nothing about equal keys — the ordering you get is the one you encoded. That determinism is what makes a pipeline run reproducible.
  • What if the payload legitimately needs to affect the ordering?
    Then put the real secondary key into the tuple explicitly — `(priority, deadline, next(counter), payload)` — so the intent is visible. Alternatively use `@dataclass(order=True)` and mark only the genuinely non-comparable fields with `field(compare=False)`. What you must not do is let an unplanned fall-through to the payload decide it for you.
  • How do you cancel an entry that has already been pushed?
    You cannot remove from the middle of a heap, so the stdlib recipe is lazy deletion: keep a dict mapping key to entry, and to cancel, mutate that entry's payload slot to a removed sentinel. Popping skips sentinels. The entry must be a mutable list rather than a tuple for the in-place edit to work, and you need periodic rebuilds if cancellations pile up.

saying these in an interview costs you the question

  • Thinks tuple comparison only looks at the first field
  • Says heapq accepts a key or comparator argument
  • Uses a random float as the tiebreak field
  • Assumes heapq is stable for equal priorities by default
  • Wraps the payload in a list to avoid comparison
  • Believes the TypeError means the priority type is wrong

context