Why does a cached total on a `list` subclass go stale when only `append` is overridden?

## The scenario An image-thumbnail worker keeps its pending jobs in a `list` subclass that caches the total pixel count so the batch sizer does not recompute it on every tick: ```python class PendingJobs(list): def __init__(self, items=()): super().__init__(items) self._total = sum(items) def append(self, item): super().append(item) self._total += item @property def total(self): return self._total ``` It is correct in the unit test, which appends. In the six-hour nightly run the queue is filled by `jobs.extend(batch)` and `jobs += retries`, trimmed with `del jobs[:100]`, and reordered with `jobs.sort()`. `total` stays at whatever the appends left behind, the sizer allocates from a stale number, and the failure shows up hours in as thumbnails produced at the wrong batch size — with no exception anywhere to point at the cause. ## Why the override is bypassed `list` is a C type. Its mutating methods manipulate the underlying pointer array directly; they do not perform a Python-level attribute lookup for `append` and call it. That is a deliberate design choice — it is what makes list operations fast — but it means a Python subclass has no interception point: - `extend`, `insert`, `remove`, `pop`, `clear`, `sort`, `reverse` — all C, all direct. - `+=` goes through `list.__iadd__`, which extends in place. - `jobs[0] = x`, `jobs[:100] = xs` and `del jobs[0]` go through the C slot functions. The general rule: **inheriting from a built-in container inherits its methods, not its call graph.** You can override any single method, but you cannot assume the built-in's other methods will call the one you overrode. The same shape appears whenever people subclass a built-in container and override one accessor or one mutator, expecting the rest of the type to funnel through it. ## Does `collections.UserList` fix it? Partly, and it is worth being precise about how much. `UserList` is written in Python and keeps the real list in an ordinary attribute named `data`. Every method is therefore overridable, and slicing and `+` return `self.__class__` instead of a plain list. But its methods delegate to the wrapped list, not to each other: its `extend` is `self.data.extend(other)`, its `__iadd__` is `self.data += other`, its `insert` is `self.data.insert(...)`. Overriding `append` on a `UserList` subclass still does not catch `extend`. So the honest answer is that `UserList` gives you the *ability* to intercept everything, at the cost of doing so explicitly: `append`, `insert`, `extend`, `pop`, `remove`, `clear`, `sort`, `reverse`, `__setitem__`, `__delitem__`, `__iadd__`, `__imul__`. Twelve overrides, and a reviewer has to notice if a thirteenth appears. It is also several times slower per operation than a built-in list, and it is not a `list` — `isinstance(jobs, list)` is false, and code that special-cases lists, including `json.dumps`, will not accept it. ## The designs that hold **Composition with a narrow API.** Do not be a list; hold one. Expose `add`, `take_batch` and `__len__` — the three operations the worker actually performs — and maintain the total inside them. Nothing else can mutate the collection, so nothing else can invalidate the cache. Everything you did not expose is a mutation that can no longer surprise you. **No cache at all.** Compute `sum(self._jobs)` on demand, or keep it as a lazily-computed value cleared by the narrow API. A derived value that is recomputed cannot be stale, and for a queue of a few thousand integers the cost is invisible next to decoding one image. **Immutable snapshots.** If the batch sizer needs a consistent view, hand it a tuple built at the moment of the decision, with the total computed alongside it. Value and total travel together and cannot drift apart. ## Proving it in a test The test that would have caught this exercises every public mutation path and asserts the derived value against a fresh recomputation, rather than testing `append` alone. That is also the review heuristic: whenever a cached attribute is invalidated in an overridden method of a container, ask what else can change the container. On a built-in subclass, the answer is almost always "a dozen things, none of which call you".