skip to content

Why can an `__init_subclass__` registry register the same handler twice, and how do you make it idempotent?

level: seniorimportance: should knowfreq 18%

answer

  1. Automatic means opt-out, not selective
  2. Grandchildren and test subclasses register too
  3. One file can yield two class objects
  4. A list cannot dedupe; a mapping can
  5. Compare identity, then raise on collision

basics

~20 s

Because the hook fires for every descendant at any depth, and because a module imported under two names defines two distinct class objects. Key the registry explicitly, skip intermediates with an opt-out header keyword, and raise on a duplicate key instead of appending.

solid answer

~50 s

Self-registration in `__init_subclass__` is opt-out, so it catches more than you intended. It fires for grandchildren and for thin subclasses created by tests or by a plugin author, so a list-based registry ends up holding a handler and its near-clone, and dispatch runs the same side effect twice. The second cause is import identity: the same file imported as `__main__` and as a package module, or reached through two `sys.path` entries, produces two different class objects that both register. The fixes are structural — make registration explicit rather than automatic (`class Tmx(Handler, key='tmx')`, with no key meaning no registration), store in a dict keyed by that key rather than appending to a list, and raise `TypeError` when a key is already taken by a different class object so a double import fails at import time instead of duplicating work later.

code

python · 22 lines
python
class Handler:
    registry = {}

    def __init_subclass__(cls, /, key=None, **kwargs):
        super().__init_subclass__(**kwargs)
        if key is None:
            return
        existing = Handler.registry.get(key)
        if existing is not None and existing is not cls:
            raise TypeError(f"duplicate handler key {key!r}")
        Handler.registry[key] = cls


class Tmx(Handler, key="tmx"):
    pass


class TmxV2(Tmx):
    pass


print(Handler.registry)

go deeper

for a junior

Know that a base class can automatically collect its subclasses, and that anything automatic collects more than you had in mind — including subclasses written in tests.

for a middle

Explain the two mechanics behind duplicates: the hook runs for descendants at any depth, and one source file imported two ways yields two distinct class objects with the same name.

for a senior

Demonstrate the production fix and the diagnosis: an explicit key that gates registration, mapping storage, an identity check that raises on collision, and reading id plus module to classify a duplicate you are already seeing.

for a principal

Take a position on implicit registration as a codebase policy — what it costs in debuggability and import-time coupling, and when explicit registration or entry points is the right default for a plugin surface other teams extend.

## The failure, concretely Take a translation-memory updater whose segment handlers self-register: a `Handler` base keeps a list, and `__init_subclass__` appends every subclass to it. The updater walks the list and lets each handler write matching segments into the memory. A 340-case regression pack that used to be clean starts reporting that every translation-memory write lands twice — the same segment updated once, then updated again — and only for some handlers. This is the classic duplicated side effect of automatic registration, and it has two independent causes that look identical from the outside. ## Cause one: the hook is opt-out and depth-unaware `__init_subclass__` runs for every descendant at any depth, not for direct children only. So: * A thin subclass — `class TmxV2(Tmx)` adding one attribute, or a test double subclassing the real handler to override a single method — registers *in addition to* its parent. Dispatch then finds two handlers claiming the same format, and each performs the write. * Abstract intermediates register too. A `BaseFileHandler` that exists only to share code has no business being in a dispatch list, but nothing stops it from landing there. * Anything defined inside the test suite registers into the same global registry and stays there for the rest of the process, which is why a bug like this often appears only under the full regression pack and not when a single case is run. ## Cause two: one source file, two class objects Class identity is per-import, not per-file. The same module executed as `__main__` *and* imported by its package name produces two distinct class objects with the same `__qualname__`. The same happens when two `sys.path` entries reach the same file by different paths, when a package is importable both as `pkg.mod` and as `mod`, and when something calls `importlib.reload`, which re-executes the class statements and fires the hook again. A registry keyed by name would overwrite and hide this; a registry that appends to a list keeps both and doubles the work. Neither is what you want: you want the second registration to be *noticed*. ## Making it idempotent The shape that survives contact with real codebases has three properties. **Registration is explicit.** Use a class-header keyword as the registry key and treat its absence as "not a registered handler". That single change removes intermediates, mixins and test subclasses from the registry without any special-casing: ```python class Handler: registry = {} def __init_subclass__(cls, /, key=None, **kwargs): super().__init_subclass__(**kwargs) if key is None: return ... ``` **The registry is a mapping, not a list.** Appending can never be idempotent; assigning under a key can be. **A collision is loud.** If the key already maps to a *different* class object, raise. That converts the double-import case into an import-time error naming both classes, which is diagnosable in seconds, instead of a duplicated write discovered by a regression pack. Comparing with `is` rather than by name matters: two class objects from a double import have identical names and different identities, and identity is the thing you care about. ## Diagnosing it when it is already happening Print the registry with `id()` and `__module__` for each entry. Two entries with the same `__qualname__` and different ids is the double-import signature; a parent and a child both present is the depth signature. `sys.modules` will show the same file under two keys in the first case. From there the fix is either an import-path correction or the guarded registry above. ## Second-order concerns A class registry holds strong references, so every registered class — and through it, its module and any closures it captured — lives for the life of the process. That is fine for a fixed set of handlers loaded at startup and a genuine leak for classes generated per request or per document; `weakref.WeakValueDictionary` is the tool when the registry must not keep its entries alive. And registration only happens if the defining module is actually imported. `__init_subclass__` cannot discover a plugin nobody imported: you still need an explicit import sweep or a distribution entry point read through `importlib.metadata` to get the modules loaded. Self-registration solves bookkeeping, not discovery. ## When to abandon self-registration If the surprises keep coming — intermediates, test subclasses, conditional imports — a class decorator that registers exactly the class it decorates is the honest alternative. It is opt-in, it is visible at the definition site, and it never fires for a subclass you did not write. The cost is that someone can forget it; the benefit is that nothing happens by accident.

  • How do you tell a double-import duplicate from a depth duplicate?
    Inspect the registry entries. Two classes with the same `__qualname__`, the same source file and different `id()` values mean the module was executed twice — check `sys.modules` for the same file under two keys. A parent and its child both present, with different names, means the hook simply fired at every depth and the registration is not selective enough.
  • Why does self-registration still miss handlers at runtime?
    The hook only runs when the module defining the subclass is imported. A handler living in a module nobody imports is invisible to the registry, so you still need explicit imports in a package `__init__`, a directory scan, or a distribution entry point read via `importlib.metadata` to load the plugins. Registration solves bookkeeping, not discovery.
  • When would a class decorator be the better mechanism here?
    When accidental capture costs more than a forgotten registration. A decorator registers exactly the class it is applied to: intermediates, mixins and test subclasses stay out by construction, and the registration is visible at the definition site. The tradeoff is that it is opt-in and easy to omit, whereas the hook is opt-out and catches everything.
  • What memory effect does a global class registry have?
    It holds strong references, so each registered class keeps itself, its module and any captured objects alive for the process lifetime. For a fixed startup set that is harmless; for classes generated dynamically per request or per document it is a steady leak, and `weakref.WeakValueDictionary` is the appropriate container.

saying these in an interview costs you the question

  • Assumes the hook fires only for direct subclasses
  • Appends to a list and calls it idempotent
  • Dedupes by class name instead of identity
  • Believes the same module cannot be imported twice
  • Ignores that test subclasses land in the global registry
  • Keeps strong class references without considering lifetime

context