Why does next(obj) ignore a next attribute set on the instance?

middleshould knowfreq 30%

answer

Protocols do not use ordinary attribute lookup
The lookup starts one level up from the object
Speed, plus keeping metaclasses straight
Implicit special-method lookup goes to type(obj)

basics

~10 s

Implicitly invoked special methods are looked up on the object's type, not on the instance. next(obj) resolves next through type(obj), so a function stored in the instance dict is skipped and next() raises TypeError.

solid answer

~40 s

Python performs implicit special method lookup on the type: next(obj) resolves __next__ starting at type(obj) and its MRO, never at obj.__dict__. So assigning obj.__next__ = some_function leaves next(obj) raising TypeError: 'X' object is not an iterator, even though the explicit call obj.__next__() works, because that is ordinary attribute lookup. The same rule covers iter(), len(), subscripting and every operator. CPython does it for speed — the method lives in a C-level slot on the type, so a for loop avoids a dict probe per iteration — and for consistency, so that a dunder on a class serves its instances while the metaclass serves the class. The fix is to put the method on a class: patch C.__next__, or give the one object its own subclass.

Python performs *implicit special method lookup* on the type: `next(obj)` resolves `__next__` starting at `type(obj)` and its MRO, never at `obj.__dict__`. So assigning `obj.__next__ = some_function` leaves `next(obj)` raising `TypeError: 'X' object is not an iterator`, even though the explicit call `obj.__next__()` works, because that is ordinary attribute lookup. The same rule covers `iter()`, `len()`, subscripting and every operator. CPython does it for speed — the method lives in a C-level slot on the type, so a `for` loop avoids a dict probe per iteration — and for consistency, so that a dunder on a class serves its instances while the metaclass serves the class. The fix is to put the method on a class: patch `C.__next__`, or give the one object its own subclass.

code

python · 12 lines

python

class Counter:
    def __iter__(self):
        return self

c = Counter()
c.__next__ = lambda: 42

print(c.__next__())        # 42 - explicit lookup finds the instance attribute
try:
    next(c)                # implicit lookup goes to type(c)
except TypeError as exc:
    print(type(exc).__name__, exc)

go deeper

for a junior

Recall that next() and for need __next__ defined on the class body, not stuck onto an object afterwards. Knowing that dunders belong on the class is enough at this level.

for a middle

Explain implicit special method lookup: next(obj) resolves through type(obj) and its MRO while obj.__next__() uses ordinary attribute lookup, and say why — the C-level slot on the type avoids a dict probe per loop iteration.

for a senior

Use it as a debugging heuristic: a test double or proxy that refuses to iterate, or a __getattr__ forwarder that drops a protocol, is this rule biting. Show how you patch the class or mint a per-object subclass instead.

for a principal

Own the API consequence: protocols are a property of types, so any wrapping layer — proxies, lazy handles, instrumentation — must enumerate the dunders it intends to support rather than forward attributes generically, and that list is part of the contract you maintain.

The rule

For every operation Python invokes implicitly — an operator, a builtin that maps onto a protocol, a statement — the special method is looked up on the object's type, not on the object. next(obj) therefore resolves __next__ starting at type(obj) and walking that class's MRO; the instance __dict__ is never consulted. The same rule governs iter(obj), len(obj), obj[k], a + b, with obj:, str(obj) and the rest of the data model. It is stated in the language reference as implicit special method lookup, and it is not a CPython quirk: it is the defined semantics.

So this does not work:

python

obj.__next__ = lambda: 42
next(obj)     # TypeError: 'X' object is not an iterator

while writing obj.__next__() explicitly does return 42, because an explicit attribute access is ordinary attribute lookup and the instance dict wins.

Why the language is built that way

Two reasons, one mechanical and one about consistency.

Mechanically, CPython stores the protocol implementations as C-level function pointers on the type object — the iterator's next method lives in a slot on the type, filled in when the class is created and refreshed whenever the class attribute is reassigned. Going through the slot is a pointer dereference; going through instance attribute lookup would mean a dict probe on every single loop iteration. Every for loop in the language pays that cost, so the fast path is not optional.

For consistency, remember that classes are themselves objects. If implicit lookup started at the instance, then for a class C the expression len(C) would find C.__len__ — the method intended for instances of C — instead of a __len__ defined on C's metaclass. Bypassing the instance keeps the two levels apart: instance behaviour is defined on the class, class behaviour is defined on the metaclass.

What this means when you want to change behaviour

Patch the class, not the instance. Assigning C.__next__ = fn updates the type and takes effect immediately for all existing instances, because the assignment refreshes the type's slot.
To affect one object only, give it its own class. Create a small subclass carrying the method and either instantiate that instead, or reassign obj.__class__ to it. That keeps the dunder on a type — one that only this object uses.
Test doubles follow the same rule. A plain mock object with a __next__ attribute configured on the instance is not iterable, which is why mock libraries in the standard library provide a magic variant that configures the supported dunders on the mock's type. If a fake refuses to loop, this rule is the first thing to check.
Wrapping and proxying is harder than it looks. A generic proxy that forwards __getattr__ will not forward implicit dunder calls, because __getattr__ is only consulted during attribute lookup and implicit lookup never performs one. A proxy must define each protocol method on its own class.

Reading the error message

The TypeError that comes back is specific: 'X' object is not an iterator means the type has no __next__. Compare it with 'X' object is not iterable, which comes from iter() and means the type offers no way to obtain an iterator at all. Two different missing methods, two different messages, and telling them apart shortens the debugging enormously — the first says "you gave for something that iter() handed back wrongly, or you called next() on a container", the second says "you gave for an object that cannot start iterating".

The exception that proves the rule

Explicit dunder calls are just attribute access, so all of these behave differently from their implicit counterparts: obj.__next__(), obj.__len__(), obj.__eq__(other). That is precisely why code should call the builtin or use the operator rather than the dunder directly — next(it), len(x), x == y — and why a subtle bug appears when someone "optimises" a hot loop by calling it.__next__() in place of next(it): it now honours an instance attribute the real protocol would ignore, and the two paths can diverge.

## The rule For every operation Python invokes *implicitly* — an operator, a builtin that maps onto a protocol, a statement — the special method is looked up on the object's **type**, not on the object. `next(obj)` therefore resolves `__next__` starting at `type(obj)` and walking that class's MRO; the instance `__dict__` is never consulted. The same rule governs `iter(obj)`, `len(obj)`, `obj[k]`, `a + b`, `with obj:`, `str(obj)` and the rest of the data model. It is stated in the language reference as *implicit special method lookup*, and it is not a CPython quirk: it is the defined semantics. So this does not work: ```python obj.__next__ = lambda: 42 next(obj) # TypeError: 'X' object is not an iterator ``` while writing `obj.__next__()` explicitly *does* return 42, because an explicit attribute access is ordinary attribute lookup and the instance dict wins. ## Why the language is built that way Two reasons, one mechanical and one about consistency. Mechanically, CPython stores the protocol implementations as C-level function pointers on the type object — the iterator's next method lives in a slot on the type, filled in when the class is created and refreshed whenever the class attribute is reassigned. Going through the slot is a pointer dereference; going through instance attribute lookup would mean a dict probe on every single loop iteration. Every `for` loop in the language pays that cost, so the fast path is not optional. For consistency, remember that classes are themselves objects. If implicit lookup started at the instance, then for a class `C` the expression `len(C)` would find `C.__len__` — the method intended for *instances* of `C` — instead of a `__len__` defined on `C`'s metaclass. Bypassing the instance keeps the two levels apart: instance behaviour is defined on the class, class behaviour is defined on the metaclass. ## What this means when you want to change behaviour * **Patch the class, not the instance.** Assigning `C.__next__ = fn` updates the type and takes effect immediately for all existing instances, because the assignment refreshes the type's slot. * **To affect one object only, give it its own class.** Create a small subclass carrying the method and either instantiate that instead, or reassign `obj.__class__` to it. That keeps the dunder on a type — one that only this object uses. * **Test doubles follow the same rule.** A plain mock object with a `__next__` attribute configured on the instance is not iterable, which is why mock libraries in the standard library provide a magic variant that configures the supported dunders on the mock's *type*. If a fake refuses to loop, this rule is the first thing to check. * **Wrapping and proxying is harder than it looks.** A generic proxy that forwards `__getattr__` will not forward implicit dunder calls, because `__getattr__` is only consulted during attribute lookup and implicit lookup never performs one. A proxy must define each protocol method on its own class. ## Reading the error message The `TypeError` that comes back is specific: `'X' object is not an iterator` means the type has no `__next__`. Compare it with `'X' object is not iterable`, which comes from `iter()` and means the type offers no way to obtain an iterator at all. Two different missing methods, two different messages, and telling them apart shortens the debugging enormously — the first says "you gave `for` something that `iter()` handed back wrongly, or you called `next()` on a container", the second says "you gave `for` an object that cannot start iterating". ## The exception that proves the rule Explicit dunder calls are just attribute access, so all of these behave differently from their implicit counterparts: `obj.__next__()`, `obj.__len__()`, `obj.__eq__(other)`. That is precisely why code should call the builtin or use the operator rather than the dunder directly — `next(it)`, `len(x)`, `x == y` — and why a subtle bug appears when someone "optimises" a hot loop by calling `it.__next__()` in place of `next(it)`: it now honours an instance attribute the real protocol would ignore, and the two paths can diverge.

Does the same rule apply to iter() and len(), or is next() special?
It applies to every implicitly invoked special method: `iter()`, `len()`, `bool()`, subscripting, arithmetic operators, `with`, `str()` and so on all resolve on `type(obj)`. Only an explicit call such as `obj.__len__()` uses normal attribute lookup, which is one reason to prefer the builtin or the operator in application code.
How would you give a single object a custom __next__ then?
Put it on a type that only that object uses: define a small subclass with the method and instantiate it, or build the subclass dynamically and reassign `obj.__class__`. Patching the shared class works too but changes every instance, which is usually not what a one-off override wants.
Why does a __getattr__-based proxy fail to forward iteration?
`__getattr__` runs only during attribute lookup, and implicit special method invocation never performs one — it reads the slot on the proxy's own type. So the proxy is not iterable no matter what `__getattr__` would have returned. A proxy has to define `__iter__` and `__next__` (and any other protocol it forwards) explicitly on its class.

saying these in an interview costs you the question

Says dunder methods are found like any other attribute
Thinks the double underscores make the name private
Claims next(obj) is just sugar for obj.__next__() lookup
Believes assigning on the class won't affect existing instances
Assumes defining __iter__ alone makes next(obj) work
Reads 'not an iterator' as meaning 'not iterable'

Why does next(obj) ignore a __next__ attribute set on the instance?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

The rule

Why the language is built that way

What this means when you want to change behaviour

Reading the error message

The exception that proves the rule

▾common follow-ups3 questions

▾red flags6 flags

§ context

▾related questions4 questions

▾related topics4 topics

The rule

Why the language is built that way

What this means when you want to change behaviour

Reading the error message

The exception that proves the rule

Why does next(obj) ignore a next attribute set on the instance?

answer

go deeper

context