A collector's next delegates to an inner iterator and a 6,800-row telemetry batch stops at row 900 with no error. What happened, and how do you fix it?

seniorshould knowfreq 35%

answer

The loop ended, it did not fail
Two very different causes, one identical signal
Delegation leaks the terminator upward
Wrap the inner next() and re-raise something loud

basics

~20 s

A StopIteration raised inside next by the delegated call, or by a helper it uses, is indistinguishable from a deliberate end-of-stream signal. The consuming for loop catches it and ends normally, so the short batch looks like a successful one.

solid answer

~40 s

Any StopIteration that escapes __next__ means "exhausted" to the consuming for loop, whether you raised it or a helper did. The loop catches it, ends normally, and the code that commits the batch runs on 900 rows with nothing logged — a try/except around the loop never fires because nothing propagates. The fix is to raise StopIteration from exactly one place: wrap every delegated next() call inside __next__, re-raise it untouched only when your own source is legitimately finished, and convert any other one into a RuntimeError (raise ... from None) so the failure is loud and rollback-able. Longer term, do not infer completeness from a loop ending: assert a row count against the manifest, or require a terminal marker record, before committing.

Any `StopIteration` that escapes `__next__` means "exhausted" to the consuming `for` loop, whether you raised it or a helper did. The loop catches it, ends normally, and the code that commits the batch runs on 900 rows with nothing logged — a `try`/`except` around the loop never fires because nothing propagates. The fix is to raise `StopIteration` from exactly one place: wrap every delegated `next()` call inside `__next__`, re-raise it untouched only when your own source is legitimately finished, and convert any other one into a `RuntimeError` (`raise ... from None`) so the failure is loud and rollback-able. Longer term, do not infer completeness from a loop ending: assert a row count against the manifest, or require a terminal marker record, before committing.

code

python · 14 lines

python

def first_reading(chunk):
    return next(iter(chunk))          # StopIteration on an empty chunk

class Collector:
    def __init__(self, chunks):
        self._chunks = iter(chunks)

    def __iter__(self):
        return self

    def __next__(self):
        return first_reading(next(self._chunks))

print(list(Collector([[1], [], [3]])))   # [1] - the empty chunk ended the loop

go deeper

for a junior

Remember the one rule behind the whole scenario: a StopIteration coming out of __next__ means "finished" to the loop, no matter who raised it or why. Do not call next() casually inside __next__.

for a middle

Explain the mechanics — delegation leaks the terminator, the loop catches it, nothing propagates — and write the fix: wrap the inner next(), re-raise only for genuine exhaustion, convert anything else into a domain error.

for a senior

Show the diagnosis path on a live batch: a count that disagrees with the manifest and no exception, then a boundary wrapper that turns the hidden signal into a traceback naming the helper. Argue why the commit must not be driven by the loop simply ending.

for a principal

Own the contract: exhaustion and truncation are indistinguishable at the protocol level, so completeness must be carried in the data — manifest counts, terminal markers, an explicit complete flag — and asserted at the transaction boundary across every ingestion path, not patched per collector.

What actually happened

The consuming for loop catches StopIteration raised by its call to __next__. It cannot tell why that exception arrived. Whether the collector deliberately signalled "the source is finished" or a helper deep inside __next__ happened to call next() on an empty inner iterator, the loop sees one thing: exhaustion. It ends normally, no exception escapes, and the code after the loop runs as if the whole 6,800-row batch had been read. In a collector that commits per batch, that is the worst possible failure shape — a silent short read that looks like success.

The classic shape is delegation plus a helper that itself uses next():

python

def first_reading(chunk):
    return next(iter(chunk))       # empty chunk -> StopIteration

class Collector:
    def __next__(self):
        return first_reading(next(self._chunks))

One empty chunk at row 900 and the batch ends there. Nothing is logged, nothing is raised, the exit status is zero.

Why the usual defences miss it

A try/except around the loop catches nothing, because nothing propagates. Error handling never runs, so a rollback keyed to an exception never fires.
except Exception does not help either. StopIteration is an Exception subclass, so a broad handler would catch it — but there is nothing to catch: the loop consumed it internally.
Row counts look plausible. 900 rows is a legitimate batch size; only comparing against the expected 6,800 reveals the truncation.
A generator-based implementation would have failed loudly. When a generator function lets a StopIteration escape its frame, Python converts it into a RuntimeError. A class-based __next__ gets no such conversion — you have to build it.

Diagnosing it

Instrument the boundary rather than the body. Increment a counter inside __next__ and log the final count next to the manifest count; the moment those disagree without an exception you know the loop ended early rather than failed. Then narrow to the culprit by wrapping the delegated call:

python

try:
    row = next(self._source)
except StopIteration:
    raise RuntimeError("inner source exhausted early") from None

If the batch now raises where it previously succeeded, the StopIteration was coming from inside — and the traceback of the converted error names the helper that produced it. Reproducing it in a unit test is cheap: feed a source containing one empty chunk in the middle and assert the collected count.

Fixing it

Raise StopIteration from exactly one place. In __next__, the only legitimate StopIteration is the one you raise yourself when your source is genuinely finished. Every delegated call gets wrapped:

python

def __next__(self):
    try:
        raw = next(self._source)
    except StopIteration:
        if self._seen != self._expected:
            raise RuntimeError(f"truncated at {self._seen}") from None
        raise
    self._seen += 1
    return self._parse(raw)

Note the two branches: exhaustion at the expected point re-raises StopIteration untouched, so the loop still terminates cleanly; exhaustion anywhere else becomes a loud error the caller's except block will see and can roll back on.

Keep helpers out of the protocol. A parsing helper should raise a domain exception on empty input, not lean on next() and let the resulting StopIteration travel. next() with a default, or an explicit emptiness check, keeps the signal from ever being minted.

The design lesson for the rollback

Even with the fix, the deeper point stands: a loop ending is not evidence that the data was complete. Exhaustion is indistinguishable from truncation at the protocol level, and it always will be — a network read that ends early and a file that ends on its last record both surface as "no more items". So the commit decision must not be inferred from control flow reaching the end of the loop. Make completeness explicit: compare the row count against a manifest, require a terminal marker record from the producer, or have the iterator expose a complete flag that the transaction boundary asserts before committing. Then a partial read fails the check and the batch rolls back, whatever caused the stream to stop.

## What actually happened The consuming `for` loop catches `StopIteration` raised by its call to `__next__`. It cannot tell *why* that exception arrived. Whether the collector deliberately signalled "the source is finished" or a helper deep inside `__next__` happened to call `next()` on an empty inner iterator, the loop sees one thing: exhaustion. It ends normally, no exception escapes, and the code after the loop runs as if the whole 6,800-row batch had been read. In a collector that commits per batch, that is the worst possible failure shape — a *silent* short read that looks like success. The classic shape is delegation plus a helper that itself uses `next()`: ```python def first_reading(chunk): return next(iter(chunk)) # empty chunk -> StopIteration class Collector: def __next__(self): return first_reading(next(self._chunks)) ``` One empty chunk at row 900 and the batch ends there. Nothing is logged, nothing is raised, the exit status is zero. ## Why the usual defences miss it * **A `try`/`except` around the loop catches nothing**, because nothing propagates. Error handling never runs, so a rollback keyed to an exception never fires. * **`except Exception` does not help either.** `StopIteration` *is* an `Exception` subclass, so a broad handler would catch it — but there is nothing to catch: the loop consumed it internally. * **Row counts look plausible.** 900 rows is a legitimate batch size; only comparing against the expected 6,800 reveals the truncation. * **A generator-based implementation would have failed loudly.** When a generator function lets a `StopIteration` escape its frame, Python converts it into a `RuntimeError`. A class-based `__next__` gets no such conversion — you have to build it. ## Diagnosing it Instrument the boundary rather than the body. Increment a counter inside `__next__` and log the final count next to the manifest count; the moment those disagree without an exception you know the loop ended early rather than failed. Then narrow to the culprit by wrapping the delegated call: ```python try: row = next(self._source) except StopIteration: raise RuntimeError("inner source exhausted early") from None ``` If the batch now raises where it previously succeeded, the `StopIteration` was coming from inside — and the traceback of the converted error names the helper that produced it. Reproducing it in a unit test is cheap: feed a source containing one empty chunk in the middle and assert the collected count. ## Fixing it **Raise `StopIteration` from exactly one place.** In `__next__`, the only legitimate `StopIteration` is the one you raise yourself when *your* source is genuinely finished. Every delegated call gets wrapped: ```python def __next__(self): try: raw = next(self._source) except StopIteration: if self._seen != self._expected: raise RuntimeError(f"truncated at {self._seen}") from None raise self._seen += 1 return self._parse(raw) ``` Note the two branches: exhaustion at the expected point re-raises `StopIteration` untouched, so the loop still terminates cleanly; exhaustion anywhere else becomes a loud error the caller's `except` block will see and can roll back on. **Keep helpers out of the protocol.** A parsing helper should raise a domain exception on empty input, not lean on `next()` and let the resulting `StopIteration` travel. `next()` with a default, or an explicit emptiness check, keeps the signal from ever being minted. ## The design lesson for the rollback Even with the fix, the deeper point stands: **a loop ending is not evidence that the data was complete.** Exhaustion is indistinguishable from truncation at the protocol level, and it always will be — a network read that ends early and a file that ends on its last record both surface as "no more items". So the commit decision must not be inferred from control flow reaching the end of the loop. Make completeness explicit: compare the row count against a manifest, require a terminal marker record from the producer, or have the iterator expose a `complete` flag that the transaction boundary asserts before committing. Then a partial read fails the check and the batch rolls back, whatever caused the stream to stop.

Why doesn't wrapping the batch loop in try/except catch this?
Because no exception ever escapes. The `for` loop consumes the `StopIteration` internally and finishes normally, so control falls through to the code after the loop — including the commit — and every `except` clause is skipped. The failure has to be turned into a propagating exception inside `__next__`, or detected by a completeness check after the loop.
How would you catch this in production rather than in code review?
Make completeness explicit and assert it at the transaction boundary: compare rows consumed against the batch manifest, or require the producer to emit a terminal marker record and treat its absence as a failure. Log the consumed count next to the expected one on every batch so a short read shows up as a metric, not as a quiet success.
Is `raise ... from None` the right choice when converting the exception?
Yes here. The `StopIteration` is a control-flow signal, not a root cause, so chaining it adds a confusing "during handling of the above exception" block to every traceback. Suppress the context and put the useful facts — rows seen versus rows expected — in the `RuntimeError` message instead.
Would writing the collector as a generator function have avoided the bug?
It would have made it loud rather than silent: when a `StopIteration` escapes a generator's frame, Python raises a `RuntimeError` instead of letting it read as exhaustion. You would have seen a crash at row 900 rather than a short commit. The underlying mistake — a helper leaking the terminator — is still yours to fix.

saying these in an interview costs you the question

Expects StopIteration to appear in the logs as an error
Blames the for loop for swallowing a real exception
Thinks try/except around the loop would have caught it
Wraps __next__ in a bare except, hiding the real bug
Treats the loop finishing as proof the batch was complete
Returns None from __next__ to signal a problem

A collector's __next__ delegates to an inner iterator and a 6,800-row telemetry batch stops at row 900 with no error. What happened, and how do you fix it?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

What actually happened

Why the usual defences miss it

Diagnosing it

Fixing it

The design lesson for the rollback

▾common follow-ups4 questions

▾red flags6 flags

§ context

▾related questions4 questions

▾related topics4 topics

What actually happened

Why the usual defences miss it

Diagnosing it

Fixing it

The design lesson for the rollback

A collector's next delegates to an inner iterator and a 6,800-row telemetry batch stops at row 900 with no error. What happened, and how do you fix it?

answer

go deeper

context