How does the Iterator pattern change when the sequence is resource-backed or remote — a database cursor, a file, or a paginated HTTP API — rather than an in-memory collection?
answer
- same interface, real I/O per step
- iterator owns a resource → must close, even on break
- single-pass; failure can strike mid-stream
- batch/prefetch or suffer N+1 round trips
- offset drifts; keyset + opaque token = portable cursor
basics
~20 sThe loop looks identical, but each next() may do real I/O. So the iterator now owns a resource that must be closed, can fail halfway through, is usually single-pass, and hides per-element cost — a plain-looking loop can issue thousands of network or disk requests.
solid answer
~50 sA resource-backed iterator keeps the same `hasNext`/`next` contract but breaks four assumptions clients make about collections: cheapness, totality, repeatability and infallibility. Design consequences: (1) **lifetime** — the iterator holds a file handle, socket, or server-side cursor (often pinning a transaction/snapshot), so it must be closeable and used inside a scoped block; abandoning a loop early must still release it. (2) **failure mid-stream** — `next()` can throw after partial consumption, so callers need idempotent processing or checkpointing, not all-or-nothing assumptions. (3) **single-pass** — re-iterating means re-querying; document it or materialize. (4) **batching** — fetch in pages/chunks behind the cursor so per-element cost amortizes, with a tunable size trading memory against round-trips. (5) **paging stability** — offset paging drifts under concurrent inserts/deletes (duplicates and skips); keyset/seek paging with a stable sort key plus an opaque continuation token is the robust form, and that token is simply externalized traversal state.
code
pseudocode · 16 lines// Resource-backed: closeable, batched, scoped
use(db.streamRows("select ... order by id", fetchSize = 500)) { rows ->
while (rows.hasNext()) { process(rows.next()) } // close guaranteed on break/throw
}
// Remote pages behind an iterator: traversal state = the token
class PageIterator(first: Token?) : Iterator<Item> {
var buf = emptyList<Item>(); var i = 0; var token = first; var more = true
hasNext() = i < buf.size || more
next(): Item {
if (i == buf.size) { val p = api.list(after = token); buf = p.items; token = p.next; more = p.next != null; i = 0 }
return buf[i++]
}
}
// offset=200&limit=50 -> duplicates/skips when rows are inserted or deleted concurrently
// after=<opaque token> -> stable, resumable, index-seekablego deeper
Note that each next() may hit disk or network, so the iterator must be closed and the loop is usually single-pass.
Add batching/fetch size to avoid N+1 round trips, scoped closing that survives break, and the fact that failures can occur mid-traversal.
Discuss offset vs keyset pagination and its consistency anomalies, opaque continuation tokens as externalized cursor state, checkpointing plus idempotent processing, and open-transaction cost.
Treat the traversal contract as a long-lived public API decision: snapshot vs live view, token opacity and expiry, server-side page ceilings, resumability across nodes, and when to switch to a push/streaming protocol with explicit backpressure.
## The abstraction still holds — and that is the risk `while (it.hasNext()) process(it.next())` reads the same whether the source is an in-memory list or 40 million rows across a network. That uniformity is Iterator's whole point, but it also **hides cost and failure**. Four assumptions that hold for collections break for resource-backed sequences: | Assumption for a collection | Reality for a resource-backed iterator | |---|---| | `next()` is cheap and non-blocking | It may perform disk or network I/O, block, and time out | | The whole sequence is available | It is produced incrementally; the tail may not exist yet | | You can iterate repeatedly | Usually single-pass; a second pass re-executes the query | | Iteration cannot fail | Any step can throw (connection reset, timeout, expired cursor) | ## 1. Resource lifetime The iterator now *owns* something: a file descriptor, a socket, a server-side cursor, often a database transaction or MVCC snapshot held open so the result set stays consistent. Consequences: - The iterator must be **closeable/disposable**, and clients must use a scoped construct (`try-with-resources`, `using`, `with`, `defer`, RAII) — including on the **early-exit path**, which is where leaks actually happen (`break`/`return`/exception out of the loop). - Long-open cursors have systemic cost: they pin a transaction, block vacuum/cleanup of old row versions, hold connection-pool slots, and can be killed by server-side idle timeouts mid-traversal. - Because internal iteration owns the loop, a `forEachRow(query) { ... }`-style API can *guarantee* the close in a `finally`. This is one of the strongest arguments for offering an internal-iteration entry point over a raw external cursor for resource-backed data. - Finalizer/GC-based cleanup is not a substitute: closing is deterministic behaviour the API must require. ## 2. Failure semantics With a collection, a loop either runs or doesn't. With I/O, `next()` can throw after 10,000 of 1,000,000 elements. So the client must decide: - **Checkpointing / resumability** — record how far you got (the continuation token) so a retry resumes rather than restarts. - **Idempotent processing** — if a retry re-delivers already-processed elements, side effects must tolerate duplicates (at-least-once processing is the realistic guarantee). - **Retry placement** — retrying `next()` transparently inside the iterator is convenient but must not silently reorder or duplicate elements; it is usually better surfaced. - **Partial-result exposure** — an API that returns "a list" implicitly promises all-or-nothing; one that returns a cursor honestly exposes partiality. ## 3. Batching, laziness and prefetch A naive remote iterator that fetches one element per `next()` produces the N+1 problem: an innocuous loop becomes N round trips, and latency dominates. The standard fix is to make batching an implementation detail of the iterator: - fetch a **page/chunk** of size K per round trip and serve K elements from memory; - K trades memory and tail latency against round trips; make it configurable, and note that JDBC-style `fetchSize`, cursor `ARRAY SIZE`, and API `page_size` are all the same knob; - **prefetch/read-ahead** (fetch page N+1 while the consumer processes page N) hides latency but wastes work when the consumer stops early; - laziness also means the sequence can be **unbounded** (a tailing log, a change feed) where `hasNext()` may block rather than answer. ## 4. Pagination as externalized traversal state For a stateless HTTP service, the server cannot hold a cursor per client, so the traversal state is handed to the client as a token. Two families: - **Offset/limit** (`?offset=200&limit=50`): simple, allows random page access, but is **unstable under concurrent writes** — an insert before your offset shifts everything right and you see an element twice; a delete shifts left and you skip one. It is also increasingly expensive on the server (the database must count and discard offset rows). - **Keyset / seek pagination** (`?after=<last_sort_key>`), usually wrapped in an **opaque continuation token**: stable under inserts/deletes, O(index seek) rather than O(offset), and the token can encode the sort key, filter set, and a snapshot/version id. Opaqueness matters — it lets you change the encoding later and stops clients from fabricating positions. Trade-offs: no random page jumps, requires a total, stable sort key (add a tiebreaker like an id to a non-unique sort column), and tokens need an expiry story. A continuation token is literally an iterator's position serialized and shipped across a boundary — the same concept that lets a client resume tomorrow on a different server. ## 5. API design guidance - **Say what you return.** A `List` promises materialized, repeatable, bounded. A cursor/stream promises lazy, single-pass, closeable. Do not return the second where callers expect the first. - **Bound the damage.** Offer a max page size and enforce a server-side ceiling; unbounded `limit` invites accidental full-table scans. - **Make cost visible.** Name things `streamRows`, `openCursor`, `pages()` rather than `getAll()`, so reviewers see the I/O. - **Consider push for high-throughput firehoses**, but then design explicit backpressure (a demand/credit protocol) since the consumer no longer paces the producer. - **Ordering and consistency:** state whether the sequence is a consistent snapshot (server-side cursor in a transaction) or a live view that may include items added mid-traversal (typical for keyset paging over a live table).
- Why does offset-based pagination produce duplicates and skipped records, and what fixes it?Offset counts positions in a result set that is recomputed per request. An insert before your offset shifts every later row right, so page N+1 repeats a row; a delete shifts left and one row is never returned. Keyset/seek pagination — "give me rows after this sort key" behind an opaque token — anchors to data instead of position, needs a total stable sort order (add an id tiebreaker), and is also cheaper server-side.
- A caller breaks out of a loop over a database-backed iterator on the first match. What must the API guarantee?That the cursor, statement, connection and any open transaction are released on the early-exit path. Either the iterator is closeable and the client uses a scoped block, or the library owns the loop internally and closes in a `finally`. Relying on the garbage collector or a finalizer is not deterministic and leaks pool connections under load.
- Where does batching belong — in the client loop or inside the iterator?Inside the iterator, as a tunable fetch/page size. That keeps the one-element-at-a-time contract clients like while amortizing round trips, and it lets you tune memory versus latency in one place. Exposing batches to clients is worth it only when they can genuinely process a batch more efficiently (bulk insert, vectorized work).
An in-memory iterator is walking along a shelf you already own. A remote iterator is phoning a warehouse for each item: same gesture, but every step costs a call, the line can drop halfway, and if you walk away without hanging up, the clerk stands there holding the line open.
saying these in an interview costs you the question
- Returning a lazy, single-pass, resource-backed sequence from a method whose name and type suggest a materialized list, then being surprised by leaks or empty second passes.
- Assuming a plain-looking `for` loop is cheap when each step is a network call — the N+1 problem hiding behind a clean abstraction.
- Using offset/limit paging over data that changes concurrently and calling the result complete.
- Relying on garbage collection or finalizers to close cursors and connections instead of scoped, deterministic close.
- Treating an interrupted traversal as all-or-nothing; without checkpointing and idempotent processing, retries either lose or duplicate work.
- Holding a server-side cursor (and its transaction) open across slow client-side processing, pinning connections and blocking cleanup of old row versions.