How do itertools.chain and itertools.chain.from_iterable differ when flattening a list of lists?
answer
- Concatenation, not interleaving
- One level of flattening only
- Star-unpacking happens before the call
- from_iterable keeps the outer lazy
- Beats sum(rows, []) copying
basics
~20 sBoth yield the items of the inner iterables one level flatter, lazily. itertools.chain(*rows) unpacks the outer sequence into arguments first, so it must be finite and in memory; chain.from_iterable(rows) takes the outer iterable itself and pulls it lazily.
solid answer
~50 s`itertools.chain(a, b, c)` returns an iterator that yields everything from `a`, then `b`, then `c`, building no intermediate list. When the iterables arrive as one container, `chain(*rows)` works, but the `*` unpacking is evaluated **before** `chain` is called: the whole outer container is walked into an argument tuple, so an expensive generator of rows is fully driven up front and an unbounded one never returns. `itertools.chain.from_iterable(rows)` takes that single outer iterable as its only argument and asks for the next inner iterable only when the previous one is exhausted, so the outer may itself be a generator or unbounded. Both flatten exactly one level — a list of lists of lists comes back as a list of lists — and both treat a `str` as an iterable of characters. Against `sum(rows, [])` or repeated `+`, chain also avoids quadratic copying.
code
python · 13 linesimport itertools
rows = [[1, 2], [3], [4, 5, 6]]
print(list(itertools.chain(*rows)))
print(list(itertools.chain.from_iterable(rows)))
def blocks():
for start in range(0, 9, 3):
print("producing block", start)
yield list(range(start, start + 3))
flat = itertools.chain.from_iterable(blocks())
print(next(flat))go deeper
Be ready to say what itertools.chain does at all: it yields the items of several iterables one after another as a single stream, and it does not build a list to do it.
Explain why the star in chain(*rows) forces the outer container to be evaluated before chain even runs, and why chain.from_iterable keeps that outer iterable lazy. Know that flattening is one level deep.
Show the judgement: reach for chain.from_iterable when rows stream from files, a cursor or a generator so peak memory stays at one inner chunk, and be able to explain the quadratic cost of sum(rows, []) on a large batch.
Own the guidance for the codebase: where laziness genuinely pays versus where materialising is simpler and cheaper to debug, and how long a chained-iterator pipeline may grow before it becomes an unreadable pull chain with unclear resource lifetimes.
### What chain actually is `itertools.chain` is lazy concatenation over iterables. `chain(a, b, c)` returns an iterator that yields every item of `a`, then every item of `b`, then every item of `c`, and stops. It allocates no intermediate container: the concatenated stream exists only as a cursor into whichever inner iterable is currently being drained. That makes it the iterator equivalent of `a + b + c` for lists, except that nothing is copied and the inputs need not be sequences at all — files, generators, sets, `dict` views and range objects all chain happily. ### The two spellings Real data rarely arrives as separate variables. It arrives as one container *of* containers: a list of pages, a generator of chunks, an iterator of records each of which is itself a list of fields. Two spellings flatten that shape: ```python itertools.chain(*rows) itertools.chain.from_iterable(rows) ``` They yield identical items, and for a short in-memory list of lists either is fine. The difference is **when the outer container is consumed**. `*rows` is argument unpacking, performed by the interpreter before `chain` ever runs. Python walks `rows` to the end, collects every element into a tuple, and passes them as positional arguments. Three consequences follow. If `rows` is a generator that reads a huge file chunk by chunk, that generator is driven to completion before a single item comes out the other side. If `rows` is unbounded, the unpacking never returns — the call hangs, and it is not `chain` that is hanging. And if `rows` has a million entries you have just materialised a million-element argument tuple, which is exactly the memory you were trying to avoid. `chain.from_iterable(rows)` — a class method on `chain` — takes the outer iterable as its one argument and pulls from it lazily, requesting the next inner iterable only once the current one is exhausted. Peak extra memory is one inner chunk plus whatever the outer iterable holds open, and the outer may legitimately be infinite because nothing forces it. ### Exactly one level of flattening Neither form recurses. `chain.from_iterable([[1, [2]], [3]])` yields `1`, `[2]`, `3` — the inner list survives untouched as an item. Arbitrarily nested structures need an explicit recursive generator; chain is not a deep-flatten tool and claiming it is one is a common interview slip. The corollary that bites in production is `str`. A string is an iterable of one-character strings, so chaining a list of names yields characters, not names, and no error is raised anywhere — the bug shows up far downstream as suspiciously many one-character values. If the inner elements might be strings, wrap or filter them deliberately. ### Why not sum(rows, []) or a += loop `sum(rows, [])` builds a brand-new list at every step and copies everything accumulated so far into it, which is quadratic in the total number of items; on a large batch it is the difference between milliseconds and minutes. `sum` also refuses `str` outright with a `TypeError` that steers you to `str.join`. A nested comprehension, `[x for row in rows for x in row]`, is linear and perfectly idiomatic — the honest comparison is *comprehension when you want a list, chain when you want a stream*. Reach for `chain.from_iterable` when the flattened result is going to be consumed once, immediately, and possibly not to the end. ### Practical properties worth stating The result is an iterator, so it is one-shot: no `len()`, no indexing, and a second `for` over it yields nothing. Chaining zero iterables (`chain()`), or iterables that are all empty, simply yields nothing rather than raising. Inner iterables are not touched until the stream reaches them, which means a file opened lazily inside a generator of rows is only opened when its turn comes — useful, and also a resource-lifetime trap if the caller abandons the chain half way and expects everything to have been closed. ### Choosing between them Use `chain(a, b)` when you genuinely have a handful of named iterables. Use `chain.from_iterable(rows)` whenever the iterables come from a container or a stream — it is shorter than `chain(*rows)`, it never has to build an argument tuple, and it keeps the outer producer lazy. Treating `chain(*rows)` as the default is the mistake: it works right up until the outer iterable is large, slow or endless, and then it fails in a way that points at the wrong line.
- Why is itertools.chain.from_iterable preferred over sum(rows, []) for joining many lists?`sum` allocates a fresh list at every step and copies everything accumulated so far, which is quadratic in the total item count and shows up badly on large batches. `chain.from_iterable` yields items one at a time with no intermediate list at all. If a real list is wanted, a nested comprehension is linear and clearer than either.
- What happens if you write itertools.chain(*rows) where rows is an endless generator?The call never returns. Star-unpacking is performed by the interpreter before `chain` runs, so Python tries to drain the generator into an argument tuple and loops forever, eventually raising `MemoryError`. `chain.from_iterable(rows)` handles the same input fine because it pulls the outer iterable one element at a time.
- Does itertools.chain flatten nested lists recursively?No. It removes exactly one level: the items of each inner iterable are yielded as they are, so a nested list stays a nested list. Deep flattening needs an explicit recursive generator that checks each item, and that check must exclude `str` or it will recurse down to single characters.
chain is a conveyor that empties several bins end to end onto one belt; from_iterable is handed the whole trolley of bins and fetches the next bin only when the current one runs dry.
saying these in an interview costs you the question
- Says chain materialises all items into a list first
- Thinks chain flattens arbitrarily deep nesting
- Uses chain(*gen) on an unbounded generator
- Believes from_iterable still needs the outer sequence in memory
- Confuses chain with parallel iteration over the inputs
- Forgets that chaining strings yields characters