When is itertools.chain.from_iterable a better flatten than a nested comprehension for a metrics scraper's per-endpoint sample lists?

## What the two forms actually are `[x for row in rows for x in row]` is a nested loop that appends every item to a new list, and it hands back that finished list. `itertools.chain.from_iterable(rows)` is different in kind: it returns an **iterator**. Nothing is flattened until something consumes it, and it holds only the sublist it is currently draining, not the concatenation. That difference is what decides between them, and it is easiest to see on a scraper that polls a set of endpoints and gets one list of samples back per endpoint: ```python import itertools def batches(): for endpoint in endpoints: yield collect(endpoint) # one list of samples per endpoint total = sum(1 for _ in itertools.chain.from_iterable(batches())) ``` Here `batches()` is a generator, so the comprehension form would still work, but it would hold every sample from every endpoint in one list at once. `chain.from_iterable` holds one endpoint's samples at a time. With an 83% cache-hit rate most of those per-endpoint lists are tiny or empty, so the difference is not the average batch — it is the total, which is precisely what the flat list would have had to hold. ## When to reach for each **Reach for `chain.from_iterable` when:** * the outer object is a stream, a file-backed generator, or anything you would rather not materialize; * the result is going straight into a consumer such as `sum`, `max`, `any`, `min` or a `for` loop, so the intermediate list would exist only to be thrown away; * you want concatenation and nothing else, and the C-level loop is worth the saved per-item bytecode. **Reach for the comprehension when:** * you want to transform items — `[f(x) for row in rows for x in row]` says that plainly, while the `chain` version needs a `map` wrapped around it; * you want to filter — a trailing `if` clause is clearer than composing another iterator; * the reader is better served by seeing the loops than by knowing one more `itertools` name; * you genuinely want the list, in which case the two are close enough in speed that clarity should decide. On 3.12 and later, list, set and dict comprehensions are inlined into the enclosing function rather than running in a separate frame, which narrowed the gap further; the remaining edge for `chain.from_iterable` on a plain concatenation is per-item interpreter overhead, not frame setup. ## The unpacking trap `itertools.chain(*rows)` looks equivalent and is not. The `*` unpacks the outer iterable into positional arguments **before** `chain` is called, so the outer iterable is fully consumed up front and every sublist is held simultaneously. On a generator of unknown length that erases the laziness you came for; on an unbounded one it never returns. `from_iterable` exists exactly to take the container instead of its unpacked contents. ## One level only, and the type trap Like the two-clause comprehension, `chain.from_iterable` removes exactly one level of nesting. Given an iterable of iterables of lists, you get lists back. The sharper trap is the element type, and it bites hardest where encodings are mixed. Both flatteners iterate whatever the sublists are. A `str` iterates as one-character strings, so a list of decoded metric names flattens into letters. A `bytes` object iterates as **integers**, so the same code over raw, undecoded lines flattens into a list of byte values. A scraper that decodes some responses and not others will therefore produce a mixture of `str` and `int` from identical-looking code, and the failure shows up far from the flatten — typically at a later join or comparison. Decide at the boundary whether you are handling `str` or `bytes`, and flatten only structures whose element type you have actually established. ## What a senior answer sounds like Do not present this as one being faster. State the categorical difference — a list versus an iterator, and the memory that follows from it — then give the rule: `chain.from_iterable` for pure concatenation, especially over a stream or straight into a consumer; a comprehension when the flatten also transforms or filters. Add the `chain(*rows)` warning, the one-level limit, and the `str` versus `bytes` element-type trap, and the question is fully answered.