Why is a list comprehension usually faster than an append loop in CPython?

### The two shapes, instruction by instruction ```python out = [] for x in xs: out.append(x * 2) ``` Every iteration of that loop does: advance the iterator, bind `x`, load the name `out`, look up the attribute `append` on it (which involves the type's lookup machinery and, without specialisation, materialising a bound method object), call it with one argument, and pop the returned `None`. The comprehension ```python [x * 2 for x in xs] ``` compiles instead to a loop whose body ends in a dedicated append instruction that acts on the partially built list already sitting on the interpreter's value stack. There is no name to load, no attribute to resolve, no call to make and no return value to discard. `dis.dis` on the two functions shows the difference directly, and it is the cleanest way to demonstrate the point in an interview. The practical effect for a trivial body is a modest speedup - a factor of well under two on modern CPython, not an order of magnitude. A useful sanity check is the classic hand optimisation: hoist the bound method out of the loop with `append = out.append` and call `append(x * 2)`. That recovers most of the difference, which tells you the cost was the repeated lookup plus the call, not anything mysterious about comprehensions. ### What is still interpreted A comprehension is not a C loop. Its body runs as ordinary bytecode, one instruction at a time, exactly like the explicit loop's body. What has moved into C is only the appending. This is the boundary that separates this idiom from something like `''.join(parts)` or `sum(data)`, where the entire iteration disappears into a single C call. Saying that comprehensions are fast because they are implemented in C is a misconception worth being able to correct. ### The 3.12 change Before CPython 3.12 a list, dict or set comprehension was compiled as a nested function: evaluating the comprehension created a function object, called it with the outer iterable, and pushed a frame. That is what gave comprehensions their own scope, so the loop variable never leaked. It also meant that for short comprehensions the frame push could be a large share of the total cost. PEP 709, shipped in 3.12, inlines those comprehensions into the enclosing function. No function object is created and no frame is pushed; the compiler instead saves and restores any enclosing local that shares a name with the iteration variable, so the isolation semantics are preserved exactly - the loop variable still does not leak. The PEP reports up to roughly a twofold improvement for comprehensions, largest on short ones where the frame overhead dominated. Two visible side effects: a comprehension no longer shows up as a separate frame in a traceback, and it no longer appears as a separate entry in profile output. Generator expressions were deliberately not inlined, since their laziness requires a real frame that outlives the expression. ### map and filter `list(map(f, xs))` moves the iteration itself into C, so it can beat a comprehension - but only when `f` is already a C-level callable, such as a built-in or an unbound method like `str.strip`. Wrap it in a lambda and each element now costs a Python function call plus the map machinery, and the comprehension wins comfortably. The rule that survives contact with real code: use `map` when you are passing a function that already exists and is implemented in C, and a comprehension otherwise. The same reasoning applies to `filter` against a comprehension with an `if` clause. ### When the difference stops mattering If the body calls a Python function, parses a string, or touches anything outside the interpreter, the per-element append overhead is a small fraction of the iteration and the two forms measure the same. That is the common case, and it is why this rewrite should be driven by clarity rather than by speed. A comprehension is the better default because it states intent - build a list from this sequence - in one expression; an explicit loop is better when the body needs several statements, error handling, or early exits. Nesting three comprehensions to avoid a readable loop is a net loss regardless of the microbenchmark. One thing both forms share: they build the whole result list in memory, so peak memory is proportional to the number of elements produced.