Why does the built-in sum() beat an equivalent Python for-loop over the same list?

### What one interpreted iteration actually costs Write the obvious loop: ```python total = 0 for x in data: total += x ``` For each element CPython executes several bytecode instructions. One advances the iterator, which is a call into the iterator object's `__next__` slot. One binds the yielded value to a local. One performs the in-place add, which for a general object goes through the number protocol and ends up in the type's addition slot. One stores the result back. Around all of that sit reference-count increments and decrements on every object touched, and the dispatch machinery of the evaluation loop itself. None of this is wasted exactly - it is what makes Python dynamic - but it is charged n times. `sum(data)` performs the identical additions and the identical iteration, except the loop counter, the iterator protocol calls and the accumulator all live in C. There is one bytecode call instruction for the whole operation. On top of that, CPython's `sum` carries fast paths: when the running total and the next item are both `int`, or both `float`, it accumulates using machine arithmetic (with overflow handling for ints) rather than allocating a new object per step. That removes most of the remaining allocation traffic. ### It is a constant factor, and that matters The distinction a good answer makes explicitly: `sum()` still visits every element, so it is still linear. Nothing about pushing a loop into C changes the number of operations - it changes the price of each one. Expect a small multiple, not an order of magnitude, and never expect it to rescue an algorithm that is doing the wrong amount of work. If a loop is slow because it is quadratic, a built-in will not save it. ### The same shape across the other built-ins - `min()` and `max()` fold in C, accept a `key` callable and a `default` for the empty case - without `default`, an empty iterable raises `ValueError`. - `any()` and `all()` fold booleans in C and short-circuit: `any` stops at the first truthy element, `all` at the first falsy one. Feeding them a generator expression means the elements after the decisive one are never produced at all. - `sorted()` runs the merge sort in C. A `key` callable is applied once per element up front - n calls, not n log n - and the comparisons then run against those precomputed keys using `__lt__`. This is why a `key` written as a Python lambda costs n Python-level calls, while `operator.itemgetter` or an unbound method keeps that work in C too. Supplying an old-style comparison function through `functools.cmp_to_key` is much worse, because then a Python call happens on every comparison. ### Where sum() is the wrong built-in Correctness first, with a caveat that dates a lot of interview advice: since CPython 3.12 `sum` applies compensated (Neumaier) summation when adding floats, so the classic demonstration that it drifts no longer reproduces on 3.12 and later. `math.fsum` is still the function that guarantees a correctly rounded total, so use it when the result is a specification rather than an estimate. `sum` refuses `str` arguments outright and tells you to use a join instead, because the quadratic concatenation trap is exactly what it would hide. Summing lists or tuples with `start=[]` is legal but builds a new sequence at every step, so it has the same quadratic character - `itertools.chain` or `list.extend` is the honest tool. And `sum` on an empty iterable returns `0`, which may or may not be the answer your caller wants. ### What the interpreter has done to narrow the gap CPython 3.11 introduced the specializing adaptive interpreter (PEP 659), which rewrites hot instructions into type-specialised forms - integer addition on two ints, for instance, stops going through the general dispatch. That materially narrowed the built-in-versus-loop gap, but it did not close it: the iterator protocol call, the store, the reference counting and the instruction fetch still happen per element. In 3.14 there is also an experimental JIT, off by default, enabled with the `PYTHON_JIT=1` environment variable and introspected with `sys._jit.is_available()`; its reported effect ranges from about ten percent slower to twenty percent faster depending on workload, so no answer should treat it as a free speedup that makes hand-written loops competitive. ### The judgement to show Swapping a loop for a built-in is worth doing when the body is trivial - an add, a comparison, a truth test - because then the overhead is the bulk of the cost. When each iteration parses a record, hits a database or calls a Python function, the per-iteration interpreter overhead is a rounding error and the rewrite buys nothing while often costing readability. Measure the isolated operation with `timeit` and locate the real hotspot with `cProfile` before rewriting anything.