Why is `s += part` in a Python loop unsafe to rely on for speed?

## The naive cost model Strings are immutable, so `s = s + part` cannot extend anything. It must allocate a buffer big enough for both operands and copy both in. Run that in a loop that appends n equal-sized pieces and the i-th iteration copies roughly i pieces' worth of characters, so the total work is proportional to 1 + 2 + ... + n — quadratic in the final length. Doubling the input quadruples the runtime. On modest data nobody notices; on a payload built from tens of thousands of fragments it turns a millisecond job into a multi-second one. ## What CPython actually does CPython contains a well-known optimization for exactly this shape. Its evaluation loop recognises an in-place add of two `str` operands where the target is a plain local name and the very next thing that happens is storing the result back into that same local. If, at that instant, the accumulated string's reference count is 1 — meaning the interpreter itself holds the only reference to it — nothing can observe the object changing, so the interpreter is free to resize the existing buffer and append into it. The observable value is identical; only the cost differs. That turns the loop from quadratic into something close to linear, which is why a quick timing experiment often fails to reproduce the textbook disaster. Since 3.11 and the specializing adaptive interpreter (PEP 659) this lives as a specialized fast path chosen at runtime for the concatenation bytecode; it is still present in 3.14. But note what it depends on: CPython's reference counting and a very specific instruction pattern. It has never been part of the language definition, no alternative Python implementation promises it, and the CPython documentation explicitly warns against depending on it. ## Everything that quietly turns it off This is the part interviewers are probing. The fast path evaporates when: * **Anything else holds a reference.** Appending the running string to a list for debugging, capturing it in a closure, passing it to a function that stores it, or even holding it in a second name lifts the reference count above 1. * **The target is not a plain local.** `self.buffer += chunk`, `d['key'] += chunk` or a global target do not match the pattern, because the result is not stored straight back into a local slot. These always build a new string. * **You are not on CPython.** Another implementation is free to do the naive thing, and typically does. The failure mode is the nastiest kind: nothing breaks, no exception is raised, no test fails. A refactor that adds one innocuous line reintroduces quadratic behaviour, and it only shows up as latency on the largest inputs — exactly the inputs you have least coverage for. ```python s = "" kept = [] for part in parts: kept.append(s) # this line alone makes the loop quadratic s += part ``` ## What to do instead Collect the pieces and join once: ```python parts = [] for chunk in source: parts.append(chunk) payload = "".join(parts) ``` `str.join` walks the sequence once to total the lengths and determine the widest character width needed, allocates the result exactly once, then copies each piece into place. Total work is proportional to the final length, and that is a documented property rather than an accident of the interpreter. `list.append` is amortized constant time, so the collection phase is linear too. When the code is shaped like a series of writes rather than a list build — a function that hands a writer to helpers — use `io.StringIO` and call `getvalue()` at the end. When the data is octets rather than text, accumulate into a `bytearray` with `+=` or `extend()`; that type *is* mutable, so its append really is an append, and you convert with `bytes(buf)` at the end. ## Proving it in an interview Do not argue from theory alone; measure the scaling. Time the loop at n, 2n and 4n pieces. Linear code doubles; quadratic code quadruples. Do it once with a bare local target and once with a second live reference, and the difference between the guaranteed algorithm and the interpreter's courtesy optimization becomes visible in a couple of seconds. ## The judgement to state out loud The strongest answer is not "`+=` is slow" — often it is not. It is: *the fast case is an unguaranteed CPython optimization that depends on a refcount you do not control, so I write the version whose cost is guaranteed.* Reserve `+=` for joining two or three fragments where clarity wins; use `"".join()` or a `io.StringIO` for anything built in a loop.