skip to content

A route-optimisation worker runs `self.routes += batch` while another thread reassigns `self.routes`; what breaks?

level: seniorimportance: should knowfreq 30%

answer

  1. Count the steps in that line
  2. Load, operate, store back
  3. Something can land between them
  4. One method call avoids the write-back

basics

~20 s

self.routes += batch is three steps: load the attribute, extend the list in place, store it back. A concurrent self.routes = [] landing between them is silently undone, because the store rebinds the attribute to the old, now-extended list.

solid answer

~50 s

Augmented assignment on an attribute is a read-modify-write, not an atomic operation. The interpreter loads `self.routes`, runs `list.__iadd__` on the object it got, and then stores that object back into the attribute. If another thread rebinds `self.routes` to a fresh list after the load, the store puts the old object back and the reassignment vanishes — a classic lost update, and one that also loses whatever the other thread appended to the new list. The fix at the language level is to remove the store: `self.routes.extend(batch)` is a single method call on the object, so there is nothing to write back. That still does not give you a multi-step invariant — if you need "drain and replace" to be one unit, take a `threading.Lock` — but it eliminates this specific window. A single-threaded regression pack will never reproduce it, because with one thread the store back is a no-op.

code

python · 10 lines
python
class Worker:
    def __init__(self):
        self.routes = [1]

w = Worker()
old = w.routes
stale = old.__iadd__([2])   # step 2: extend in place
w.routes = []               # another thread rebinds
w.routes = stale            # step 3: the store back
print(w.routes)             # [1, 2] - the reset is gone

go deeper

for a junior

Take away the core fact: += on an attribute both changes the object and writes it back to the attribute. Knowing there is a write-back step at all is the piece most people miss, and it explains surprises far beyond threading.

for a middle

Explain the load-modify-store sequence and show that extend collapses it to a single call. Be able to say why the same line is harmless on a local name and risky on an attribute, a global or a container slot.

for a senior

Diagnose it: describe the interleaving that loses the reset, name the fix that removes the store, and state its limit — compound invariants still need a lock. Say how you would test for it, since deterministic suites never will.

for a principal

Decide the concurrency style rather than patching the line: message-passing between workers, ownership of mutable state by a single thread, or explicit locking with a documented invariant. Factor in that a free-threaded interpreter turns rare interleavings into routine ones.

### Read the line as bytecode, not as English `self.routes += batch` reads like one action. It compiles to three: **load the attribute**, **apply the in-place operation**, **store the attribute**. `dis.dis` on `o.v += [2]` shows a load-attribute instruction, the in-place binary op, and then a store-attribute instruction. Every interleaving argument about this line follows from that shape. ### The window Thread A loads `self.routes` and gets list *L*. Before A reaches its store, thread B executes `self.routes = []`, binding the attribute to a fresh list *M*, and appends its own work to *M*. Thread A now runs its store and rebinds `self.routes` back to *L*. Two things are lost: B's intent to reset, and every item B put into *M*. Nothing raised, nothing logged, and the totals are simply wrong. Note that the mutation half is not the problem. `list.__iadd__` extended *L* correctly. The damage is done by the write-back, which is the part of augmented assignment people forget exists. ### Why the regression pack was green A 340-case suite that exercises the worker single-threaded cannot fail here: with one thread the load and the store bracket nothing, and rebinding the attribute to the object it already held is a no-op. Correctness under interleaving is not a property any deterministic single-threaded case can observe. If this class of bug matters, the tests that find it are the ones that run the operation concurrently and assert on a total, not the ones that assert on a single call. ### The language-level fix ```python self.routes.extend(batch) # one call on the object; no store back ``` `list.extend` mutates the object the attribute currently points at and returns `None`, so there is no value to write anywhere. If thread B rebinds the attribute concurrently, thread A's items go into whichever list A loaded — you may still lose work, but you no longer resurrect a stale binding, which is the more destructive failure. Two other shapes are worth knowing: * **Accumulate locally, publish once.** Build the batch in a local list — where `+=` is entirely safe, because a local name is not shared — and hand it over with a single operation at the end. * **Guard the whole compound action.** If the invariant spans more than one operation ("take the current routes and replace them with an empty list"), no choice of operator saves you; that needs a `threading.Lock` or a `queue.Queue` handing work between threads. ### Do not lean on "the GIL makes it atomic" The GIL guarantees that one bytecode instruction is not torn, not that three of them run as a unit; the interpreter may switch threads between them. So the window described above exists in a normal build too. What changed recently is its size: the free-threaded build shipped as experimental in 3.13 and became officially supported in 3.14 under PEP 779, and there the threads are genuinely parallel, so read-modify-write races that used to reproduce once a week reproduce constantly. Code written against "the GIL protects me" is exactly the code that breaks when someone evaluates that build. ### The same store-back, a different symptom Remove threads and the store-back still bites: if `routes` is a read-only `property`, `self.routes += batch` extends the underlying list and *then* raises `AttributeError` because there is no setter. Same three steps, same lesson — the mutation has already landed when the store fails. ### Choosing between `+=`, `extend` and `+` * `+=` on a shared attribute: mutates plus a write-back you probably did not want. Reach for it on locals. * `.extend(...)`: mutation only. The right default for a container that other code already holds. * `+`: a new list, no mutation, no aliasing — the right default when the input must be left alone, but it is quadratic if you do it in a loop, since each step copies everything accumulated so far. ### What a senior answer sounds like Name the three steps, name the lost update, say which operator removes the store, and be explicit about the limit of that fix: it closes one window, it does not give you a transaction. Then say how you would have caught it — concurrent stress over the accumulate path, not another deterministic case in the suite.

  • Is `self.routes.extend(batch)` therefore thread-safe?
    It removes the store-back window, and the list stays internally consistent, but that is not the same as safe. Any invariant spanning two operations — read the routes, then replace them — still needs a `threading.Lock`. Treat `extend` as "one less race", not as a synchronisation primitive, and never build a design on assumed atomicity of an interpreter detail.
  • Why did a 340-case regression pack never catch this?
    Every case ran single-threaded, and with one thread `+=` on an attribute is correct: the store back rebinds the attribute to the object it already held. Interleaving-dependent faults need tests that actually interleave — run the accumulate path from several threads and assert on the total, ideally many times, rather than adding another deterministic case.
  • When would you still prefer `self.routes = self.routes + batch`?
    When you want a snapshot: `+` builds a new list so existing readers holding the old object keep a stable view. It costs a full copy, and inside a loop it is quadratic, so it suits a publish-once-per-cycle pattern rather than per-item accumulation. It also does not remove the store-back race — it only changes what the store installs.

saying these in an interview costs you the question

  • Says `x += y` is one atomic bytecode operation
  • Claims the GIL makes `+=` on shared state thread-safe
  • Treats `+=` and `extend` as identical for shared attributes
  • Thinks only integer counters suffer lost updates
  • Says a green single-threaded suite proves thread safety
  • Blames the list rather than the write-back step

context

resolving links…