skip to content

A nightly report generator built 6,800 row buffers by repeating one template list; how do you prove the rows are shared and fix it?

level: seniorimportance: should knowfreq 33%

answer

  1. Equality cannot see this defect
  2. Ask how many distinct objects exist
  3. Set of id values over the rows
  4. Outer copy duplicates pointers only
  5. Construct inside the comprehension, not before

basics

~10 s

Prove it with identity: collect id() for the rows and see one distinct value, or check rows[0] is rows[-1]. Fix it by constructing each row inside a comprehension instead of repeating one object.

solid answer

~40 s

Reach for identity, not equality — equality cannot distinguish 6,800 real rows from one row seen 6,800 times. `len({id(r) for r in rows})` collapsing to `1`, or `rows[0] is rows[6799]` returning `True`, settles it in one line; `sys.getrefcount(rows[0])` showing a count in the thousands is corroboration. The cause is that the template was evaluated once and repetition filled every slot with that one reference, so a repair pass that rewrites one row's encoding rewrites the batch. The fix is to build each row where it is used: `[make_row() for _ in range(n)]`, or `[list(template) for _ in range(n)]` if a template genuinely seeds each row. Copying the outer list is not a fix — it duplicates the pointer array while every pointer still addresses the same row.

code

python · 10 lines
python
def build_rows(n):
    template = ["", "utf-8"]
    return [template] * n

rows = build_rows(6800)
assert len(rows) == 6800
assert len({id(r) for r in rows}) == 1

rows[0][1] = "latin-1"
assert rows[6799][1] == "latin-1"

go deeper

for a junior

Recall that identity, not equality, tells you whether two names refer to the same object, and that a targeted write showing up everywhere points at one shared object rather than a data bug.

for a middle

Be able to write the proof yourself: an id set over the rows, or an identity check between the first and last row, then rebuild the rows inside a comprehension rather than repeating one template.

for a senior

An interviewer expects the full diagnosis narrative — why the value-based tests passed, why an outer-level copy is not a fix, and how you would confirm it in a live process rather than only in a notebook.

for a principal

Own the prevention: an identity invariant asserted at construction, and a policy on repetition over mutable elements, because the failure mode is silent data corruption that no equality-based test will ever catch.

### The symptom and why it misleads The reported failure is that a repair pass over a 6,800-row batch fixes an encoding mismatch on a single row and every other row changes with it. That shape — *a targeted write becomes a global write* — is almost always aliasing, and in Python the most common origin is sequence repetition over a mutable template. The misleading part is that everything before the first mutation looks correct. `len(rows)` is 6,800. Every row has the right shape and the right initial values. A test that builds the batch and asserts on its contents passes. The defect is invisible to equality by construction: a list of 6,800 references to one row compares equal, element by element, to a list of 6,800 genuinely independent rows with identical contents. ### Proving it Identity is the only tool that separates the two. ```python def build_rows(n): template = ["", "utf-8"] return [template] * n rows = build_rows(6800) assert len(rows) == 6800 assert len({id(r) for r in rows}) == 1 assert rows[0] is rows[6799] rows[0][1] = "latin-1" assert rows[6799][1] == "latin-1" ``` The `id()` set is the sharpest instrument: it answers "how many distinct objects are actually in here?" in one expression, and it scales — on a healthy batch it returns 6,800. `rows[0] is rows[-1]` is the fastest smoke test. `sys.getrefcount(rows[0])` returning a number in the thousands rather than a small number is a third confirmation, useful when you only have a REPL attached to a running process and want to check without materialising a set of 6,800 ids. One caution when using `id()` in diagnosis: an id is only unique among **live** objects, so comparing ids of objects that may already have been freed is meaningless. Inside a container that holds every row alive, it is exact. ### Fixing it Build each row where it is needed, so the constructing expression is re-evaluated per row: ```python def build_rows(n): return [["", "utf-8"] for _ in range(n)] rows = build_rows(6800) rows[0][1] = "latin-1" assert rows[6799][1] == "utf-8" assert len({id(r) for r in rows}) == 6800 ``` If a template genuinely seeds each row, copy it per row inside the comprehension — `[list(template) for _ in range(n)]` — so each slot gets its own object. If rows are nested more than one level deep, a per-row shallow copy still shares the level below it, and you either rebuild recursively or use a deep copy. ### The fixes that are not fixes **Copying the outer list.** `rows[:]`, `list(rows)` or an outer-level shallow copy duplicates the pointer array and nothing else; every duplicated pointer still addresses the same row. This is worth being explicit about in the postmortem, because it is the intuitive first patch: ```python rows = [["", "utf-8"]] * 3 outer_copy = list(rows) outer_copy[0][1] = "latin-1" assert rows[2][1] == "latin-1" assert outer_copy[0] is rows[0] ``` **Rebinding the offending row.** Assigning `rows[0] = [...]` fixes exactly one slot and leaves the other 6,799 sharing. The symptom moves rather than disappearing, which is worse than the original bug because it looks fixed. **Calling a factory outside the comprehension.** `[make_row()] * n` is the same defect wearing a helper function: the call runs once and its single result is repeated. The call must be inside the loop. ### Preventing the next one The durable protection is a test that asserts on identity rather than on values, because a value-based test cannot fail on this bug. A single assertion at construction — the number of distinct row ids equals the number of rows — pins the invariant permanently and costs one line. Alongside it, treat `* n` applied to a mutable literal, a call, or a name bound to a mutable object as a review-blocking pattern; it is mechanically greppable, and the failure it produces is silent, delayed and data-corrupting rather than an exception you would catch in staging. Finally, note where the encoding detail fits: it is only the reason someone finally looked. The batch had been carrying one shared buffer for as long as the code existed; every earlier per-row write had been corrupting the batch the same way, and nobody noticed because the writes happened to set every row to the same value anyway.

  • Why did the existing test suite pass while the batch was corrupt?
    Because the tests asserted on values. A list of 6,800 references to one row compares equal to a list of 6,800 independent rows with the same contents, so every equality assertion at construction time passes. The divergence only appears after an in-place write, which happened in a later pass. An identity assertion — distinct row ids equal to the row count — would have failed immediately.
  • Would copying the batch before the repair pass have helped?
    No, not an outer-level copy. `list(rows)` or `rows[:]` duplicates the pointer array; each copied pointer still addresses the same row object, so the repair pass mutates the same shared buffer. You would have to rebuild the inner rows — `[list(r) for r in rows]` — or, for deeper nesting, copy recursively.
  • How would you check this in a live process without building a set of 6,800 ids?
    Compare two ends directly — `rows[0] is rows[-1]` — which is O(1) and conclusive when it returns True. `sys.getrefcount(rows[0])` is a second cheap signal: a row referenced by every slot shows a count in the thousands instead of a small number. Sampling a handful of ids is enough to confirm before you commit to the full scan.

saying these in an interview costs you the question

  • Compares rows with == and concludes they are distinct
  • Copies the outer list and declares the sharing fixed
  • Rebinds the one bad row and closes the ticket
  • Blames the encoding conversion instead of the shared buffer
  • Suspects a threading race before checking object identity
  • Moves the factory call outside the comprehension anyway

context