skip to content

In pandas 3.0, if you take `email = customers['email']` and change a value in `email`, does the `customers` DataFrame change?

level: juniorimportance: must knowfreq 55%

answer

  1. derived objects behave as copies
  2. one statement updates one object
  3. column access is indexing too
  4. write to the frame itself

basics

~20 s

No. In pandas 3.0 Copy-on-Write makes every indexing result, including a column taken with [], behave as a copy, so editing it leaves the parent DataFrame untouched; to change the frame, write to the frame itself.

solid answer

~40 s

No. In pandas 3.0 Copy-on-Write is the only mode: the result of any indexing operation, including `customers['email']`, and the output of any method behaves as if it were a copy. Writing `email.iloc[0] = 'x'` changes `email` only. In 2.x the same code could silently update `customers` too, because a column Series was often backed by the frame's own data. The rule now is that one statement updates one object, so to change the frame you write to the frame, for example `customers.loc[0, 'email'] = 'x'`, or assign the edited column back with `customers['email'] = email`. Selecting the column is still cheap, because pandas shares the data until the first write.

code

python · 17 lines
python
import pandas as pd

customers = pd.DataFrame(
    {
        "customer_id": [101, 102, 103],
        "email": ["[email protected]", "[email protected]", "[email protected]"],
    }
)

email = customers["email"]
email.iloc[0] = "[email protected]"      # changes email only

print(email.iloc[0])               # [email protected]
print(customers.loc[0, "email"])   # [email protected] - the frame is untouched

customers["email"] = email          # write back to the frame on purpose
print(customers.loc[0, "email"])   # [email protected]

go deeper

for a junior

Recall the one-line rule: in pandas 3.0 anything you get by indexing or from a method behaves as a copy, so to change a frame you write to the frame.

for a middle

Explain that the copy is lazy: the column shares the frame's data until one of them is written, which is why selecting stays cheap while edits stay separate.

for a senior

Show you can spot 2.x code that silently relied on a column edit reaching the frame, and say how you would rewrite it to assign the result back.

for a principal

Frame the rule as a predictability gain: a reviewer can now tell from one statement which object changes, which shapes the team's style for helpers and pipelines.

## The rule pandas 3.0 enforces In pandas 3.0 **Copy-on-Write** (CoW) is the only mode. Its user-facing rule has two halves, both stated in the 3.0 release notes: 1. The result of **any indexing operation** (a column taken with `customers["email"]`, a row slice, a `loc` or `iloc` selection) and of **any method that returns a new Series or DataFrame** behaves *as if* it were a copy. 2. As a consequence, the only way to change an object is to modify **that object itself**. So when you write `email = customers["email"]` and then `email.iloc[0] = "[email protected]"`, the change lands in `email` and nowhere else. `customers` keeps the old address. One statement updates one object. ## What changed from pandas 2.x Before 3.0, whether a derived object was a copy or a **view** (an object reading the same memory as its parent) depended on the exact operation, and the rules were hard to predict. Taking a single column with `[]` usually handed back a Series backed by the frame's own data, so writing into it reached the frame as well. The pandas user guide shows exactly this: under the old behaviour, `subset = df["foo"]` followed by `subset.iloc[0] = 100` changed `df` too. | Code | pandas 2.x without CoW | pandas 3.0 | |---|---|---| | `s = df["col"]; s.iloc[0] = v` | could change `df` as well | changes `s` only | | `view = df[:]; df.iloc[0, 0] = v` | `view` could see the new value | `view` keeps the old value | | `df.iloc[0, 0] = v` | changes `df` | changes `df` | The last row matters: Copy-on-Write does not stop you modifying a frame in place. It stops a write from spreading to *other* objects that were derived from it, or that it was derived from. ## How to change the frame on purpose When the goal really is to change `customers`, the statement has to target `customers`: - set the cells through the frame's own setter, for example `customers.loc[0, "email"] = "[email protected]"` (the assignment forms themselves are covered in the lesson on assigning values); - or edit the extracted column and assign it back as a whole with `customers["email"] = email`; - or build a new frame with a method and rebind the name, as in `customers = customers.assign(email=cleaned)`. ```python email = customers["email"] email.iloc[0] = "[email protected]" # only email changes customers["email"] = email # now the frame changes, on purpose ``` Each of these is a statement whose target is the frame you want changed, which is the only kind of write that reaches it. ## Why the subset is still cheap "Behaves as a copy" does not mean pandas copies the column when you select it. Under the hood the new Series shares the frame's data, and pandas keeps track of the fact that two objects share it. Only when one of them is written does pandas copy the shared data first, so the other object keeps its values. That deferred copy is the *copy-on-write* mechanism the mode is named after, and it is why selecting a column, renaming or dropping columns costs almost nothing until something is modified. ## Misreadings to avoid - **"A selected column is a live view."** That was often true in 2.x; in 3.0 it is never visible to you. - **"I need `.copy()` after every selection."** That habit mostly existed to silence `SettingWithCopyWarning`, which pandas 3.0 removed; the subset is already independent. - **"CoW forbids in-place changes."** You can still write into a frame; only the spill-over to other objects is gone. - **"The write to the subset fails."** It succeeds; it simply stays in the subset. ## Where old code breaks When reading older code, look for places that relied on the 2.x spill-over. A helper that receives `customers["email"]`, lower-cases it in place and returns nothing used to work by accident, because its Series wrote straight into the frame. In pandas 3.0 the cleaned values stay inside the helper's Series and the caller's frame is unchanged, with no error to tell you. The fix is to have the helper return the cleaned Series and assign it back to the column. Note the limit of the rule: if the helper is handed the whole `customers` frame and writes into it, Python passed it the very same object, so the caller still sees the change; CoW only separates objects that were derived from one another.

  • Does `customers.iloc[0, 1] = 'x'` still change `customers` in place under Copy-on-Write?
    Yes. Copy-on-Write only stops a write from reaching other objects. Setting a value on the frame itself through `loc`, `iloc`, `at` or `[]=` still changes that frame. If no other object shares its data, pandas writes into the existing memory; if another object, such as an earlier `customers[:]`, still shares it, pandas copies first so that the other object keeps the old values.
  • If you take `snapshot = customers[:]` and then change `customers`, does `snapshot` see the change?
    No. In pandas 3.0 the rule works in both directions: `snapshot` behaves as a copy of `customers` as it was. Because the two still share data when you write to `customers`, pandas copies before writing, so `snapshot` keeps the old values. In 2.x a slice like this could reflect later edits to the parent.

  • A column taken with [] is a live view, so edits flow back to the frame
  • pandas 3.0 eagerly copies the whole column the moment you select it
  • You still need .copy() after selecting a column to protect the frame
  • Copy-on-Write stops you from modifying a DataFrame in place at all
  • Writing to the extracted column now raises an error

Assumes pandas 3.0; in 2.x a column taken with [] could share the frame's data so edits reached it, in 3.0 Copy-on-Write makes it behave as a copy.