In pandas 3.0, why does `customers[customers['balance'] < 0]['status'] = 'review'` leave `customers` unchanged, and what is ChainedAssignmentError?
answer
- two indexing steps, one statement
- the temporary is thrown away
- a warning, not an exception
- reference counts, named subsets
basics
~20 sThe first indexing step returns a temporary that behaves as a copy; the value lands in it and is discarded. pandas 3.0 flags this with ChainedAssignmentError, a warning, not an exception; assign in one loc step instead.
solid answer
~40 sThe line runs as two operations: `customers[mask]` builds a temporary DataFrame, then `['status'] = 'review'` writes into that temporary, which is thrown away. In pandas 3.0, under Copy-on-Write, an indexing result always behaves as a copy, so chained assignment never updates the original. pandas detects the pattern through the temporary's reference count and emits `ChainedAssignmentError`, which is a `Warning` subclass, so the code keeps running. The same warning covers inplace methods on a column, like `customers['status'].fillna('active', inplace=True)`. The fix is one statement targeting the frame, such as `customers.loc[mask, 'status'] = 'review'`. Detection misses writes into a named subset, which change only that subset.
code
python · 18 linesimport warnings
import pandas as pd
from pandas.errors import ChainedAssignmentError
customers = pd.DataFrame(
{"balance": [120.0, -40.0, 15.0], "status": ["active", "active", "active"]}
)
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
customers[customers["balance"] < 0]["status"] = "review" # two steps
print(customers["status"].tolist()) # ['active', 'active', 'active']
print(any(w.category is ChainedAssignmentError for w in caught)) # True
customers.loc[customers["balance"] < 0, "status"] = "review" # one step
print(customers["status"].tolist()) # ['active', 'review', 'active']go deeper
Recall that two indexing steps in one assignment write into a throwaway object in pandas 3.0, and that one loc assignment is the fix.
Explain getitem then setitem on a temporary, and that ChainedAssignmentError is a Warning subclass, so the program keeps running with unchanged data.
Show you know the detection's blind spots, named subsets and PyPy, and how you would turn the warning into a test failure during an upgrade.
Treat the warning as a migration gate: decide whether CI fails on it and how to find the silent named-subset writes it cannot see.
## What the statement really does Chained assignment means updating a frame through **two indexing steps in one statement**: ```python customers[customers["balance"] < 0]["status"] = "review" ``` Python runs this as two operations. First a **getitem**, `customers[customers["balance"] < 0]`, builds a temporary DataFrame holding the matching rows. Then a **setitem**, `temp["status"] = "review"`, writes into that temporary. The temporary is never bound to a name, so it is discarded at the end of the line, and with it the only object that received the new value. In pandas 3.0 every indexing result behaves as a copy under Copy-on-Write, so there is no case in which the temporary writes through to `customers`. The release notes say chained assignment "will stop working", and the user guide says it will "consistently never work". ## What ChainedAssignmentError is Because the statement silently does nothing, pandas tries to detect it and emits **`pandas.errors.ChainedAssignmentError`**. Despite the name, it is a **`Warning` subclass**, not an exception: - the statement completes and the program keeps running, so it is easy to scroll past in a notebook; - its message says the value is being set on a copy "through chained assignment", that this "never works to update the original", and suggests a single-step `.loc[row_indexer, col_indexer] = value` instead; - a second message covers **inplace methods** called on a column, such as `customers["status"].fillna("active", inplace=True)`, and suggests `customers.fillna({"status": "active"}, inplace=True)` or `customers["status"] = customers["status"].fillna("active")`. The single-step `loc` assignment is the fix; its forms are covered in the lesson on assigning values. ## How it differed in pandas 2.x | Statement | pandas 2.x | pandas 3.0 | |---|---|---| | `df[mask]["b"] = v` | did not update `df`; the classic `SettingWithCopyWarning` case | does not update `df`; `ChainedAssignmentError` | | `df["b"][mask] = v` | often updated `df` | does not update `df`; `ChainedAssignmentError` | | `df["b"].fillna(0, inplace=True)` | often updated `df` | does not update `df`; `ChainedAssignmentError` | | `df.loc[mask, "b"] = v` | updates `df` | updates `df` | The mask-first form was already broken in 2.x, because a boolean selection builds new arrays. The column-first forms are the dangerous ones in a migration: code that worked in 2.x stops updating anything in 3.0. `SettingWithCopyWarning` itself is removed in pandas 3.0, because there is no longer an ambiguous case to warn about. ## The limits of detection pandas spots chained assignment by looking at the **reference count** of the object being written into: a temporary created inside the same statement has fewer references than a named object. That design has blind spots worth knowing: 1. **Named intermediates are not chained assignment.** `negatives = customers[mask]` followed by `negatives["status"] = "review"` is a legitimate write to `negatives`. No warning fires, and `customers` is unchanged. If the author expected `customers` to change, the bug is silent. 2. **The check is switched off on PyPy**, where reference counts do not work the same way. 3. **It is a warning**, so the default warnings filter can show it once per location, and logging setups can swallow it. A test suite can turn it into a failure with a warnings filter whose action is `"error"` for `ChainedAssignmentError`, which makes the column-first 2.x forms impossible to merge. ## Diagnosing a dead assignment When a cleaning step "runs but changes nothing" after an upgrade: - look for two `[]`, `loc` or `iloc` steps on the left of one `=`; - look for `inplace=True` on a method called on `df[col]`; - look for a named subset that is modified and then never assigned back. Rewrite each as one statement whose target is the frame you want to change. ## A worked migration case A 2.x cleaning notebook flags customers in debt with `customers["status"][customers["balance"] < 0] = "review"`. On 2.x the flag appeared, so downstream reports filtered on it. On pandas 3.0 the same cell prints a `ChainedAssignmentError` warning and the status column stays `"active"` everywhere; the report downstream quietly shows no customers to review. Nothing crashed, which is exactly why reading the warning matters. The one-line fix is `customers.loc[customers["balance"] < 0, "status"] = "review"`, and the same cell then behaves identically on 2.x and 3.0, which makes it safe to commit before the upgrade lands.
- Why does `negatives = customers[mask]` followed by `negatives['status'] = 'review'` raise no warning?Because it is not chained assignment: `negatives` is a named object and writing into it is legitimate, so it changes. pandas only flags writes into a temporary created in the same statement, which it detects from the reference count. `customers` stays unchanged, so if the author meant to update it, the bug is silent and only a test or a data diff will catch it.
- How do you rewrite `customers['status'].replace('active', 'open', inplace=True)` for pandas 3.0?Make the frame the target. Either call the method on the frame with a per-column mapping, `customers.replace({'status': {'active': 'open'}}, inplace=True)`, or drop `inplace` and assign the result back: `customers['status'] = customers['status'].replace('active', 'open')`. The original line only changes a temporary column and emits ChainedAssignmentError.
- How can a test suite make sure no chained assignment slips through?Turn the warning into a failure, for example with a warnings filter whose action is 'error' for `pandas.errors.ChainedAssignmentError` in the test configuration. Then any two-step write on an unnamed temporary fails the test. It will not catch named-subset writes or code that only runs on PyPy, where the check is disabled.
- ChainedAssignmentError is an exception, so the bad line stops the program
- df['b'][mask] = v still works in 3.0 because the column comes first
- SettingWithCopyWarning still marks chained writes in pandas 3.0
- If no warning appears, the subset write must have reached the original frame
- Calling .copy() on the temporary makes the chained write update the frame
Assumes pandas 3.0; in 2.x the column-first form often updated the frame and SettingWithCopyWarning flagged risky cases, in 3.0 chained assignment never updates and warns with ChainedAssignmentError.