skip to content

In pandas, a `df.apply(func, axis=1)` whose function returns a tuple yields one column of tuples; how do you get two real columns instead?

level: seniorimportance: should knowfreq 38%

answer

  1. list-likes stay packed by default
  2. a returned Series expands
  3. result_type only on axis=1
  4. expand, reduce, broadcast
  5. two expressions beat one apply

basics

~20 s

With the default result_type=None, a function returning a tuple or list gives a Series of those objects. Pass result_type="expand" to spread them into columns, or return a Series whose index becomes the column names; better still, compute each column vectorized.

solid answer

~30 s

`apply(axis=1)` infers the result shape from what the function returns. A tuple or list is list-like, and with `result_type=None` pandas keeps it packed, so I get one object-dtype Series of tuples. `result_type="expand"` turns each list-like into columns labelled 0, 1, ..., which I can assign straight into `df[["dti", "prime"]]`. Returning `pd.Series({"dti": ..., "prime": ...})` expands too, and names the columns. `"reduce"` keeps list-likes packed in a Series, and `"broadcast"` keeps the frame's original shape and labels. In review I would replace the whole apply with two column expressions, because both outputs are plain arithmetic.

code

python · 25 lines
python
import pandas as pd

loans = pd.DataFrame({
    "income": [52000, 90000, 31000],
    "debt": [26000, 18000, 3100],
    "score": [610, 720, 680],
})


def ratios(row):
    return row["debt"] / row["income"], row["score"] >= 650


packed = loans.apply(ratios, axis=1)
print(packed.dtype)  # object: a Series of tuples

wide = loans.apply(ratios, axis=1, result_type="expand")
print(list(wide.columns))  # [0, 1]

loans[["dti", "prime"]] = wide
vectorized = loans.assign(
    dti=loans["debt"] / loans["income"],
    prime=loans["score"] >= 650,
)
print(vectorized.dtypes)

go deeper

for a junior

Recall that apply with axis=1 calls the function once per row and that returning a tuple does not create two columns.

for a middle

Explain the four result_type values, that they only act on axis=1, and how a returned Series expands into named columns.

for a senior

Diagnose an object column of tuples on sight, fix it with expand or a Series return, then remove the apply when the outputs are column arithmetic.

for a principal

Set the review expectation that row-wise apply needs a stated reason, such as a one-record external call, and that derived columns default to column expressions.

## How `apply(axis=1)` decides the shape of its result `DataFrame.apply(func, axis=1)` calls `func` once per row, passing the row as a Series indexed by the column names. pandas then has to assemble the returned values into a result. The `apply` docstring states the rule: with `result_type=None`, **list-like results are returned as a Series of those**, but if the function returns a **Series, those are expanded to columns**. The return type is inferred from the first result the function produces. So a function that returns two derived values as a tuple does not give two columns: ```python def ratios(row): return row["debt"] / row["income"], row["score"] >= 650 loans.apply(ratios, axis=1) # one Series of tuples, dtype object ``` Trying to assign that to two columns is where people get stuck, and unpacking the tuples afterwards is a second slow pass. ## The four `result_type` values `result_type` only acts when `axis=1`. | `result_type` | Function returns a list-like | Function returns a Series | |---|---|---| | `None` (default) | a Series of list-likes | a DataFrame, columns from the Series index | | `"expand"` | a DataFrame, columns `0, 1, ...` | a DataFrame, columns from the Series index | | `"reduce"` | a Series of list-likes | still a DataFrame: the returned Series is expanded | | `"broadcast"` | a DataFrame with the original index and columns | a DataFrame with the original index and columns | Notes on each: - **`"expand"`** is the direct fix for the tuple case. Its columns are numbered, so either rename them or assign by position: `loans[["dti", "prime"]] = loans.apply(ratios, axis=1, result_type="expand")`. - **Returning a Series** from the function, such as `pd.Series({"dti": ..., "prime": ...})`, expands even with the default and gives named columns. The price is building one extra Series per row. - **`"reduce"`** is the opposite of expand for list-likes: it keeps a Series of tuples or lists. A function that returns a Series is still expanded into columns. - **`"broadcast"`** forces the result back onto the input frame's shape, keeping its index and columns, so the function's output length must fit the number of columns. ## Why the column-wise answer is usually better Every variant above still calls Python once per row and builds a row Series for each call. When the outputs are arithmetic or comparisons, two column expressions do the same job without any per-row call: ```python loans = loans.assign( dti=loans["debt"] / loans["income"], prime=loans["score"] >= 650, ) ``` This also gives each new column its natural dtype directly: float64 for the ratio and bool for the flag. With `apply`, each row arrives as one Series with a single dtype, so a frame that mixes text and numbers hands the function `object` rows, and the expanded columns' dtypes depend on that inference. When per-row Python really is needed, for example a call to an external scoring function that takes one record, `apply(axis=1, result_type="expand")` is still preferable to `iterrows()` and appending to lists, because pandas assembles the frame once. ## Related arguments worth knowing 1. **`raw=True`** passes each row as a NumPy array instead of a Series. It skips building row Series, but you lose access by column name. 2. **`args=` and keyword arguments** are forwarded to the function, which avoids wrapping it in a lambda for a threshold parameter. 3. **`axis=0`**, the default, passes each **column** instead; a reduction there returns one value per column, and `result_type` has no effect. ## When the per-row call has to stay Some derivations really are per record: a pricing or scoring function from another team that takes one applicant and returns several figures. Then keep the call, but let pandas do the assembly: - Pass fixed parameters with `args=(...)` or keywords instead of a closure, which keeps the function testable on its own. - Ask for `result_type="expand"` and rename the numbered columns once, or have the function return a Series with the final names. - Assign the expanded frame in one step, never row by row inside a loop. ## Diagnosing the tuple column in review - A column printed as `(0.5, False)` with dtype `object` is the signature of a list-like return with the default `result_type`. - Fix it at the source with `result_type="expand"` or a Series return, not by splitting the tuples afterwards. - Then ask whether the apply is needed at all: if both values come from column arithmetic, replace it with two expressions.

  • Does `result_type="expand"` change anything for `df.apply(func)` with the default `axis=0`?
    No. The docstring says `result_type` only acts when `axis=1`. With `axis=0` the function receives each column, and pandas shapes the result from what it returns: a scalar per column gives a Series indexed by the column names.
  • What does `raw=True` change in `apply(axis=1)`?
    Each row reaches the function as a NumPy array instead of a Series, which avoids building a Series per row and is faster for numeric reductions. The function can no longer index by column name, and the result dtype is inferred from the first returned value.

  • Expects a returned tuple to become separate columns by default
  • Uses result_type with axis=0 and expects it to apply
  • Splits a tuple column afterwards instead of expanding at the source
  • Treats apply with result_type="expand" as vectorized
  • Assumes expanded columns are named after the tuple's variables