In pandas, what do the index, columns and dtype arguments of the Series and DataFrame constructors do when the data already carries labels?
answer
- labelled input versus positional input
- select by key, or name by position
- a label the data lacks
- Series from a dict keeps its keys
- one dtype for every column
basics
~20 sWith labelled data (dict keys, record keys, Series indexes), index= and columns= select and reorder by label, and an absent label gives NaN; with unlabelled arrays and lists they name positions. dtype= forces one dtype on every column.
solid answer
~40 sThe rule is labelled versus positional input. `pd.Series({"b": 2, "a": 1})` keeps the keys as its index in insertion order, and `pd.Series(d, index=["b", "z"])` **pulls out** the matching keys, gives `z` a `NaN` and therefore turns an integer Series into `float64`; `pd.Series([2, 1], index=["b", "z"])` just labels the values by position. A DataFrame works the same way: with a dict or list of dicts, `columns=` keeps only the named keys in that order, so a misspelt name silently gives an all-NaN column, or an empty frame if no name matches. With a 2-D array or a list of lists, `columns=` and `index=` name positions and a wrong length raises `ValueError`. `dtype=` takes a single dtype applied to every column and raises when a value cannot convert.
code
python · 15 linesimport pandas as pd
prices = {"b": 2, "a": 1, "c": 3}
print(pd.Series(prices).index.tolist())
print(pd.Series(prices, index=["b", "z"]).to_dict())
print(pd.Series([2, 1], index=["b", "z"]).to_dict())
records = [{"order_id": "A1", "qty": 2}, {"order_id": "A2", "qty": 1}]
print(pd.DataFrame(records, columns=["order_id", "Qty"]))
print(pd.DataFrame({"qty": [1, 2]}, columns=["Qty"]).shape)
try:
pd.DataFrame({"a": [1, 2], "b": ["x", "y"]}, dtype="int64")
except ValueError as exc:
print("ValueError:", exc)go deeper
Recall that a Series built from a dict uses the keys as its index, and that columns= on records picks keys rather than renaming them.
Explain the labelled-versus-positional rule, why a missing label gives NaN and float64, and why dtype= is one type for all columns.
Show how you catch a silent all-NaN column from a misspelt name, and build mixed-dtype frames from typed Series instead of dtype=.
Consider whether shared loaders should assert expected columns after construction, since the constructor itself never flags a missing key.
## Labelled input versus positional input Every pandas constructor takes optional `index=` (row labels), `columns=` (column labels, DataFrame only) and `dtype=` arguments. What `index=` and `columns=` **do** depends on whether the data you pass already carries labels: - **labelled input** has its own keys: a dict, a list of dicts, a dict of `Series`, a `Series`. Here the arguments **select and reorder** by label. - **positional input** has no keys: a NumPy array, a list of lists, a list of tuples, a plain list. Here the arguments **name** the positions, one label per position. Most construction surprises come from forgetting which of the two you passed. ## Series from a dict A `Series` is a one-dimensional labelled array. Built from a dict, it **keeps the keys as its index**, in the dict's insertion order: - `pd.Series({"b": 2, "a": 1, "c": 3})` has index `["b", "a", "c"]`, not a sorted one. - `pd.Series(d, index=["b", "c", "z"])` pulls out the values whose keys match those labels; `z`, absent from the dict, gets `NaN`. Because `int64` cannot hold `NaN`, an integer Series becomes `float64`. A missing key raises nothing. - `pd.Series([2, 1, 3], index=["b", "c", "z"])` is positional: the first value gets `b`, the second `c`, the third `z`, and a length mismatch raises `ValueError: Length of values (3) does not match length of index (2)`. - `pd.Series(5.0, index=["a", "b"])` repeats a scalar across the given labels. ## DataFrame: columns= and index= on each input | Input | `columns=` | `index=` | |---|---|---| | dict of lists | selects and orders keys | names rows; length must match | | list of dicts | selects and orders keys | names rows; length must match | | dict of Series or of dicts | selects and orders keys | selects rows by label | | 2-D array, list of lists or tuples | names columns; count must match | names rows; count must match | Some consequences worth knowing: 1. **A typo is silent with labelled input.** `pd.DataFrame(records, columns=["order_id", "Qty"])` keeps `order_id` and adds an all-NaN `Qty`, because the records carry `qty`. If **no** requested name matches a dict of lists, the result is an empty frame with those column labels and zero rows. 2. **A count mismatch is loud with positional input.** `pd.DataFrame(np.zeros((3, 2)), index=["a", "b"])` raises `ValueError: Shape of passed values is (3, 2), indices imply (2, 2)`, and three names for two columns of a list of lists raise `ValueError: 3 columns passed, passed data had 2 columns`. 3. **A dict of Series is aligned.** Each Series brings its own index, and the frame's row labels are the union of them, with `NaN` where a Series lacks a label; passing `index=` then keeps only the listed labels. How labels line up is the subject of the lesson on axis labels and alignment. ## dtype=: one type for everything `dtype=` on the DataFrame constructor accepts **a single dtype** and applies it to every column. It is useful for a homogeneous block, such as `pd.DataFrame(matrix, dtype="float32")`, and it fails loudly otherwise: - `pd.DataFrame({"a": [1, 2], "b": ["x", "y"]}, dtype="int64")` raises `ValueError: invalid literal for int() with base 10: 'x'`. - `pd.DataFrame({"a": [1, 2], "b": ["1", "2"]}, dtype="int64")` succeeds and converts the numeric strings. - It does not accept a per-column mapping; for mixed types, build each column as a `Series` with its own dtype and put them in a dict, as in `{"qty": pd.Series(q, dtype="Int64"), "price": pd.Series(p, dtype="float64")}`. Converting columns after the frame exists, with `astype` and its dict form, is the lesson on casting. ## A checklist for the constructor call - Know whether your input is labelled; if it is, treat `columns=` as a **filter**, not as names. - After building, compare `df.columns` with what you asked for and look for unexpected all-NaN columns in `df.info()`. - Use `index=` on a dict of lists to label rows, but expect a `ValueError` if the count differs from the list length. - Reach for `dtype=` only when one type truly fits every column. An interviewer who asks this is checking that you will not "rename" a column with `columns=` and lose its data, and that you can explain why `pd.Series(d, index=...)` turned integers into floats.
- How do you rename columns at construction time instead of selecting them?With labelled input you cannot do it through `columns=`, because that argument filters by key. Rename the keys before building, or build first and call `rename(columns={...})` or `set_axis`. With positional input such as a list of lists, `columns=` does name the columns, since there are no keys to match.
- Why does pd.DataFrame({'qty': [1, 2]}, columns=['Qty']) have zero rows rather than two rows of NaN?With a dict of lists, the row count comes from the selected columns. No key matches `Qty`, so no column contributes a length, and pandas builds an empty frame with the requested label. If one requested key matches, its length sets the rows. A list of dicts differs: each record is a row, so the same call on two records gives two rows of NaN.
- columns= always renames the columns of the input data
- pd.Series(d, index=...) raises KeyError for a label not in the dict
- A Series built from a dict sorts its keys
- dtype= on the DataFrame constructor accepts a per-column dict
- A wrong-length index on a NumPy array is padded with NaN