In scikit-learn, which CV splitter keeps all of one patient's rows in a single fold?

## The failure being prevented When several rows describe the same underlying entity — 12 slices from one patient's scan, 40 sessions from one user, a dozen photos of one product — those rows are not independent. A row-level split puts some of an entity's rows in training and the rest in test. The model can then identify the entity rather than the phenomenon and score well for the wrong reason. The symptom is a cross-validation number that is excellent and a deployed model that is mediocre, because in production every entity is new. This is one of the most common ways a promising model dies, and it produces no warning at all — the code runs perfectly. ## Group-aware splitters All of them take a `groups` array of length `n_samples`, holding one label per row, and guarantee that no label spans the train/test boundary of a split. - **`GroupKFold(n_splits=5)`** — the K-fold analogue. It partitions the *groups* into folds and tries to balance the number of samples per fold, since groups vary in size. Every group appears in exactly one test fold. - **`GroupShuffleSplit(n_splits=..., test_size=...)`** — repeated random draws at the group level; use it when you want many independent train/test draws rather than a partition, or a specific test fraction. - **`StratifiedGroupKFold(n_splits=5)`** — keeps groups disjoint *and* tries to preserve class proportions across folds. This is what you want when the entity structure and class imbalance both matter, which in medical data is nearly always. The balance is approximate; with strongly class-correlated groups, perfect stratification is not achievable while keeping groups whole. - **`LeaveOneGroupOut()`** — one fold per group. Natural when groups are few and substantively interesting: train on four hospitals, test on the fifth, and read per-hospital generalization directly. - **`LeavePGroupsOut(n_groups=p)`** — the combinatorial generalization; the number of splits explodes quickly. ## How groups reaches the splitter This is the mechanical detail interviews probe, because it is not where people expect. `groups` is not a constructor argument. It travels through the *fit* path: ``` cv = GroupKFold(n_splits=5) for tr, te in cv.split(X, y, groups=patient_ids): ... cross_val_score(model, X, y, groups=patient_ids, cv=cv) search = GridSearchCV(model, grid, cv=cv) search.fit(X, y, groups=patient_ids) ``` Omitting it raises rather than silently degrading — a group splitter refuses to run without group labels — which is the one part of this failure mode the library does protect you from. ## Choosing what the group is The hard part is domain work, not API work. The group is whatever unit will be entirely unseen at prediction time: - Medical imaging → patient, never image or slice. - Recommenders → user, when the model must serve new users; if it only ever serves known users, a temporal split may be the honest one instead. - Documents chunked for retrieval → source document, not chunk. - Sensor fleets → device or installation, since each has its own calibration. - Augmented data → the original sample, so an image and its rotated copy cannot straddle the split. Get the unit wrong and the splitter enforces the wrong invariant with total confidence. ## Consequences to expect A correct group split usually makes the score *drop*, sometimes a lot. That is the point: the earlier number was measuring memorization. Expect higher fold-to-fold variance too, since groups differ from each other more than random rows do — and that variance is itself the useful signal about how the model behaves on an unseen entity. Group sizes are rarely equal, so folds carry different row counts and, with few large groups, different class balances. `StratifiedGroupKFold` mitigates the latter. If a handful of groups dominate the dataset, consider whether per-group weighting or capping rows per group belongs in the training procedure. ## Interaction with the rest of the workflow Group structure has to be respected at *every* holdout boundary, not just in cross-validation: the final test set must also be group-disjoint, or the leak simply moves. And any preprocessing that aggregates across rows must be fitted inside the fold, since a statistic computed over all rows carries information about the held-out groups.