How do you set a decision tree's minimum leaf size on a 900-row cohort with only 36 positive cases?

## Count events, not rows A 900-row cohort with a 4% outcome rate holds 36 positive cases. Every leaf-size decision on this dataset is really a decision about how those 36 events are divided, because the majority rows are plentiful and carry little information about where the positive region lies. This is the reframing an interviewer wants to hear. A candidate who talks about 900 rows will pick a leaf floor that sounds reasonable — 20, or 50 — and never notice that the resulting leaves contain one or two events. ## What each extreme actually does **Floor of 1.** The tree may split until every leaf is pure, isolating individual patients. Training accuracy is 100%, which is a statement about lookup, not prediction. Each leaf reports a probability of exactly 0 or 1, and those numbers have no usable uncertainty attached: a leaf built around one positive patient says "100% risk" for anyone who lands there. Change the random seed of a cross-validation split and the tree's shape changes wholesale, because a single event moving between folds re-routes an entire branch. **Floor of 50.** Now at most 900/50 = 18 leaves exist. Better — but the expected positives per leaf is 0.04 * 50 = 2. A leaf with 2 out of 50 reports 4% and a leaf with 5 out of 50 reports 10%, and the difference between those two leaves is well within what resampling would produce by chance. You have controlled tree size without making any individual leaf's estimate trustworthy. ## Sizing from the minority class backwards The usable rule is to fix the number of events you need behind a prediction and divide by the prevalence: ``` leaf_size ~= required_events / prevalence ``` Want at least 10 positives supporting each probability estimate at a 4% rate? That is 10 / 0.04 = 250 rows per leaf, so about three leaves across the whole cohort — a tree with one or two splits. This feels shockingly small, and it is the correct conclusion. Thirty-six events do not support a fifteen-leaf decision structure, whatever the training score says. Clinical prediction work has long used a similar order-of-magnitude discipline: the number of events, not the number of records, bounds how many effects you can estimate. ## The other knobs, and their limits - **Class weights.** Upweighting the minority class changes the impurity arithmetic so the tree stops ignoring the rare outcome. Combined with a minimum *weighted* fraction of samples per leaf, the floor then applies to weighted mass rather than raw rows. This is worth doing — but note carefully what it does not do: it creates no new events. The information ceiling set by 36 positives is unchanged. - **Maximum depth.** A cap of 2 or 3 is a reasonable belt-and-braces limit here, but it is blunter than the leaf floor, because a depth-3 branch carrying few rows is still allowed to make a claim about them. - **Cost-complexity pruning.** Perfectly usable, and often preferable to guessing a floor: grow a modest tree, prune it, and let cross-validated performance pick the size. On data this small, wrap the whole procedure — growth and pruning together — inside the resampling, or the pruning decision leaks the validation data. ## Validating the choice Use repeated stratified cross-validation so each fold keeps roughly 4% positives; with 36 events, unstratified folds can easily produce a fold with two positives in it. Two things are worth inspecting beyond the headline score. First, the spread of leaf probability estimates across repeats: if the same input region is assigned 5% in one repeat and 40% in another, the leaves are too small regardless of the mean score. Second, the tree structure itself: if the root split changes feature across folds, you are reading noise, and no amount of leaf-size tuning fixes it. ## The senior conclusion Sometimes the answer to "how do I set the leaf size" is "this dataset does not support a tree with interesting structure". A stump or two-split tree that a clinician can read, with honest wide uncertainty on its leaf rates, is a better deliverable than a fifteen-leaf tree tuned to a cross-validation score computed on 36 events. Saying that out loud — and proposing either collecting more events or moving to a lower-variance model — is the answer that distinguishes someone who has shipped a model on rare outcomes from someone who has only tuned hyperparameters.