Elastic net has two dials, the mixing ratio and the penalty strength — what does each change?

## Two dials that do different jobs An elastic net objective is ``` objective = loss + lambda * [ alpha * sum |w_j| + (1 - alpha) * sum w_j^2 / 2 ] ``` There are two numbers to choose, and they are not interchangeable. **The mixing ratio `alpha`** sets the *character* of the penalty: what fraction of it is the absolute-value (L1) term and what fraction is the squared (L2) term. It answers "what kind of shrinkage?" At `alpha = 1` the penalty is pure L1 — maximal sparsity, exact zeros, but a solution that can be non-unique and unstable on correlated columns. At `alpha = 0` it is pure L2 — every coefficient shrunk smoothly, nothing zeroed, a strictly convex objective with a unique solution. In between you get both effects, weighted. **The overall strength `lambda`** sets *how much* penalty is applied. It answers "how hard?" At `lambda = 0` you recover the unpenalised fit; as `lambda` grows, all coefficients shrink and — with any non-zero L1 share — more of them reach exactly zero, until at large enough `lambda` only the intercept survives. ## Why you cannot tune them independently The L1 component of the penalty is `alpha * lambda` and the L2 component is `(1 - alpha) * lambda`. Both scale with the same `lambda`, so changing `alpha` at a fixed `lambda` changes *both* how much L1 pressure and how much L2 pressure the fit feels. Drop `alpha` from 1.0 to 0.1 and you have cut the sparsity-inducing pressure to a tenth while introducing a large smooth-shrinkage term. The consequence in practice: **the best `lambda` moves when `alpha` moves.** A strength tuned at a pure L1 fit is usually far too small once most of the penalty has become L2. Any honest search sweeps a full range of `lambda` *for each* `alpha` you try, rather than fixing one and tuning the other. The usual shape of the search is a coarse grid over the ratio — something like 0.1, 0.5, 0.9, 1.0 — crossed with a fine sweep of the strength for each. The reason the ratio grid can stay coarse and the strength sweep must be fine is empirical but very consistent: cross-validated error is typically **flat in `alpha` and steep in `lambda`**. Getting the strength badly wrong ruins the model; getting the ratio a little wrong usually costs a rounding error in score, though it can change the selected feature set a lot. ## What each dial buys in a real problem Consider a 50,000-column ad-context design where you want a short, defensible list of predictors. A pure L1 fit gives you one — but refit it on a different month, or a bootstrap resample, and a different set of columns comes back, because many of the columns are near-duplicates and the L1 objective has no strong preference among them. Turning the ratio down slightly, to a mostly-L1 mix with a small L2 share, changes very little about the score and a great deal about the *selection*: the added squared term makes the objective strictly convex, so there is exactly one optimum, and near-duplicate columns share weight rather than fighting over it. The selected set becomes something you can reproduce and defend. Meanwhile the strength dial is doing something completely different: it is deciding how much you believe your data. Too small and you have a high-variance fit that memorises the sample; too large and you shrink real signal toward zero and underfit. That bias–variance trade lives entirely on `lambda`; the ratio does not resolve it. ## Reading a flat surface honestly When the cross-validated error surface is nearly flat across ratios, do not over-read the argmin — the winning cell is often within noise of several others. Two defensible policies: pick the ratio whose selected feature set is most stable across folds or resamples, or pick the highest L1 share whose score is statistically indistinguishable from the best, on the grounds that it ships the smallest model. Either is better than reporting the ratio that happened to win by 0.0004. ## Common mistakes worth avoiding Fixing `lambda` from an earlier pure-L1 experiment and then scanning `alpha` is the classic error — it compares mixtures at a strength that only suited one of them. Treating the ratio as a direct sparsity knob ("0.5 means half the coefficients are gone") is the second. And reporting a single best cell without ever looking at how many coefficients survived at that cell means you know your score but not your model.