Your team defaults to an RBF-kernel SVM on every tabular problem. When is that default wrong?

## Restate the question before answering it "Which kernel?" is really "what does similarity mean for these objects?" A kernel method has one assumption at its heart: points the kernel scores as similar should tend to share a label. Every kernel is a different, specific claim about that, and the default is wrong exactly when its claim is false of your data. ## What each standard kernel claims - **Linear** (`x . z`): the classes are close to separable by a flat boundary in the original coordinates. Common with wide, sparse data where there are already more features than rows and there is little left for a lift to add. It is also the only option that hands you a weight per feature, which matters when someone will ask why a decision was made. - **Polynomial** (`(x . z + c)^d`): interactions among features up to order `d` matter, globally and uniformly across the input space. Degree `d` is the prior's strength — `d = 2` says pairs of features interact, `d = 3` adds triples. Raising `d` multiplies capacity fast, and high degrees are numerically awkward because kernel values swing across orders of magnitude as the dot product moves either side of one. - **RBF** (`exp(-gamma * ||x - z||^2)`): similarity decays smoothly with Euclidean distance, and the target function is locally smooth. This is the most flexible and the least committal, which is why it is the reflexive default — and why it is also the one whose assumption is most often quietly violated. ## Where the RBF default actually breaks **Heterogeneous tabular data.** Squared Euclidean distance over a table containing an age in years, an income in currency units, three one-hot indicator blocks and an ordinal rating is a sum of incomparable quantities. Any distance you compute is dominated by whichever columns happen to have the widest numeric range, and "nearby" stops meaning anything about the domain. Trees never compute a distance at all; they threshold one feature at a time, which is why gradient-boosted ensembles are usually the stronger reflex on this shape of data. **Very high dimension with many irrelevant features.** As dimension rises, pairwise Euclidean distances between random points concentrate: the nearest and the farthest neighbour become nearly equidistant. Feed that into `exp(-gamma * ||x - z||^2)` and the kernel returns nearly the same value for every pair, so the similarity has little to discriminate with. Irrelevant features make it worse, because each one contributes to the distance without contributing to the label. A kernel has no built-in feature selection — every column enters the distance with equal weight. **Explanation requirements.** A kernel model has no weight vector in the original space to read. If the deliverable includes per-feature effects for a regulator, a clinician or a product owner, you either accept a model-agnostic explanation layer on top or you choose a family that is transparent by construction. **Operational shape.** The trained model is not a self-contained weight vector; it carries its support vectors and evaluates the kernel against them for each prediction. That is a different artefact to ship and version than a linear model's coefficients, and it is worth being deliberate about rather than discovering at deployment. ## Where kernels are the right call Structured inputs are the strongest case: sequences, strings, sets, graphs and other objects that are not naturally a fixed row of numbers. A domain-designed kernel can express "these two protein sequences share motifs" or "these two graphs have similar substructure" directly, without anyone having to invent a column set that captures it. That is the situation where the implicit feature space is genuinely doing work that no explicit table could. Small-to-moderate datasets with dense numeric features on a common scale — sensor arrays, spectra, image descriptors — are the classic tabular case where an RBF kernel earns its keep, because there Euclidean distance really does mean something. ## What to standardise instead of a kernel The defensible team-level policy is not "use kernel X" but "justify the similarity assumption". Concretely: start from a linear baseline so you can see what the lift is worth; escalate to a kernel only when the baseline's residuals show structure it cannot express; keep a tree ensemble as the parallel candidate on any heterogeneous table; and require that a chosen kernel's hyperparameters were searched on held-out data rather than inherited from a previous project, since a kernel width tuned on one feature space is meaningless on another. The failure mode a lead should be most alert to is not picking the wrong kernel — it is picking one by habit and never checking whether its similarity claim survives contact with the data.