When should you use LinearSVC or SGDClassifier instead of SVC in scikit-learn?

## Why SVC stops scaling `SVC` is a Python wrapper over libsvm. It solves the dual formulation of the support-vector problem, which is expressed entirely in terms of kernel evaluations `K(xᵢ, xⱼ)` between pairs of training points. The number of such pairs is quadratic in the sample count, and the SMO-style optimizer visits many of them repeatedly. Scikit-learn's documentation states the practical consequence directly: fit time scales at least quadratically with the number of samples and it is hard to scale beyond a few tens of thousands of samples. `cache_size` (default 200, in MB) controls how much of the kernel matrix is held in memory. Raising it to 1000 or 2000 on a machine with headroom is the cheapest real speedup available for a borderline dataset, because it reduces recomputation of kernel entries. It does not change the asymptotics — a 500,000-row `SVC` fit is not going to finish because you gave it more cache. Prediction cost is proportional to the number of support vectors, available afterwards as `n_support_` and `support_vectors_`. On noisy data with a small `C`, a large fraction of training points end up as support vectors, so inference gets slow too — a detail people miss when they benchmark training only. ## The linear alternatives **`LinearSVC`** wraps liblinear rather than libsvm. Because the kernel is fixed to linear, the problem can be solved in the primal, in terms of the weight vector, so the cost scales roughly with `n_samples * n_features` instead of quadratically. Scikit-learn's docs describe it as having more flexibility in penalties and losses and scaling better to large numbers of samples. It is the right first move whenever a linear boundary is adequate. **`SGDClassifier(loss='hinge')`** fits the same linear SVM objective with stochastic gradient descent: one sample at a time, a fixed number of epochs, cost linear in samples per epoch. It also exposes `partial_fit`, so it can learn from data that never fits in memory at once, and it accepts `loss='log_loss'` if you want a logistic model instead. The catch is that SGD is genuinely sensitive to feature scaling and to the learning-rate schedule; a badly scaled SGD fit can look much worse than the equivalent `LinearSVC` for reasons that have nothing to do with the model class. **Kernel approximation.** If you need nonlinearity but cannot afford `SVC`, `sklearn.kernel_approximation` provides `Nystroem` and `RBFSampler`, which map inputs into an explicit finite-dimensional feature space whose inner products approximate the RBF kernel. Fit `LinearSVC` or `SGDClassifier` on that expansion and you get most of the kernel's expressiveness at linear cost. The `SVC` docstring itself points at this path for large datasets. ## The differences that change results, not just speed Swapping `SVC(kernel='linear')` for `LinearSVC` is not a pure performance change: - **Loss.** `LinearSVC` defaults to `loss='squared_hinge'`; `SVC` uses hinge. Different loss, different fitted boundary. - **Intercept.** liblinear folds the intercept in as a synthetic feature, so it is regularized; `intercept_scaling` exists to blunt that. libsvm's intercept is not penalized. - **Multiclass.** `LinearSVC` does one-vs-rest, fitting one classifier per class. `SVC` fits one-vs-one internally — `n_classes * (n_classes - 1) / 2` binary problems — and then aggregates, with `decision_function_shape='ovr'` controlling only the shape of what `decision_function` hands back, not how the model was fitted. - **Probabilities.** Neither gives probabilities for free; `LinearSVC` has no `predict_proba` at all. - **Dual formulation.** `LinearSVC` has a `dual` parameter, which since scikit-learn 1.5 defaults to `'auto'` and picks the primal or dual solver based on the shape of the data. So a candidate who reports that accuracy shifted after the swap is not necessarily doing something wrong — those are two different estimators that happen to be adjacent. ## Where SVC is still the right answer Small-to-medium datasets — thousands to low tens of thousands of rows — with genuine nonlinearity and a modest feature count. That is a real and common regime, and `SVC(kernel='rbf')` with tuned `C` and `gamma` is a strong baseline there. Two parameter notes: `gamma='scale'` is the default and computes `1 / (n_features * X.var())`, which makes the kernel width adapt to the data's spread; and the whole family is highly sensitive to feature scaling, since the kernel is a distance in input space. ## What the interviewer is listening for The answer that lands is a size threshold plus a mechanism: "past roughly tens of thousands of rows the kernel solver's quadratic-or-worse fit cost dominates, so I move to a primal linear solver, and if I need the nonlinearity I approximate the kernel explicitly first." A candidate who only says "SVMs are slow on big data" has read that somewhere; a candidate who names `LinearSVC`, `SGDClassifier` and `Nystroem` and knows the loss and multiclass differences has actually had to make the swap.