You will learn how models learn from data, how to keep them from overfitting, how to evaluate them honestly, and when to reach for which classical algorithm. This is the 'explain bias-variance', 'precision or recall here?' core of every DS/MLE screen, probed independent of any framework.
Start with a regularized linear or logistic baseline for a cheap number to beat, then reach for gradient-boosted trees, the default on mixed numeric-and-categorical tabular data. Data shape and hard constraints override that default.
A 1-to-5 star rating is an ordinal target: the values are ordered, but the gaps between them are not guaranteed equal. Regression uses the order and assumes equal spacing; five-class classification keeps the classes but discards the order.
Build one row per time step whose features use only past values — lags such as 1, 7 and 14 steps back, plus rolling means or spreads over trailing windows — and whose label is the value at the forecast horizon.
Because it sets the bar almost for free. Seasonal-naive predicts each future hour with the load observed at the same hour one week earlier, capturing the daily and weekly cycles without any training; a model that cannot beat it is adding nothing.
A discriminative classifier models p(y|x) - the label given the features - directly. A generative classifier models the joint p(x,y), normally as a class prior p(y) times a per-class feature distribution p(x|y), and inverts it to score labels.
Capacity is how large and flexible a model's space of possible fitted functions is. Too little capacity and the model cannot represent the real pattern, which is underfitting. Too much and it reproduces the training sample's noise, which is overfitting.
A held-out score is an average of errors over a sample of rows. It estimates future error only if future rows are independent draws from that same distribution. When the population or the relationship moves, the number stops applying.
Bias is the gap between the true value and the average prediction a model makes when it is retrained on different training samples. Variance is how much that prediction moves from one training sample to the next.
Training error is optimistic because the fitting procedure tuned the model to those exact rows, absorbing their random noise as if it were signal. On fresh rows the noise does not repeat, so error rises.
A validation curve plots training score and held-out validation score against one hyperparameter that controls model capacity. Pick the value where validation peaks; a widening gap between the two lines past that point is the overfitting signal.
At the far left the penalty is largest and every coefficient is exactly zero, so the model predicts a constant. Moving right, predictors enter one by one, and the far right is the unpenalised least-squares fit.
Early stopping halts an iterative fit once a held-out validation score stops improving, and keeps the best-scoring iterate. Each extra iteration lets the model absorb finer detail from the training rows, so stopping sooner limits effective capacity and cuts variance.
Elastic net penalises the sum of absolute coefficients (L1) and the sum of squared coefficients (L2) together. The L1 part drives weak predictors to exactly zero; the L2 part keeps correlated predictors together and makes the fit stable.
Ridge with lambda = 0 reproduces the ordinary least squares fit. As lambda grows, every slope coefficient shrinks smoothly toward zero and the model flattens toward a constant, trading variance for bias. No coefficient reaches exactly zero at finite lambda.
An L1 or L2 penalty adds up coefficient sizes, and a coefficient's size depends on the units of its predictor. Standardising puts every predictor on a common spread so one lambda penalises them all comparably.
Bucket the predictions by score, then compare each bucket's mean predicted probability with the fraction of rows in it that were actually positive. Plotted, that is a reliability diagram: a calibrated model sits on the diagonal.
The four cells count true positives, false positives, true negatives and false negatives - one per pairing of predicted and actual label. Accuracy is (TP + TN) divided by all four counts; the error rate is 1 minus accuracy.
Because a model that never predicts 'defective' already scores 99.7% on that data. The majority class supplies almost every row, so accuracy measures performance on good units and says nothing about whether a single defect was ever caught.
With true classes as rows and predicted classes as columns, a class's recall is its diagonal cell divided by its row total, and its precision is that same diagonal cell divided by its column total.
Precision is the share of predicted positives that are truly positive; recall is the share of actual positives the model catches. Loosening the decision rule labels more examples positive, which raises recall and usually lowers precision.
Draw n rows with replacement from the sample of n and train on that draw. Roughly 37% of the original rows are never drawn; score the model on those. Repeat a few hundred times and average the errors.
Exported files usually arrive sorted — by label, by date, by customer. Cutting such a file into contiguous blocks gives folds that are not representative, sometimes single-class, so every fold score misleads. Shuffling row order first breaks that ordering.
Fitting a scaler on all rows lets the test rows' mean and standard deviation shape the transform, so the test set is no longer unseen and the score is optimistic. Fit on training rows only, then apply it to test.
Report the mean, 0.81, together with the spread: the fold standard deviation is about 0.06, giving a standard error of roughly 0.03. Write it as 0.81 plus or minus 0.03 over five folds, never as the best fold's 0.89.
Stratified k-fold gives every fold roughly the same class proportions as the whole dataset. Plain k-fold fails when a class is rare: some folds get very few or zero minority rows, so fold scores swing or go undefined.
Yes. Linear here means linear in the coefficients, not in the inputs. A squared term is just another column, so the same least-squares fit still applies, even though the curve in the input bends.
They differ in how many rows each gradient step averages. Full-batch uses all n rows for one update per epoch; stochastic uses one row, giving n updates; mini-batch averages a chunk of B rows, giving about n/B updates.
Log loss is convex in a logistic model's weights, so the error surface has one global minimum and no local minima. Every run that converges lands on the same coefficients, which is why random restarts add nothing.
The sigmoid squashes any linear score into the open range 0 to 1, monotonically: large positive scores approach 1, large negative scores approach 0, and a score of 0 maps to 0.5. The result is read as the event probability.
Logistic regression scores each point with one weighted sum. The sigmoid is monotone, so a 0.5 probability cut is exactly the rule score above zero, and the set where the score equals zero is flat: a line, plane or hyperplane.
Distance adds up per-feature gaps, so a feature measured in large units, like income in dollars, swamps one measured in small units, like age in years. Standardising puts every feature on a comparable scale so each can contribute.
Naive Bayes assumes that once the class label is known, the features carry no further information about each other. That licenses replacing the whole likelihood P(features | class) with a product of one-feature terms P(x_i | class).
Laplace smoothing adds one pseudo-count to every word-class pair before dividing, so no estimated likelihood is exactly zero. Without it, a word never seen in a class drives that class's whole product to zero whatever the other words say.
Only the training points lying exactly on the margin boundary - the support vectors - hold the hyperplane in place. Every other point sits strictly further away, so removing it leaves the same widest street and the same fitted boundary.
A support vector machine optimises one hyperplane, which has exactly two sides, so its objective can only express a positive and a negative class. Forty categories are covered by training one SVM per class pair: 780 models.
Every split tests one feature against one threshold, so each cut is a line perpendicular to one axis. A boundary that depends on two features jointly can only be tiled by many small axis-parallel steps.
Bagging trains many copies of one model on bootstrap resamples of the training rows, then averages their predictions. Averaging cancels the errors that differ from copy to copy, cutting variance; the systematic error every copy shares survives, so bias stays.
Gradient boosting starts from a single constant prediction, then repeatedly fits a shallow tree to the errors the model still makes and adds that tree to the running total. Earlier trees are never refitted.
Level-wise growth splits every leaf at the current depth before going deeper, producing balanced trees. Leaf-wise growth repeatedly splits whichever leaf anywhere in the tree promises the largest loss reduction, producing deeper, lopsided trees that fit harder and overfit sooner.
Four pre-pruning limits: a maximum tree depth, a minimum node size to split, a minimum number of samples per resulting leaf, and a minimum impurity decrease per split. Any split breaking one of them is refused.
DBSCAN calls a point core when at least minPts points lie within radius eps of it. A non-core point that sits inside some core point's eps-radius is a border point. Anything else is noise and gets no cluster.
The point is split softly between the two components: given the fitted model, there is a 60 percent posterior chance the first component generated it and 40 percent the second. It is an ambiguity score, not a label.
Each Lloyd iteration does two things: assign every point to its nearest centroid, then move each centroid to the mean of the points assigned to it. The loop repeats until assignments stop changing.
A dendrogram is a tree recording the order and distance at which clusters merge. Agglomerative clustering starts with every observation as its own cluster and repeatedly merges the two closest ones, drawing each merge at the distance it happened.
PCA picks the first principal component as the unit-length direction in the mean-centred data along which the projected points have the largest variance. Each later component maximises the remaining variance while staying orthogonal to all earlier ones.
Split the timestamp into the calendar parts demand depends on: hour of day, day of week, an is-weekend flag, month or quarter, and a public-holiday flag. The raw timestamp alone only lets a model learn a trend.
Choose the entity and a reference date, keep only that entity's events inside a window ending at the reference date, then collapse them into one row: counts, sums, means, maxima, distinct counts, and days since the last event.
Impute in most cases. Deleting the rows discards a fifth of the data and has no equivalent at scoring time, where every request must return a prediction. Drop the column only if it is nearly empty or adds no lift.
Education level has a real order - high-school below bachelor below master below PhD - so integer codes carry that information. Job family has no order, so codes 0-3 invent a ranking the model takes literally. Encode unordered categories as one-hot indicators instead.
Z-score standardisation subtracts the mean and divides by the standard deviation, producing mean 0, standard deviation 1, and no fixed bounds. Min-max scaling rescales values into a fixed range such as 0 to 1 using the training minimum and maximum.
Other features reconstruct the removed attribute. ZIP code, school, job history and purchase patterns correlate with it, so the model relearns it indirectly. Dropping the column removes your ability to measure the disparity, not the disparity itself.
The model has learned a non-monotone relationship: average predicted demand is highest at cold temperatures and at hot temperatures, and lowest in between. The curve summarises the model's average behaviour over the whole dataset, not any single building.
An interpretable-by-design model is one whose fitted structure is itself the explanation: a sparse weighted sum, a depth-3 tree, a short rule list. Post-hoc methods leave the model opaque and build a separate, approximate account of it.
Permutation importance shuffles one column's values across held-out rows, breaking that column's link to the label, then re-scores the already-trained model. The drop from the baseline score, averaged over several shuffles, is the feature's importance.
Demographic parity requires the same positive-prediction rate in every group, ignoring the true outcome. Equalized odds requires equal true-positive and false-positive rates in every group, comparing errors only among people who share a true label.
Support is the share of all baskets containing the rule's items. Confidence is the share of baskets holding the left-hand side that also hold the right-hand side. Lift divides confidence by the right-hand side's own support.
It describes each item by its own features, builds a profile vector from the features of items that user liked, and ranks candidates by similarity between profile and item vector. No other user's data is involved.
Explicit ratings are stated preferences on a scale and can express dislike. Implicit signals only record actions, so they are abundant and cheap but noisy, one-class, and count-valued: no interaction is not a stated dislike.
No. A ranked list depends only on the order the scores induce, and any strictly increasing transform of every score leaves that order identical. The wrong numbers cost you elsewhere: thresholds, expected-value arithmetic, anything reading the score as a probability.
Matrix factorization learns a short vector of latent factors for every user and every film, then predicts a rating as the dot product of the two vectors plus bias terms. Every empty cell can be filled that way.
An agent's action values are only estimates. With probability epsilon it acts randomly so it keeps sampling actions its estimates currently undervalue, which corrects an unlucky early estimate instead of locking onto the first action that looked good.
An MDP is written as states, actions, a transition function giving the probability of the next state, a reward function, and a discount factor gamma. Together they define a sequential decision problem an agent solves by choosing actions.
For every state and action, the model must give the probability of each possible next state and the expected reward. Value iteration averages over all successors in every update, so it plans inside the model rather than learning from experience.
Policy gradient methods tune the policy's parameters directly, so an action is sampled from a distribution. Value-based control has to maximise a value estimate over all actions at every step, which is intractable when the action is a real-valued vector.
V(s) is the expected return from state s when the agent follows its policy from there on. Q(s,a) is the expected return from taking action a in state s first, then following the same policy.