Regression & Statistical Modeling

The standard error of a regression coefficient measures how much that estimate would move around if you refit the model on other samples from the same process. A smaller standard error means a more precisely estimated effect.

Ordinary least squares picks the coefficients that make the sum of squared residuals as small as possible. A residual is an observed value minus the value the fitted line predicts for it, so the misses are vertical.

Nothing in the data supports the model's shape out there. The arithmetic still prints a number and a finite interval, but that interval covers only sampling error under an assumed straight line, not the risk that the true relationship bends.

Divide the coefficient by its standard error to get the t-statistic, then compare it against a t distribution with n minus the number of estimated coefficients. A magnitude near 2 or more clears the usual 5% bar.

Set both partial derivatives of the squared-residual sum to zero to get the two normal equations. Solving them gives slope = sum of (x - xbar)(y - ybar) over sum of (x - xbar) squared, and intercept = ybar minus slope times xbar.

The Saturday coefficient is the estimated difference in the outcome between Saturday and Monday, with the model's other predictors held fixed. It is a contrast against the omitted baseline day, not Saturday's own average level.

One more year of schooling is associated with roughly a 7% higher wage, holding the other predictors fixed. Because the outcome is logged, the coefficient reads as an approximate percent change; the exact figure is exp(0.07) - 1, about 7.25%.

The four region dummies add to 1 in every row, reproducing the intercept's column of ones exactly. The design matrix loses full rank, so no unique least-squares coefficients exist. This is the dummy variable trap.

It is the promotion effect when the weekend indicator equals zero, that is on weekdays only. The interaction coefficient is the extra promotion effect on weekends, so the weekend promotion effect is the sum of the two.

It is a price elasticity: a 1% price increase is associated with roughly a 1.4% fall in quantity, holding the other predictors fixed. Because the size exceeds 1, demand is elastic, so raising price lowers revenue.

Readings from one patient are correlated, so three readings carry far less information than three different patients. Counting them as independent inflates the sample size, shrinks the standard errors and p-values, and manufactures significance that is not there.

Heteroscedasticity means the variance of the regression errors is not the same for every observation: it changes systematically, usually with a predictor. Homoscedasticity, the classical assumption, is the opposite - one common error variance for every row.

An outlier has a large residual: its response sits far from the fitted line. An influential observation is one whose removal visibly changes the fitted coefficients. A point can be either, both, or neither.

Multicollinearity means the predictors are strongly linearly related to each other, so the fit cannot tell their separate effects apart. It inflates coefficient standard errors and makes individual coefficients unstable, while overall fit and predictions stay largely intact.

A structureless horizontal band: residuals scattered randomly around zero across the whole fitted range, with roughly constant vertical spread and no curve, funnel or clustering. Any visible shape means the model is missing something.

R-squared is the share of the outcome's total variation that the fitted model accounts for, computed as 1 minus the residual sum of squares over the total sum of squares. It measures fit on the data used, not correctness.

Least squares can always set the new coefficient to zero and reproduce the previous fit, so the minimised residual sum of squares can only tie or shrink. Since R-squared is 1 minus that sum over a fixed total, it never falls.

It tests whether every slope coefficient is zero at once, meaning the model does no better than an intercept-only model that predicts the outcome's mean. A small p-value says at least one predictor carries signal, without saying which.

Fit the model with and without the three terms on identical rows and run a nested F-test: the drop in residual sum of squares per added parameter, divided by the full model's residual variance, on 3 and n-k-1 degrees of freedom.

Yes, when the model's job is to identify and size drivers in a noisy outcome rather than to predict individuals. The acceptance bar comes from the decision the model supports, not from any fixed R-squared threshold.

Record the user as right-censored at 21 days: the subscription lasted at least 21 days, and the eventual cancellation time is unknown. Deleting the row biases survival downward; coding it as a cancellation invents an event that never happened.

Probability is p, a number between 0 and 1. Odds is p/(1-p), the event weighed against its complement, and runs from 0 to infinity. A probability of 0.8 is odds of 4, or 4-to-1.

Counts are non-negative integers, usually skewed, and their spread grows with their level. Poisson regression models the log of the expected count, so fitted values stay positive and each predictor acts multiplicatively rather than adding a fixed number of events.

At each event time, divide the events by the number still at risk just before it and multiply the surviving fractions: S(t) = product of (1 - d_i / n_i). Censored subjects shrink the risk set without creating a step.

Annual-plan customers cancel at 1.8 times the instantaneous rate of the reference group, at every time point the model considers. A hazard ratio compares rates among those still subscribed; it is not a probability.