How does matrix factorization predict a missing rating in a users-by-films matrix?
answer
- two skinny matrices, not one enormous one
- one short vector per user, one per film
- prediction is a dot product plus offsets
- loss sums over observed cells only
- L2 shrinks thinly-rated users' vectors
basics
~20 sMatrix factorization learns a short vector of latent factors for every user and every film, then predicts a rating as the dot product of the two vectors plus bias terms. Every empty cell can be filled that way.
solid answer
~50 sThe ratings matrix is mostly empty, so instead of storing it we approximate it as the product of two skinny matrices: a `users x k` factor matrix and a `films x k` factor matrix, with `k` far smaller than either side. The prediction for user `u` on film `i` is `r_hat = mu + b_u + b_i + dot(p_u, q_i)`, where `mu` is the global mean rating, `b_u` and `b_i` are user and item offsets, and `p_u`, `q_i` are the two `k`-dimensional factor vectors. Training minimises squared error **over the observed ratings only** — blanks are missing, not zero — plus an L2 penalty on the factors so cells with two ratings do not produce enormous vectors. Once fitted, any user-film pair is one dot product away, which is what makes the model useful: it scores pairs nobody has ever rated.
go deeper
Be ready to write the prediction as a global mean plus a user offset plus an item offset plus the dot product of two short vectors, and to say that training uses only the cells that actually contain a rating.
Explain why the low-rank assumption is what buys generalisation, count the parameters against the number of cells, and say what the L2 penalty does to a user who has rated three films.
Show judgment about what the model cannot do: no vector for an unseen user, factor dimensions that are not interpretable across runs, and squared error on ratings not being the same objective as a good ranked list.
Own the framing decision — whether rating prediction is even the right objective for the product, what the low-rank assumption implicitly asserts about your users, and when a factorization model is the wrong tool for the catalogue you have.
## The shape of the problem A collaborative-filtering dataset is a table with users down the side and items across the top, and a rating in a cell when that user rated that item. The Netflix Prize matrix is the canonical example: roughly 100 million 1-to-5 star ratings, several hundred thousand subscribers, and on the order of ten thousand films. Multiply the sides together and you get billions of cells, so about 99% of the table is empty. The task is to guess what belongs in the empty cells. You cannot learn one free parameter per cell — there are billions of cells and only 100 million observations, so such a model has nothing to generalise from. Matrix factorization imposes a structural assumption instead: **the table is approximately low rank**. Taste is not arbitrary. If a handful of underlying dimensions (how much a film leans towards slow character drama, how much towards spectacle, how mainstream it is) explain most of what people like, then the whole table is close to the product of a small user matrix and a small item matrix. ## The model Pick a latent dimension `k` — typically tens to a few hundred. Give every user `u` a vector `p_u` of `k` numbers and every item `i` a vector `q_i` of `k` numbers. The predicted rating is ``` r_hat(u, i) = mu + b_u + b_i + sum over f of p_u[f] * q_i[f] ``` where `mu` is the global mean rating, `b_u` is how far this user's ratings sit above or below that mean, and `b_i` is the same for the item. The dot product `sum p_u[f] * q_i[f]` is the *interaction* term: it is large and positive when the user's taste vector and the item's content vector point the same way in the latent space, and negative when they oppose. Stacking the vectors gives matrices `P` (users x k) and `Q` (items x k), and the whole prediction matrix is `P Q^T` plus biases. Parameter count drops from `users x items` to `k * (users + items)` — on a catalogue of 50,000 films and a million users with `k = 50`, that is about 52 million numbers instead of 50 billion. ## The loss Fit by minimising squared error **on the observed cells only**: ``` loss = sum over observed (u,i) of (r_ui - r_hat(u,i))^2 + lambda * (||p_u||^2 + ||q_i||^2 + b_u^2 + b_i^2) ``` Two details matter and both are commonly got wrong in interviews. First, **the sum runs over observed ratings, not over the full grid**. Treating blanks as zeros would tell the model that every unwatched film deserves a 0-star prediction, which is false — the user simply has not seen it. (The implicit-feedback setting changes this deliberately, with weights, but in the explicit-rating setup blanks are dropped.) Second, **regularization is not optional**. A user with three ratings would otherwise get a factor vector that reproduces those three ratings exactly and predicts nonsense everywhere else. The L2 penalty shrinks lightly-supported vectors towards zero, which for such a user collapses the prediction back onto the bias terms — a sensible fallback. ## What the factors mean Nothing fixed. The factorization is identified only up to an invertible transform: replace `P` with `P R` and `Q` with `Q R^-T` and the product is unchanged, so "dimension 7" has no stable meaning across two training runs. Individual dimensions sometimes correlate with something human-readable after the fact, but treating them as discovered genres is over-claiming. What *is* meaningful is the geometry: films close together in factor space are liked by the same people, and a user's vector is a point in that same space. ## What you get out Scoring is cheap. For recommendations you compute the dot product of one user vector against all item vectors, drop the items already seen, and take the top N. Both matrices are small enough to hold in memory, and the item matrix changes only when you retrain, which is why factorization models were the production workhorse of rating prediction for years. ## Where it stops A user or item with no interactions has no factor vector to learn from — the model can only fall back on biases. Accuracy is also not the same as a good recommendation list: minimising squared error on ratings optimises the numbers, not the ordering of the top of the list, and those two objectives are not identical.
- Why not fill the empty cells with zeros and factorize the dense matrix?A blank means "not seen", not "rated zero". Filling with zeros makes the model fit an overwhelming majority of fabricated low ratings, and since the matrix is around 99% empty those fabrications dominate the loss and drag every prediction towards the fill value. In the explicit-rating setup you sum the error over observed cells only.
- Do the individual latent dimensions correspond to genres?Not reliably. The solution is only defined up to an invertible transform of the factor space, so `P R` and `Q R^-T` give identical predictions and dimension indices are not stable across runs. Some directions correlate with interpretable traits after the fact, but you cannot promise a stakeholder that factor 3 is "comedy".
- How do you turn the fitted factors into a top-10 list for one user?Score that user's vector against every item vector with a dot product, add the item biases, remove items the user has already interacted with plus anything filtered by business rules, and take the highest ten. At catalogue scale you replace the exhaustive scan with an approximate maximum-inner-product search over the item matrix.
Instead of memorising which of a million diners liked which of fifty thousand dishes, you learn a short taste profile per diner and a short flavour profile per dish, and predict the match by lining the two profiles up.
saying these in an interview costs you the question
- Says blanks are filled with zeros before factorizing
- Claims each latent dimension is a discovered genre
- Thinks a larger latent dimension always predicts better
- Says the full dense matrix must fit in memory
- Skips regularization and lets thinly-rated users overfit