When would you use Gaussian, multinomial or Bernoulli likelihoods in a naive Bayes classifier?

## The shared skeleton Every naive Bayes variant computes the same thing: ``` log_score(class) = log P(class) + sum over features j of log P(x_j | class) ``` and predicts the argmax. The class prior comes from label counts, and the per-feature independence is assumed throughout. What varies between the named variants is only the family used for `P(x_j | class)` — so the choice is a modelling decision about what kind of quantity each feature is. ## Gaussian likelihood — continuous features For a continuous feature, assume that within each class it follows a normal distribution, and estimate a mean and a variance for that feature in that class from the training rows of that class alone. For three machine-sensor readings and two classes, that is 3 x 2 means and 3 x 2 variances, plus the prior. There is no covariance matrix: independence means each feature gets its own one-dimensional density, which amounts to a diagonal-covariance Gaussian per class. Two practical notes. First, a feature that is constant inside a class gives a variance of zero and an infinite density, so implementations floor every variance with a small positive constant — the Gaussian counterpart of smoothing a zero count. Second, the normality is assumed *within a class*, not overall; a feature that is bimodal across the whole dataset is fine if each class sits on one mode. What breaks it is a feature that is bimodal *inside* one class — say a sensor that idles at 20 and spikes at 80 in the same failure mode. The fitted mean lands near 50 with a huge variance, putting its density peak where the class never actually is. Remedies: transform the feature, split the class, or discretise the reading into bins and switch that column to a categorical likelihood. ## Multinomial likelihood — counts Here each feature is a non-negative count and each class has a probability distribution over the vocabulary. A document contributes ``` sum over distinct terms t of count(t) * log P(t | class) ``` so a term occurring five times contributes five times its log-likelihood: repetition is evidence. Estimation is the count ratio with additive smoothing so no term is exactly zero. The consequence worth stating in an interview is that multinomial scoring only touches terms that are **present**. A term absent from the document contributes nothing at all. For a long newswire story that is fine, because presence carries plenty of signal. ## Bernoulli likelihood — binary flags Bernoulli treats each feature in a fixed feature set as an independent yes/no event, with `p_j = P(feature j present | class)` estimated as the fraction of that class's training rows in which it occurs. Scoring runs over **every** feature, not just the present ones: ``` sum over j of [ x_j * log p_j + (1 - x_j) * log (1 - p_j) ] ``` The second term is the whole point: an absent feature contributes `log(1 - p_j)`, which is a strong penalty for a class where that feature is usually present. For mobile malware triage over 200 present/absent app-permission flags, the fact that an app does **not** request network access is genuinely informative, and Bernoulli encodes it while multinomial would let it pass silently. Smoothing applies here too, as `(rows_with_feature + alpha) / (rows_in_class + 2 * alpha)`; the `2 * alpha` reflects two possible outcomes rather than a vocabulary of size V. Note also that Bernoulli discards repetition entirely — five occurrences and one occurrence look identical. ## Choosing between them - Continuous measurements, roughly unimodal per class: Gaussian. - Counts where how many times matters, and documents long enough that presence carries the signal: multinomial. - Binary indicators, or short items over a modest feature set where absence is informative: Bernoulli. Binarising counts and feeding them to multinomial is not the same as Bernoulli: you lose the repetition information either way, but only Bernoulli adds the absence terms back. On very short texts, where there is barely any repetition to exploit, the explicit modelling of zeros is why Bernoulli can be competitive; on long documents multinomial generally has the advantage. ## Mixing families Because the log-score is a sum of independent per-feature terms, nothing forces one family across the whole feature set. A model over two continuous sensor readings and five binary flags can use Gaussian terms for the first two and Bernoulli terms for the rest and simply add them, keeping one shared class prior. That flexibility is a good sign that the candidate understands the skeleton rather than three memorised recipes.