skip to content

Conditioning & Bayes' Rule

How conditioning reshapes probability: the multiplication rule, independence, the law of total probability and Bayes' theorem on medical-test and Monty Hall problems. A perennial interview trap.

on this pageshow

explore

questions

15

In the Monty Hall problem, why does switching doors win two-thirds of the time?

level: juniorimportance: must knowfreq 64%

answer

  1. your first pick never improves
  2. the host knows and never reveals the car
  3. 1/3 stays put, 2/3 has to go somewhere
  4. two of three cases: host is forced
  5. picture 100 doors instead of three

basics

~20 s

Your first pick wins only one time in three, so two times in three the car is behind another door. The host, who knows where it is, opens a losing door and concentrates that 2/3 onto the single door left.

solid answer

~50 s

Split the world at the moment of your first choice. With probability 1/3 you picked the car, and with probability 2/3 it is behind one of the two doors you did not pick. The host then opens a losing door from that pair -- crucially, he knows what is behind them and never opens the car. That reveal gives you no information about your own door, whose probability stays at 1/3, but it collapses the whole 2/3 onto the single remaining unopened door. So switching wins 2/3 of the time and staying wins 1/3. In Bayes terms: if you picked door 1 and the host opened door 3, then P(open 3 | car behind 1) = 1/2 while P(open 3 | car behind 2) = 1, and that 2-to-1 likelihood ratio is exactly what doubles door 2's odds. The answer depends entirely on the host's rules, so state them.

code

python · 15 lines
python
import random

def play(switch):
    doors = [0, 1, 2]
    car = random.choice(doors)
    pick = random.choice(doors)
    opened = random.choice([d for d in doors if d != pick and d != car])
    if switch:
        pick = next(d for d in doors if d != pick and d != opened)
    return pick == car

trials = 100000
stay = sum(play(False) for _ in range(trials)) / trials
swap = sum(play(True) for _ in range(trials)) / trials
print(round(stay, 3), round(swap, 3))  # about 0.333 0.667

go deeper

for a junior

Know the answer is switch, that it wins two times in three, and be able to say in one sentence that your original door keeps its 1/3 because the host could always show a goat.

for a middle

Prove it, not just assert it: enumerate the three car positions or run the likelihood ratio, and state the host assumptions that the result depends on before you compute anything.

for a senior

Use it to make the general point that a posterior depends on the data-generating process. Show the random-host variant where the identical visible scene yields 1/2, and connect it to selection effects in real data.

for a principal

Treat it as a lesson about specifying how observations were selected before modelling them. Be ready to argue that an unstated collection rule is the hidden assumption that most often invalidates an analysis.

## The setup, stated precisely Three doors. One hides a car, two hide goats. You choose a door. The host then opens one of the other two doors, always revealing a goat, and offers you the chance to switch to the remaining closed door. Should you switch? The answer is yes: switching wins with probability 2/3, staying with probability 1/3. But this is only true under a specific set of host rules, and naming them is half of a good answer: 1. The host always opens a door (he is not choosing whether to make an offer based on what you picked). 2. The host knows where the car is and never opens it. 3. When your first pick happens to be the car and he has a free choice of two goat doors, he picks between them at random. Change any of these and the number changes. ## The one-line argument Your initial pick is correct with probability 1/3. That probability was fixed before any door was opened, and the host's action gives you no evidence about your own door, because he can always open a goat door no matter what you chose. So your door stays at 1/3, the remaining 2/3 has to live somewhere, and after the reveal there is exactly one other closed door to hold it. ## Enumerating the three cases Suppose you always pick door 1. Each car position has probability 1/3. - Car behind 1: host opens 2 or 3. Staying wins, switching loses. - Car behind 2: host must open 3. Switching wins. - Car behind 3: host must open 2. Switching wins. Two of the three equally likely cases reward switching. Notice where the asymmetry comes from: in two of three cases the host has **no choice** about which door to open, and that forced move is the information leaking into the game. ## The Bayes version Say you picked door 1 and the host opened door 3. Write `C_i` for the car being behind door i, each with prior 1/3. ``` P(host opens 3 | C_1) = 1/2 (free choice between doors 2 and 3) P(host opens 3 | C_2) = 1 (forced -- he cannot open your door or the car) P(host opens 3 | C_3) = 0 (he never opens the car) ``` Applying Bayes' rule: ``` P(C_1 | opens 3) = (1/3 x 1/2) / (1/3 x 1/2 + 1/3 x 1) = (1/6)/(1/2) = 1/3 P(C_2 | opens 3) = (1/3 x 1) / (1/2) = 2/3 ``` In odds form the prior odds of door 2 against door 1 are 1:1, the likelihood ratio for the observed reveal is 1 / (1/2) = 2, and the posterior odds are 2:1 in favour of switching. ## The variant that flips the answer Suppose the host does **not** know where the car is, opens one of the other two doors at random, and it happens to reveal a goat. Now `P(opens 3 and it is a goat | C_1) = 1/2`, `P(... | C_2) = 1/2`, and `P(... | C_3) = 0`. The two surviving cases are equally likely, so the posterior is 1/2 for each closed door and switching gains nothing. The physical scene on stage is identical -- your door closed, a goat visible, one door left -- but the probability is different because the **process that produced the evidence** is different. That is the deep lesson of the puzzle and the reason interviewers keep asking it: you condition on how the data was generated, not merely on what you see. A second variant: if the host opens a door only when your first pick was correct, switching always loses. If he opens one only when your pick was wrong, switching always wins. Without the host's rule the question has no answer at all. ## Why intuition fails, and how to fix it Most people reason 'two doors left, so it is 50-50'. That would be right if the two remaining doors had arrived at that state symmetrically. They did not: your door was protected from being opened by your choice, the other survivor was protected by the host's knowledge. The standard intuition pump is to scale up. With 100 doors, you pick one -- probability 1/100 -- and the host opens 98 goat doors, deliberately skipping one. Almost nobody thinks that is now 50-50; it is obvious that the host's single spared door carries the 99/100. Monty Hall is the same structure with the numbers shrunk until the asymmetry stops being visible. ## How to answer in the room Give the 1/3-versus-2/3 split first, state the host assumptions explicitly, then offer either the three-case enumeration or the Bayes calculation as proof, and finish with the random-host variant to show you understand that the answer hinges on the host's rule rather than on the number of closed doors.

  • What if the host forgets where the car is, opens a door at random, and it happens to show a goat?
    Then it really is 50-50 and switching gains nothing. The reveal now has likelihood 1/2 under both surviving hypotheses instead of 1/2 versus 1, so the likelihood ratio is 1 and the odds do not move. Same visible scene, different data-generating process, different posterior -- which is the whole point of the puzzle.
  • Where exactly does the information come from, if the host always shows a goat?
    From the cases where he had no choice. When your first pick is wrong -- two times in three -- only one goat door is available to him, so which door he opens is dictated by where the car is. That forced move is informative; the free choice he makes in the remaining one-third of cases is not.
  • How do you show the same result with 100 doors?
    You pick one door, so you are right with probability 1/100. The host opens 98 goat doors and leaves exactly one closed. Your door keeps its 1/100 because it was never at risk of being opened, so the surviving door carries 99/100. Switching wins 99 times in 100.

Imagine 100 doors: you pick one, and the host opens 98 goat doors, carefully leaving one closed. Your door is still the 1-in-100 guess you made blind; his spared door carries everything else.

saying these in an interview costs you the question

  • Says two doors remain so it must be 50-50
  • Ignores that the host knows where the car is
  • Claims the host's reveal updates your own door
  • Cannot state the host rules the answer depends on
  • Thinks the answer holds even for a random reveal

context

open as a page

Why can two mutually exclusive events with nonzero probability never be independent?

level: juniorimportance: must knowfreq 80%

basics

~20 s

Mutually exclusive means the two events cannot both happen, so P(A and B) = 0. Independence requires P(A and B) = P(A) times P(B), which is strictly positive when both probabilities are. Exclusivity therefore forces maximal dependence, not independence.

open as a page

Using the law of total probability, what is P(defective) if line A makes 60% of units at 2% defective and line B 40% at 5%?

level: juniorimportance: must knowfreq 78%

basics

~10 s

The overall defect rate is 3.2%. The law of total probability weights each line's defect rate by that line's share of production: 0.60 * 0.02 + 0.40 * 0.05 = 0.032.

open as a page

A test with 99% sensitivity and 95% specificity flags a disease with 1% prevalence: how likely is a positive to be real?

level: middleimportance: must knowfreq 78%

basics

~10 s

About 17%. In 10,000 people, 100 have the disease and 99 of them test positive, while 495 of the 9,900 healthy people also test positive. Rare conditions make positives mostly false positives.

open as a page

How does the chain rule factor P(landed, started, finished) for a three-step signup funnel?

level: middleimportance: must knowfreq 62%

basics

~20 s

P(landed) times P(started given landed) times P(finished given landed and started). The chain rule turns a joint probability into a product of conditionals, each measured on the survivors of the previous step, no independence needed.

open as a page

What is the expected number of fair coin flips until the first HH, by first-step conditioning?

level: middleimportance: must knowfreq 50%

basics

~20 s

Six flips on average. Condition on the next flip using two states — no progress, or a trailing head. Solving E0 = 1 + 0.5E1 + 0.5E0 and E1 = 1 + 0.5*E0 gives E1 = 4 and E0 = 6.

open as a page

How do prior odds and a likelihood ratio combine to interpret a positive workplace drug test?

level: middleimportance: should knowfreq 46%

basics

~20 s

Posterior odds equal prior odds times the likelihood ratio. With 2% of staff using, prior odds are 1:49; a 95%-sensitive, 95%-specific test has a positive likelihood ratio of 19, giving posterior odds of 19:49, about 28%.

open as a page

In the two-children problem, why does 'at least one is a girl' give 1/3, not 1/2?

level: middleimportance: should knowfreq 34%

basics

~20 s

Because 'at least one is a girl' leaves three equally likely families, girl-boy, boy-girl and girl-girl, of which one has two girls. Naming a specific child, such as the elder, leaves two cases and gives 1/2.

open as a page

What is the prosecutor's fallacy when a DNA database cold hit has a one-in-a-million match probability?

level: seniorimportance: should knowfreq 36%

basics

~10 s

It is swapping P(match given innocent) for P(innocent given match). A one-in-a-million random-match probability is not a one-in-a-million chance of innocence: searching a 500,000-profile database expects about half an innocent match by chance alone.

open as a page

What does it mean for two variables to be conditionally independent given a third?

level: seniorimportance: should knowfreq 44%

basics

~10 s

Two variables are conditionally independent given a third when, once that third is fixed, they carry no information about each other: P(A and B given C) = P(A given C) times P(B given C).

open as a page

How does the law of total variance split customer spend variance into within- and between-segment parts?

level: seniorimportance: should knowfreq 38%

basics

~20 s

It writes Var(Y) = E[Var(Y | S)] + Var(E[Y | S]): the average spread inside segments plus the spread of the segment means. The first term is within-segment noise, the second is how far apart the segments sit.

open as a page

How do you choose the segment partition when forecasting an aggregate rate as a weighted sum?

level: principalimportance: should knowfreq 30%

basics

~20 s

Pick segments that are mutually exclusive, exhaustive, assignable before the outcome is known, and whose conditional rates are more stable than the aggregate. Then forecast rates and mix separately, and stop splitting once cells get too thin.

open as a page

Two coins, one fair and one double-headed: you draw one blind and flip three heads, so what is the chance you hold the biased one?

level: juniorimportance: nice to knowfreq 30%

basics

~20 s

8/9, about 89%. Each coin started equally likely. Three heads has probability 1 under the double-headed coin and 1/8 under the fair one, so the posterior is 1 divided by 1 plus 1/8, which is 8/9.

open as a page

How can three events be pairwise independent but not mutually independent?

level: middleimportance: nice to knowfreq 28%

basics

~20 s

Pairwise independence constrains only pairs; mutual independence also requires the three-way product rule. Flip two fair coins and take 'exactly one head' as a third event: every pair is independent, yet all three cannot occur together.

open as a page

In gambler's ruin on a fair game, why is the chance of reaching N before 0 from k equal to k/N?

level: seniorimportance: nice to knowfreq 26%

basics

~20 s

Conditioning on the next bet gives P(k) = 0.5P(k-1) + 0.5P(k+1), so each value is the average of its neighbours — a straight line. The boundaries P(0) = 0 and P(N) = 1 force P(k) = k/N.

open as a page