skip to content

Monte Carlo Simulation

Estimating a probability or expectation by generating many random draws and averaging, with error shrinking like 1/sqrt(n), plus inverse-transform sampling. Used when no closed form is in reach.

on this pageshow

questions

6

How would you estimate pi by Monte Carlo simulation with uniform random points?

level: juniorimportance: must knowfreq 66%

answer

  1. an area ratio, not a geometry formula
  2. sample uniformly inside the unit square
  3. count points with x*x + y*y <= 1
  4. the hit fraction estimates pi/4
  5. error shrinks like one over sqrt(n)

basics

~20 s

Draw many independent points uniformly in the unit square and count the fraction that land inside the quarter circle of radius 1. That fraction estimates pi/4, so multiply it by 4. Accuracy improves like 1/sqrt(n).

solid answer

~40 s

Sample `n` points with both coordinates drawn independently from Uniform(0,1). A point counts as a hit when `x*x + y*y <= 1`, meaning it falls inside the quarter circle of radius 1. The quarter circle has area `pi/4` and the square has area 1, so the probability a random point is a hit is exactly `pi/4`. The hit fraction is therefore an unbiased estimate of `pi/4`, and `4 * hits / n` estimates pi. The estimate is random: its standard error is about `1.64/sqrt(n)`, so a thousand draws gives roughly the first decimal and a million draws roughly three decimals. Quadrupling the draws halves the typical error. Always report the estimate with that uncertainty rather than as a bare number.

code

python · 14 lines
python
import random

def estimate_pi(n, seed=0):
    rng = random.Random(seed)
    hits = 0
    for _ in range(n):
        x = rng.random()
        y = rng.random()
        if x * x + y * y <= 1.0:
            hits += 1
    return 4.0 * hits / n

for n in (1000, 100000, 1000000):
    print(n, estimate_pi(n))

go deeper

for a junior

Be ready to set this up from scratch: uniform draws in the square, the hit test xx + yy <= 1, hit fraction times 4. Say out loud that more draws means a better estimate.

for a middle

Explain why the hit fraction is unbiased for pi/4 and quantify the error as roughly 1.64/sqrt(n), so quadrupling draws halves it. Distinguish seeding for reproducibility from anything that improves accuracy.

for a senior

Show you would never ship a simulated number bare: attach a Monte Carlo margin, pin the seed for auditability, and say how many draws the decision actually needs before running any.

for a principal

Own the framing that simulation trades analyst time for compute and precision at a punishing rate. Argue when a rough half-percent answer today beats an exact derivation next week, and when it does not.

## The idea in one line Monte Carlo estimation replaces a quantity you cannot compute in closed form with the average of many random draws that have that quantity as their expected value. The dart-throwing estimate of pi is the smallest complete example of the pattern, which is why it is the standard opening question. ## The construction Take the unit square: all points `(x, y)` with `0 <= x <= 1` and `0 <= y <= 1`. Its area is 1. Inside it sits the quarter of the unit circle: the set of points with `x*x + y*y <= 1`. A full circle of radius 1 has area pi, so a quarter of it has area `pi/4`, approximately 0.7854. Now draw a point uniformly at random from the square, meaning `x` and `y` are independent Uniform(0,1) draws. "Uniform" means the probability of landing in any sub-region equals that region's area divided by the square's area. Since the square has area 1, the probability of landing inside the quarter circle is exactly `pi/4`. Define an indicator variable `I` that equals 1 when the point is a hit and 0 otherwise. Then `E[I] = P(hit) = pi/4`. Repeat the draw `n` times independently and average: p_hat = (number of hits) / n pi_hat = 4 * p_hat Because the average of independent copies has the same expectation as one copy, `E[p_hat] = pi/4` and `E[pi_hat] = pi`. The estimator is unbiased: it is centred on the truth for every `n`, small or large. ## How accurate is it Each draw is a Bernoulli trial with success probability `p = pi/4 = 0.7854`, so its variance is `p(1-p) = 0.1685` and its standard deviation is about 0.4105. Averaging `n` independent draws divides the standard deviation by `sqrt(n)`, and multiplying by 4 multiplies it by 4. So the typical size of the error in `pi_hat` is about 1.642 / sqrt(n) That gives roughly 0.05 at n = 1,000, 0.005 at n = 100,000, and 0.0016 at n = 1,000,000. Two consequences follow, and both are what an interviewer is listening for. First, the error shrinks like `1/sqrt(n)`, not like `1/n`. To halve the error you need four times as many draws; to gain one more correct decimal digit you need a hundred times as many. Monte Carlo is cheap to write and expensive to make precise. Second, the estimate is a random variable. Running the same code twice with different randomness gives different answers, and a run that returns 3.19 from a thousand draws is not buggy, it is simply within ordinary sampling noise (0.05 error is one standard error). Reporting a Monte Carlo number without a stated margin is the classic weak answer. ## Reproducibility Fixing the random seed makes the run repeatable: the same seed replays the same stream of draws, so a colleague or a test can reproduce your figure exactly. It does not make the figure more accurate. A seeded run is one draw from the distribution of possible answers, just a draw you can find again. Seeds belong in the code and in the report; they are a reproducibility control, never an accuracy control. ## Why this pattern generalises Nothing in the argument used circles. To estimate the area of any region contained in a box you can sample uniformly in, sample points in the box and multiply the hit fraction by the box's volume. More generally, to estimate `E[f(X)]` for any quantity you can simulate, draw `X_1, ..., X_n` and average `f(X_i)`. Probabilities are the special case where `f` is an indicator: `P(A)` is just `E[1 if A else 0]`. That is why simulation is the tool of choice when the rules of a system are easy to code but hopeless to integrate: a game whose outcome depends on a chain of dice rolls, a queue with irregular arrivals, a portfolio of interacting rules. You do not need the answer's formula, only the ability to play the system out many times. ## What to say out loud Name the expectation being estimated, state that the estimator is unbiased, give the `1/sqrt(n)` error rate with a concrete number of draws, and mention the seed as a reproducibility device. That is a complete answer in under a minute.

  • How many draws do you need before the estimate is reliable to two decimal places?
    The standard error is about `1.64/sqrt(n)`. For an error near 0.005 you need `n` around `(1.64/0.005)^2`, roughly 100,000 draws. Getting to three decimals needs about 100 times more again. Quoting that arithmetic shows you understand that precision is bought in factors of four and a hundred, not in a few extra thousand draws.
  • Does fixing the random seed make the estimate more accurate?
    No. A seed makes the run reproducible: the same seed replays the same draws, so the number can be regenerated exactly. Accuracy is controlled only by the number of draws and by variance-reduction technique. A seeded run is a single draw from the distribution of possible answers, and it can happen to be an unlucky one.
  • Why does this approach extend to shapes and integrals with no formula?
    The method only ever estimates an expectation. Sample uniformly in any box you can draw from, and the hit fraction times the box volume estimates the enclosed region's volume. Replacing the indicator by any function `f` estimates `E[f(X)]`. Nothing in the argument depends on the region being a circle or on the dimension being two.

It is like measuring what share of a dartboard is red by throwing darts blindfolded and counting the red hits, instead of measuring the shape with a ruler.

saying these in an interview costs you the question

  • Says the error shrinks like 1/n instead of 1/sqrt(n)
  • Reports a simulated number with no uncertainty attached
  • Thinks a fixed seed makes the estimate more accurate
  • Calls a 3.19 result from 1,000 draws a bug
  • Cannot say which expectation the average is estimating

context

open as a page

How does inverse-transform sampling turn uniform draws into a target distribution?

level: middleimportance: must knowfreq 54%

basics

~10 s

Apply the target's inverse CDF to a Uniform(0,1) draw: X = F^-1(U) has exactly the distribution F. An exponential with rate lambda comes out as -ln(U)/lambda. Discrete variables use a cumulative-probability ladder instead.

open as a page

How do you decide how many draws a Monte Carlo simulation needs?

level: middleimportance: should knowfreq 48%

basics

~20 s

Pick the precision the decision actually needs, then buy draws to reach it. Simulation error falls like 1/sqrt(n), so halving the error costs four times the draws and one extra decimal digit costs a hundred times.

open as a page

A rejection sampler for a truncated normal accepts only 3% of proposals; why?

level: seniorimportance: should knowfreq 34%

basics

~20 s

The envelope is far too loose. Rejection sampling accepts a proposal with probability 1/M, where M is how much the envelope must be inflated to cover the target, so a box dwarfing the truncated region wastes nearly every draw. Accepted draws remain exact.

open as a page

When would you choose Monte Carlo simulation over an exact analytic calculation?

level: principalimportance: should knowfreq 30%

basics

~20 s

Simulate when the system's rules are easy to code but hopeless to integrate, when the problem is high-dimensional, or when the whole output distribution matters rather than a mean. Prefer the closed form when one exists: it is exact, instant and auditable.

open as a page

How does variance reduction cut Monte Carlo error without buying more draws?

level: seniorimportance: nice to knowfreq 26%

basics

~20 s

Variance reduction reshapes what you average rather than how much. Antithetic variates pair each uniform U with 1-U so their errors cancel; control variates subtract a correlated quantity whose true mean is known; common random numbers reuse one stream across competing designs.

open as a page