Why does targeting a retention discount by churn risk differ from targeting by uplift?
answer
- who responds is not who changes
- the offer can backfire on some
- four quadrants of response
- persuadables versus sure things
- difference of two retention probabilities
basics
~20 sA churn-risk model ranks who will leave; an uplift model ranks whose behaviour the discount changes. Those are different people: the highest-risk customers often leave regardless, and some contented ones cancel only because the offer reminded them the subscription exists.
solid answer
~50 sUplift is the difference a treatment makes for a unit: `uplift(x) = P(stay | discount, x) - P(stay | no discount, x)`. A churn model estimates an outcome level, not a change, so ranking by it answers a different question. Sorting customers by uplift splits them into four groups: persuadables, who stay only if offered; sure things, who stay either way and simply cost you the margin; lost causes, who leave either way; and sleeping dogs, whom the offer actively pushes out because it reminds them they are paying. High churn risk is dense in lost causes and low churn risk is dense in sure things, so a risk-ranked campaign spends most of its budget where it changes nothing and some of it where it does harm. Building an uplift model also needs something a churn model does not: a randomized untreated arm to difference against.
go deeper
Be able to state that uplift is the change a treatment causes rather than the likelihood of the outcome, and that the two rank customers differently.
Name the four response quadrants and explain why a discount aimed at the highest-risk customers mostly buys outcomes that would have happened anyway.
Show how you would build and check it: a randomized control arm kept in production, incremental value per treated customer, and a slice-level check for negative effects.
Own the tradeoff of permanently withholding treatment from a control group, and make the budget argument for spending only where behaviour actually changes.
## Two different quantities A churn model estimates `P(leave | x)` — a level. An uplift model estimates a difference between two levels under two conditions: `uplift(x) = P(stay | treated, x) - P(stay | untreated, x)` That is a conditional treatment effect on a binary outcome. The two quantities have no fixed relationship: a customer can be at high risk with zero uplift, at low risk with high uplift, or anywhere else in the plane. Ranking by one is not a proxy for ranking by the other, which is why an accurate churn model can produce a poor targeting policy. ## The four quadrants Slice the population by what happens under each condition and you get four groups, which is the standard vocabulary of uplift modelling. - **Persuadables** stay if treated and leave if not. All of the campaign's value lives here. Their uplift is positive. - **Sure things** stay either way. Treating them changes nothing about retention and gives away the discount margin for free. Uplift zero. - **Lost causes** leave either way. The offer is wasted breath, though at least it is cheap — no discount is redeemed by someone who leaves. Uplift zero. - **Sleeping dogs** stay if left alone and leave if treated. A retention email arrives, the customer is reminded they have a subscription they barely use, and they cancel. Uplift negative. No individual customer can be labelled with certainty, because you observe only one condition per customer. The quadrants describe conditional averages over similar customers, not observable classes. ## Why risk ranking misallocates budget Think about where each quadrant sits on the churn-risk axis. The very highest risk is full of people whose reasons for leaving a discount does not touch — they moved, the product does not fit, they finished the project the tool was for. These are lost causes. The very lowest risk is full of sure things. Persuadables are typically in the middle, wavering on price. A campaign that pours the whole budget into the top risk decile therefore buys the least movable slice of the population, and it will still report excellent-looking numbers, because retention among a treated high-risk group is compared against nothing. Sleeping dogs make the picture worse. They are invisible to any evaluation that only looks at the treated, since a customer who cancels after the email looks like a customer who was going to cancel anyway. Only a treated-versus-untreated comparison within that slice exposes a negative effect, and only an uplift model is looking for one. ## What uplift modelling requires The hard requirement is a randomized untreated arm. Uplift is a difference between two conditions, so the training data must contain comparable customers under both. In practice this means a holdout group that is permanently excluded from campaigns — a real, ongoing cost that has to be defended to whoever owns the revenue target. Without it there is no counterfactual, and any 'uplift' model degenerates into a response model wearing a different name. Given such data, several routes exist: model each arm's response and difference them, model the effect directly with a learner built for it, or transform the outcome so that ordinary regression on the transformed label targets the difference. The choice matters less than the discipline around it. ## Deciding whom to treat Ranking by uplift is only half the decision; depth is the other half. Each treated persuadable delivers an expected incremental retention gain, and each treated customer costs the discount they redeem. Extend down the ranked list while incremental value per treated customer exceeds cost, and stop there — treating the entire base means paying sure things and provoking sleeping dogs to no benefit. This also reframes the reporting. Retention among the treated is the wrong headline, because it rewards selecting people who were going to stay. Incremental retention per discount granted is the number that reflects what the campaign actually caused. ## Common traps Calling a churn score an uplift score is the big one. A close second is evaluating a targeting change without a control group, so that a policy which merely picks safer customers looks like a policy that saves more customers. A third is ignoring the possibility of negative effects entirely: teams that have never checked for sleeping dogs usually do not know whether they have them. ## The interview answer in one breath Churn risk ranks who is at stake; uplift ranks whom you can move. The budget belongs to persuadables, is wasted on sure things and lost causes, and does damage among sleeping dogs — and separating the four requires an untreated control arm, both to train the model and to prove the policy worked.
- What is a sleeping dog and why does it matter more than its size suggests?A sleeping dog is a customer the treatment makes worse — a quiet subscriber who cancels because the retention email reminded them the subscription exists. It matters because it flips the sign of the campaign's value on that slice, and a response-ranked campaign cannot see it: the customer looks like an ordinary churner either way. Only a treated-versus-untreated comparison inside that slice reveals the negative effect.
- What data does an uplift model need that a churn model does not?A randomized untreated arm inside the training data — comparable customers who did not receive the campaign. A churn model learns from outcomes alone, but uplift is a difference between two conditions, so the untreated side has to exist and be comparable. Operationally that means permanently holding a control group out of campaigns, which costs real short-run revenue and has to be defended.
- How would you show the business that uplift targeting beat the old risk-based list?Randomize between the policies: assign customers to the uplift-targeted list, the risk-targeted list, or an untreated control, and compare incremental retention per discount granted rather than raw retention among the treated. The old list usually wins on raw retention precisely because it selects likely stayers, so the incremental number is the one that settles the argument.
Judging a campaign by who responded is like judging a doctor by which patients recovered. Some would have recovered untreated, and a few were made worse by the treatment.
saying these in an interview costs you the question
- Ranks customers by predicted churn and calls it uplift
- Judges the campaign by retention among the treated only
- Ignores that a message can make some customers worse
- Trains an uplift model with no untreated control group
- Assumes highest risk means highest opportunity