How would you set a team policy for change-aversion claims that override a negative A/B result?

Change aversion is a genuine phenomenon and organisations that ignore it will kill good redesigns. The failure mode is the opposite one: the claim becoming an all-purpose override for inconvenient results. Setting policy here is a leadership problem about evidence standards, not an analysis technique. ## Why the claim needs governing at all Three properties make it dangerous. It is always available: any negative result on a user-facing change can be attributed to disruption. It is asymmetric in who reaches for it: the person proposing it is usually the person whose work is at stake, and nobody argues change aversion when their feature wins. And it is convenient in timing: it is raised after the result arrives, which means it is a post-hoc rescue rather than a prediction. Any explanation with those three properties will be over-used unless the organisation makes it costly to invoke. ## Make the claim falsifiable, in writing, before it counts The core of the policy is that the claim must be stated as a prediction with an accompanying way to be wrong. A usable form has three components. - **A mechanism.** Which population holds which habit, and what specifically the change forces them to relearn. A claim that cannot name the habit is not a claim. - **A predicted signature.** Change aversion makes strong commitments: the deficit is concentrated where prior habit exists and largely absent among users with no prior exposure; it shrinks as the affected users accumulate exposure; and it scales with how entrenched the prior habit was. Each of those is checkable and each can come back negative. - **A stated bar for recovery.** How much of the deficit must close, over what amount of exposure, for the launch to proceed. Agreeing this before looking at the extra evidence is what stops the bar from being redefined to whatever the data happens to show. A claim missing any of the three does not get to change a decision. That single rule does most of the work, because a post-hoc excuse cannot supply a prediction it never made. ## Weigh the two errors honestly, and note they are not symmetric Shipping a genuine loser under a change-aversion banner produces a permanent, quiet cost: the regression is now the baseline, it is folded into every subsequent measurement, and nobody re-opens it. Killing a genuine winner is visible, annoying, and recoverable — the idea can be reworked and re-tested. That asymmetry argues for the burden of proof sitting with the claim rather than against it. It does not argue for banning the claim, which would systematically block improvements that require users to relearn anything. ## Price the transition, do not wave it away Even a fully recoverable deficit is a real bill: the per-user loss multiplied by the affected population multiplied by the adaptation period, landing on the most tenured users. Part of that bill is irreversible, because some fraction of frustrated users leave during the dip and never complete the adaptation curve, converting a temporary effect into a permanent one for them. Policy should require that number to be estimated and accepted explicitly, rather than treating recoverability as though it made the interim free. ## Close the loop, or the policy is theatre The accountability mechanism is the one thing teams skip. If a launch proceeded on a change-aversion argument, a long-term re-read of the metric should be scheduled at ship time with a named owner and a date, and its result should return to the same forum that approved the launch — including when it is unflattering. Two effects follow. The obvious one is that individual wrong calls get caught. The more valuable one is calibration: after a handful of these, the organisation knows empirically how often the argument was right on its surfaces, and that base rate is far better evidence for the next decision than anyone's intuition. ## What the policy should not do It should not require certainty. Recovery evidence is often partial and noisy, and demanding proof would collapse into a ban. It should not be applied only to negative results — the mirror discipline, treating a large early lift as suspect until it is shown to persist, keeps the standard symmetric and keeps the policy from reading as a tool for blocking other people's launches. And it should not be so heavy that it is routed around; a one-page written prediction and a scheduled re-read is about the right weight for a decision of this size.