Your real-user monitoring dashboard shows healthy loading metrics, yet the same pages have a high abandonment rate and users report the site as slow. How can RUM systematically under-report the worst experiences, and what would you change?

## The shape of the bias Every RUM dataset is a sample of the sessions that were *able to report*. That sounds harmless until you notice what determines the ability to report: the measurement code has to have loaded and run, and the page has to have lived long enough for the value to be finalised and the report to leave the device. Both conditions are correlated with the page being fast. The sample is therefore not random — it is filtered by exactly the variable you are trying to measure. This is survivorship bias, and it makes a slow site look acceptable. ## Where the samples disappear **Abandonment before the metric exists.** A loading metric such as LCP is not final until the browser stops finding larger candidates, which in practice is when the user interacts or the page is hidden. A visitor who waits four seconds, sees nothing useful and closes the tab may leave no loading sample at all — or leave one that is truncated at the moment of departure. The result is a dataset that contains the loads people were patient enough to sit through. **Instrumentation shipped inside the app bundle.** If the RUM snippet initialises after the framework mounts, every session where the bundle arrived late, failed to parse, hit a chunk-loading error or was blocked outright reports nothing. Those are among your worst sessions. The instrumentation and the failure share a cause. **Consent gating.** Where the measurement beacon is only allowed after a consent choice, users who bounce at the banner never contribute. Since the banner sits in front of a page that is still loading, the slowest experiences are again the ones that go unrecorded. **Blockers and old clients.** Content blockers frequently remove third-party analytics endpoints, and very old browsers may lack the measurement APIs entirely. Both correlate with slower devices. **Lost beacons.** A report that is sent as the tab goes away can be dropped if it is issued the wrong way — a normal asynchronous request during teardown is not guaranteed to complete, and the sessions where teardown happens abruptly are, again, the frustrated ones. **Aggressive sampling.** Sampling one session in fifty to keep costs down is legitimate, but it thins the tail fastest. There may be plenty of data to state the typical case and almost none to characterise the slow slice, which is the slice that matters. ## Diagnosing how big the hole is You cannot see the missing sessions directly, so measure them by difference: 1. **Reconcile counts.** Compare the number of RUM navigations against a source that does not depend on your JavaScript — server access logs, edge or CDN request counts, or a server-side page-view counter. A large gap is the size of your blind spot, and it should be tracked as a metric in its own right. 2. **Break the gap down.** If the shortfall is concentrated on particular templates, regions, device classes or entry points, that is not measurement noise; that is where your worst experiences live. 3. **Record incompleteness explicitly.** Tag reports with whether the page was hidden before the metric settled, so you can count truncated sessions rather than discarding them silently. 4. **Cross-check against behaviour.** Bounce or abandonment rates that disagree with a healthy performance dashboard are the classic tell that the dashboard is describing survivors. ## Fixing the collection - **Load measurement first and separately.** Put the collection code early in the document, independent of the application bundle, so it exists before anything can go wrong with the app. Its own failure modes should not be the app's failure modes. - **Report when the page is hidden, not on unload.** Finalise and send at the point the page becomes hidden, using a delivery mechanism designed to survive teardown, so departures produce data instead of silence. - **Send partial data.** A truncated LCP with a flag saying the user left at 4.2 seconds is far more informative than nothing. So is a plain "navigation started, never completed" event. - **Sample by session, not by event, and keep the tail.** Uniform session sampling preserves the shape of the distribution; sampling that drops long sessions or slow pages destroys exactly the evidence you need. - **Watch for consent and blocker effects.** Know what share of traffic can never report, and reason about that population separately rather than pretending it does not exist. ## What to say in the interview The insight interviewers are listening for is that a performance dataset has a *selection process*, and the selection is not independent of performance. Anyone can read a p75 off a dashboard; the senior move is to ask which sessions were eligible to appear in it, quantify the ones that were not, and change the collection so the answer improves.