When would you refuse to trust the asymptotic standard errors and tests from a likelihood fit?

## Why this is a judgment question Every likelihood-based standard error, Wald test and nested-model comparison is the payoff of one approximation: near its maximum, the log-likelihood looks like a quadratic bowl, and the estimate is approximately Normal with variance given by the inverse information. This is a **limit** statement. The senior skill is not reciting it but knowing the situations where the approximation is thin enough that reporting the numbers would mislead a decision. ## Failure mode 1: too little data per parameter Asymptotic results describe behaviour as the sample grows with the model held fixed. A rich model fitted to a modest dataset is exactly the opposite regime. Symptoms: intervals of the form estimate plus or minus two standard errors under-cover; test statistics are too liberal; the fit moves substantially when a few observations are removed. What matters is not raw row count but *effective* information — in a rare-event setting, thousands of rows carrying a handful of events give the information of a very small sample, and the parameter governing those events is the one whose errors are least trustworthy. **Response**: prefer intervals obtained by inverting the likelihood-ratio statistic, which follow the actual shape of the log-likelihood, or calibrate by resampling. Simplify the model until the information supports it. ## Failure mode 2: boundary estimates The quadratic expansion needs the estimate to sit in the **interior** of the parameter space so the log-likelihood can curve away in both directions. When a variance component is estimated at zero, a probability at one, or a mixing weight at an endpoint, the surface is one-sided. Consequences: a symmetric interval can extend into impossible values; the observed information may be singular or ill-defined; and the nested-model comparison no longer has its usual chi-square reference — for a single non-negative variance component tested at zero the correct null distribution is a 50:50 mixture of a point mass at zero and a chi-square with 1 degree of freedom, so using the plain chi-square is conservative rather than merely approximate. **Response**: reparameterise so the boundary is pushed to infinity where that is meaningful, use the correct boundary reference distribution, or move to an interval that respects the parameter's range. ## Failure mode 3: weak identifiability If some combination of parameters barely affects the fit, the log-likelihood has a nearly flat ridge, the information matrix has an eigenvalue near zero, and its inverse explodes. You see enormous standard errors, correlations near plus or minus one between estimates, results that swing with the optimiser's starting point, and a Hessian that fails to be negative definite at the reported solution. The numbers are not merely imprecise — they are numerically unreliable, and small changes in the data or the algorithm change them a lot. **Response**: diagnose rather than report. Drop or combine the offending parameters, add a penalty or prior that regularises the flat direction, or design data collection that varies along it. Never present the raw inverse of a near-singular matrix as an uncertainty statement. ## Failure mode 4: misspecification The identity between the variance of the score and the negative expected curvature — the information equality — holds only when the model is right. Under misspecification the estimate still converges to a well-defined quantity (the parameter minimising divergence from the truth), but its variance is no longer the inverse information. The consequence is systematic: with dependence such as clustered or serially correlated observations, nominal errors are typically **too small**, so tests reject too often. **Response**: a sandwich variance estimator that combines the score's empirical variance with the curvature, or a resampling scheme that respects the dependence structure — resampling clusters, not rows. ## Failure mode 5: statistics that are not invariant Even with plenty of data, a Wald-style test built on the estimate and its standard error depends on how the parameter is written; testing a coefficient and testing its exponential are not the same test. In binary-outcome models the Wald statistic can even *shrink* as the estimate moves further from the null value, an anomaly known as the Hauck-Donner effect, so a very strong effect can produce an unimpressively small statistic. The likelihood-ratio version is invariant to reparameterisation and does not suffer this. **Response**: prefer likelihood-ratio-based tests and intervals when the two disagree, and treat a disagreement between them as a signal, not a rounding difference. ## Turning this into practice As the person responsible for what the organisation believes, the useful move is a standing checklist attached to any likelihood-based result: how much effective information per parameter, is any estimate at a boundary, what is the condition of the information matrix, what dependence structure does the data have, and do the likelihood-ratio and Wald conclusions agree. Where the checklist trips, the report should carry the interval that survives scrutiny rather than the default one plus a caveat nobody reads. The deeper point to convey in an interview is that asymptotic likelihood theory is a superb default precisely because its failure modes are enumerable — and someone has to own the enumeration.