How do you judge whether public compensation data is usable for the level you are interviewing at?

## Why the data needs auditing at all Public compensation aggregators and crowdsourced pay databases are genuinely useful, and they are also the least controlled data you will use for anything important. Entries are submitted voluntarily by individuals, with no verification, in whatever structure the site asks for. The result is a dataset with three predictable biases: - **Self-selection.** People are more likely to submit a number they are pleased with, so the upper part of a sample is usually better represented than the lower part. - **Level ambiguity.** Many entries carry a title the submitter typed rather than a rung the employer administers, which reintroduces the whole title-versus-level problem at the data layer. - **Composition drift.** One entry reports cash only, the next reports a whole package, and the site displays them in the same column. None of that makes the sources unusable. It makes them a sample you audit before you draw a band from it. ## The audit, as a short checklist Run these questions over every source before any figure from it lands on the benchmark sheet: 1. **How many entries survive level-tagging?** An aggregate over hundreds of untagged entries is worth less than a dozen entries you can tie to a described rung. If you started with 23 entries for a senior-level security engineering role and only 9 carry a level you can map, your real sample is 9. 2. **Do they share a location tier?** Entries from different tiers widen the band for a reason that has nothing to do with the rung. 3. **Do they share a company stage?** A pooled sample spanning early-stage and large employers mixes packages of different composition. 4. **What does each total include?** Cash, cash plus variable pay, or a whole package including an equity component. 5. **How recent are the entries?** Bands move. Old entries describe a band that may no longer exist, and the sites rarely make staleness prominent. 6. **Is one employer dominating?** A sample where most entries come from a single company is that company's band, not the market's. 7. **Which direction does the bias run?** Self-reported samples usually skew upward; an employer-stated band usually skews toward the part of the range the employer would prefer to pay. ## Percentiles beat a median A single median invites you to treat one number as the answer. A percentile spread shows how much the sample disagrees with itself, and disagreement is information: a tight spread across level-tagged entries in one tier is a strong band, and a spread so wide that the upper end is nearly double the lower end usually means the sample is still pooling rungs, tiers or stages. Since your target and walk-away are expressed as percentile positions, the spread is also the thing you are actually using. ## Triangulate across kinds of source Use more than one kind of source rather than more of one kind: - **Crowdsourced aggregators** give volume and percentiles with self-selection bias. - **A band stated by a compensation partner** is authoritative about that employer's own range for that rung, with a known direction of interest. - **Peers at your level in your market** give small, high-quality samples with detail no site captures — but be careful with a single friend's number, which is one point, not a band. - **Published ladder descriptions** do not carry pay, but they let you map rungs, which is what makes every other source usable. When two kinds of source disagree sharply, the usual cause is a level mapping error rather than a market anomaly. Re-check the rung before you re-check the market. ## When the sample is genuinely thin Niche roles in smaller markets often yield too few level-tagged entries to support percentiles at all. Three honest responses, in order: 1. **Widen deliberately and record it.** Add an adjacent tier or an adjacent stage, and write on the sheet that you did, so the resulting band carries its own caveat. 2. **Move from a point to a range.** Set the target as a region of the band rather than a percentile point, and keep the floor conservative because your evidence is weaker. 3. **Say what you do not know.** A thin sample honestly labelled is stronger than a confident number nobody can defend — including to yourself, three weeks later, when you have forgotten where it came from. What you must not do is fill the gap by reverting to a title-keyed aggregate. That produces the illusion of precision from exactly the data problem the audit exists to catch.