When is a shadow-model membership build the wrong tool against a deployed classifier?

## The chair This is the red-teamer's question, not the defender's. Somebody has to decide whether to spend a week assembling a same-distribution corpus and training several stand-ins, and then has to stand behind whatever number comes out. The failure mode is committing to the build, getting a result, and being unable to say what it establishes. ## The three costs, stated honestly **1. A sample you can defend.** The build assumes the adversary can obtain rows drawn from the same population as the target's training data. For a consumer credit-limit model, public lending-registry and census microdata plausibly do describe that applicant population. "Plausibly" is doing work: what has to match is not a marginal distribution or two but the feature schema and encoding, the class balance, the level of label noise, and — importantly — the skew. A target trained on a book of business concentrated in one segment is not well imitated by a population-wide public sample, and the stand-ins will over-fit in a different pattern. **2. Several training runs.** The rule needs the same record seen included and excluded across runs to separate the systematic member effect from one run's accidents. That is compute, and it scales with how many stand-ins you decide you need — a number an engagement lead will ask you to justify. **3. A response rich enough to carry the signal.** The discriminator is fitted on the shape of a returned distribution. The vantage assumed here is a full probability vector over the decision classes. Take that away and the build does not become impossible, but its yield falls and its per-record cost rises. ## When it is the wrong tool - **The distributional assumption cannot be defended.** If you cannot name a source and say why it matches, the build's headline number is uninterpretable and you have spent the week for nothing. - **The target generalizes well on the axis you care about.** Membership leakage is a consequence of imperfect generalization; a model trained on an enormous, well-regularised corpus has little average gap to exploit, and every membership attack — cheap or expensive — degrades together. Note the qualifier *average*: atypical records, rare subgroups and duplicated rows still stand out even when the aggregate gap is small, so "we trained on forty million rows" is a statement about the average and not an immunity claim. - **A cheaper per-record test already answers the engagement's question.** The stand-in family is the expensive branch of this attack class; a build that costs several models and a corpus has to buy something the cheap branch does not — a rule calibrated offline, reusable, split per class, needing no calibration point from the target. If the engagement only needs to know whether *this* endpoint leaks at all, spend accordingly. - **You cannot state a base rate.** An attack result is an advantage over the base rate of membership among the candidates being tested. If nobody can say what fraction of your candidate list was plausibly in the training set, the number has no denominator and the finding will not survive review. ## Reading the result you produce The direction of the claim is where these findings usually go wrong. A separation measured on your own held-out stand-in records is a property of *your* models. It is what the rule achieved under a distributional match you constructed, which is the friendliest possible condition. Two columns are then missing before it says anything about the deployed model: whether it still separates on responses the target returns, and what the base rate among candidates was. There is a second subtlety worth carrying. A rule with a modest average advantage may be confidently right about only a handful of records — usually the unusual ones — and near chance on the rest. For a privacy finding, that handful is often the whole story, because it is the atypical record whose membership means the most about a person. Reporting only the average can understate the finding as easily as overstate it. ## What a good answer sounds like "Before I build, I need a data source I can defend as same-distribution, a reason several stand-ins are affordable, and a candidate list with a known base rate. If the endpoint returns a full probability vector and the training population is approximable, this buys a reusable per-class rule. If any of those three is missing, I say so and pick a cheaper test rather than producing a number I cannot interpret."