Why can an attacker train shadow models against a credit-decision API without holding any of its training rows?

## The claim being tested A membership-inference question asks one bit about one record: *was this row in the model's training set?* One family of attacks answers it by thresholding a signal the target returns for the candidate record. The family this question is about does something different: the adversary first builds their own models — **shadow models**, or stand-ins — and uses them to learn what a trained-on response looks like, before ever pointing anything at the target. The engineer's instinct on first hearing this is that the attack is not real, because "they would need our training data to build shadow models, and they do not have it." That instinct is wrong, and understanding why is the whole point of the leaf. ## What the adversary actually assumes Set the scene concretely. A consumer credit-limit-increase decision model is exposed to partner banks over an ordinary API. Send an applicant row, get back a probability vector over the decision classes. No weights, no gradients, no explanations, no per-record loss. The adversary here holds: - **ordinary API access** to the target — the same access a partner bank has; - **their own compute**, enough to train several models of their own; - **a sample of applicant-shaped rows drawn from the same population** the target's training data came from — public lending-registry aggregates, census microdata, or any other source describing the same applicants with the same kinds of fields. That third item is the assumption people mis-state. It is *distributional*, not *set-level*. The adversary is not trying to guess which rows we trained on; they are trying to learn a general fact about how a model of this kind behaves on rows it has seen versus rows it has not. ## Why same-distribution data is enough Membership leakage is a consequence of imperfect generalization. A trained model fits what it saw more tightly than what it did not: it is more confident, and more often right, on training members. That difference in behaviour is a property of *models trained on this kind of data*, not a property of our particular rows. So the adversary can reproduce the phenomenon on their own side. They train stand-in models on their own sample, deliberately holding some records out of each one. Now, for every record they own, they know the ground truth — this stand-in saw it, that stand-in did not — and they can observe what each stand-in returns for it. Those observations are **labelled training data for a discriminator**: examples of a member response and examples of a non-member response, generated for free because the adversary controlled the experiment. The discriminator they fit is a decision rule over responses. It encodes something like "a response this peaked, this well-margined, for this predicted class, is characteristic of a record the model trained on." That statement does not mention our dataset anywhere, which is exactly why it can be carried over and applied to responses our API returns. ## What "same distribution" has to mean It is a real assumption, not a free pass, and it is the thing to interrogate in an interview. The stand-ins must over-fit *in a similar way* to the target for the learned rule to transfer. That needs a comparable feature schema and encoding, a comparable class balance, comparable label noise, and comparable data volume relative to model capacity. It does **not** need identical rows, and it does not need the same architecture: the rule is about the shape of confidence, and models of different families trained on similar data show similar member-versus-non-member behaviour. Where it breaks is where the target's training population is genuinely peculiar — a book of business skewed to a segment the public sample under-represents, an in-house labelling convention no outsider replicates, or a training set orders of magnitude larger than anything the adversary can imitate. Then the stand-ins learn a different confidence pattern and the transferred rule performs near chance. ## What it costs, and what it buys The cost is the stand-in sample plus the compute to train several models — a data-assumption-and-compute cost, not a per-call bill. What it buys is a **reusable, offline-calibrated discriminator**: once fitted, it is a rule the adversary can apply to responses from the real target, without ever having had ground truth about the target's set. And note the direction of the conclusion. Success proves that our model's responses look like a stand-in's responses to its own members. It does not prove the adversary has our data, has our model, or can read any record out of it. The output is one bit per candidate — and whether that bit is a disclosure depends on what membership *means* about the person behind the row. ## The one-line answer Shadow models are trained on data the adversary already has or can obtain from the same population; they exist to teach a general rule about over-fit behaviour, not to reconstruct our training set. "They would need our data" is not a defence.