With only generation access to a fine-tuned code model, which training strings come back verbatim?

## The set is smaller and stranger than people expect Asked what an attacker can pull out of a model with ordinary generation access, engineers give one of two wrong answers: "nothing, it is a statistical model" or "anything, given enough queries". Both are wrong in the same way — they treat recoverability as uniform across the corpus. It is not. It concentrates sharply, in two families, for two different reasons. ## Family one: duplication A span that appeared many times across the training text has an outsized claim on the objective. Every copy is another place where getting that exact sequence right lowers loss, and there is no competing pressure to abstract it, because the same characters really do follow the same prefix every time. This is why boilerplate is the most reliably extractable content in any corpus: licence headers, generated file preambles, a config block copied into forty services, a fixture pasted into every test module. The relationship is monotone in the useful direction — more copies, more reliable recovery — which is what makes duplication count the single most predictive attribute you can know about a span. ## Family two: entropy The second family is counter-intuitive and it is where the real damage lives. Take a string with no internal structure — a random-looking key, a long opaque identifier, a machine-generated token embedded in a fixture. Language modelling cannot predict it from anything. There is no rule to learn, no neighbouring example to generalise from, no pattern to compress it into. So if the trained model finds that string unusually unsurprising, that can only be because the parameters retain something specific to it. Storage is the only mechanism available. The practical consequence is the one that surprises reviewers: a structureless literal seen three times can be more recoverable than an ordinary comment seen ten times. Count and entropy trade against each other, and secrets are exactly the content that sits at the extreme of the second axis. ## What sets the ceiling Model capacity. A larger fine-tune over the same corpus can hold more, so the recoverable set grows with parameter count. This is a fact about the training run, not about the deployment, and it is one of the few levers an owner actually has — alongside the corpus itself. ## What the adversary controls, and what they do not The attacker in this picture has nothing special: prompt in, continuation out, no weights, no gradients, no returned log-probabilities. What they spend is queries and the specificity of the prefixes they choose. A prefix that genuinely preceded the span in the corpus pulls hardest, which is why someone who uses the product — who knows the house conventions, the file layout, the naming style — is a stronger adversary than a stranger at the same query budget. But that budget only changes the harvest rate. It samples a set whose membership was decided when the run ended. Ten times the queries recovers more of the same set sooner; it does not cause an unstored string to appear. This asymmetry is the whole reason the defensive question is a corpus question. ## Reading a memorization audit The attached table is the shape such a result comes in, and the row worth arguing about is the fourth. Prose seen roughly eight times is recovered three times in forty; a high-entropy literal seen three times is recovered seven times in forty. A reviewer who ranks rows by copy count alone reads that table backwards. Note also what the last row does not establish: one recovery out of forty single-occurrence literals is a small number, not zero, and the sampling was over spans somebody chose to test. ## Where this leaves a defender You cannot enumerate the recoverable set, but you can characterise its inputs, and both are auditable without touching the model: which classes of string entered the corpus, and how often each class was duplicated. That is why the mitigation conversation is always about the training data and never about the endpoint. Metering the API, hiding scores, limiting query volume — all of these slow a harvest of a set that remains exactly as full as it was. ## The summary an interviewer is listening for Duplication and entropy, both fixed at training; capacity sets the ceiling; the attacker's budget sets the rate, not the contents; ordinary prose paraphrases and does not return.