An adversary reads a released malware classifier and infers a fact true of one tenant's whole fleet — does a per-record privacy bound stop that?

## The question the guarantee answers, and the one it does not A differential-privacy bound is a statement about a swap: take the training corpus, add or remove **one record**, and an observer of the released model cannot tell which corpus produced it by more than a bounded factor. Everything the guarantee protects is something that *changes* under that swap. A fact about a cohort does not change under that swap. If a tenant organisation's endpoint fleet is dominated by a particular configuration, that remains true after any single trace is deleted, and it remains true after quite a lot of them are. So the mechanism has nothing to say about it. This is not a flaw that better parameters fix; it is the design. A mechanism that hid corpus-level regularities would be a mechanism that prevented learning, and learning corpus-level regularities is what training is. ## Why group privacy does not rescue it The natural next move is to reach for the group argument: treat the tenant as a group of k records and chain the per-record bound k times. Two things go wrong. First, the arithmetic. At tenant scale k is in the thousands. The privacy parameter scales with k and the additive failure term degrades exponentially in it, so the chained statement is vacuous long before you get there — it permits everything. Second, and more fundamental: even a genuinely tenant-level accounting only bounds what changes when that **whole tenant** is present or absent. If the property in question is shared across many tenants, it survives removing this one too, and remains outside any per-contributor guarantee at any granularity. There is no unit small enough to be affordable and large enough to hide a population regularity. ## What the adversary is actually doing This is inference at a coarser granularity than membership. The adversary reading only the released weights is not asking 'was this trace in the training set'; they are asking 'what was true of the corpus this model was fitted on'. Cohort-level properties leak because the model was fitted to them, and the model file is the summary. The vantage requires no query channel and no per-row auxiliary knowledge — a released or on-device checkpoint is sufficient — which makes it a very cheap question to ask. And the payoff can matter more commercially than a single record would. That an organisation's estate skews a particular way, that one contributor's data is the reason a class exists at all, that a customer segment behaves distinctively — these are the facts a competitor or a counterparty actually wants, and none of them are records. ## What actually helps Because the gap is structural, the mitigations are structural: - **Do not put the distinguishing attribute in.** If the tenant identity, or a feature that proxies it, is never a training signal, the model has less cohort structure to give away. This is the cheapest lever and the most often skipped. - **Partition.** Train per-tenant or per-segment models so a released artefact summarises only the cohort entitled to see it. This costs a great deal in data efficiency and is sometimes still the right call for a shared-corpus product. - **Move the unit up, with eyes open.** Accounting with the tenant as the unit does bound what one tenant's whole contribution changes — and if the tenant's contribution is why the model works, that bound is expensive in exactly the way that matters. - **Control at a different layer.** Contractual terms, aggregation thresholds on anything reported back, and simply not releasing the checkpoint outside the trust boundary are all real controls, and none of them are the privacy parameter. ## The reporting obligation The direction of the claim has to be right in writing. A small privacy parameter establishes that an adversary gains little from the presence or absence of one record. It does not establish that nothing about a contributing organisation is inferable, and a model card that lets a reader draw the second conclusion from the first is misleading even when every number in it is correct. The honest form names the unit and then names the exclusion: cohort-level and population-level properties are outside the scope of the guarantee, by construction.