The face encoder ships on-device but the match threshold stays server-side — what does that split still bound?

## Where the line actually falls A split deployment looks reassuring on an architecture diagram: the heavy model runs on the device, the sensitive decision stays in the data centre. The useful discipline is to ignore the diagram's boxes and ask a single question of every component — *did we hand this to the user?* Whatever shipped is the adversary's. Whatever the server computes for itself, from data the client did not supply, is still yours. Nothing else on the diagram is load-bearing. ## What the shipped half gives away With the encoder in hand, an adversary has an exact, differentiable copy of the function that turns an input into the representation your server will judge. That means: - unlimited local evaluation, at no cost and with no rate limit; - the gradient of any objective they choose with respect to the input, which is what makes searching for an input with a chosen output tractable rather than blind; - no transfer loss, because it is the deployed function itself and not an imitation of it; - complete invisibility, because none of this touches your service. So the encoder is not a control. It is a component the adversary now owns and can study for as long as they like. ## What the server half still holds The server retains three things that were never shipped: **The enrolled template.** The adversary does not have the reference the comparison is made against. This matters, and it is the most valuable half of the split — an attacker who wants to be accepted *as a specific person* has to produce something that lands near a target they cannot see. **The threshold.** How close is close enough is a policy decision the client never learns, and probing it costs real attempts against a real account. **The decision itself.** Whether to allow the action, demand a second factor, or refuse — plus everything the server knows that the model does not: the device, the account's history, the amount at stake. ## The failure mode this split invites The split is only worth something if the server *verifies*. The tempting design is for the app to compute the representation, or worse the verdict, and send it up; the server then compares or simply believes. Once the client is the adversary's, a submitted value carries no evidence about how it was produced. It does not attest that a camera was present, that a live person was in front of it, or that the shipped encoder was the thing that ran. It is a number the adversary chose. That is the difference between a *check* and a *report*, and after weights ship, only checks count. This is also why binding a submission to something the client cannot forge — a hardware-backed attestation of the device and app, a server-generated challenge that must be answered fresh — is what upgrades the remaining control from "we hope the client is honest" to something an adversary must actually defeat. Without such a binding, the server is not bounding the attacker; it is bounding a well-behaved user. ## Why rate limits look better than they are here A per-attempt limit on the login endpoint is a genuine control against someone guessing online. It is a much weaker one here, and the reason is where the work happens. An attacker holding the encoder does all of their searching offline: thousands or millions of local evaluations, none of which you observe. They arrive at your endpoint only when they already have a candidate they believe will clear the threshold. From the endpoint's point of view that is one attempt, or a handful, indistinguishable from a user with a smudged camera lens. The limit still caps how many *finished* candidates they can present, which is not nothing — it prices the residual uncertainty about the template and the threshold. But it never sees the search, and reasoning about it as though it did badly overstates the protection. ## Stating the bound honestly A correct one-line statement of this deployment's threat model reads: *the adversary possesses the encoder and may evaluate it without limit; the only quantity bounding them is what the server decides from the enrolled template, the threshold, and evidence the client cannot mint.* Anything you claim beyond that has to point at a specific server-side check, and each such check has to be robust to a client that lies about everything it reports. ## Partial possession is the general case The useful generalisation: shipped deployments are usually *partial* handovers. The encoder ships and the threshold does not; the ranking model ships and the eligibility rules do not; the detector ships and the escalation policy does not. In each case the security argument survives exactly as far as the part you kept, and collapses at the boundary of the part you sent.