The model shipped inside a past mobile app release was extracted — what does shipping a replacement buy you?

## The instinct, and why it misfires The reflex on learning that a shipped model was extracted is to treat it like a leaked secret: rotate it, ship a new one, close the ticket. The analogy breaks in two places. **A weight file has no revocation.** Rotating a key works because a verifier stops accepting the old one at a moment you control. A copied model file is not presented to any verifier — the adversary runs it themselves. There is nothing to refuse. The old copy keeps working as long as it is *useful*, and its usefulness is set by whether your service still behaves the way it did. **You do not control the client population.** A new build reaches users on their update schedule, not yours. Some fraction never updates. So even where the new model does change the picture, the change arrives as a long tail rather than a cutover, and during that tail the old file and the old client protocol are both live. ## What is actually still yours The same thing as before the incident: whatever the server decides for itself. That means remediation has to be phrased as *changes to what the server accepts*, and the new model is only one input to that. Practical shapes this takes, from most to least reliable: - **Add a decision the shipped model cannot compute.** A check that runs server-side on evidence the client does not produce is not defeated by holding an old encoder, because holding it says nothing about that check. - **Bind submissions to something unforgeable.** A hardware-backed attestation of device and app build, or a fresh server-issued challenge, means the server is no longer accepting anonymous values of unknown origin. - **Re-enrol or re-key the references the comparison uses.** If the adversary's advantage came from anything derived from stored references, changing those is the part that expires their work. - **Retire old client versions on a published schedule.** This is what finally makes "we shipped a new model" true, and it costs you the users who cannot or will not update. Notice that only the last of these mentions the model at all. ## Reasoning about the tail Treat the exposure window as a distribution, not a date. Useful questions for the review: what fraction of active sessions run the affected build today, what does that curve look like at thirty and ninety days, what is the floor it flattens to, and what is the oldest version the service still accepts at all. If the answer to the last one is "anything that ever shipped", the extraction is permanent by construction and that — not the extraction — is the finding to escalate. ## What the incident should change on paper Before the extraction, the deployment's documented threat model may well have said black-box: an adversary reaches the model through the app, so they are metered and observed. That was never true once weights shipped, and the extraction merely demonstrated it. So the register entry changes twice over. The *access class* becomes white-box, permanently, for this and every future release that ships weights. And any robustness figure quoted for this system has to be the white-box figure, because that is the operating point. A black-box number here was measuring the effort of unzipping an application, which was never a security property. ## The judgment call inside the response There is a real tension worth naming. Cutting off old client versions quickly shortens the exposure but locks out users on old devices, which in a consumer banking context is a support and fairness problem, not just an engineering one. Leaving them on indefinitely means the extracted artifact is exploitable forever. The honest position is to pick a retirement date, publish it, and accept the cost — and to make the *server-side* additions immediately, since those help the tail without requiring anyone to update. ## The one-sentence version Rotating a model that has already been handed out does not remove the adversary's copy; it only matters to the extent that your server stops honouring what that copy produces, and that is a decision you make in the data centre, on a schedule set by your installed base rather than by your release.