What does recovering a model's weights 'only up to symmetry' leave an attacker holding?

## Where the symmetry comes from Two distinct rearrangements leave a piecewise-linear network's output completely unchanged. **Permutation.** The hidden units within a layer have no intrinsic order. Swap two of them — swapping the corresponding rows of the incoming weight matrix and the corresponding columns of the outgoing one — and every input produces exactly the same output as before. For a layer of n units there are n factorial such rearrangements, all indistinguishable from outside. **Positive rescaling.** A rectified piecewise-linear unit is positively homogeneous: scaling its pre-activation by a positive constant scales its output by the same constant. So multiply one unit's incoming weights and its bias by some positive c, divide its outgoing weight by c, and the two changes cancel exactly. Again nothing observable moves. Because both operations are invisible in the function, no attack that works from input-output behaviour alone can distinguish between them. An algebraic solve for the parameters therefore recovers an **equivalence class**, not the one weight matrix sitting on your servers. ## Reading the limit in the right direction This is a place where candidates commonly draw the wrong conclusion, in both directions. **The wrong optimistic reading:** 'they only got the weights up to symmetry, so they did not really get the weights.' Functionally this is empty. Every member of the class computes the identical function on every input — not approximately, identically. The attacker can serve it, fine-tune it, inspect it, and study it offline for further attacks. Symmetry costs them nothing they wanted. **The wrong pessimistic reading:** treating symmetry as a defect of the attack that makes the recovered model unreliable. It is not a source of error. The recovered parameters are exact within the precision achieved; symmetry is about which representative of an exactly-equivalent family you land on, not about how close you got. ## Where it genuinely changes what you can say The place symmetry actually bites is on the defender's side, in questions about provenance and evidence. If you are asked whether a model somebody else is serving is yours, comparing weight matrices entry by entry is not a valid test. A parameter-recovered model is *expected* to differ entry by entry — the units may be in a different order and each may be scaled differently — while computing precisely the same function. So an entry-wise mismatch is not evidence of independence, and you would need either a canonicalisation that quotients out the known symmetries or a behavioural comparison to say anything. The same caution applies to the reverse claim. Two models that agree closely in behaviour are not thereby proven to be copies either; agreement can come from similar tasks and overlapping data. Symmetry is a reason an entry-wise test is invalid, not a licence to substitute a behavioural test and treat it as proof. ## What an interviewer is listening for Three things, in this order. First, the mechanism — that permutation and positive rescaling are exact invariances of the computed function, so behaviour cannot see them. Second, the honest impact — functionally zero for the attacker. Third, the one place it does matter — that you cannot compare weights entry by entry to decide whether something was taken. A candidate who reports the symmetry as though it were a partial protection has understood the phrase but not the consequence.