A creative-tooling service loads community checkpoints on credentialed hosts - which risk would you detect?

## Two risks, two completely different detection stories The practical version of the non-substitutability argument is about **evidence**. A team that has decided to use community checkpoints will eventually ask what would happen if one of them were malicious, and the honest answer depends entirely on which of the two channels the adversary used. ### The loader channel is a normal detection problem When reading a published file is an execution event, everything about the incident is familiar. There is a moment: the file was opened. There is a process: whatever read it. There is a host: usually a developer machine or a build runner, and in a creative-tooling setting frequently one holding cloud credentials, storage tokens or a package-publishing key. And there is evidence of the kind normal instrumentation collects - a child process, a write outside the expected path, a connection to somewhere nothing else on that host talks to, a credential used from an unusual context. This is good news, and it should be said plainly: the loader channel is defensible with security engineering you already own. It is also worth noting where it actually lands. Candidate checkpoints are opened during *evaluation*, which is upstream of every review gate a policy imposes on production. A control that says "only approved checkpoints run in production" does not touch the machine where the file was first read. ### The weights channel produces essentially no evidence Now the other one. A network with a conditional behaviour trained into it does nothing at load. It spawns nothing, opens nothing and writes nothing, because it is a tensor of numbers being read into memory. At inference it behaves like the model it appears to be, on every input except the ones the publisher chose - and your monitoring is watching aggregates, which are normal by design. So consider what each of your existing signals would say: - **Endpoint and network telemetry on the loading host**: silent. Correctly silent. Nothing happened there. - **Aggregate quality dashboards**: flat. The attack requires them to be flat. - **Error rates, latency, resource use**: unchanged. A conditional adds no measurable cost. - **Output review or user reports**: the only channel with any chance, and only if the triggered behaviour is something a user notices *and* escalates *and* someone connects to the model rather than to a prompt or a bad seed. The conclusion a senior engineer should reach out loud: **"our monitoring did not fire" is a statement about the loader channel.** It is not a statement about the model. Reporting it as one is the same substitution error as accepting a format rule as a clean bill of health, made at run time instead of review time. ### The responses differ too, and that surprises people Suppose you confirm the loader channel fired. The model is irrelevant to the response. You have a compromised host: contain it, rotate every credential that process could reach, determine what those credentials touched, and treat any artefact that machine produced afterwards as suspect. The weights may be perfectly honest, and that changes nothing about the work. Now suppose you confirm the weights channel instead. There is nothing on the host to clean - the machine behaved correctly the entire time. The work is model-shaped: stop shipping that checkpoint and everything derived from it, decide whether outputs already produced need review, and ask the harder question of what downstream systems were letting the model's output decide. Credential rotation buys you nothing here, and an endpoint sweep will come back clean and be misread as reassurance. Because the responses share almost no steps, a team that has only rehearsed one will handle the other badly. It is worth having said, before an incident, which of the two you are actually equipped for. ### What you can do about the invisible one Since detection is weak, the leverage moves elsewhere: reduce what the model's output is allowed to do without a check, narrow which sources are eligible for deployments where the output has consequence, and shrink the set of hosts permitted to open unreviewed files so the loader channel is small even before the format rule closes it. None of that detects a backdoor. It limits what one is worth, which is the honest goal when detection cannot be made reliable. ### The answer in one line *You will see the file executing and you will not see the weights lying, so design the review and the response around that asymmetry rather than around a single verdict.*