A vendor screens with a small classifier and generates with a frontier model: defect or design limit?

## The chair you are sitting in Somebody has to write a sentence into a tracker: *defect, or design limit*. The evidence is not in dispute. A request submitted through the host product's form field, restated so that none of the vocabulary the screening classifier was trained on appears, passed the screen and was answered by the generator. The question is what that fact **is**, and it is an adjudication, not a measurement. ## It is a claims question first A defect is a deviation from a claim. So the first artefact to read is not the classifier, it is whatever the vendor has told the host and the host's tenants that this layer does. | If the claim is… | Then this evidence is… | | --- | --- | | "the layer prevents category X reaching the model" | a defect, measured directly against that sentence | | "the layer reduces the volume of category X" | a design limit, and consistent with the claim | | nothing written down anywhere | the real finding — an unstated assurance that people are relying on | The third row is the common one and it is the most useful thing a write-up can surface. Teams routinely operate as though a screening layer were a boundary while their contracts say something weaker or say nothing. Forcing the claim to be written down is what converts a recurring argument into a decision. ## Who owns it The capability gap belongs to whoever chose the pairing. The vendor picked a compact classifier and a frontier-scale generator; the generator is licensed, not owned, so its comprehension moves on somebody else's schedule while the classifier moves only when the vendor funds retraining. That means the person who signs "design limit" is signing on behalf of a property that will drift, and drift in one direction. This matters more than usual because the feature is embedded multi-tenant. A design limit the vendor accepts is inherited by every tenant of the host product. Those tenants never saw the tradeoff, and in most cases the host did not either. "Accepted risk" written by one party and carried by hundreds is exactly the kind of decision that should have a name attached to it rather than being closed quietly by whoever triaged the ticket. ## The cost argument, stated honestly The reflex response is to add the filed restatement to the classifier's training data. That is cheap, and it works for that neighbourhood. What it does not do is close the class, because the class is not a set of surfaces — it is the region where a request stays outside a small model's learned boundary while remaining legible to a much larger one. Funding rounds of relabelling against an open-ended class buys neighbourhoods at a per-neighbourhood price, forever. So a defensible write-up says: here is the class, here is the pairing that produces it, here is what one round of retraining is expected to buy and what it is not. It does **not** promise that the layer can be made to hold the class, because that promise is the thing that keeps getting made and broken. ## What nobody should expect the dashboards to do One uncomfortable property of this class is that it is invisible from the layer's own instrumentation: a submission that carries none of the trained vocabulary is, from the screen's point of view, an ordinary clean request. Whichever label is chosen, it should be chosen knowingly rather than defaulted into by an absence of signal, because an absence of signal is precisely what this construction produces. ## How to write the decision A usable adjudication has four parts: 1. **The class**, stated as a property of the pairing, not as the restatement that was used — the restatement is the most perishable part of the finding. 2. **The claim it is being measured against**, quoted, or the observation that no such claim exists in writing. 3. **The label and its owner** — a named person who accepts "design limit" on behalf of the tenants who inherit it. 4. **A remeasurement date**, because the gap moves when either model version moves and the label is only true as of a version pair. ## When you should refuse to pick If the wording of the assurance is being decided in the same meeting as the label, the adjudication is circular — the answer becomes whatever the claim is retroactively edited to say. The right move is to state the class and hand the claim question back to whoever owns the customer-facing wording. That is not indecision; it is the only order in which the two questions can be answered honestly.