In TextAttack, an attack recipe pairs a transformation that proposes candidate rewrites with a set of constraints those candidates must pass. What role do the constraints play in the search, and what happens to your reported success rate if you relax them?

## The composition, named Text has no continuous box to clip into, so TextAttack replaces the value range and the movable-feature mask with an explicit four-part object. A `textattack.Attack` is built from a **transformation** (proposes candidate rewrites, for example word substitutions at chosen positions), a list of **constraints** (rejects candidates), a **search method** (decides which surviving candidate to take next), and a **goal function** (queries the target model and says when the attack has succeeded). A published recipe is nothing more than a specific choice of those four, and the CLI runs one with `textattack attack --recipe <name> --num-examples 100`. Constraints come in recognisable families. *Preservation* constraints ask whether the rewrite still means and reads like the original — `WordEmbeddingDistance` on the substituted word, `UniversalSentenceEncoder` on the whole sentence, a part-of-speech check on the swap. *Capability* constraints limit how much may change at all — a cap on the fraction of words perturbed, a rule forbidding repeated modification of the same position or modification of stopwords. Both families are **your assumption about the attacker and the domain**, not a fact about either. ## Where they sit, and why that is the good news Constraints gate the candidate set at each step of the search, before the goal function spends target-model queries. That is structurally better than the numeric libraries, where the equivalent rule is code you bolt on after generation: here the feasible set is declared, inspectable and inside the loop. It is also the reason a TextAttack result is reportable at all — you can state exactly what capability and what preservation rule you assumed. ## What it costs The cost profile is the opposite of what people expect. The naive assumption is that the target model dominates. In practice a transformation can propose hundreds of candidates per position, and a semantic-similarity constraint is itself an encoder forward pass **per candidate**, so the constraint stack routinely dominates wall-clock time — a run that is mysteriously slow against a small fast local model almost always has an expensive constraint evaluating a very wide candidate set. The levers are ordering cheap constraints before expensive ones so most candidates die early, capping the candidates considered per step, caching encoder outputs, and setting an explicit query budget so a single stubborn example cannot consume the run. Budget arithmetic worth doing before you launch: candidates per position times positions times examples is the constraint-evaluation count, and it is usually one to two orders of magnitude larger than the target-model query count. ## Where the number misleads **Success is defined relative to the constraint set.** Loosening a similarity threshold reliably raises the reported rate, and part of that rise is not adversarial at all: the rewrite genuinely changed the sentence, so the model's new prediction may be *correct* and the original label is stale. That is exactly the invalid-row failure from the tabular case arriving through a different door. Two runs under different constraint stacks measure different things and must never be presented as a before-and-after about the model. **The denominator hides a second trap.** TextAttack's summary reports successful, failed and *skipped* attacks, where skipped means the model already got the example wrong, and the attack success rate is computed over the attacked examples only. A weak model skips a large share of the set, so its success rate is computed over the small favourable subset it originally got right. Comparing two models on that figure is comparing two different denominators; **accuracy under attack**, reported in the same summary, is the figure that stays comparable. ## What I would check The per-constraint rejection rate: a constraint that rejects essentially every candidate means the recipe is doing no search and the low success rate is an artefact of your own filter. Where the wall-clock actually went, so the cost story in the report is true. The skipped count alongside the success rate, and original accuracy beside accuracy under attack. And a human read of a sample of successful rewrites — the only real check that the meaning survived, and the one that catches a stale-label inflation no metric will show you.