LLM & GenAI Attack Techniques

An outright override announces disobedience, which is the one shape refusal training and screening layers have seen most. Spans that carry supply a reason the task legitimately changed, so the model reclassifies its situation instead of defying its orders.

The validator checked the field's shape for a screen that only displayed it: length, character set, encoding. Prose passes all of that unchanged, and the assistant now downstream of the same bytes reads prose as something to act on.

The scan judged the string when nothing could act on it. At submission it was a rejected field value. It became an instruction only when a different component re-read it as operational input it was expected to act on.

Stripping HTML removes tags, scripts and attributes and keeps the text — and the text is the injection. A sanitiser built to stop a browser executing markup is a no-op against natural-language sentences a model reads as directive.

The chat box is the surface everyone watches - transcript-logged, sampled and hardened first. A display name or an attachment filename was designed as data, reaches the same model context, and nobody re-reads that path.

A per-message screen scores one message at a time against a threshold, and in a gradual escalation no single message is objectionable. The objectionable object is the whole transcript, which that screen never looks at as one thing.

A model's refusal is a trained propensity, not an enforced rule. Hundreds of consistent in-context examples showing the model complying supply competing evidence about how this exchange goes, and enough of them outweigh that propensity.

A refusal is a learned response to how a request looks, not an enforced rule. A persona card restates the request as a character's line in a scene, a shape the model was mostly trained to continue rather than decline.

A refusal is text the model generates token by token, and it usually begins a characteristic way. Constraining the answer's first words and forbidding qualifiers removes those openings, so the trained tendency to decline has much less to latch onto.

Capability generalises further than safety training does. The model still follows and answers the low-resource form competently, but the alignment data barely covered it, so the refusal behaviour that fires in a widely spoken language never engages.

The target thread is not in the assistant's context, so the payload cannot simply emit it. It must first make the model look the thread up, then send it. A one-step 'send everything' reaches only what is already loaded — usually a summary.

Displaying is enough. If the answer carries a reference the viewing client resolves on its own — a preview it expands, a resource it embeds — the request goes out while the answer is being drawn. Nobody clicks.

The forbidding line names one operation, repeating, and a translation is a different operation. Nothing marks the preamble as secret; there is only a sentence about one verb sitting beside the text, and the request never uses that verb.

An output screen matches on shape - credential patterns, key formats, blocks of JSON. A plain-English sentence about which operations a bot has matches none of those, so capability talk leaves as ordinary help text while carrying an inventory.

A per-reply cap bounds one response, not the total. Each submission is a fresh run over the same hidden context, so many submissions yield many capped slices, reassembled outside the system. Exposure counts per campaign, not per reply.

The prompt-injection label describes only how the instruction arrived: as supplier text the workflow read and treated as an instruction. It says nothing about why that workflow could release a payment hold, which is where the expensive half sits.

Only the half you ship. Your repository holds the system prompt, the surfaces and what the app does with returned text. The refusal that framing got past is trained behaviour in a supplier's hosted model, with no release of yours.

The work is the property that makes an assistant read the span as instruction, and that property is channel-independent. The supplier already writes every free-text field of the relationship, so a second carrier costs one more ordinary submission.

It proves the framing landed once, against one deployment, at one moment. Refusal is a trained propensity sampled at generation time, so a failed retry means that attempt was declined - not that the finding is wrong or the behaviour gone.

It bought coverage of one arrival path against one wording, which is real but narrow. The workflow's authority to commit a settlement write from supplier-supplied text is untouched, so the same instruction re-authored into another supplier carrier still reaches it.

Because the danger lives in the pair, not either tool. A per-capability review judges each on its own and never sees the combination, and the combination is assembled later, when a deployment wires a private-context read and a network-reaching write into the same session.

A per-request token cap bounds one model call, not one submission. An accepted upload fans out into extraction, a call per unit of content, an aggregation pass and retries, so dozens of individually compliant calls are billed to the operator.

Session teardown discards the conversation context, not the durable store. Anything the assistant extracted and saved during that session outlives it by design, so a planted claim is read back into later, unrelated conversations.

Causing a new call adds an operation nobody asked for. Supplying arguments leaves the operation exactly as planned and changes only what it acts on. The second is much quieter, because the call itself still looks routine.

Because the check compares each action against the recorded objective, and the recorded objective is what was edited. It tests consistency, not authenticity, so after the edit every action genuinely serves the goal on file and the verifier signs each one off.

Closeness to the query decides. A first-stage retriever scores each chunk by how near it sits to the question and has no notion of who wrote it, so a forum post and an official page compete on similarity alone.

A metadata predicate is only as trustworthy as where its value comes from. When status, date or type are copied out of the submitted document, the passage's author sets them too, so narrowing removes honest rivals and keeps the plant.

A review and a retrieval consume different artefacts. The reviewer reads an added hunk inside the whole page; the model receives one chunk cut from that page, alone. Approval evidences that a person read the file, not the fragment.

A reranking stage drops planted text shaped against an embedding model, which reads as nonsense, and promotes planted text written to read as the ideal answer. It scores relevance to the query, never who wrote the passage.

First-stage similarity search embeds query and passage separately and ranks on vector distance alone. That score never reads the text for fluency, authorship or truth, so text fitted to the encoder's geometry can outrank prose written for a human reader.

It measured harm categories in text: toxicity, violence, self-harm and the like, and found none. The answer was ordinary prose. Its effect came from the parser that gave part of it structural meaning — a property the classifier never scores.

Streaming sends tokens to the client as they are produced, so a screen that scores the finished answer returns its verdict after the opening has already been delivered. It can stop the remainder; it cannot recall what shipped.

A refusal describes the final answer and nothing else. A screen pointed at the answer never reads the deliberation panel or the trace record beside it, so refused substance and the model's stated objection sit outside its reach.

The score is hidden; the reply is not. A canned block card, a hedged answer and a normal answer are three visible outcomes, so every reply says which band the request landed in, and a few replies bracket the line.

The screen and the generator read the same string for different purposes. A term-based screen matches surface wording it was trained on; a capable generator resolves paraphrase, referents and framing, so meaning survives a restatement that contains no trained term.

A screen is a lossy view of stored text. Codepoints such as zero-width format characters and Unicode tag characters produce no glyph, but they are still stored, still copied, and still turned into tokens, so the model reads them.

The screen compares codepoints, not appearance. A Cyrillic letter drawn like a Latin one is a different codepoint, so the word no longer equals any stored term and the comparison returns no match, while the glyphs on screen are unchanged.

The reviewer read the document as rendered; extraction deliberately recovers what rendering omits — form-field values, annotation contents, off-canvas or notice-sized runs. Those are two different texts, and only the extracted one reaches the model.

The scan inspects audio bytes, and at that moment the instruction text does not exist. Transcription manufactures it afterwards. A clean verdict on the stored file says nothing about the transcript the model later reads.

Pasting an image vouches for why the user wanted it, not for who wrote the words in it. A multimodal model reads every legible sentence in the frame, so a planted directive reaches the context as content the model may follow.