Defending against prompt injection

Treat retrieved pages, emails, documents, tool outputs, and user-uploaded files as untrusted data, even when they contain text claiming to be system instructions. Keep instructions and data in distinct channels or fields, label provenance, minimize the data sent to privileged agents, and never let content grant itself permissions or select secrets to reveal. Enforce authorization and destination allow-lists outside the model, require confirmation for sensitive writes, and test indirect-injection cases where malicious instructions arrive through a trusted connector.

Prompt injection is the top-ranked risk in the OWASP Top 10 for LLM applications for a structural reason: an LLM has one input stream, and everything in it — your system prompt, the user's request, a scraped web page, a tool result — is just tokens. Any text the model reads can try to act like instructions. That is why the UK NCSC argues that prompt injection is not SQL injection: with SQL you could fix the vulnerability class with parameterized queries; with LLMs there is no equivalent boundary inside the model, so defenses must assume some injections get through.

The threat model that matters for agents

Direct injection ("ignore previous instructions") is the easy case. The dangerous one for agents is indirect injection: malicious instructions arriving through content the agent was legitimately asked to process — a web page it retrieved, an email it summarized, an issue comment it triaged. Simon Willison's "lethal trifecta" names the condition to watch for: an agent that combines (1) access to private data, (2) exposure to untrusted content, and (3) the ability to communicate externally can be manipulated into exfiltrating the private data. If your agent has all three, an injection is not a wrong answer — it is a breach.

Defenses that survive contact

Because the model layer cannot be fully hardened, effective defense is architectural — contain what a compromised model can do:

Separate instructions from data. Delimit and label untrusted content; tell the model retrieved text is observation, never command. This raises the bar but is not sufficient on its own.
Break the trifecta. Deny at least one leg: strip external-send tools from agents that read untrusted content, or wall private data off from them. The Design Patterns for Securing LLM Agents paper (arXiv:2506.08837) systematizes this — patterns like plan-then-execute and context minimization all work by constraining what the untrusted-content-reading component is able to trigger.
Enforce authorization outside the model. Allow-lists for destinations, scoped credentials, and permission checks live in code the model cannot rewrite. Content must never be able to grant itself capability.
Gate irreversible actions. Sends, payments, deletes, and config changes get human confirmation — so a successful injection can propose but not complete the damaging step.
Test indirect paths. Red-team through your connectors: plant instructions in a test email, calendar invite, or web page and verify the agent surfaces rather than obeys them.

The mindset

Design as if injection will succeed occasionally, and make success boring: least privilege, broken trifecta, external authorization, and approval gates turn "the model got fooled" from an incident into a non-event.

Sources: OWASP LLM01 — Prompt Injection · NCSC — Prompt injection is not SQL injection · Willison — The Lethal Trifecta · Design Patterns for Securing LLM Agents (arXiv:2506.08837).

Defending against prompt injection

The threat model that matters for agents

Defenses that survive contact

The mindset

Related notes