Your copilot reads mail the attacker can write

The risky sentence is no longer only the one a person types into a chat box. It is the sentence already sitting in a ticket, a PDF, a pull request, or a web page the assistant was asked to summarize. If the assistant can do more than talk — if it can read the mailbox, open the drive, file a refund, or query a production system — then untrusted text is not content. It is a candidate instruction, executed with the user’s authority.

Researchers gave this a name in 2023: indirect prompt injection. Academic work, including a widely cited paper on compromising real applications that had wired large language models to tools, showed the shape of the problem without needing a cartoon jailbreak. The user asks for a summary. The document contains directions the user did not write. The model follows the document. The damage, when there is damage, is whatever the connected tool was willing to do.

A confused deputy with a corporate login

Security teams have met this character before. A confused deputy is a program that holds privilege and can be talked into using it for someone else. The classic deputy was a web server with a powerful token. The new deputy is an assistant that inherited a person’s mailbox and a fistful of plugins, then treated every paragraph it read as a possible order.

That is why the argument “we told the model not to obey instructions in documents” is not a control. Models are suggestible. Filters catch some attempts and miss others, and the attempts do not have to look like a slogan. A policy sentence that says the model should ignore rude requests does not survive contact with a long document the business actually needs summarized. The cartoon version of this bug is a single line demanding special treatment. Real incidents are duller. They look like email.

The blast radius is the token, not the paragraph. A summarizer that can only return text to the person who asked can embarrass you. It can also hide a poisoned recommendation: pay this invoice, trust this vendor, reset this factor. A summarizer that can send, delete, pay, or change a security setting can do the thing itself. Product teams add those tools because they are the demo. Attackers do not need to break the model. They need the model to be believed by a tool that was built to skip a click.

What the enterprise version looks like

By now the assistant is not a novelty tab. It is wired to mail, chat, tickets, file stores, and sometimes to internal APIs that were previously reached only by a human with a change ticket. Each connector is a new deputy. The security review that would have been obvious for a new service account is often skipped because the feature shipped as a checkbox in a suite the company already pays for.

Questions worth asking the owner of that suite, in writing:

• Can a document, message, or web page cause the assistant to call a tool, or can it only propose text?
• Which tools can move money, send a message, change permissions, or read a store broader than the file the user opened?
• Does a sensitive tool wait for a human confirmation that shows the action in the user’s own words, not the model’s?
• Whose credentials does the tool use: the user’s, a shared service account, or a tenant-wide key?
• Are tool calls logged somewhere a person can search after something looks wrong?
• What happens to content that came from outside the company? Is it labeled untrusted, or mixed into the same channel as the system prompt?

If the answer to the first question is “it can call the tool,” the rest of the review is mandatory. If the answer is “we are not sure,” that is also an answer. Turn the tool off until someone is sure.

Controls that survive a clever paragraph

No filter closes this. Classifiers help. They are not the boundary. The boundary is what the assistant is allowed to touch when it is wrong, because it will be wrong.

Keep untrusted content in a different channel from instructions. That is an architectural choice: the text of an email is data the model may quote, not a system message it must obey. Vendors implement this with uneven success. Ask them to show you the channel split, not a slide that says “guardrails.”

Scope every tool to the smallest action. A plugin that can “manage email” is four plugins pretending to be one, and three of them should not exist. Reading a single thread is not the same privilege as sending as the user or forwarding the entire mailbox.

Require a human confirmation before sending, paying, deleting, changing authentication, or opening a new external connection. The confirmation has to describe the effect. A dialog that says “the assistant wants to continue” is not a confirmation. A dialog that says “send this invoice to this new account” is.

Do not give a summarizer a standing production credential. If a workflow needs production data, put a narrow, audited query in front of it and keep the credential with the query service, not with the model host. Treat the model’s output as untrusted when it flows into another system. A ticket filed by an assistant should not be able to authorize a deploy.

Log the tool call: who asked, what document was in context, what action was attempted, whether a human approved it. You will need that record the first time a refund goes to the wrong place and nobody remembers clicking yes.

OWASP’s Top 10 for Large Language Model Applications has listed prompt injection at the top of the practitioner list for a reason. It is not an exotic research topic anymore. It is a procurement question. Buy the assistant the way you would hire a junior administrator. Write down the systems it may touch. Assume it can be talked into a bad idea by a document it was asked to read. Then decide whether that bad idea is allowed to leave the chat window.

This briefing describes the impact of indirect prompt injection and the defensive boundaries that matter. It does not include attack payloads.