Multi-Agent Systems: Patterns, Failure Modes and Trust Boundaries

A multi-agent system divides a task among several model-driven components, each with its own instructions, tools and context. The idea is attractive: specialists can be simpler to prompt, run in parallel and keep their contexts small. The cost is added complexity, more model calls and, if designed carelessly, a larger attack surface. This article covers the common patterns, how they fail, and the security boundaries that matter.

Three common patterns

Orchestrator and workers

A coordinating agent breaks the task into subtasks, delegates each to a worker agent, and combines the results. Workers may have different tools: one searches documents, another runs code. This is the most widely used pattern because control stays centralized.

Pipeline

Agents run in sequence, each taking the previous one's output: for example, research, draft, review, publish. It is easy to understand and test, but errors propagate downstream and a bad early stage poisons everything after it.

Debate or critic loops

One agent proposes and another critiques, iterating until they agree or a limit is hit. This can improve quality on reasoning tasks but multiplies cost and can converge on a shared mistake.

When multiple agents are not worth it

Splitting work across agents adds coordination overhead, and models are often better served by one agent with good tools. Consider multiple agents when:

• Subtasks are independent and can run in parallel.
• Different parts need different tools or permissions (a reason that is also a security benefit).
• A single context would become too large to manage.

If none applies, a single agent or a simple workflow is usually cheaper, faster and easier to debug.

How multi-agent systems fail

• Compounding errors. A small mistake by one agent is treated as fact by the next.
• Lost context. The orchestrator's summary omits a detail the worker needed.
• Runaway loops. Agents keep delegating or re-checking each other.
• Conflicting actions. Two agents modify the same resource.
• Cost blowup. Each message is a model call, so token use grows quickly.
• Opaque behavior. With many agents talking, finding the cause of a bad outcome is hard without good logs (observability).

Security: every message is untrusted input

The key principle is that a message from another agent is data, not a command. If agent A reads a web page containing an injected instruction and passes its summary to agent B, then B receives the attacker's text with the apparent authority of a colleague. This is how a low-privilege agent can be used to push a high-privilege one into acting.

Trust boundary rules

1. Separate privileges by agent. The agent that reads untrusted content (web, email, uploaded files) should have no tools that send, delete or pay. The agent that performs actions should never read raw untrusted content directly.
2. Pass structured data, not free text. Have a reading agent return a fixed JSON schema (fields, enums, bounded strings). Validate it in code before the acting agent sees it. Structure removes most of the room for hidden instructions.
3. No privilege inheritance. A worker's permissions come from its own role, not from whoever called it.
4. Authorize at the tool, not in the prompt. The tool checks the caller's identity and permissions regardless of what any agent claims.
5. Gate irreversible actions. Require human approval on any externally visible or destructive step, no matter which agent requests it (human-in-the-loop design).
6. Bound the system. Maximum depth of delegation, maximum total steps, token and spend budgets for the whole task, not just per agent.

A safer reference design

A pattern that works for many applications:

• A reader agent with read-only tools processes untrusted content and returns validated structured output.
• A planner/orchestrator reasons only over that structured output.
• An actor agent with narrowly scoped tools executes approved steps, with human approval for sensitive ones.
• A logging layer records every message and tool call with a shared task ID.

This mirrors a long-standing security idea: keep the component that handles hostile input separate from the component that holds power.

Testing

Test the whole system, not individual agents. Include adversarial documents that try to instruct downstream agents, tasks that should hit step limits, and cases where one worker fails. Measure task success, cost per task and the rate of unapproved sensitive actions (which should be zero). For general guidance see Evaluating and Testing AI Agents.