Multi-Agent Systems: Orchestrator-Workers, Handoffs and Trade-Offs
Multi-agent LLM system design: orchestrator-worker, handoffs and parallel subagents, shared state, failure handling, cost, and when a single agent is better.
A multi-agent system splits work across several LLM agents, each with its own instructions, tools and context, coordinated by code or by another agent. It can help when a task is broad enough to parallelise, or when separate roles need separate context and permissions. It also multiplies cost, latency and failure modes. This guide covers the main coordination patterns, how state and failures are handled, and how to decide whether you need more than one agent at all.
Why split into several agents
- Parallelism: research ten sources at once instead of one after another.
- Context isolation: each subagent reads its own large material and returns a short summary, keeping the coordinator's context small.
- Specialisation: different prompts, tools and even models per role (a cheap model for extraction, a strong one for planning).
- Permission boundaries: a billing agent holds refund tools; the general agent doesn't.
Every extra agent adds model calls, tokens and new ways to fail (miscommunication, duplicated work, loops). A single agent with good tools is often enough; reach for multiple agents when one context or one sequential loop clearly can't do the job.
Coordination patterns
Orchestrator and workers
Good for research, large code changes and analysis across many documents. The orchestrator's task descriptions must be precise (objective, scope, output format, budget), or workers duplicate effort or wander.
Handoffs (router and specialists)
Other patterns
| Pattern | How | Use |
|---|---|---|
| Pipeline | Fixed sequence of specialised agents (extract → analyse → write) | Predictable multi-stage processing |
| Generator and critic | One agent drafts, another reviews against criteria, loop a bounded number of times | Quality-sensitive writing and code |
| Debate / voting | Several agents answer independently; aggregate or let them argue | Hard judgments; expensive |
| Shared blackboard | Agents read and write a common state store | Loosely coupled long-running work; hard to debug |
State and communication
- Pass summaries, not transcripts: workers return compact, structured results; the coordinator doesn't need their full reasoning.
- Shared artifacts in storage: large outputs (files, tables) go to a store and are passed by reference.
- Typed messages: define the handoff payload (task, constraints, results, confidence) like an API contract.
- Durable orchestration: for long jobs, persist the plan and each subtask's status so a crash resumes; a workflow engine or task queue suits this.
Failure modes and controls
| Failure | Control |
|---|---|
| Workers duplicate or overlap work | Precise task boundaries; orchestrator tracks assignments |
| Endless delegation or ping-pong handoffs | Max depth and max handoffs; repeated-handoff detection |
| Cost explosion | Budgets per task, per worker, and in total; cap parallelism |
| One worker fails or hangs | Timeouts, retries for idempotent work, partial results allowed |
| Error propagation (a wrong worker result poisons the answer) | Ask for sources and confidence; verify key claims |
| Privilege escalation via another agent | Each agent gets only its own tools; content from other agents is untrusted input |
Cost and latency
Single agent research task : ~8 steps × 6K tokens context ≈ 50K tokens, ~60 s sequential
Orchestrator + 4 workers : planner 10K + 4 × 40K (each reads its own sources)
+ merge 15K ≈ 185K tokens, ~25 s with workers in parallel
→ ~3-4× the tokens for ~2.5× faster and broader coverage. Worth it for
high-value research, not for a password reset.Evaluating multi-agent systems
Evaluate the end result first, then the coordination: did the orchestrator decompose sensibly, did workers stay in scope, how many tokens per successful task? Trace every agent's calls under one parent trace id so a failure can be followed across agents. See LLM evaluation and LLMOps.
Where it’s used
An orchestrator spawns parallel searchers that each read many pages and return condensed findings.
Triage plus specialist agents with separate tool permissions, and handoff to humans.
A planner splits a change across files or services, with test-running workers verifying each part.
In an interview
- Justify multiple agents (parallelism, context isolation, permissions) or argue for one.
- Pick a pattern: orchestrator-workers, handoffs or a pipeline.
- Define compact, typed handoff payloads and durable state.
- Cap depth, handoffs and budgets, and trace across agents.
Frequently asked questions
When should I use multiple AI agents?
When work can be parallelised, when subtasks need large separate contexts, or when roles need different tools and permissions. Otherwise a single agent with good tools is cheaper, faster and easier to debug.
What is the orchestrator-worker pattern?
A lead agent plans a task, delegates focused subtasks to worker agents that run in parallel, and combines their condensed results into the final answer.
Further reading
- Building effective agentsAnthropic
- How we built our multi-agent research systemAnthropic Engineering