Scope
What happens when one agent isn’t enough: the topologies for wiring several LLM-driven agents together (orchestrator-worker, hierarchical, network/swarm, handoffs, debate/voting), why delegation quality is the dominant failure mode, the brutal cost/reliability tradeoff (~15× the tokens of a chat), how to evaluate non-deterministic multi-agent trajectories, and — honestly — the live 2026 debate over whether you should build these at all. Multi-agent suits breadth-first, read-only, parallelizable work and hurts on interdependent, decision-consistent work. For the single-agent loop, see 08 - Agents & Agentic Systems; for deterministic code-orchestrated pipelines, see 09 - Workflow & Orchestration Patterns.
An orchestrator/lead agent fans out subagents to work in parallel, then synthesizes their results — the canonical multi-agent shape.
Related: Gen AI Engineer, 08 - Agents & Agentic Systems, 09 - Workflow & Orchestration Patterns, 13 - Evaluation & Evals
1. The concept: when (and why) to add a second agent
A multi-agent system is several LLM-driven agents — each with its own context window, prompt, and tool set — coordinating to solve one task. The point is not “more agents = smarter.” The point is that some tasks have structure a single context window can’t hold: too many independent paths to explore, too much intermediate data, or sub-tasks that genuinely benefit from a fresh, focused context.
The honest framing, which the rest of this note keeps returning to:
Multi-agent helps on breadth, hurts on depth
Multi-agent systems shine on breadth-first, read-only, parallelizable work — research, “find everything about X across many sources,” fan-out enumeration — where subagents explore independent paths simultaneously and the combined findings exceed one context window. They hurt on interdependent, decision-consistent work — most coding, refactors, anything where one agent’s choice must stay coherent with another’s — because separate context windows let agents act on conflicting assumptions. This is the real consensus underneath the loud disagreement in §10.
Before reaching for multiple agents, exhaust the cheaper tiers: a single agent (§08 - Agents & Agentic Systems), then a code-orchestrated workflow (§09 - Workflow & Orchestration Patterns). Add agents only when the task is genuinely parallel and the value clears the ~15× token bill (§7).
2. Orchestrator-worker / supervisor
The dominant topology, and the one with the most framework support in 2026. A lead (orchestrator/supervisor) agent owns the user-facing task: it decomposes the goal, spawns worker subagents with crisp objectives, runs them in parallel, and synthesizes their results. Workers never talk to each other or the user — they report up to the lead.
Anthropic’s research system is the reference implementation: a Claude Opus lead spawns 3–5 Claude Sonnet subagents at once; each subagent in turn fans out its own parallel tool calls. On internal research evals this beat single-agent Opus by 90.2%.
The "~80% of variance is token usage" thesis
Anthropic found that token usage alone explains ~80% of the performance variance across their research evals; number of tool calls and model choice were the other two factors. Read carefully: multi-agent works mainly because it spends enough tokens to solve the problem — multiple agents with multiple context windows simply pour far more inference at a problem than one agent can. That’s a feature for breadth-first research and a liability everywhere the spend isn’t justified (§7). It also means: if a single agent with a bigger budget would do, you don’t need the topology.
In the OpenAI Agents SDK, this is the agents-as-tools (manager) pattern: agent.as_tool() wraps a specialist as a callable tool the orchestrator invokes like any function. The manager keeps the conversation; the specialist answers a bounded sub-question and returns.
- Pros: clear control flow; parallelism is natural; each worker gets a clean, focused context; easy to reason about responsibility.
- Cons: the lead is a bottleneck and single point of failure; synthesis quality caps the whole system; token cost scales with worker count.
- Caveats: workers can’t see each other’s work — if sub-tasks are interdependent, this topology silently produces incoherent results. Use it only when sub-tasks are genuinely independent.
- Risk mitigation: give each worker a self-contained objective (everything it needs in its prompt — see §6); cap worker count and depth; have the lead validate/merge rather than blindly concatenate; checkpoint long runs so a worker crash doesn’t restart everything.
- In practice: the default starting topology. If you’re unsure which shape to use, this is it.
3. Hierarchical
Orchestrator-worker, recursed: a top-level lead delegates to mid-level coordinators, which delegate to leaf workers — a tree. Fits problems with natural decomposition (a “research director” → “section leads” → “fact gatherers”).
| Pros | Scales to large tasks; each layer reasons at one altitude; mirrors human org structure |
| Cons | Latency and token cost compound per layer; errors propagate downward; harder to debug |
| Caveats | Most frameworks (and Anthropic’s own guidance) cap useful delegation depth — beyond one or two levels, coordination overhead usually outweighs the gain |
Depth has sharply diminishing returns
Each extra layer multiplies tokens and latency while adding more places for context to fragment. In practice, one level of delegation (a flat orchestrator-worker) handles most real work; reach for a second level only when the task tree is genuinely deep and the branches are independent.
- Risk mitigation: bound the tree depth explicitly; summarize upward at each layer so parent context stays small; treat any layer that just forwards messages as a smell to flatten.
4. Network / swarm
No fixed hierarchy — agents are peers that route work to whichever agent is best suited, dynamically. Control isn’t centralized; it flows to the relevant specialist.
This is what handoffs (an OpenAI Agents SDK primitive) implement at the topology level: rather than a manager calling a specialist as a tool and keeping control, one agent transfers control to a peer that takes over the conversation. Critically, on a handoff the full message history is transferred — the receiving agent sees the entire conversation as if it had been there all along.
Handoffs vs. agents-as-tools — pick by who owns the conversation
- Agents-as-tools (supervisor, §2): specialist answers a bounded sub-question, manager keeps the user-facing conversation. Use when you need a helper, not a successor.
- Handoffs (network, this section): specialist becomes the conversation. Use when routing itself is the workflow — e.g. a triage agent hands a billing question to the billing agent, which then owns the thread.
- Pros: flexible routing; no orchestrator bottleneck; specialists own their domain end-to-end.
- Cons: emergent, hard-to-predict control flow; loops and ping-pong handoffs are real failure modes; harder to trace and evaluate.
- Caveats: “swarm” can degrade into agents talking past each other. Without disciplined handoff conditions, you get oscillation.
- Risk mitigation: define explicit, mutually-exclusive handoff triggers; cap total handoffs per task; log every transfer for trace inspection (§8); prefer handoffs only where routing is genuinely the job.
5. Debate / voting
Several agents independently attempt the same task, then results are combined — by majority vote, by a judge agent, or through rounds of debate where agents critique each other and revise. A breadth-of-reasoning pattern rather than breadth-of-search.
- Pros: can lift accuracy on hard reasoning/factual tasks; surfaces disagreement you’d otherwise miss; a critic round catches errors a single pass won’t.
- Cons: multiplies cost by the number of debaters/rounds; agents can converge on a shared wrong answer (correlated errors); a weak judge poisons the result.
- Caveats: closely related to single-model self-consistency (
n>1sampling + vote, see 02 - Decoding, Sampling & Reasoning Controls) — often the cheaper version of the same idea. Reach for multi-agent debate only when agents bring different tools/contexts, not just different samples. - Risk mitigation: ensure debaters have genuinely diverse contexts or roles (else errors correlate); use a stronger model as judge than as debater; cap rounds; eval whether debate actually beats single-pass on your task — it frequently doesn’t.
6. Delegation quality — the #1 failure mode
Vague delegation is the single biggest reason multi-agent systems fail
The most common failure is not topology choice — it’s the lead handing a subagent a fuzzy objective. A subagent only has the context you put in its prompt; it cannot see the lead’s reasoning, the user’s full intent, or its sibling agents’ work. Vague instructions (“research the competitive landscape”) produce duplicated effort, gaps, and subagents wandering off-scope.
Crisp delegation means each subagent’s brief carries:
- A precise objective — what specifically to find/do, and what “done” looks like.
- Explicit boundaries — what’s in scope and (just as important) what’s not, so siblings don’t overlap.
- The output format and detail level expected — so the lead can merge results without a second round.
- Enough self-contained context — everything the subagent needs, because it can’t ask follow-ups mid-task and can’t see the rest of the system.
This is the multi-agent expression of context engineering (03 - Prompt & Context Engineering): the lead is engineering each worker’s entire context window. Anthropic’s guidance is explicit that subagents need objectives, output formats, tool guidance, and clear task boundaries — get this wrong and no topology saves you.
7. Cost & reliability tradeoffs
The decision is rarely “can it work” — it’s “is it worth ~15× the bill.”
| Tier | Token usage (rough) |
|---|---|
| Single chat turn | 1× (baseline) |
| Single agent (tool loop) | ~4× a chat |
| Multi-agent system | ~15× a chat |
Multi-agent ≈ 15× the tokens of a chat
Anthropic reports agents use ~4× the tokens of a chat, and multi-agent systems ~15×. That spend is the source of the performance (§2’s 80%-of-variance thesis) — but it only pays off where the task value is high and the work genuinely parallelizes. For most tasks the economics don’t clear, which is exactly why §10’s skeptics push back.
Reliability costs compound too: more agents = more independent points of failure, more non-determinism, and stateful coordination that’s hard to recover mid-run. A 10-step single agent that’s 95% reliable per step finishes ~60% of the time; spread that across coordinating agents and the math gets worse before error handling helps.
- Mitigation: reserve multi-agent for high-value, parallelizable, error-tolerant work; checkpoint state for resumability; set hard caps on agents/depth/handoffs/total tokens; degrade gracefully (partial results beat a crashed run). Tie all of this to 15 - Cost, Latency & Performance.
8. Evaluating multi-agent trajectories
Multi-agent runs are non-deterministic and path-dependent — the same task can succeed via different routes, and a right answer can come from a flawed trajectory. Scoring only the final output misses how it got there.
- Outcome eval: did the system reach a correct/complete result? (necessary, not sufficient)
- Trajectory eval: were the right subagents spawned with sound objectives? was delegation non-overlapping? did handoffs fire correctly and terminate? was token spend justified per branch?
- LLM-as-judge over the full trace handles the “many valid paths” problem better than exact-match — judge the trajectory’s quality, not its identity. (See 13 - Evaluation & Evals.)
- Log everything: per-agent token usage, tool calls, handoff transitions, and the lead’s synthesis input. Anthropic emphasizes that you must trace agent decisions to debug them — you cannot fix what you can’t see.
Eval whether multi-agent even helps
The most valuable eval is the comparison: run the task single-agent vs. multi-agent on a representative set. If multi-agent doesn’t clearly win on quality, the 15× cost is pure waste — fall back to the simpler tier.
9. Production hardening
- Determinism boundaries: wrap the non-deterministic agent core in deterministic, code-controlled orchestration where you can (09 - Workflow & Orchestration Patterns) — let agents decide what, let code enforce limits and retries.
- State & resumability: long parallel runs fail partway; checkpoint so a single worker crash doesn’t discard hours of work.
- Caps everywhere: max agents, max depth, max handoffs, max tokens, max wall-clock — runaway fan-out is a cost incident waiting to happen.
- Idempotent / gated side effects: workers that write data or call external APIs need confirmation gates and idempotency, since the same step may retry.
- Observability: structured traces per agent (see §8) are non-negotiable in production — debugging emergent multi-agent behavior without them is hopeless.
- Graceful degradation: return partial results with provenance rather than failing the whole task when one branch dies.
10. ⚠️ The live debate: should you build this at all?
This is genuinely contested in 2026, and a senior signal is being able to state both sides honestly.
Anthropic (pro) vs. Cognition (con) — and what they actually agree on
Anthropic — “How we built our multi-agent research system”: multi-agent works for them. The Opus-lead/Sonnet-workers research system beat single-agent Opus by 90.2%, driven by parallel token spend (§2). Their domain is research — breadth-first, read-only, embarrassingly parallel.
Cognition — “Don’t Build Multi-Agents” (Walden Yan): in 2025–26, running agents in collaboration produces fragile systems. Two principles of context engineering: (a) share context across all components, and (b) avoid conflicting decisions by ensuring every action sees the full record of prior actions. Multi-agent violates both: subagents work from separate context windows and take actions on conflicting assumptions never established upfront. Their recommended default is a single-threaded linear agent — continuous context, no dispersed decision-making. Their domain is coding — interdependent, decision-consistent.
The consensus hiding under the disagreement: they’re not really contradicting each other — they’re describing different task shapes. Multi-agent suits read-only / parallelizable / breadth-first work (research) and hurts on interdependent / decision-consistent work (most coding), precisely because shared context matters there and separate windows fragment it. Pick the architecture from the task, not the hype.
When NOT to reach for multi-agent
- The sub-tasks are interdependent — later steps depend on decisions made in earlier ones. (Most coding, refactors, anything where coherence matters.)
- Shared context is load-bearing — agents would need to see each other’s full work to stay consistent. Separate windows guarantee drift.
- A single agent with a bigger budget would do — if you just need more tokens/tools, don’t add agents (§2).
- A deterministic workflow fits — fixed steps don’t need agentic decision-making at all (09 - Workflow & Orchestration Patterns).
- The task is low-value or cost-sensitive — 15× tokens (§7) rarely pays off here.
- You can’t evaluate or debug it — without trajectory traces (§8) you’re flying blind.
When it’s worth it
- Breadth-first research / enumeration / discovery across many independent sources.
- Genuinely parallel sub-tasks with little cross-dependency.
- High-value, error-tolerant work where the 15× spend buys real quality.
11. Pros / cons / when (summary)
| Topology | Use when | Avoid when |
|---|---|---|
| Orchestrator-worker (§2) | Independent parallel sub-tasks; breadth-first research | Sub-tasks interdependent |
| Hierarchical (§3) | Deep, naturally-decomposable trees | Coordination overhead > gain (usually >1 level) |
| Network / swarm (handoffs, §4) | Routing is the workflow; specialist owns the thread | You just need a helper (use agents-as-tools) |
| Debate / voting (§5) | Hard reasoning with diverse agent contexts | Plain self-consistency (n>1) is cheaper |
| None — single agent / workflow | Interdependent, decision-consistent, or low-value work | (this is the default; justify deviating) |
12. Caveats & risk mitigation
- The architecture follows the task, not the trend. Most tasks don’t need multiple agents; start single-agent and escalate only when breadth/parallelism genuinely demand it.
- Shared-context loss is the silent killer. Separate context windows mean agents act on conflicting assumptions — fatal for interdependent work (Cognition’s core point). If coherence matters, prefer a single continuous context.
- Vague delegation dominates failures (§6). Crisp objectives + boundaries + self-contained context per subagent, every time.
- Cost is ~15× (§7) and it’s the source of the performance, not a side effect. Budget for it explicitly or don’t build it.
- Determinism is rare even before multi-agent — non-determinism compounds across coordinating agents. Don’t promise reproducible runs; assert on outcomes via evals (13 - Evaluation & Evals).
- You can’t debug what you can’t trace. Structured per-agent observability is a prerequisite, not a nice-to-have.
- Eval multi-agent against single-agent on your own task. If it doesn’t clearly win, the complexity and cost are waste.
13. In practice
# Orchestrator-worker (supervisor): subagents as tools, lead keeps the conversation.
# Each worker gets a CRISP, self-contained brief — vague delegation is failure mode #1.
research_worker = Agent(
name="source_researcher",
instructions=(
"Find primary sources on the SPECIFIC sub-topic you are given. "
"Return: 3-5 sources with one-line summaries and URLs. "
"Scope: ONLY your assigned sub-topic — do not cover sibling topics."
),
tools=[web_search],
)
lead = Agent(
name="research_lead",
instructions=(
"Decompose the question into INDEPENDENT sub-topics. "
"Delegate each to a researcher with a precise, bounded objective. "
"Synthesize and validate results — do not blindly concatenate."
),
tools=[research_worker.as_tool( # agents-as-tools = supervisor pattern
tool_name="research_subtopic",
tool_description="Research one independent sub-topic in depth.",
)],
)
# Handoff (network): control TRANSFERS, full history goes with it.
billing = Agent(name="billing", instructions="Own billing questions end to end.")
triage = Agent(
name="triage",
instructions="Route to the right specialist; hand off when routing IS the task.",
handoffs=[billing], # billing takes over the conversation, sees full history
)
# Reserve all of this for breadth-first / parallelizable / read-only work.
# For interdependent or decision-consistent work, use ONE agent with continuous context.14. Sources
- Anthropic — How we built our multi-agent research system (90.2% uplift; token usage ≈ 80% of variance; ~15× tokens; orchestrator-worker with parallel subagents; delegation guidance)
- Cognition — Don’t Build Multi-Agents (Walden Yan) (context-engineering principles: share context, avoid conflicting decisions; single-threaded agent as default; why multi-agent is fragile for coding)
- OpenAI Agents SDK — Agent orchestration (handoffs & agents-as-tools)
- OpenAI API — Orchestration and handoffs