Building Multi-Agent Systems: When to Use Them and How to Do It Right
Multi-agent systems are not inherently better than single-agent systems. You should only introduce the additional coordination costs of multiple agents when you face three specific scenarios: context pollution, parallel execution needs, or specialized tooling requirements. This guide will help you determine whether your project actually needs a multi-agent architecture and, if it does, how to decompose and orchestrate it correctly.
Image Source: Unsplash
Why You Should Start with a Single Agent
Core question: Why shouldn’t I just build a multi-agent system from the start to solve my problem?
A well-designed single agent, paired with the right set of tools, can accomplish far more than most developers expect. Multi-agent systems introduce significant overhead. Every additional agent you add represents another potential point of failure, another set of prompts that needs to be maintained, and another source of unpredictable behavior.
There is a very common pattern in practice: teams spend months building elaborate multi-agent architectures, assigning separate agents for planning, execution, review, and iteration, only to find that context is lost at every single handoff. The agents end up spending more tokens coordinating with each other than they spend actually executing the work. Test data consistently shows that for equivalent tasks, multi-agent implementations typically consume 3 to 10 times more tokens than single-agent approaches.
This overhead comes from three distinct places. First, context is duplicated across agents because each one needs enough background to do its job. Second, agents must exchange messages to coordinate their actions. Third, results must be summarized and compressed whenever they are passed between agents during handoffs.
“
Reflection: I have seen numerous teams equate “multi-agent” with “a more advanced architecture,” as if having more agents inherently makes the system smarter. Reality is exactly the opposite. If you have not yet optimized your single agent’s prompts and tool configuration, adding more agents will just amplify your problems three to tenfold. Get the simple approach working first; complexity is the absolute last thing you should add.
Section summary: Multi-agent systems are not the default option. They are a specific tool for specific constraints. Until you have concrete evidence that a single agent cannot meet your needs, do not introduce multi-agent complexity.
A Decision Framework for Multi-Agent Systems
Core question: Under what specific circumstances does a multi-agent system actually deliver genuine value?
Multi-agent architectures only provide value when they solve specific constraints that a single agent fundamentally cannot overcome. This means multi-agent architectures should be reserved strictly for cases where the benefits are clear and justify the added cost.
The following three patterns represent the scenarios where we consistently observe positive returns on the investment of added complexity.
Context Protection
Core question: How do you prevent model performance from degrading when the context window gets filled with irrelevant information?
Large language models have finite context windows, and as context grows, response quality predictably degrades. When an agent’s context accumulates information from one subtask that has absolutely no relevance to subsequent subtasks, you get “context pollution.” Subagents solve this by providing isolation. Each subagent operates in its own clean context, focused exclusively on its specific task.
Scenario: Order lookups and technical diagnosis in a customer support system
Imagine a customer support agent that needs to retrieve order history while simultaneously diagnosing a technical issue. If every order lookup dumps thousands of tokens of order details into the context window, the agent’s ability to reason about the technical problem is severely diluted.
The problem with the single-agent approach is that everything accumulates in the exact same conversation context:
# Single agent: everything piles up in the same context
conversation_history = [
{"role": "user", "content": "My order #12345 isn't working"},
{"role": "assistant", "content": "Let me check your order..."},
# Tool result adds 2000+ tokens of order history
{"role": "user", "content": "... (order details, past purchases, shipping info) ..."},
{"role": "assistant", "content": "Now let me diagnose the technical issue..."},
# Context is now polluted with order details the agent doesn't need
]
The multi-agent approach separates the order lookup from the technical diagnosis entirely:
from anthropic import Anthropic
client = Anthropic()
class OrderLookupAgent:
def lookup_order(self, order_id: str) -> dict:
# Separate agent with its own isolated context
messages = [
{"role": "user", "content": f"Get essential details for order {order_id}"}
]
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
messages=messages,
tools=[get_order_details_tool]
)
# Returns only the essential, refined summary
return extract_summary(response)
class SupportAgent:
def handle_issue(self, user_message: str):
if needs_order_info(user_message):
order_id = extract_order_id(user_message)
# Get only what's needed, not the full history
order_summary = OrderLookupAgent().lookup_order(order_id)
# Inject a compact summary, not the full raw context
context = f"Order {order_id}: {order_summary['status']}, purchased {order_summary['date']}"
# Main agent context stays clean and focused
messages = [
{"role": "user", "content": f"{context}\n\nUser issue: {user_message}"}
]
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=2048,
messages=messages
)
return response
The order lookup subagent processes the massive order history and extracts a tight summary. The main support agent receives only the 50 to 100 tokens it actually needs, keeping its context window entirely focused on the technical problem.
Context isolation works best when all of the following conditions are met simultaneously:
Image Source: Unsplash
Parallelization
Core question: How can multiple agents working simultaneously improve result quality when you need to cover a massive information search space?
Running multiple agents in parallel allows you to explore a significantly larger search space than a single agent could ever cover on its own. This pattern has proven to be particularly valuable for search and research-heavy tasks.
Scenario: Multi-dimensional topic research
Suppose you need to research a complex query like “global electric vehicle market trends.” A single agent can only search through dimensions sequentially, one after another. A multi-agent setup can simultaneously dispatch subagents to investigate technology roadmaps, regulatory policies, market competition, and supply chain logistics all at the exact same time.
import asyncio
from anthropic import AsyncAnthropic
client = AsyncAnthropic()
async def research_topic(query: str) -> dict:
# Lead agent breaks the query into distinct research facets
facets = await lead_agent.decompose_query(query)
# Spawn subagents to research each facet concurrently
tasks = [
research_subagent(facet)
for facet in facets
]
results = await asyncio.gather(*tasks)
# Lead agent synthesizes all the distinct findings
return await lead_agent.synthesize(results)
async def research_subagent(facet: str) -> dict:
"""Each subagent operates in its own independent context window"""
messages = [
{"role": "user", "content": f"Research topic: {facet}"}
]
response = await client.messages.create(
model="claude-sonnet-4-5",
max_tokens=4096,
messages=messages,
tools=[web_search, read_document]
)
return extract_findings(response)
There is a critical misunderstanding here that needs to be addressed: the core benefit of parallelization is thoroughness, not speed. Multi-agent systems typically consume 3 to 10 times more tokens than single-agent approaches for equivalent tasks. While parallelism does reduce total execution time compared to running all that same work sequentially, the sheer increase in total computation means the multi-agent system often takes longer overall than a streamlined single-agent system.
What you are trading your tokens and time for is more comprehensive result coverage. If your goal is raw response speed, optimizing your single agent is almost always the better path. If your scenario genuinely demands searching every corner of an information space, then the extra cost of parallel multi-agents is justified.
“
Reflection: Many people hear the word “parallel” and immediately think “faster.” But in the multi-agent context, the primary value is “more thorough.” If you are optimizing for latency, a single agent is usually the right answer. If you are optimizing for completeness—leaving no stone unturned—then the extra overhead of parallel multi-agents becomes a worthwhile investment.
Section summary: Parallelization is suited for scenarios requiring broad coverage of an information space. The tradeoff is higher token consumption and often longer total execution time, in exchange for more comprehensive results.
Specialization
Core question: How does splitting work by toolsets, system prompts, and domain expertise actually improve an agent’s reliability?
Different tasks sometimes require fundamentally different tool sets, system prompts, or domains of expertise. Rather than handing a single agent access to dozens of tools, you can deploy specialized agents equipped only with focused toolsets that match their specific responsibilities. This targeted approach significantly improves reliability.
Specialization manifests across three distinct layers:
Tool Set Specialization
When an agent has access to too many tools, its performance suffers. There are three clear warning signs that indicate you need tool specialization:
-
Quantity overload. Once an agent has 20 or more tools, its ability to consistently select the correct one drops noticeably. -
Domain confusion. When tools span multiple unrelated domains—such as mixing database operations, external API calls, and file system operations—the agent gets confused about which domain applies to a given request. -
Performance degradation. If adding a new tool causes performance on existing tasks to drop, the agent has hit its operational capacity for tool management.
System Prompt Specialization
Different tasks require different personas, constraints, or behavioral instructions. When you combine these, they conflict. A customer support agent needs to be empathetic and patient. A code review agent needs to be precise and critical. A compliance-checking agent needs rigid rule-following. A brainstorming agent needs creative flexibility.
When a single agent is forced to switch between these conflicting behavioral modes, it produces inconsistent results. Separating them into specialized agents with tailored system prompts solves this friction.
Domain Expertise Specialization
Certain tasks require deep domain background context that would completely overwhelm a generalist agent. A legal analysis agent might need extensive context about case law and regulatory frameworks. A medical research agent might need specialized knowledge about clinical trial methodology. Rather than dumping all of this domain context into one massive prompt, specialized agents can carry only the focused expertise relevant to their specific responsibilities.
Scenario: Cross-platform integration systems
Imagine an integration system that needs to operate across a CRM platform, a marketing automation platform, and a messaging platform. Each platform has 10 to 15 relevant API endpoints. A single agent facing 40+ tools will frequently select the wrong one, confusing similar operations across different platforms. Splitting this into specialized agents with focused toolsets and tailored prompts resolves the selection errors immediately:
from anthropic import Anthropic
client = Anthropic()
class CRMAgent:
"""Handles customer relationship management operations"""
system_prompt = """You are a CRM specialist. You manage contacts,
opportunities, and account records. Always verify record ownership
before updates and maintain data integrity across related records."""
tools = [
crm_get_contacts,
crm_create_opportunity,
# 8-10 CRM-specific tools
]
class MarketingAgent:
"""Handles marketing automation operations"""
system_prompt = """You are a marketing automation specialist. You
manage campaigns, lead scoring, and email sequences. Prioritize
data hygiene and respect contact preferences."""
tools = [
marketing_get_campaigns,
marketing_create_lead,
# 8-10 marketing-specific tools
]
class OrchestratorAgent:
"""Routes requests to the appropriate specialized agents"""
def execute(self, user_request: str):
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
system="""You coordinate platform integrations. Route requests to the appropriate specialist:
- CRM: Contact records, opportunities, accounts, sales pipeline
- Marketing: Campaigns, lead nurturing, email sequences, scoring
- Messaging: Notifications, alerts, team communication""",
messages=[
{"role": "user", "content": user_request}
],
tools=[delegate_to_crm, delegate_to_marketing, delegate_to_messaging]
)
return response
This pattern mirrors effective professional collaboration. Experts who wield tools perfectly matched to their roles collaborate far more effectively than generalists who try to maintain expertise across every single domain.
However, specialization introduces routing complexity. The orchestrator must correctly classify requests and delegate them to the right agent, and misrouting leads directly to poor results. Maintaining multiple specialized agents also increases your prompt maintenance overhead. Specialization works best when domains are clearly separable and routing decisions are unambiguous.
Signs You Have Outgrown a Single-Agent Architecture
Core question: What concrete signals indicate you have actually hit the limits of a single agent?
Beyond the general framework outlined above, there are specific, observable signals that suggest a single-agent pattern is no longer sufficient:
“
Reflection: Pay close attention to the sequence of recommended actions in that table: “first try X, then consider Y.” Before jumping to a multi-agent solution, single-agent optimizations like context compaction and tool search might be entirely sufficient. This reinforces a foundational principle: exhaust the simple solution completely before adding complexity.
It is important to note that these thresholds will shift as models improve. The current numbers represent practical guidelines based on today’s capabilities, not fundamental, immutable constraints.
Context-Centric Decomposition
Core question: Should you divide multi-agent work by what needs to be done, or by what context is required?
When you adopt a multi-agent architecture, the single most important design decision you will make is how to divide work between your agents. In practice, teams frequently make this choice incorrectly, leading to coordination overhead that completely negates the theoretical benefits of a multi-agent design.
The key insight is to adopt a context-centric view rather than a problem-centric view when decomposing work.
Problem-centric decomposition (usually counterproductive): Dividing work by the type of labor—one agent writes features, another writes tests, a third reviews code—creates constant, heavy coordination overhead. Every single handoff loses critical context. The test-writing agent lacks the knowledge of why certain implementation decisions were made. The code reviewer lacks the context of the exploration and iteration process that led to the final code.
Context-centric decomposition (usually effective): Dividing work by context boundaries means that an agent handling a feature should also handle its tests, because it already possesses all the necessary context to do both well. Work should only be split when the context can be truly, cleanly isolated.
This principle emerges directly from observing failure modes in real-world multi-agent systems. When agents are split by problem type, they engage in a “telephone game.” Information passes back and forth, and every handoff degrades fidelity. In one experiment using agents specialized by software development role (planner, implementer, tester, reviewer), the subagents spent more tokens coordinating with each other than they spent on the actual implementation work.
Here is a clear comparison of effective versus problematic decomposition boundaries:
Image Source: Unsplash
“
Unique Insight: The principle of “context-centric decomposition” is not limited to multi-agent AI systems. Think about how human organizations are structured. The most painful collaboration almost always happens between people who need the exact same information but have been siloed into different departments. The logic underlying multi-agent system decomposition is identical to human organizational design: minimize unnecessary information transfer, and let the person—or agent—that holds the context make the decisions that require that context.
The Verification Subagent Pattern
Core question: How do you make a separate agent reliably validate another agent’s output without failing into the same traps?
Out of all multi-agent patterns, the verification subagent is the one that consistently works well across the widest variety of domains. This is a dedicated agent whose sole, exclusive responsibility is testing or validating the main agent’s work.
It is worth noting that more capable orchestrator models (like Claude Opus 4.5) are increasingly able to evaluate subagent work directly without needing a separate verification step. However, verification subagents remain highly valuable in three situations: when you are using a less capable orchestrator model, when the verification step requires specialized tools, or when you want to enforce explicit, mandatory verification checkpoints in your workflow.
Verification subagents succeed because they naturally sidestep the telephone game problem. Verification, by its very nature, requires minimal context transfer. A verifier can perform blackbox testing on a system without needing the full history of how that system was built.
Implementation Details
The main agent completes a unit of work. Before it proceeds to the next step, it spawns a verification subagent, handing it the artifact to verify, clear success criteria, and the tools needed to perform the verification.
The verifier does not need to understand why the artifact was built the way it was. It only needs to determine whether the artifact meets the specified criteria.
from anthropic import Anthropic
client = Anthropic()
class CodingAgent:
def implement_feature(self, requirements: str) -> dict:
"""Main agent implements the feature"""
messages = [
{"role": "user", "content": f"Implement: {requirements}"}
]
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=4096,
messages=messages,
tools=[read_file, write_file, list_directory]
)
return {
"code": response.content,
"files_changed": extract_files(response)
}
class VerificationAgent:
def verify_implementation(self, requirements: str, files_changed: list) -> dict:
"""Separate agent verifies the work in isolation"""
messages = [
{"role": "user", "content": f"""
Requirements: {requirements}
Files changed: {files_changed}
Run the test suite and verify:
1. All existing tests pass
2. New functionality works as specified
3. No obvious errors or security issues
You MUST run the complete test suite before marking as passed.
Do not mark as passing after only running a few tests.
Run: pytest --verbose
Only mark as PASSED if ALL tests pass with no failures.
"""}
]
response = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=4096,
messages=messages,
tools=[run_tests, execute_code, read_file]
)
return {
"passed": extract_pass_fail(response),
"issues": extract_issues(response)
}
def implement_with_verification(requirements: str, max_attempts: int = 3):
for attempt in range(max_attempts):
result = CodingAgent().implement_feature(requirements)
verification = VerificationAgent().verify_implementation(
requirements,
result['files_changed']
)
if verification['passed']:
return result
requirements += f"\n\nPrevious attempt failed: {verification['issues']}"
raise Exception(f"Failed verification after {max_attempts} attempts")
Where Verification Subagents Excel
Verification subagents are highly effective for the following applications:
-
Quality assurance: Running test suites, linting code, validating outputs against predefined schemas. -
Compliance checking: Verifying that documents meet strict policy requirements, checking outputs against regulatory rules. -
Output validation: Confirming that generated content meets exact specifications before it is delivered to the end user. -
Factual verification: Having a separate agent independently verify claims or citations in generated content.
The Early Victory Problem
Core question: Why do verification agents frequently “cheat” by passing incomplete tests, and how do you force them to be thorough?
The most significant failure mode for verification subagents is marking outputs as passing without performing thorough testing. The verifier runs one or two tests, observes that they pass, and prematurely declares success.
The root cause of this problem is that the agent defaults to taking shortcuts. It observes partial evidence of success and extrapolates that to a complete pass, rather than actually executing the full, exhaustive verification workflow.
There are four primary mitigation strategies to counteract this behavior:
“
Reflection: The “early victory” failure mode closely mirrors a known issue in human auditing. Auditors sometimes spot-check a few samples and issue a “no significant issues found” report. Whether dealing with humans or AI, when the verification work itself is time-consuming and tedious, there is a strong cognitive pull toward “seeing good signs and stopping.” The solution is identical in both domains: replace vague mandates like “ensure there are no problems” with precise directives like “execute these 5 specific checks and report the explicit result of each.”
Practical Summary and Action Checklist
Decision Action Checklist
Before introducing a multi-agent architecture, walk through this list and confirm every item:
-
[ ] Is the single agent truly optimized? Have prompts been thoroughly iterated? Are tool configurations logical and streamlined? -
[ ] Does a genuine constraint exist that only multi-agent can solve? Is it specifically context pollution, a parallelization opportunity, or a specialization need? -
[ ] Have single-agent optimizations been attempted? Have context compaction and tool search tools been evaluated first? -
[ ] Is the decomposition based on context boundaries, not problem types? Confirm you are not splitting tightly coupled sequential phases into different agents. -
[ ] Are there clear verification points? Can subagents independently validate work without requiring the full context of how it was built? -
[ ] Have you accepted the cost tradeoffs? Are you prepared for 3-10x token overhead, potentially longer total execution times, and higher maintenance complexity?
One-Page Summary
Frequently Asked Questions
Q1: Are multi-agent systems faster than single-agent systems?
Not necessarily. While parallelization reduces time compared to running all work sequentially, the sheer increase in total computation (3-10x more tokens) means multi-agent systems often have a longer total execution time than a well-optimized single-agent system. The primary benefit of parallelization is result thoroughness, not speed.
Q2: How many tools can a single agent handle before it struggles?
Selection accuracy typically begins to drop when an agent has more than 15-20 tools. However, before splitting the agent, you should first try using a Tool Search tool, which allows the model to discover tools on demand rather than loading all definitions upfront. This can reduce token usage by up to 85% and improve accuracy, potentially eliminating the need to split the agent at all.
Q3: Why does splitting agents by “planner, implementer, tester, and reviewer” perform poorly?
Because these sequential phases share massive amounts of context. The implementer lacks the planner’s exploration context, the tester lacks the implementer’s design rationale, and the reviewer lacks the entire iteration history. Every handoff acts as a “telephone game,” degrading information fidelity, and the coordination overhead ends up exceeding the actual work overhead.
Q4: What is the difference between a verification subagent and having the main agent review its own work?
A main agent reviewing its own work is susceptible to “confirmation bias”—it has a tendency to approve its own output. A verification subagent operates in a completely independent context, free from the preconceptions of the build process, making it much more likely to catch objective errors. That said, highly capable orchestrator models are getting better at self-review, which may reduce the need for separate verifiers over time.
Q5: When does context isolation yield the best results?
Context isolation is most effective when a subtask generates a high volume of context (over 1,000 tokens), most of that generated information is irrelevant to the main task, the subtask has clear and objective criteria for what to extract, and the operation is fundamentally a lookup or retrieval action.
Q6: What happens if the orchestrator routes requests to the wrong specialized agent?
Routing errors are the primary risk of the specialization pattern. You can mitigate this by ensuring domain boundaries are crystal clear with no ambiguity, writing explicit routing rules in the orchestrator’s system prompt, and potentially adding a validation step for the routing decision itself. If domain boundaries remain fuzzy, specialization is likely the wrong pattern to apply.
Q7: Is the “early victory” problem unique to verification agents?
The tendency to take shortcuts is a general behavioral pattern, but it is most dangerous in verification scenarios because the entire value of the step depends on actually finding problems. In other scenarios, shortcutting causes lesser harm. The solution remains the same everywhere: replace vague instructions with precise, concrete, and un-skippable directives.
Q8: Where exactly does the token overhead of a multi-agent system come from?
It comes from three distinct sources. First, every agent needs its own context, leading to duplicated background information. Second, agents must exchange messages to coordinate their actions. Third, results must be summarized and compressed whenever they are handed off from one agent to another. The combination of these three factors leads to the standard 3-10x token multiplier for equivalent tasks.
