Design hierarchical agent architectures

Introduction

When a single AI agent tries to handle a complex, multi-step task end-to-end, it quickly runs into context overload, tangled responsibilities, and brittle failure modes — one wrong turn in a long reasoning chain corrupts the entire result. Hierarchical agent architectures solve this by organizing agents into strategic, tactical, and operational tiers, each operating at its own level of abstraction. By the end of this lesson, you will be able to design a three-level agent hierarchy, define the shared state schema that flows across those levels, and apply the separation-of-concerns principle that prevents each tier from interfering with the others.

Key Terminology

• Hierarchical Agent Architecture — a multi-agent design that organizes agents into strategic, tactical, and operational tiers, each operating at its own level of abstraction so that no single agent must handle an entire complex task end-to-end.
• HierarchyLevel — a str, Enum with values STRATEGIC, TACTICAL, and OPERATIONAL that pins each agent to exactly one tier at definition time, making the tier an explicit, checkable property rather than an implicit convention.
• AgentDefinition — a dataclass that captures every agent's identity and wiring: its level, capabilities, subordinates (agents it manages), reports_to parent, tools, and system_prompt, which together drive the graph's routing and instruction logic.
• Separation of Concerns — the principle that restricts each tier's cognitive scope so that strategic agents never touch implementation details, tactical agents don't micromanage execution, and operational agents focus solely on their assigned specialized work.
• HierarchicalState — a TypedDict that carries the full request lifecycle — from user_request and strategic_plan through team_assignments, team_results, worker_results, and final_response — across all three tiers of the graph.
• Reducer Annotation — the use of Annotated[T, reducer_fn] on state fields so that fan-out branches can write results independently; operator.or_ merges team_results dictionaries across teams and operator.add concatenates worker_results and errors lists without branches clobbering each other.

Concepts

Why Single Agents Break at Scale

A single agent solving a complex, multi-step task must hold the entire problem — goal interpretation, execution planning, domain-specific reasoning, error recovery, and synthesis — inside one context window. This leads to three compounding failure modes: context overload as the reasoning chain grows, tangled responsibilities where goal-setting logic and implementation details interfere with each other, and brittle cascades where one wrong turn corrupts every downstream step. The failures are not random; they are structural. A single scope is simply the wrong unit of reasoning for tasks that naturally decompose into distinct phases with different expertise requirements.

Hierarchical agent architectures resolve this by breaking the monolith vertically. Rather than giving one agent more context, the hierarchy assigns different levels of abstraction to different agents, each of which operates on a smaller, well-defined scope.

The Three-Tier Model and What Each Level Owns

The architecture divides work across three tiers, each with a non-overlapping responsibility:

• Strategic agents interpret the user's goal, decompose it into a coordinated plan, and assign work to teams. They never touch implementation details — their job is direction.
• Tactical agents translate that strategic direction into domain-specific execution plans and coordinate the operational workers beneath them. They don't second-guess the strategy or micromanage individual tool calls.
• Operational agents execute specialized tasks with deep expertise in a narrow domain. They focus solely on their assigned work and report results upward.

The HierarchyLevel enum (STRATEGIC, TACTICAL, OPERATIONAL) makes this tier assignment an explicit, checkable property at definition time — it is stored on every AgentDefinition alongside subordinates and reports_to, giving the graph's routing logic a typed, inspectable record of the hierarchy's shape (see Code Walkthrough).

This separation is the load-bearing design principle: restricting each agent's cognitive scope enables parallel team execution and creates clean failure-containment boundaries. A failure in one operational agent does not corrupt the strategic plan; it surfaces as an entry in the errors list and can be handled at the appropriate tier.

Shared State and Safe Fan-Out with Reducer Annotations

All three tiers communicate through a single HierarchicalState TypedDict that flows through the entire graph. Fields span the full lifecycle — user_request, strategic_plan, team_assignments, team_results, worker_results, shared_context, and final_response — so any agent at any level can read the context it needs without a separate communication channel.

The critical design challenge in a fan-out hierarchy is concurrent writes: when multiple operational agents execute in parallel, each must be able to append its result without overwriting a sibling's. HierarchicalState solves this with reducer annotations. team_results is typed as Annotated[Dict[str, dict], operator.or_], which merges dictionaries from parallel team branches. worker_results and errors are typed as Annotated[List[dict], operator.add], which concatenates lists. These reducers are declared once on the state schema and applied automatically by LangGraph at merge time — no branch needs to know about the others, and no manual locking is required. The result is a state object that is safe to write from independent branches and always reflects the union of all completed work.

Code Walkthrough

Now that you've seen why single agents break at scale, the three-tier model and what each level owns, and shared state and safe fan-out with reducer annotations, this walkthrough turns them into working code.

A well-designed hierarchy assigns each level a distinct scope: the strategic level interprets goals and coordinates teams, the tactical level translates strategy into domain-specific execution plans, and the operational level executes specialized tasks with deep expertise. The separation-of-concerns principle is the load-bearing idea — strategic agents never touch implementation details, tactical agents don't second-guess direction or micromanage execution, and operational agents focus solely on their assigned work. This restricts each agent's cognitive scope, enables parallel team execution, and creates clean failure-containment boundaries.

The code below defines the foundational building blocks: a HierarchyLevel enum for the three tiers, an AgentDefinition dataclass capturing every agent's properties, and a HierarchicalState TypedDict that flows through the entire graph.

Code snippetpython
1from typing import TypedDict, List, Optional, Annotated, Dict, Any
2from dataclasses import dataclass
3from enum import Enum
4import operator
5
6class HierarchyLevel(str, Enum):
7    STRATEGIC = "strategic"
8    TACTICAL = "tactical"
9    OPERATIONAL = "operational"
10
11@dataclass
12class AgentDefinition:
13    name: str
14    level: HierarchyLevel
15    description: str
16    capabilities: List[str]
17    subordinates: List[str]   # agents this one manages
18    reports_to: Optional[str] # supervising agent
19    tools: List[str]
20    system_prompt: str
21
22class HierarchicalState(TypedDict):
23    user_request: str
24    request_id: str
25    strategic_plan: Optional[dict]
26    team_assignments: List[dict]
27    team_results: Annotated[Dict[str, dict], operator.or_]
28    worker_results: Annotated[List[dict], operator.add]
29    shared_context: Dict[str, Any]
30    dependencies: List[dict]
31    blocked_tasks: List[str]
32    tactical_synthesis: Dict[str, str]
33    final_response: Optional[str]
34    current_level: HierarchyLevel
35    active_team: Optional[str]
36    errors: Annotated[List[dict], operator.add]

HierarchyLevel pins each agent to a tier at definition time. AgentDefinition records an agent's name, tier, capabilities, subordinates, reports_to parent, available tools, and system prompt — the last two fields drive the graph's routing and instruction logic. HierarchicalState uses Annotated with operator.or_ on team_results (dictionary merge across teams) and operator.add on worker_results and errors (list concatenation), so fan-out branches can write results independently without clobbering each other.

To verify the scaffold is wired correctly, instantiate a minimal AgentDefinition with level=HierarchyLevel.STRATEGIC, set subordinates to two names, and assert that a freshly initialized HierarchicalState contains empty worker_results and errors lists — if both assertions pass, the type annotations and reducer semantics are correctly aligned before you add any agent logic.

Do's and Don'ts

Having walked through designing hierarchical agent architectures above, the following Do's and Don'ts distill it into practice.

Do's

1. ✓Do assign each agent a HierarchyLevel at definition time — pinning agents to STRATEGIC, TACTICAL, or OPERATIONAL in AgentDefinition is what enforces separation-of-concerns at the schema level; without it, nothing prevents a strategic agent from drifting into implementation details and collapsing the abstraction boundaries the hierarchy is built on.
2. ✓Do use Annotated with the correct reducer for each HierarchicalState field — operator.or_ on team_results enables dictionary merging across fan-out branches, while operator.add on worker_results and errors enables list concatenation, so parallel teams can write independently without clobbering each other's outputs.
3. ✓Do verify the scaffold before adding agent logic — instantiate a minimal AgentDefinition with level=HierarchyLevel.STRATEGIC and assert that a fresh HierarchicalState has empty worker_results and errors lists; if those assertions pass, your type annotations and reducer semantics are correctly wired and safe to build on.

Don'ts

1. ✗Don't let strategic agents touch implementation details or let operational agents second-guess direction — violating the tier boundaries collapses the separation-of-concerns principle that keeps cognitive scope narrow, parallel execution clean, and failures contained to the level where they occur.
2. ✗Don't use a plain Dict or List instead of Annotated[..., operator.or_] / Annotated[..., operator.add] for fan-out fields — without the reducer annotation, concurrent branches writing to team_results or worker_results will clobber each other's data rather than merge, producing silently incomplete results.
3. ✗Don't leave subordinates and reports_to mismatched in your AgentDefinition instances — if agent A lists agent B in subordinates but B's reports_to points elsewhere (or is None), the graph's routing and instruction logic has no coherent supervision chain, and delegation calls will fail or route to the wrong tier.