Scope
The composable patterns for wiring LLM calls, tools, and control flow into reliable systems — Anthropic’s five workflow patterns (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer) built on the augmented LLM, plus the production plumbing that makes them survive failure: durable execution vs checkpointers, human-in-the-loop, and idempotency. The dividing line: a workflow runs LLMs/tools through predefined code paths; an agent lets the LLM dynamically direct its own steps. This note is the workflow side. For the agent side see 08 - Agents & Agentic Systems and 10 - Multi-Agent Systems.
Related: Gen AI Engineer, 08 - Agents & Agentic Systems, 10 - Multi-Agent Systems, LangGraph Senior Prep
1. The building block: the augmented LLM
Every pattern below composes one primitive — an LLM augmented with retrieval, tools, and memory. Not a raw completion call; a call that can search, invoke functions, and read/write state.
┌──────────────────────────┐
input ──▶│ Augmented LLM │──▶ output
│ ┌──────┐ ┌──────┐ ┌────┐ │
│ │retrie│ │tools │ │mem │ │
│ │ -val │ │ │ │ory │ │
│ └──────┘ └──────┘ └────┘ │
└──────────────────────────┘
Mental model
Don’t reach for a framework first. Anthropic’s headline advice: start with the simplest thing — a single augmented LLM call — and add orchestration only when it measurably helps. Most “agent” failures are over-engineering: a workflow (predefined path) would have been cheaper, faster, and more debuggable than a free-roaming agent.
The standard integration layer for these augmentations is MCP (Model Context Protocol) — one protocol to expose tools, resources, and prompts to any model, instead of N bespoke adapters. See 05 - Model Context Protocol (MCP).
Workflow vs agent — the one distinction that matters
Workflow = LLMs and tools orchestrated through predefined code paths (you wrote the graph). Agent = the LLM dynamically directs its own process and tool use (it decides the path at runtime). Patterns 1–3 are squarely workflows; 4 (orchestrator-workers) and 5 (evaluator-optimizer) sit on the boundary and shade into agentic. Prefer workflows when the steps are predictable — you trade flexibility for reliability and you can test it.
2. Pattern 1 — Prompt chaining
What: Decompose a task into a fixed sequence of LLM calls; each call consumes the previous output. Insert programmatic gates between steps to check the intermediate result and bail early if it fails.
in ─▶ [LLM: step 1] ─▶ ◇gate ─pass─▶ [LLM: step 2] ─▶ [LLM: step 3] ─▶ out
└─fail──▶ exit / retry
When: The task cleanly splits into fixed subtasks known ahead of time — e.g. outline → check outline meets criteria → write draft → translate. Trades latency for accuracy, because each call does something simpler.
| Pros | Cons |
|---|---|
| Each step is simple and accurate | Latency adds up (serial) |
| Gates catch errors early, cheaply | Rigid — steps are fixed at design time |
| Easy to test step-by-step | Wrong decomposition caps quality |
In practice
Add a validation gate after any step whose output the next step depends on (e.g. “is this valid JSON / does it contain the 3 required fields”). A cheap deterministic check beats letting a bad intermediate poison the whole chain.
3. Pattern 2 — Routing
What: Classify the input, then dispatch it to a specialized downstream path (specialized prompt, model, or sub-workflow) tuned for that category.
┌─▶ [refund handler]
in ─▶ [router/classifier] ─┼─▶ [tech-support handler]
└─▶ [general handler] ──▶ out
When: Inputs fall into distinct categories that are each handled better in isolation — customer-support intents, easy-vs-hard questions (route cheap queries to a small model, hard ones to a frontier model). Lets you optimize each path without one mega-prompt trying to do everything.
| Pros | Cons |
|---|---|
| Specialized prompts/models per class | Misroute sends input down the wrong path |
| Cost control (cheap model for easy inputs) | Adds a classification hop (latency + cost) |
| Separation of concerns, easier to evolve | Needs categories that are actually separable |
The classifier is a single point of failure
A wrong route is often worse than no route. Keep the category set small and well-separated, eval the classifier on its own, and add a default/fallback path for low-confidence or unknown inputs.
4. Pattern 3 — Parallelization
What: Run LLM work concurrently in one of two shapes:
- Sectioning — split the task into independent subtasks, run in parallel, aggregate. (e.g. one call per document section; one call drafts while another screens for policy violations.)
- Voting — run the same task multiple times and aggregate by majority/threshold for higher confidence. (Related to self-consistency, see 02 - Decoding, Sampling & Reasoning Controls.)
SECTIONING VOTING
in ─┬▶ [LLM A] ─┐ in ─┬▶ [LLM run 1] ─┐
├▶ [LLM B] ─┼▶ aggregate ├▶ [LLM run 2] ─┼▶ vote ─▶ out
└▶ [LLM C] ─┘ └▶ [LLM run 3] ─┘
When: Subtasks are independent (sectioning → latency win) or you want multiple looks for confidence/safety (voting → quality win). A classic safety use: separate calls each check a different guardrail rather than overloading one prompt.
| Pros | Cons |
|---|---|
| Lower wall-clock latency (sectioning) | Higher token cost (N calls) |
| Higher confidence / coverage (voting) | Aggregation logic can be tricky |
| Each call focuses on one concern | Only works when subtasks are truly independent |
5. Pattern 4 — Orchestrator-workers
What: A central orchestrator LLM dynamically breaks a task into subtasks at runtime, delegates each to a worker LLM, then synthesizes the results. Unlike sectioning, the subtasks are not predetermined — the orchestrator decides them based on the input.
in ─▶ [orchestrator LLM] ──┬─▶ [worker 1] ─┐
(decides subtasks) ├─▶ [worker 2] ─┼─▶ [synthesizer] ─▶ out
└─▶ [worker N] ─┘
When: Complex tasks where you can’t predict the decomposition up front — e.g. a coding change that touches an unknown number of files, or research that spawns sub-queries depending on what’s found. This is where workflows start becoming agentic (the LLM chooses the structure).
| Pros | Cons |
|---|---|
| Handles unpredictable decomposition | Harder to test (paths vary per run) |
| Scales to variable subtask counts | Orchestrator errors fan out to all workers |
| Workers stay focused/parallelizable | More tokens, more latency, more to observe |
Sectioning vs orchestrator-workers
Both fan out. The difference is who decides the subtasks: sectioning = you (fixed, in code); orchestrator-workers = the model (dynamic, at runtime). Reach for orchestrator-workers only when the subtasks genuinely can’t be enumerated ahead of time — otherwise the simpler, testable sectioning wins.
6. Pattern 5 — Evaluator-optimizer
What: A generate→critique→refine loop. One LLM (optimizer) produces a candidate; a second LLM (evaluator) scores it against criteria and returns feedback; the optimizer revises. Loop until the evaluator passes it or a budget is hit.
in ─▶ [optimizer: generate] ─▶ [evaluator: critique] ─pass─▶ out
▲ │
└────── feedback ──────────┘ (fail → revise)
When: There are clear evaluation criteria and iterative refinement adds measurable value — literary translation (evaluator catches nuance), complex search (evaluator decides if more rounds are needed), code that must pass a spec. The signal that it fits: a human reviewer could articulate why a draft is wrong, and a second pass demonstrably improves it.
| Pros | Cons |
|---|---|
| Quality climbs with each round | Each loop = full cost + latency again |
| Mirrors human draft→review cycles | Needs a bound — loops can run away |
| Evaluator failures are inspectable | Evaluator only as good as its criteria |
Always cap the loop
Unbounded refine loops burn tokens and can oscillate without converging. Set a max-iterations ceiling and a “good enough” threshold, and log every round so you can see whether quality is actually improving or just churning. The evaluator and optimizer can be the same model with different prompts, but distinct criteria help.
7. Durable execution vs checkpointers
Once a workflow runs longer than one request — multi-step, waits on humans, calls flaky tools — you need it to survive a crash and resume. Two architectures dominate, and conflating them is a common interview trap.
| Checkpointer (LangGraph-style) | Durable execution (Temporal-style) | |
|---|---|---|
| Unit saved | state between nodes/steps | full event history of every activity |
| On resume | re-enters the node after the last checkpoint and re-runs it (LLM calls, API calls, interrupts re-fire) | replays history to rebuild in-memory state, resumes at the exact point of failure without re-running completed work |
| Granularity | does not save state inside a node | activity-level; each unit’s result is recorded |
| Mental model | ”snapshot between steps" | "deterministic replay to failure” |
Why "checkpoints ≠ durable execution"
A checkpointer persists state between steps but re-executes the current node on resume — so a node that already charged a credit card can charge it again. Durable execution replays recorded results and won’t re-run a completed activity. This is the crux: checkpointers give you resumability with re-execution; durable execution gives you exactly-once-style continuation. A common production pattern layers them — Temporal as the outer durable orchestrator, a LangGraph StateGraph inside each activity as the inner agent logic.
8. Idempotency — the prerequisite for safe replay
Both models above can re-run code (checkpointer always; durable execution during replay of non-recorded steps). Therefore any step with an external side effect must be idempotent or you get duplicates — double charges, double emails, double rows.
Idempotency is non-negotiable for side-effectful steps
Any tool that writes external state must carry an idempotency key tied to the workflow/run state, so a retry or replay collapses to a single effect. Concretely:
- Pass a stable key (e.g.
run_id + step_id) to payment/email/DB-write APIs that support dedup.- Push side effects to safe boundaries — in LangGraph, code before an
interrupt()re-runs on resume, so don’t charge the card before the human approves; do it after.- Prefer upserts over inserts; make writes check-then-act atomic.
This is why “just add retries” is dangerous advice without idempotency — retries multiply effects that aren’t dedup-safe.
9. Human-in-the-loop (HITL)
A natural extension of checkpointing: the workflow persists state, pauses indefinitely awaiting human input, then resumes from the pause point. Four interaction modes:
| Mode | What the human does | Use for |
|---|---|---|
| Approve / reject | gate a risky action before it runs | spend, deletes, external comms |
| Edit state | modify the agent’s draft/plan before continuing | correct a wrong tool arg or plan |
| Review tool call | inspect/approve a specific tool invocation | high-blast-radius tools |
| Provide input | answer a clarifying question | ambiguous requests |
In LangGraph this is the interrupt / resume mechanism: interrupt() pauses and surfaces a payload; resuming re-enters that node.
Resume re-runs the node — guard the pre-interrupt code
On resume, LangGraph re-enters the node where
interrupt()was called, so code before the interrupt can run again. Keep pre-interrupt side effects idempotent or move them past the interrupt. Same idempotency rule as §8 — HITL pauses are just very long-running steps.
10. ⚠️ Framework consolidation (high-churn — verified 2026-06-22)
The orchestration framework landscape consolidated hard in late 2025. Know the current state — it’s a live interview signal.
Microsoft: AutoGen + Semantic Kernel → Microsoft Agent Framework
Released public preview Oct 1, 2025. Microsoft merged AutoGen’s multi-agent orchestration with Semantic Kernel’s enterprise foundations (state, telemetry, type safety, middleware) into one open-source SDK, adding graph-based workflows. AutoGen and Semantic Kernel are now in maintenance mode — bug/security fixes only, no new features — pushing the community to Agent Framework. GA targeted for end of Q1 2026. If you’re starting fresh on the MS stack, start on Agent Framework, not AutoGen/SK.
OpenAI: Swarm → Agents SDK
Swarm (Oct 2024) was an experimental/educational multi-agent framework. In March 2025 OpenAI replaced it with the production Agents SDK (
openai-agentsPython + a TypeScript counterpart). Swarm is deprecated; its README points to the Agents SDK. The SDK adds built-in tracing, guardrails (input/output validation), and native Responses-API tools (web/file search, computer use, code interpreter) as first-class. Migration is hours, not a rewrite (renamed classes, different handoff API).
Selection heuristic
- No framework — start here; a single augmented LLM + a few
if/loops covers most workflows and is the easiest to debug.- LangGraph — explicit graph control, checkpointers, HITL interrupts; good when you want to see and test the path. (See LangGraph Senior Prep.)
- Temporal (+ inner LangGraph) — when you need true durable execution: long-running, crash-proof, exactly-once-style side effects.
- Microsoft Agent Framework — Azure/enterprise stack, .NET + Python.
- OpenAI Agents SDK — OpenAI-centric, want built-in tracing/guardrails.
Frameworks hide token usage and add layers — adopt only when the abstraction earns its complexity.
11. Pros / cons / when (pattern picker)
| Situation | Pattern |
|---|---|
| Fixed, decomposable steps with checks | Prompt chaining (§2) |
| Distinct input categories, specialized handling | Routing (§3) |
| Independent subtasks → speed | Parallelization / sectioning (§4) |
| Multiple looks → confidence/safety | Parallelization / voting (§4) |
| Subtask count/shape unknown until runtime | Orchestrator-workers (§5) |
| Clear criteria + refinement adds value | Evaluator-optimizer (§6) |
| Steps unpredictable, LLM should choose freely | Agent — see 08 - Agents & Agentic Systems |
12. Caveats & risk mitigation
- Default to the simplest pattern. Anthropic’s core message: most teams over-build. A workflow you can read and test beats an autonomous agent you can’t. Add orchestration only when it measurably improves outcomes.
- Every extra LLM call costs tokens + latency. Parallelization and evaluator-optimizer multiply spend; orchestrator-workers fan it out unpredictably. Tie pattern choice to budgets — see 15 - Cost, Latency & Performance.
- Idempotency before retries. Adding retries/replay without idempotency keys causes duplicate side effects (§8). This is the most common production bug in long-running agent workflows.
- Checkpoint ≠ durable execution. Don’t promise exactly-once continuation from a checkpointer that re-runs nodes on resume (§7). Know which guarantee your runtime gives.
- HITL pre-interrupt code re-runs. Side effects before an
interrupt()fire again on resume — move them after the human gate (§9). - Routers and orchestrators are single points of failure. A misclassification/bad decomposition fans out to everything downstream. Eval them in isolation and add fallback paths.
- Framework lock-in & churn. The space consolidated twice in one year (§10). Centralize model/orchestration config; keep business logic framework-agnostic so a migration is hours not a rewrite.
- Observability is mandatory. Multi-step workflows fail in non-obvious places. Trace every step (inputs, outputs, gate decisions, loop counts) — see 13 - Evaluation & Evals.
13. In practice
# Prompt chaining with a gate (workflow — predefined path)
outline = llm(f"Outline an article on: {topic}")
if not passes_gate(outline): # cheap deterministic check
return reject("outline failed criteria")
draft = llm(f"Write the article from this outline:\n{outline}")
final = llm(f"Polish for tone:\n{draft}")
# Evaluator-optimizer with a HARD cap (always bound the loop)
candidate = optimizer(task)
for _ in range(MAX_ROUNDS): # never unbounded
verdict = evaluator(candidate, criteria)
if verdict.passed:
break
candidate = optimizer(task, feedback=verdict.feedback)
# Idempotent side effect — safe under replay/retry (§8)
charge(amount, idempotency_key=f"{run_id}:{step_id}") # dedup at the API# Human-in-the-loop: do the side effect AFTER the gate, not before (§9)
plan = llm(f"Plan the refund for: {ticket}")
approval = interrupt({"review": plan}) # node re-runs on resume — keep above idempotent
if approval.approved:
issue_refund(ticket, idempotency_key=f"{run_id}:refund")14. Sources
- Anthropic — Building Effective Agents
- LangChain — Durable execution (LangGraph)
- Diagrid — Why checkpoints aren’t durable execution
- Microsoft Learn — Agent Framework overview
- Microsoft Learn — AutoGen → Agent Framework migration guide
- VentureBeat — Microsoft retires AutoGen, debuts Agent Framework
- OpenAI — New tools for building agents (Agents SDK)
- GitHub — openai/swarm (deprecated, points to Agents SDK)
- Model Context Protocol — spec