Scope
What an “agent” actually is (and isn’t), the augmented-LLM building block it’s made of, the control loop that makes it agentic, the reasoning patterns layered on top (ReAct, planning, Reflexion, evaluator-optimizer), why tool design matters more than model choice, and when not to build one. Anthropic’s framing is the through-line: most production “agents” are correctly workflows, and you “start simple, add agency only when it demonstrably helps.” For the orchestration patterns themselves see 09 - Workflow & Orchestration Patterns; for many agents talking see 10 - Multi-Agent Systems.
Related: Gen AI Engineer, 05 - Model Context Protocol (MCP), 09 - Workflow & Orchestration Patterns, 10 - Multi-Agent Systems, 13 - Evaluation & Evals
The core agent loop: reason → act (call a tool) → observe the result → repeat until done.
1. The concept: workflow vs agent (the distinction interviewers want)
Anthropic draws a sharp line, and getting it right is a senior signal:
| Workflow | Agent | |
|---|---|---|
| Control | LLMs and tools orchestrated through predefined code paths you wrote | LLM dynamically directs its own process and tool use |
| Steps | Fixed, known in advance | Unpredictable number, decided at runtime |
| Who decides “what next” | Your code | The model |
| Predictability / cost | High / low | Low / high |
Most "agents" in production are workflows — and should be
If you can draw the control flow as a flowchart before running it, it’s a workflow, not an agent. That’s usually the right answer: workflows are cheaper, faster, and far easier to debug. An agent earns its name only when the task is open-ended enough that you genuinely can’t pre-specify the path. Anthropic’s headline guidance: “find the simplest solution possible, and only increase complexity when needed.”
The defining property of an agent: it runs a loop where the model itself chooses the next action, and you don’t know ahead of time how many iterations it will take.
2. The building block: the augmented LLM
Every agent and workflow is composed of the same primitive — a model with three capabilities bolted on:
| Augmentation | What it adds | Covered in |
|---|---|---|
| Tools | Act on the world: call APIs, run code, edit files, search | this note + 05 - Model Context Protocol (MCP) |
| Memory | Carry state across steps/sessions (scratchpad, files, vector store) | 03 - Prompt & Context Engineering |
| Retrieval | Pull relevant context on demand (RAG, just-in-time fetch) | 07 - RAG |
Mental model
An agent is not a special model. It’s a plain LLM in a loop, given tools and a stopping condition. Everything below is about how that loop is steered, not about a different kind of intelligence.
MCP (Model Context Protocol) is the emerging standard for the tools-and-context layer — a uniform way to expose tools/resources to any model. See 05 - Model Context Protocol (MCP).
3. The agent control loop
The mechanism that makes a system “agentic”:
1. Receive task + tool definitions
2. LOOP:
a. Model reasons about current state
b. Model emits a tool call (or a final answer)
c. If final answer → STOP
d. Execute the tool, capture the result (observation)
e. Append observation to context → go to (a)
3. Return final answer
Pros: handles tasks you can’t fully specify upfront; recovers from intermediate failures; composes tools the way a human would.
Cons: unbounded step count, so unbounded cost and latency; the result is non-deterministic; one bad step can cascade. The single most important design decision is when the loop stops (§9).
The loop never self-terminates safely on its own
Without an explicit stop condition and an iteration cap, a confused agent will loop forever, re-trying variations of a failing action and burning tokens. Always cap iterations.
4. ReAct (reason → act → observe)
The foundational pattern (Yao et al., 2022). Instead of answering directly, the model interleaves Thought → Action → Observation at every step:
Thought: I need the current price of X. I'll search.
Action: search("X price 2026")
Observation: $142.30 as of June 2026
Thought: That answers it.
Action: finish("$142.30")
Pros: the explicit “Thought” makes reasoning legible and debuggable; grounding each action in a fresh observation cuts hallucination versus pure chain-of-thought; it’s the default loop most frameworks implement.
Cons: verbose (every step re-emits reasoning into context); can get stuck repeating an action that keeps failing; quality depends heavily on tool result phrasing.
Caveat — ReAct is interleaved reasoning
ReAct’s strength is that thinking and acting alternate. That’s also why it pairs awkwardly with reasoning models that already think internally — you can end up with two layers of reasoning. See 02 - Decoding, Sampling & Reasoning Controls on not bolting manual chain-of-thought onto a reasoning model.
5. Planning & decomposition
Two stances on when the model plans:
| Approach | How it works | Trade-off |
|---|---|---|
| Plan-then-execute | Model writes a full plan first, then executes each step | Cheaper, more controllable, auditable plan up front — but brittle when reality diverges from the plan |
| Interleaved (ReAct-style) | Plan one step at a time, react to each observation | Adapts to surprises — but no global view, can wander, costs more |
In practice: plan-then-execute suits tasks with a knowable structure (a build pipeline, a report with fixed sections); interleaved suits genuinely exploratory work (debugging, research). Hybrids — a coarse plan that’s revised at checkpoints — are common.
A plan is a hypothesis, not a contract
Plan-then-execute fails silently when an early step’s output invalidates the plan but the agent marches on anyway. Re-validate the plan against actual observations at checkpoints, or let the agent revise it.
6. Reflexion (self-critique)
Reflexion (Shinn et al., 2023) extends ReAct with a self-evaluation + memory step: after a trajectory, the agent verbally critiques its own output and stores that reflection to inform the next attempt. The “gradient” is linguistic feedback held in context — improvement happens within the context window, not by retraining.
Reported gains were large: ~91% pass@1 on HumanEval and a ~20-point absolute lift over a plain ReAct agent on HotPotQA.
Pros: turns a one-shot failure into a learning loop without any fine-tuning; the reflection is human-readable.
Cons: needs a usable feedback signal (test results, a checker, a reward); self-critique can be wrong or over-confident — the model grading itself shares its own blind spots; adds iterations (more cost/latency).
Reflexion needs an external grounding signal to be reliable
Self-critique against a real signal (unit tests pass/fail, a compiler error, a retrieval check) is powerful. Self-critique against nothing but the model’s own opinion drifts. Wire it to something objective where you can.
7. Evaluator-optimizer loop
A two-role pattern, closely related to Reflexion: one LLM call generates a candidate (the optimizer/actor), a second evaluates it against criteria and returns feedback (the evaluator), and the loop repeats until the evaluator is satisfied or a cap is hit.
Pros: strong when you have clear evaluation criteria and iterative refinement measurably helps (translation, code that must pass tests, copy that must hit a rubric); separating the critic from the author catches errors the author won’t.
Cons: only worth it when feedback genuinely improves the output — otherwise it’s pure overhead; the evaluator is itself an LLM and can be wrong; two calls per round multiplies cost.
This is a workflow pattern (you wrote the two-role loop) more than an autonomous agent — but it’s the engine inside many agentic refinement steps.
8. Tool design / the Agent-Computer Interface (ACI)
Most agent failures trace to the tool interface, not the model
Anthropic’s emphasis: you spend real effort on the human prompt; spend as much on the Agent-Computer Interface — the tool names, descriptions, schemas, and especially error messages the model sees. A capable model with badly-described tools fails; a smaller model with crisp tools often succeeds.
Concrete ACI rules:
- Write tool docs like you’re onboarding a junior engineer. Describe when to use the tool, what each argument means, edge cases, and an example call. The model only knows what the description tells it.
- Make error messages actionable. “Invalid input” teaches nothing; “
datemust be ISO-8601 like2026-06-22; gotJune 22” lets the agent self-correct on the next loop. - Choose formats the model writes well. Prefer shapes that are hard to get subtly wrong; give the model enough output tokens to “think” before committing to a tool call.
- Few thoughtful tools beat many overlapping ones. Start with a small set targeting your highest-impact workflows; expand only when evals show a gap. Too many similar tools cause wrong-tool selection.
- Test tools in isolation (a “tool workbench”). Exercise each tool with representative inputs before wiring it into the loop — most “the agent is dumb” bugs are tool bugs.
9. When to use an agent (and when not to)
Run the four-question gate before reaching for an agent:
| Check | Build an agent only if… |
|---|---|
| Complexity | The task is multi-step and you genuinely can’t specify the path in advance |
| Value | The outcome justifies the higher cost and latency |
| Viability | The model is actually capable at this task type |
| Cost of error | Mistakes are catchable and recoverable (tests, review, rollback) |
A “no” on any line means drop to a simpler tier — a single LLM call or a fixed workflow.
Start simple, add agency only when it demonstrably helps
Don’t open with a fully autonomous agent. Try a single call, then a fixed workflow, and reach for an agent loop only when a measured limitation forces it. Each step up the ladder costs more and is harder to debug.
10. Termination conditions & guardrails
The loop needs hard limits, because the model won’t reliably impose them itself:
- Step / iteration cap. A maximum number of loop turns — the backstop against infinite loops.
- Token / cost budget. Stop (or compact) when spend crosses a ceiling.
- Explicit done signal. A
finish/submittool, or a stop sequence, so “done” is unambiguous rather than inferred. - Human-in-the-loop gates. Require approval before irreversible or high-blast-radius actions (sending email, deleting data, spending money, pushing code).
- Sandboxing. Run tools (especially code execution) in an isolated environment with restricted egress, an executable allowlist, and resource limits — tool calls are untrusted model output.
- Checkpoints + retries. Snapshot state so a failed step can retry without restarting the whole run.
11. Cons, risks & mitigations
Agents cost roughly 4× the tokens of a single chat call — and that's the floor
Every loop turn re-sends the growing context (prior thoughts, tool results). Anthropic’s own framing notes the multi-step, multi-call nature makes agents materially more expensive than a chat completion. Budget for it and tie spend to 15 - Cost, Latency & Performance.
| Risk | Why it bites | Mitigation |
|---|---|---|
| ~4× token cost vs a chat call | Context regrows every turn | Iteration caps, token budgets, context editing/compaction, cheaper model for sub-steps |
| Compounding errors | One wrong step poisons every later step | Checkpoints + retries, evaluator/critic step, human gates on high-stakes actions |
| Non-determinism / hard to debug | Different path each run; no flowchart to step through | Log every Thought/Action/Observation, replay traces, assert on meaning via evals not exact strings |
| Runaway loops | Model retries a failing action forever | Hard step cap + a “no progress” detector |
| Unsafe tool use | Tool calls are untrusted output | Sandboxing, allowlists, dry-run/confirm on destructive tools |
| Poor tool interface | Model can’t tell which tool or how | Invest in tool docs/error messages; test on a tool workbench (§8) |
12. ⚠️ The 2026 shift: “agent harness” over “autonomous agent by default”
The field has cooled on dropping a fully autonomous agent into production as the first move. The 2026 framing:
Context engineering + just-in-time retrieval + a deliberate harness
- Context engineering is now the central skill: the limiting factor on long agentic runs is the context window, so the work is curating what’s in context at each step — pruning stale tool results, compacting history, and pulling only what’s needed. (See 03 - Prompt & Context Engineering.)
- Just-in-time retrieval beats front-loading: rather than stuffing everything in upfront, the agent fetches context when a step needs it, keeping the window lean and the model focused.
- The “agent harness” framing wins over “autonomous agent”: you design the scaffold — the tools, the loop bounds, the checkpoints, the gates, the context management — and the model operates within it. The intelligence is in the harness as much as the model.
The practical upshot: senior work on agents in 2026 is less “pick a clever reasoning pattern” and more “engineer the harness and the context so a capable model can’t go badly wrong.”
Sources
- Anthropic — Building Effective Agents
- Anthropic — Writing effective tools for AI agents
- Anthropic — Effective context engineering for AI agents
- ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2022)
- Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 2023)
- The 4 single-agent patterns: ReAct vs Plan-and-Execute vs ReWOO vs Reflexion