Building LLM Agents — A Reference Guide

Last verified: early 2026. The agent ecosystem moves fast — treat specific product names, model names, and prices as snapshots, not gospel. The patterns and decision frameworks underneath are stabilizing.

A linear, build-up-from-zero guide to designing and architecting LLM agent systems. Starts with the absolute basics (what is an agent?) and builds up to multi-agent orchestration, context engineering, RAG, evals, security, and production concerns.

The concepts are framework-agnostic, but the worked examples lean on LangGraph and the OpenAI Python SDK, with notes throughout on how each concept maps to the Claude Agent SDK and the OpenAI Agents SDK. The principles travel across all of them; the choice of harness is mostly about which one fits your existing stack.

How to read this guide

• Newcomers (you've called the OpenAI or Claude API once or twice): start at Chapter 1, read top to bottom.
• Intermediate (you've built a tool-calling agent): skim Part I, start at Chapter 5.
• Experienced (you've built multi-agent systems): jump to Part IV (Chapter 13) for architecture patterns and decision frameworks. Then read Part V, especially Chapters 18, 20, and 23 — the additions most often missing from production agents.

Each chapter is short — most run 300–700 words, the deeper ones (long-term memory, routing, evals, security) stretch toward 1500 when the topic earns it. Each chapter stands alone as a reference. Use the index below to jump to a specific topic, and the glossary when a term needs a refresher.

---

Index

Part I — Foundations

1. What is an agent? — vs chatbot vs workflow
2. The anatomy of an LLM call — messages, tools, schemas
3. Tools — the agent's hands — designing them well, structured outputs, idempotency
4. MCP — tools as a protocol — portability, isolation, trust boundary
   • 4a — MCP implementation patterns (deep dive) — client-host-server, Streamable HTTP, primitives, client-side vs hosted connector, auth/OAuth, the traced chokepoint, tool-surface management

Part II — The Single Agent

5. The execution loop — ReAct, recursion, termination
6. State and messages — the canonical state
7. Prompts as code — composition and pitfalls

Part III — Context, Memory & Knowledge

8. Three kinds of state — conversation, session, long-term
9. Context & cache engineering — token budgets, layout, prompt-cache discipline
   • 9a — Prompt caching (deep dive) — breakpoints, TTL economics, the multipoint conversation breakpoint, verification (Anthropic + OpenAI)
10. Long-term memory with vector stores — retrieval and pollution
11. Retrieval-augmented generation (RAG) — knowledge corpora, hybrid search, rerankers
12. State recovery and resumability — checkpointers, async and durable runs

Part IV — Multi-Agent Architecture

13. When to split (and when not to)
14. Routing patterns — classifiers, supervisors, handoffs vs tool calls vs workers
15. The merge-vs-split tightrope
16. Shared state across agents

Part V — Production Concerns

17. Streaming responses
18. Human-in-the-loop — interrupts, approval gates, edit-and-resume
19. Reliability — retries, validation, idempotency
20. Guardrails, prompt injection & agent security
21. Cost and latency optimization — with a worked example
22. Observability — the minimum viable
23. Evals & regression testing — golden sets, trajectory evals, LLM-as-judge

Part VI — Practice and Patterns

24. Common anti-patterns
25. Decision frameworks
26. Reference architecture
27. Shipping checklist
28. Tips and tricks (verified patterns)
29. Modern agent patterns and harnesses — incl. multimodal and computer-use

Reference

• Glossary — every term used in the guide, one or two sentences each.
• Worked example: a todo agent, end to end — one complete agent built across the load-bearing chapters, ~280 lines of Python you can fork.

---

Mapping concepts to the major SDKs

The guide is framework-agnostic in concept, but the same idea has different names in different SDKs. Quick cheat sheet:

Concept	LangGraph	OpenAI Agents SDK	Claude Agent SDK
Agent loop	Graph with a tool node	Agent.run()	query() loop
Tools	@tool / bind_tools	function_tool / Tool	tool definitions, MCP servers
Multi-agent handoff	Edges between nodes	handoff()	Sub-agents via tool calls
Persistent state	Checkpointer	Session	Conversation files / hooks
Human-in-the-loop	interrupt()	interruption_handler	Pre-tool-use hooks
Guardrails	Custom node	Guardrail	Hooks + permission checks
Tracing	LangSmith integration	Built-in tracing	OpenTelemetry hooks

When the worked examples in this guide use one SDK's vocabulary, the same pattern works in the others — the prose flags any meaningful differences.

---

What this guide is NOT

• A LangGraph or SDK API reference (read each project's official docs for that)
• A prompt engineering guide (focused on architecture, not prompt tricks — though Chapter 9 touches on context engineering)
• A debugging playbook (focused on doing it right, not fixing what's wrong)
• A vector DB benchmark (uses "vector store" generically)
• An academic survey of RAG techniques (Chapter 11 is practical)

What this guide IS

• A linear progression from "what is an agent?" to "how do I architect a multi-agent system?"
• A set of named patterns and anti-patterns you can reference in design discussions
• A decision framework for the questions that come up over and over (split vs merge, tool vs param, etc.)
• A working mental model that survives framework churn

Sources and influences

This guide cross-references the most current public material from major LLM providers and frameworks. Where these sources disagree, the guide takes the position that's held up best in practice and explains the trade-off.

Foundational essays

• Anthropic, Building Effective Agents (Dec 2024) — the canonical short essay on agent design patterns
• Anthropic, Effective Context Engineering for AI Agents (Sept 2025) — the follow-up that reframes prompt engineering as context engineering for agentic workflows

Provider documentation (current)

• OpenAI: Function Calling · Structured Outputs · Prompt Caching
• Anthropic: Tool Use · Strict Tool Use · Prompt Caching · Extended Thinking

Standards and protocols

• Model Context Protocol (MCP) — open standard for tool/resource sharing across agents and clients
• OWASP LLM Top 10 — canonical threat list for agent security

Agent frameworks

• LangChain, LangGraph — graph-based agent orchestration
• OpenAI, Agents SDK (Mar 2025) — opinionated Python/TypeScript SDK with handoffs, guardrails, and tracing
• Anthropic, Claude Agent SDK — programmatic SDK built around Claude Code's agent capabilities

The framework choice matters less than the principles. Pick the one that fits your existing stack; the patterns in this guide work across all of them.