Skip to content

README.md

Latest commit

 

History

History
135 lines (92 loc) · 8.81 KB

README.md

File metadata and controls

135 lines (92 loc) · 8.81 KB
Raw

Building LLM Agents — A Reference Guide

Last verified: early 2026. The agent ecosystem moves fast — treat specific product names, model names, and prices as snapshots, not gospel. The patterns and decision frameworks underneath are stabilizing.

A linear, build-up-from-zero guide to designing and architecting LLM agent systems. Starts with the absolute basics (what is an agent?) and builds up to multi-agent orchestration, context engineering, RAG, evals, security, and production concerns.

The concepts are framework-agnostic, but the worked examples lean on LangGraph and the OpenAI Python SDK, with notes throughout on how each concept maps to the Claude Agent SDK and the OpenAI Agents SDK. The principles travel across all of them; the choice of harness is mostly about which one fits your existing stack.

How to read this guide

  • Newcomers (you've called the OpenAI or Claude API once or twice): start at Chapter 1, read top to bottom.
  • Intermediate (you've built a tool-calling agent): skim Part I, start at Chapter 5.
  • Experienced (you've built multi-agent systems): jump to Part IV (Chapter 13) for architecture patterns and decision frameworks. Then read Part V, especially Chapters 18, 20, and 23 — the additions most often missing from production agents.

Each chapter is short — most run 300–700 words, the deeper ones (long-term memory, routing, evals, security) stretch toward 1500 when the topic earns it. Each chapter stands alone as a reference. Use the index below to jump to a specific topic, and the glossary when a term needs a refresher.

Index

Part I — Foundations

  1. What is an agent? — vs chatbot vs workflow
  2. The anatomy of an LLM call — messages, tools, schemas
  3. Tools — the agent's hands — designing them well, structured outputs, idempotency
  4. MCP — tools as a protocol — portability, isolation, trust boundary

Part II — The Single Agent

  1. The execution loop — ReAct, recursion, termination
  2. State and messages — the canonical state
  3. Prompts as code — composition and pitfalls

Part III — Context, Memory & Knowledge

  1. Three kinds of state — conversation, session, long-term
  2. Context & cache engineering — token budgets, layout, prompt-cache discipline
  3. Long-term memory with vector stores — retrieval and pollution
  4. Retrieval-augmented generation (RAG) — knowledge corpora, hybrid search, rerankers
  5. State recovery and resumability — checkpointers, async and durable runs

Part IV — Multi-Agent Architecture

  1. When to split (and when not to)
  2. Routing patterns — classifiers, supervisors, handoffs vs tool calls vs workers
  3. The merge-vs-split tightrope
  4. Shared state across agents

Part V — Production Concerns

  1. Streaming responses
  2. Human-in-the-loop — interrupts, approval gates, edit-and-resume
  3. Reliability — retries, validation, idempotency
  4. Guardrails, prompt injection & agent security
  5. Cost and latency optimization — with a worked example
  6. Observability — the minimum viable
  7. Evals & regression testing — golden sets, trajectory evals, LLM-as-judge

Part VI — Practice and Patterns

  1. Common anti-patterns
  2. Decision frameworks
  3. Reference architecture
  4. Shipping checklist
  5. Tips and tricks (verified patterns)
  6. Modern agent patterns and harnesses — incl. multimodal and computer-use

Reference

Mapping concepts to the major SDKs

The guide is framework-agnostic in concept, but the same idea has different names in different SDKs. Quick cheat sheet:

Concept LangGraph OpenAI Agents SDK Claude Agent SDK
Agent loop Graph with a tool node Agent.run() query() loop
Tools @tool / bind_tools function_tool / Tool tool definitions, MCP servers
Multi-agent handoff Edges between nodes handoff() Sub-agents via tool calls
Persistent state Checkpointer Session Conversation files / hooks
Human-in-the-loop interrupt() interruption_handler Pre-tool-use hooks
Guardrails Custom node Guardrail Hooks + permission checks
Tracing LangSmith integration Built-in tracing OpenTelemetry hooks

When the worked examples in this guide use one SDK's vocabulary, the same pattern works in the others — the prose flags any meaningful differences.

What this guide is NOT

  • A LangGraph or SDK API reference (read each project's official docs for that)
  • A prompt engineering guide (focused on architecture, not prompt tricks — though Chapter 9 touches on context engineering)
  • A debugging playbook (focused on doing it right, not fixing what's wrong)
  • A vector DB benchmark (uses "vector store" generically)
  • An academic survey of RAG techniques (Chapter 11 is practical)

What this guide IS

  • A linear progression from "what is an agent?" to "how do I architect a multi-agent system?"
  • A set of named patterns and anti-patterns you can reference in design discussions
  • A decision framework for the questions that come up over and over (split vs merge, tool vs param, etc.)
  • A working mental model that survives framework churn

Sources and influences

This guide cross-references the most current public material from major LLM providers and frameworks. Where these sources disagree, the guide takes the position that's held up best in practice and explains the trade-off.

Foundational essays

Provider documentation (current)

Standards and protocols

Agent frameworks

  • LangChain, LangGraph — graph-based agent orchestration
  • OpenAI, Agents SDK (Mar 2025) — opinionated Python/TypeScript SDK with handoffs, guardrails, and tracing
  • Anthropic, Claude Agent SDK — programmatic SDK built around Claude Code's agent capabilities

The framework choice matters less than the principles. Pick the one that fits your existing stack; the patterns in this guide work across all of them.