AI guide
【One-Line Pitch】
A practitioner's field guide for engineers who need to move AI agents from clever demos to live, multimodal, multi-agent systems that hold up in production. Read it if you're building real-time conversational or agentic products and care about architecture, interoperability, and operations—not just prompt tricks.
【Book Arc】
- **Opening (~0%–10%)**: Sets the ambition—agents that perceive, reason, and act in real time—and frames the whole journey from foundation-model basics to enterprise-grade platforms. Establishes why static prompting is no longer enough.
- **Early (~10%–35%)**: Builds the conceptual floor: a five-stage evolution of agents (classic deliberative/reactive/hybrid designs → foundation models → RAG → tool-using agents → multi-agent orchestration), a working definition of an agent, and the perception–reasoning–action loop.
- **Middle (~35%–55%)**: Gets concrete on the mechanics—how tools are declared (function declarations, docstrings, OpenAPI/JSON, SDK-based definitions) and how the agent execution loop orchestrates model calls and tool results. This is the "build it yourself before using a framework" stage.
- **Late (~55%–85%)**: Moves into real-time multimodal interaction and agentic architecture: live APIs, video/tools/system instructions, agent frameworks, multi-agent collaboration, and the interoperability protocols (A2A, MCP) that let agents and tools talk to each other.
- **Ending (~85%–100%)**: Productionization—an integrated reference architecture, async and long-running tasks, UX design, then AgentOps, evaluation/testing, tool management at scale, governance, security/privacy/responsible AI, and deployment/scaling.
【Key Takeaways】
- **Agents are systems, not prompts** (Early): The book defines an agent as an autonomous, foundation-model-powered system that perceives, decides, and acts via tools toward a goal—shifting attention from clever prompts to the full sense–think–act cycle.
- **History still pays off** (Early): Classic deliberative, reactive, and hybrid agent architectures supply the blueprints; LLMs simply drop a far stronger reasoning engine into them. Understanding this makes modern design choices legible rather than magical.
- **The FM transition trades transparency for power** (Early): Compared with rule-based classical agents, FM-powered agents reason probabilistically, plan dynamically, and pick tools from natural-language descriptions—but become opaque boxes that are harder to debug and explain.
- **Tool declarations are the real interface** (Middle): An agent is only as capable as its tools, and the model learns tools through descriptions—name, purpose, inputs, outputs, and worked examples. Well-written declarations (few-shot style) are a design skill, not boilerplate.
- **Standardize before you scale** (Middle): Moving from ad-hoc docstrings to formal OpenAPI/JSON definitions buys predictability, robustness, and integration—and sets up the later protocol chapters (MCP for tools, A2A for agent-to-agent).
- **The execution loop is the heart of orchestration** (Middle): The back-and-forth of query → tool call → result → model response is the pattern every framework abstracts; building it manually first makes frameworks understandable rather than mysterious.
- **Real-time and multimodal raise the bar** (Late): Live, bidirectional interaction with video, tools, and system instructions demands architecture decisions that differ sharply from request/response chatbots.
- **Production is an operations problem** (Ending): AgentOps, rigorous evaluation, tool management at scale, governance, and security are treated as first-class engineering concerns—the gap between PoC and live system lives here.
【Reading Tips】
- Deep-read Chapters 1 and the tool/execution-loop material; that foundation makes the framework and protocol chapters feel like shortcuts rather than new material.
- Skim the historical evolution if you already know agent taxonomy—but keep the "then vs. now" comparison table in mind as a design checklist.
- Treat the protocol chapters (A2A, MCP) as reference material to revisit when you actually wire systems together; don't try to memorize specs on first pass.
- Note that this is an Early Release: some chapters are marked unavailable in the excerpts, so expect gaps and check the final edition for production chapters.
- Keep a running list of your own tool declarations and evaluation criteria as you read—the book rewards readers who apply it to a live project.
【Coverage Limits】
The excerpts cover the opening, Chapter 1's agent foundations and tool mechanics, and the table of contents; most later chapters (multimodality, real-time APIs, frameworks, protocols, production) are listed but not excerpted, so this guide's treatment of those stages is structural rather than detailed.
Passage locations
Excerpt 1
are also available for most titles ( https://oreilly.com ). For more information, contact our corporate/institutional sales department: 800-998-9938 or corpo...
View in text
Excerpt 2
y principles of reasoning and planning we’ve just discussed. Stage 1: The Foundation Models and Prompts In the Generative AI era, it all started with the Lar...
View in text
Excerpt 3
the old systems for a massive gain in power and flexibility. Note We’ve covered the important history of legacy agents, but to keep things clear from this po...
View in text
Excerpt 4
in the functionDeclarations list to complete your Math tool. While writing JSON files by hand is a great way to understand the structure of a tool, in a real...
View in text