Digital Library

Build an AI Agent (From Scratch) MEAP V05 (Jungjun Hur, Younghee Song)（Z-Library）

,

Share E-Book
Read Without Registration

Build an AI Agent (From Scratch) MEAP V05 (Jungjun Hur, Younghee Song)（Z-Library）

Author

,

AI
Language English

Build a working AI agent that can reason, plan, and execute multi-step tasks! LLM-powered AI agents are the next leap in applied AI, capable of reasoning and collaboration to achieve even complex, multi-step goals. Using new protocols like MCP and A2A, agents can use software tools, retrieve relevant knowledge, and adapt to feedback. This book guides you step by step in creating an AI agent from the ground up, with clear, detailed explanations you can follow to build your own custom assistants! In Build an AI Agent (From Scratch) you will learn how to: Implement a ReAct (Thought → Action → Observation) loop Use MCP to integrate tools calls into your agent’s workflow Agentic RAG for relevant responses Create memory modules that store facts, context, and evolving goals Enable agents to plan, reflect, and self-correct Build specialized agents, including a code execution agent Design multi-agent systems In Build an AI Agent (From Scratch), bestselling author Jungjun Hur and AI expert Younghee Song guide you through creating a complete research assistant agent framework. You’ll learn how agents function under the hood—all without hidden abstractions, black boxes, or framework lock-in. You will implement each piece as you develop a mental model of how agents really work. about the reader For Python developers and AI practitioners. All examples run on a standard laptop. about the authors Jungjun Hur is an AI and data engineer with experience in e-commerce and AI industries, where he has built production-ready AI applications and LLM-powered features. He is the author of the bestselling book Practical AI Application Development Using LLMs.

Format PDF
Size 15.3 MB
10
Views
(First 20 pages)

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

Page 1
(This page has no text content)
Page 2
MEAP Edition Manning Early Access Program Build an AI Agent (From Scratch) Agents that reason, plan, and act autonomously Version 5 Copyright 2026 Manning Publications For more information on this and other Manning titles go to manning.com. © Manning Publications Co. To comment go to liveBook
Page 3
welcome Thank you for purchasing the MEAP edition of Build an AI Agent (From Scratch). To get the most out of this book, you don’t need to be an expert in AI or machine learning. If you are comfortable with the basics of Python—things like writing simple functions and classes—know how to call and use APIs, and have some familiarity with Git and GitHub, you’ll be more than ready to follow along. That’s all the background you need. When I first started exploring the world of AI agents, I found myself overwhelmed. Every week, it seemed like a new agent framework, research paper, or tool was being released. Even as an engineer, I struggled to make sense of the rapid changes. I realized that if I only stayed a user of these frameworks, I would never truly understand how they worked or how to evaluate which ones to trust. That’s when I decided to dig deeper—and ultimately, that path led me to writing this book. At the heart of this book is a hands-on project: together, we’ll build a simple but educational agent framework called scratch_agents and use it to create agents that solve problems step by step. As we build this framework, you’ll also discover that the essence of an agent lies in delivering the right context to an LLM so that its intelligence can be fully utilized. We call this process Context Engineering. Agent frameworks are essentially designed to manage this challenge effectively, and through this book you’ll learn how to design, implement, and experiment with this idea in practice. This book is not about giving you yet another tool to memorize. It’s about empowering you to understand the inner workings of agent frameworks so you can go beyond surface- level usage. By the end of this journey, you won’t just be using existing frameworks—you’ll know how to open them up, understand their trade-offs, and even build your own when needed. That’s the kind of confidence and clarity this book aims to give you. I hope this book helps you cut through the noise in this fast-moving field and gives you both confidence and clarity in working with AI agents. Please share your thoughts and questions in the liveBook discussion forum—your feedback will be invaluable in making this book the best it can be. —Younghee Song & Jungjun Hur © Manning Publications Co. To comment go to liveBook
Page 4
brief contents PART 1: BUILDING YOUR FIRST LLM AGENT 1 What is an AI agent? 2 The brain of AI agents: LLMs 3 Enabling actions: Tool use 4 Implementing a basic ReAct agent PART 2: DEVELOPING ADVANCED AGENT CAPABILITIES 5 Building knowledge bases with RAG 6 Adding memory to your agent 7 Planning and reection for complex tasks 8 Empowering agents with code execution 9 Orchestrating multi-agent systems 10 Evaluating agents Appendix A. OpenAI API Key © Manning Publications Co. To comment go to liveBook
Page 5
Part 1: Building your first LLM agent The term "AI agent" gets used broadly, but the underlying idea is simple. An LLM serves as the brain, tools give it a way to act on the world, and a loop ties the two together so the agent can keep working until a task is done. This part focuses on assembling those three elements from scratch so that by the end, you have a small working agent. We skip frameworks on purpose. Once you have built the internals yourself, any framework you pick up afterward becomes easy to read, because you already know what is happening underneath. Chapter 1 defines what an agent actually is and draws a clear line between agents and the chatbots or automation scripts they are often confused with. It establishes the mental model that carries through the rest of the book. Chapter 2 is about talking to an LLM directly. You will learn how to send messages, shape behavior through a system prompt, and parse what comes back, building the thin communication layer that every later chapter depends on. Chapter 3 gives the LLM a way to reach into the outside world through tools and function calling. You will convert ordinary Python functions into specifications that the model can understand and invoke. Chapter 4 ties everything together with a ReAct loop, the cycle of thinking, acting, and observing that turns a one-shot model call into an agent that keeps going until the job is finished. By the end of Part 1 you will have a working agent of your own, and with it a feel for the pieces every agent needs, whatever labels future frameworks stick on them: the conversation record that tracks what has been said and done, the tool abstraction that lets the model take real actions, the execution state that carries across turns of the loop, and the layer that speaks to the model itself. That small agent becomes the foundation for everything in Parts 2 and 3, and it keeps running unchanged no matter how much we add on top of it. © Manning Publications Co. To comment go to liveBook 1
Page 6
1 What is an AI agent?  This chapter covers The landscape of AI agents today  LLMs as the decision-making core of agents  Workflows vs agents and when to use each  GAIA benchmark for measuring agent performance  Context engineering for building effective agents  You may have heard of agent-building frameworks like LangGraph, CrewAI, AutoGen, or OpenAI Agents. These frameworks make it easy to build agents quickly, but they also hide what's actually happening inside. This book takes a different approach: we'll build agents from scratch, understanding every component before relying on any framework. Why build from scratch? Because agent development is fundamentally about debugging failures. When your agent gives a wrong answer or gets stuck in a loop, you need to understand exactly what went wrong. Did the LLM (Large Language Model) misinterpret the context? Did a tool return unexpected results? Was crucial information missing? Without understanding how agents work internally, diagnosing these problems is difficult, regardless of which tools you use. By building each component yourself, you'll develop the mental model needed to troubleshoot any agent system, whether you built it or inherited it. © Manning Publications Co. To comment go to liveBook 2
Page 7
This chapter establishes the foundation for everything that follows. We'll start by surveying the landscape of AI agents, from personal assistants to specialized coding tools, to understand what we're building toward. Then we'll examine how LLMs serve as the "brain" of an agent and what distinguishes a true agent from a simple workflow. We'll introduce GAIA (General AI Assistants), the benchmark we'll use throughout this book to measure our progress, and explore context engineering, the discipline that determines whether an agent succeeds or fails. By the end, you'll have a clear mental model of what agents are, when to use them, and the principles that will guide our implementation journey. 1.1 The age of AI agents AI agents are rapidly transforming how we interact with technology. From personal assistants that help with everyday tasks to sophisticated systems that handle complex professional work, agents are becoming an integral part of both consumer products and enterprise solutions. Before diving into how to build them, let's survey the landscape of AI agents to understand what we're working toward. Personal AI Agents are the most familiar form of AI agents today. Services like ChatGPT, Claude, and Gemini started as conversational chatbots where you asked a question, and they provided an answer. But these systems have evolved dramatically. Modern personal agents can analyze uploaded documents, search the web for current information, generate images, write and execute code, and even help with shopping decisions. They serve as general-purpose assistants that adapt to whatever task you bring to them, learning your preferences and communication style over time. Customer-Facing Agents operate on behalf of businesses, interacting directly with customers in real time. These agents go beyond simple FAQ chatbots. They can access company policies, retrieve customer data, process transactions, and make decisions based on business rules. When you interact with a support agent that checks your order status, processes a refund, or helps troubleshoot a product issue, you're increasingly likely to be communicating with an AI agent. These systems must balance helpfulness with strict adherence to company guidelines and regulatory requirements. Specialized Agents tackle domain-specific tasks that require deep expertise or extended processing time. Coding agents like Claude Code, Cursor, and Codex CLI can navigate codebases, implement features, fix bugs, and refactor code. These are tasks that previously required hours of developer time. Deep Research agents can conduct comprehensive investigations across hundreds of sources, synthesizing findings into detailed reports. Because these tasks often take significant time to complete, specialized agents frequently operate asynchronously, notifying users when results are ready. When connected to proprietary enterprise data, these agents become vertical solutions tailored to specific industries or use cases. © Manning Publications Co. To comment go to liveBook 3
Page 8
Despite their different applications and interfaces, all these agents share a common foundation: they are powered by Large Language Models. Understanding how LLMs work and how they enable agent capabilities is essential for anyone who wants to build effective agents. In this book, we use "AI agent" and "LLM agent" interchangeably, since virtually all modern AI agents are powered by LLMs. 1.2 Understanding LLM agents All the agents we explored share something remarkable: they rely on LLMs as their decision-making core. But how does a system designed to predict the next word become capable of completing multi-step tasks autonomously? Let’s first unpack what makes LLMs uniquely suited to power agents, then examine the three essential components that transform an LLM into an agent. This will help you understand the agent loop that forms the foundation for everything we'll build in this book. 1.2.1 What is an LLM? An LLM (Large Language Model) is a language model trained on nearly all publicly available text from the internet. While modern LLMs have evolved into multimodal models that process images, audio, and video, this book uses the term "LLM" to refer to all these variants, since they share the same foundational architecture. The core principle of LLMs is simple: learn to predict the next word by processing massive amounts of text data. Given the sentence "Because it's summer vacation, I didn't go to ______," most would expect the blank to be filled with "school." Through large-scale training on this simple principle, LLMs develop an understanding of language structure, context, and meaning. LLMs process text by breaking it into tokens, which are typically words or word fragments. The model predicts the next token based on all preceding tokens. Earlier approaches, like rule-based systems and reinforcement learning, also attempted to create intelligent agents. What sets LLMs apart is their generalization capability—the ability to handle diverse tasks without task-specific retraining. Figure 1.1 demonstrates this. First, LLM translates "banana" using an instruction and two examples (apple, orange)—this is a few-shot learning. Meanwhile, with zero-shot learning, the LLM succeeds with just the instruction, without any examples. © Manning Publications Co. To comment go to liveBook 4
Page 9
Figure 1.1 Example of a language model’s generalization capability. Recently, "reasoning models" have emerged that analyze tasks or formulate plans before producing results. The reasoning model doesn't answer immediately, but first generates thinking tokens starting with <think>. These advances enable LLM agents to tackle complex and unfamiliar problems effectively. 1.2.2 What is an LLM Agent? A representative example of an LLM agent is the research agent. When a user asks, "Summarize the 2024 Nobel Physics winners' research," the agent gathers information from multiple sources, analyzes findings, and produces a comprehensive report. As shown in figure 1.2, the research agent doesn't simply generate an answer from its training data. It searches for 2024 Nobel Prize information, identifies the winners (Geoffrey Hinton and John Hopfield), explores academic papers, and consults Wikipedia. It then synthesizes all this information into a coherent report detailing the winners' achievements and their impact on AI. What's worth noticing here is who decided each of these steps. Which source to check first, whether the search results were sufficient, what to look for next, when to start writing the report — none of these decisions were hardcoded. The LLM made them on the fly, based on the context at each moment. © Manning Publications Co. To comment go to liveBook 5
Page 10
Figure 1.2 User requests flow through the research agent, which branches into multiple searches and synthesis. CORE DEFINITION: AUTONOMOUS CONTROL FLOW This example reveals the essence of an agent. An LLM agent is a program that autonomously decides what actions to take and when to stop based on its current context and goals. By control flow, we mean the sequence of “what to do next.” In traditional software, this flow is spelled out in the code. If statements, for loops, and the order of function calls were all decided by the developer in advance. In an LLM agent, part of this decision-making authority is handed over to the LLM. © Manning Publications Co. To comment go to liveBook 6
Page 11
ELEMENTS THAT REALIZE AUTONOMY: LLM + TOOL + LOOP For autonomous control flow to work in practice, three structural elements come together. The LLM serves as the agent's brain. It understands the current situation and decides what to do next. Tools extend the agent's action space into the external world—web search, code execution, database access. The Loop is the execution structure that lets these decisions happen repeatedly until the goal is achieved. To summarize: the LLM makes autonomous decisions possible, tools expand the action space, and the loop unfolds it over time. HOW THE AGENT LOOP WORKS The Loop is necessary because it's difficult to know in advance which tools will be needed or how many steps will be required to complete a task. Figure 1.3 shows the research task about the 2024 Nobel Prize winners from figure 1.2, viewed through the lens of the Agent Loop. Figure 1.3 The LLM Agent's decision loop is an iterative process of LLM decision-making and tool use. The process follows a continuous loop: 1. The LLM evaluates the current context and determines whether a tool is needed. If more information is required, it decides which tool to use based on what's missing. 2. The selected tool is executed—whether it's searching for "2024 Nobel Physics," looking up "Hinton, Hopfield," finding academic papers, or accessing Wikipedia for background knowledge. 3. The tool's results are added back to the LLM's context, enriching its understanding of the topic. This accumulated context allows the LLM to make more informed decisions in subsequent iterations. 4. The LLM decides whether to continue or stop. If it determines that sufficient information has been gathered to answer the user's question comprehensively, it generates the final report. Otherwise, it returns to step 1 for another iteration. © Manning Publications Co. To comment go to liveBook 7
Page 12
This iterative process of reasoning, acting, and observing enables the agent to handle tasks of unpredictable complexity from simple queries requiring a single search to complex research requiring dozens of information sources. 1.3 Workflow versus agent When integrating LLMs into applications, developers face a fundamental architectural choice: how much control should the LLM have over the execution flow? This decision shapes everything from system reliability to operational costs. Figure 1.4 illustrates seven distinct levels of agency, progressing from left to right with increasing autonomy. The three rows in the figure represent different aspects of control: who produces the current output (current step), who decides what happens next (next step), and who defines the available options (available steps). As we move rightward, the LLM gains more control over each of these dimensions. Figure 1.4 Progression of agency levels in LLM applications. These seven levels can be broadly categorized into two architectural approaches: workflows and agents. In workflows, developers predefine the execution flow and use LLMs to perform specific steps within that structure. In agents, LLMs dynamically determine their own processes, deciding which actions to take and when to stop. Understanding this distinction is essential for choosing the right approach for your use case. 1.3.1 Workflow: Developer-defined flow A workflow is a system where developers explicitly design the sequence of operations, with LLMs executing specific steps within that predefined structure. The key characteristic is predictability: given the same input, the system follows the same path through the workflow. © Manning Publications Co. To comment go to liveBook 8
Page 13
LLM CALL The most basic pattern is a single LLM call, where one prompt goes in, and one response comes out. This delegates a specific task to the model while keeping everything else under developer control. Examples include text classification, summarization, or answering a straightforward question. Despite its simplicity, a well-crafted single call can handle surprisingly complex tasks. CHAIN A chain connects multiple LLM calls in a predefined sequence, where the output of one step becomes the input for the next. The developer designs this flow in advance, and the LLM executes each step sequentially. This pattern leverages a fundamental principle: LLMs perform best with focused, well- defined tasks. Rather than asking a model to "analyze this document and create a presentation," a chain might break this into discrete steps: extract key points, organize by theme, generate slide content, and refine for clarity. Each step is simpler and more likely to succeed than attempting everything at once. ROUTER A router introduces conditional logic where the LLM decides which predefined path to take next. Given a user query, the model might classify it and route it to the appropriate handler. Billing questions go to one workflow, technical support to another. While the LLM makes a decision here, it's choosing from a fixed set of options that the developer has explicitly defined. The available paths, and what happens along each path, remain under developer control. This makes routers a workflow pattern rather than an agent pattern. The LLM influences the route but doesn't control the journey. 1.3.2 Agent: LLM-directed flow An agent is a system where the LLM dynamically directs its own processes and tool usage, maintaining control over how it accomplishes tasks. Rather than following a predetermined path, the agent decides what to do next based on the current context and its progress toward the goal. TOOL USE WITH A MULTI-STEP LOOP The defining pattern of an agent combines two capabilities: the ability to use external tools and the autonomy to continue operating until the task is complete. Since LLMs can only generate text, they cannot directly interact with external systems, access real-time information, or perform precise calculations. Tools bridge this gap by exposing external functionalities (web search, code execution, database queries, API calls) as callable functions. The LLM examines the available tools, decides whether one would help, and if so, specifies which tool to call with what parameters. © Manning Publications Co. To comment go to liveBook 9
Page 14
What transforms tool use into an agent is the loop. Rather than making a single tool call and stopping, the agent operates in a cycle: assess the current state, decide on an action, observe the result, and repeat until the task is complete. The LLM itself determines when to continue and when to stop. This multi-step loop is the core mechanism that enables agents to handle tasks of unpredictable complexity. A simple query might resolve in one step; a complex research task might require dozens of iterations across multiple tools. TOOL CREATION At the highest level of autonomy, the agent doesn't just select from available tools. It creates new ones. This typically involves generating code to implement functionality that doesn't exist in the predefined toolkit. For example, if an agent needs to process data in a specific format that no existing tool handles, it might write a custom parsing function, execute it, and use the results to continue its task. This capability allows agents to extend their own abilities dynamically, adapting to requirements that weren't anticipated when the system was designed. 1.3.3 Combining workflows and agents in practice The distinction between workflows and agents isn't always binary in production systems. In practice, the most effective architectures often combine both approaches, using workflows to provide structure and predictability while embedding agents at specific points where flexibility is needed. AGENTS AS NODES IN A WORKFLOW A common pattern is to design an overall workflow with well-defined stages, but implement one or more of those stages as an agent. Consider a document processing pipeline: 1. Document intake (workflow): Validate format, extract metadata 2. Content analysis (agent): Research context, gather related information, synthesize findings 3. Quality review (workflow): Check against compliance rules, format output 4. Delivery (workflow): Route to the appropriate destination The agent operates freely within its designated stage, making multiple tool calls, following chains of inquiry, and adapting to what it discovers. But the overall process maintains the predictability of a workflow. If the agent fails or produces unexpected results, the workflow can catch this at the quality review stage. © Manning Publications Co. To comment go to liveBook 10
Page 15
WHY COMBINE? This hybrid approach offers several advantages. First, it contains complexity. Agent behavior is inherently less predictable, so limiting where agents operate makes the overall system easier to reason about and debug. Second, it optimizes costs. Agent loops can be expensive due to multiple LLM calls, so using them only where their flexibility is genuinely needed keeps operational costs manageable. Third, it enables graceful degradation. If an agent component fails, the workflow structure allows for fallback behaviors or human escalation at defined points. The architectural choice isn't "workflow or agent" but rather "where in this system does agent behavior provide enough value to justify its costs?" This pragmatic framing will serve you well as you design your own LLM-powered applications. 1.4 Tasks that require agents Before jumping into agent development, we need to ask the right questions in the right order. Two key decisions shape whether an agent is the appropriate solution: 1. Does this task require an LLM at all? 2. If an LLM is needed, does it require an agent, or will a workflow suffice? Answering "no" at either stage means choosing a simpler, more cost-effective approach. Let's examine the criteria for each decision, then look at a benchmark that captures tasks where agents truly excel. 1.4.1 Tasks that require an LLM Before considering the use of an LLM agent, you first need to determine whether the task itself requires an LLM. This foundational question matters because LLMs introduce significant overhead in terms of computational costs and potential for errors. This overhead may be unnecessary if the task can be solved with simpler, more deterministic approaches. For instance, if your task involves basic data processing with predictable inputs and outputs, traditional programming logic will be faster, cheaper, and more reliable than any LLM-based solution. There are two main criteria for making this decision: Tasks involving unstructured data. If a task requires analyzing unstructured data like text, images, or audio, it's a strong candidate for LLM usage. Traditional programming excels at structured data with clear schemas, but struggles with the ambiguity inherent in natural language or visual content. While models that handle diverse data formats are generally called Multimodal LLMs (or LMMs), they are fundamentally based on LLMs, so this book collectively refers to them as LLMs. © Manning Publications Co. To comment go to liveBook 11
Page 16
Input diversity. If user input is limited or the requested tasks are narrowly scoped, using a specialized ML model or rule-based system might be more cost-effective. Similarly, if the number of potential tasks is small and predefined, it's more efficient in terms of both cost and latency for a developer to hard-code the logic than to use an LLM. LLMs shine when inputs are unpredictable and varied, requiring flexible interpretation that can't be anticipated in advance. 1.4.2 Conditions for using agents Once you've determined that an LLM is necessary, the next question is how to use it. Will a single LLM call suffice, or does the task require the iterative reasoning and tool use that define an agent? Consider the following question: "If Eliud Kipchoge could maintain his marathon world record pace indefinitely, how long would it take him to run from Earth to the Moon?" Answering this question requires multiple steps. First, you need to search for who Kipchoge is and find his marathon world record. Then, you check Wikipedia for the distance between Earth and the Moon. Finally, you perform calculations to divide the distance by the speed and derive the final answer. Anyone can do this task manually. However, the process of navigating between multiple websites to gather information, converting units, and performing calculations is time- consuming and error-prone. This kind of multi-step research task is exactly where agents shine. As excitement around agents grows, there's a common trap people fall into: applying agents to every task simply because they're new and powerful. But just as you don't need a complex web framework to build a simple static website, choosing the right tool is key to building efficient systems. Agents come with inherent trade-offs. First, costs are higher. Multiple LLM calls multiply API expenses compared to single requests. Second, latency increases. Response time accumulates with each reasoning step. Third, errors propagate. Mistakes in early steps cascade through the entire process. Consider a customer service query that could be handled by a single LLM call costing $0.01. Processing the same query with an agent requiring 10 calls would cost $0.10. That's a 10x difference. When you're handling thousands of requests daily, this isn't just a technical decision; it's a business decision. So when should you use an agent? You can decide based on these three criteria: Task complexity. If it's difficult to predict how many steps a task will require, agents have the advantage. "Find the population of region A" is straightforward. In contrast, "Analyze how LLM agents will shape the future" requires gathering diverse information and synthesizing complex relationships. It's a task that's hard to predefine with static logic. Task value. Since agents require multiple calls and dynamic decision-making, they cost more and respond more slowly. Therefore, the value of completing the task should outweigh the additional cost and latency. © Manning Publications Co. To comment go to liveBook 12
Page 17
Error cost and detectability. LLMs are prone to making incorrect decisions, and the risk increases with more calls. If errors lead to critical consequences, an agent might not be the best choice. Detectability also matters. In domains requiring specialized knowledge, users or developers may not even realize when an agent has produced faulty results. 1.4.3 GAIA: An agent gym The Kipchoge question we examined earlier is actually from a benchmark called GAIA. In 2023, Meta and HuggingFace released GAIA (General AI Assistants), a dataset that systematically collects tasks requiring agents. GAIA consists of question-answer pairs. Each question requires multi-step reasoning, web searches, calculations, and more. They're difficult to solve with a single LLM call, and it's not easy to predefine a clear workflow either. These are problems that naturally require an agentic approach. GAIA was released in 2023, and since then, model capabilities have improved significantly, with more challenging benchmarks emerging. However, GAIA problems range from straightforward to genuinely difficult, even for current models, making it well-suited for learning agent development. This book uses GAIA for three specific reasons. Clear answers enable fast feedback. The core of agent development is "identifying when it fails and reducing those failures." When answers are clear, you can immediately verify whether you're heading in the right direction. You can rapidly iterate through cycles of experimentation and improvement without ambiguous evaluation criteria. It's optimal for practicing the agent development cycle. The process of building an agent goes like this: First, determine whether an agent would help solve a specific problem. If there's potential, develop a prototype. Then observe when the agent fails, analyze the cause, and improve. And you repeat this observe-analyze-improve cycle. GAIA provides an appropriate difficulty level to experience this entire cycle from start to finish. No domain knowledge required. Most GAIA problems center on web search and information synthesis. We all perform tasks in daily life that involve "searching, gathering information, and organizing it." You can intuitively understand the problems and solution processes without specialized knowledge in medicine or law. Throughout this book, we'll use GAIA to track changes in agent performance as we add new techniques in each chapter. You'll see firsthand how each component, such as tool use, memory, and planning, actually improves the agent's problem-solving capabilities. GAIA gives us a way to measure agent performance. But measurement alone doesn't improve an agent. What actually makes an agent better at solving these problems? The answer lies in how we engineer the context the LLM receives. © Manning Publications Co. To comment go to liveBook 13
Page 18
1.5 Context engineering Before diving into agent development, let us clarify two terms that are often used interchangeably. A prompt is the input text that a user sends to an LLM. It can be further divided into system prompts, which specify how the model should respond and behave (such as "You are a friendly travel guide"), and user prompts (or messages), which contain the user's actual requests. Context is a broader concept that encompasses all the information an LLM references when generating a response. This includes the system prompt, user messages, previous conversation history, tool execution results, retrieved documents, and more. In this sense, prompts are just one component of context. Think of context as the LLM's working memory. Just as you can't solve a math problem if someone only tells you half the equation, an LLM can't complete a task if crucial information is missing from its context. An LLM predicts the next token based solely on the information within its context window. The model generates text by leveraging patterns learned during training to make use of the information in the context. This simple fact is why context engineering is so critical in agent development. 1.5.1 Why agents fail Agent failures can be attributed to two main causes. The first is insufficient model intelligence. This includes cases where the model fails to solve complex mathematical problems or makes errors in logical reasoning. Addressing this issue requires using a more powerful model or waiting for advances in model capabilities. The second is a lack of necessary information. Even if a model is intelligent enough, it cannot produce correct answers if the information needed to complete the task is not in the context. For example, if someone asks, "What is our company's vacation policy?" but the company policy document is not in the context, even the most capable model cannot provide an accurate answer. Interestingly, a significant portion of real-world agent failures stems from the second cause: information deficiency. An LLM can only be as smart as the context it is given. This is precisely why "what information to include in the context" is a key factor that determines agent performance. 1.5.2 From prompt engineering to context engineering In the early days of LLM adoption, prompt engineering was the primary focus. Practitioners concentrated on how to write system prompts that would make models respond more accurately and which instructions would yield outputs in the desired format. Techniques like "Let's think step by step" and "You are an expert" emerged during this period. © Manning Publications Co. To comment go to liveBook 14
Page 19
However, the landscape changed as agents began using tools. When an agent performs a web search, the results are added to the context. When it executes code, the execution results are added. When it queries a database, the query results are added. Context began to change dynamically, evolving beyond static system prompts. This shift raised a new question: "How do we maintain only the essential information needed for task completion in the context?" The approach to answering this question is context engineering. Figure 1.5 LLMs can only produce accurate, high-quality responses when sufficient information is provided in the context. Context engineering is the discipline of providing the information an LLM needs to perform its tasks at the right time and in the right form. It goes beyond simply writing good prompts to comprehensively designing how tool execution results should be processed, how much conversation history to retain, and when to retrieve external knowledge. 1.5.3 Bigger context is not always better The context windows of recent LLMs have grown dramatically. Most models now support context windows ranging from hundreds of thousands to a million tokens. So would filling the entire context window with all available data yield the best responses? Unfortunately, no. Recent research has reported that model performance degrades as context length increases. This phenomenon is sometimes called Context Rot. The "Lost in the Middle" effect is particularly well documented, where models tend to miss important information located in the middle of long contexts. © Manning Publications Co. To comment go to liveBook 15
Page 20
Figure 1.6 Even with large context windows, longer inputs can degrade model performance(Source: https://research. trychroma. com/context- rot). This provides an important insight. To maximize model performance, we should not simply provide more information but rather selectively provide only highly relevant information. When the context is filled with unnecessary information, crucial details can get buried, or the model's attention can become dispersed. 1.5.4 Five context engineering strategies Context engineering can be broadly categorized into five strategies. Each strategy is covered in detail across various chapters of this book. Generation: Utilizing LLM-generated outputs within the context. This includes generating plans for complex tasks or reflecting on completed work to revise strategies. Beyond simply retrieving external information, this enables the LLM to structure and evolve the context on its own. Retrieval: Fetching necessary information from external sources and adding it to the context. This includes web searches, database queries, file reading, and similar document retrieval from vector databases. This is a core strategy that enables agents to access up-to- date information or domain-specific knowledge not present in their training data. © Manning Publications Co. To comment go to liveBook 16
The above is a preview of the first 20 pages. Register to read the complete e-book.

Recommended for You

Loading recommended books...
Failed to load, please try again later

Tip the Site

Scan the WeChat Pay or Alipay code to tip. No login required.

WeChat Pay
Alipay
← Back to List