Name: Agent Engineering Tools, Workflows, and Feedback (Jungjun Hur, Younghee Son, Roberto Infante etc.)（Z-Library）
Author: Unknown Author

Page 1

A practical progression from basic agent concepts to dependable, tool-using agentic workflows AGENT ENGINEERING TOOLS, WORKFLOWS, AND FEEDBACK AI Agents Micheal Lanham Peter Belcak Roberto Infante Jungjun Hur Younghe Song

Page 2

Save 45% on any one of the books listed below by using the code AGENTENG45 at checkout. Get all 4 ebooks for $99.99, here. Save 50% on 2 or more of these books, here.

Page 3

Agent Engineering Tools, Workflows, and Feedback Copyright 2027 Manning Publications

Page 4

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com © 2027 Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was awware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid- free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. ∞ The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 ISBN 9781638359265 Printed in the United States of America

Page 5

iii contents introduction iv What is an AI agent? 2 Chapter 1 from Build an AI Agent (From Scratch) by Jungjun Hur and Younghee Song Building tool-based agents with LangGraph 19 Chapter 11 from AI Agents and Applications by Roberto Infante Errand: Breaking complex workflows into units 46 Chapter 3 from Agent Design Patterns by Peter Belcak Building robust agents with evaluation and feedback 63 Chapter 7 from AI Agents in Action, Second Edition by Micheal Lanham

Page 6

iv introduction How do you transform an LLM into an agent? Add tools and a loop that lets the model choose actions and continue working toward a goal—even when the path cannot be mapped out in advance. This sampler, featuring four chapters from Manning books, explores when agents make sense, how they act, how complex behavior can be organized, and how their results can be evaluated and improved. “What is an AI agent?,” from Build an AI Agent (From Scratch) by Jungjun Hur and Younghee Song, starts with explaining the three elements behind agent auton- omy. Then, you’ll compare agents with developer-defined workflows, consider when to use each, and examine the GAIA benchmark and context-engineering strategies. This solid foundation will help you recognize when a task calls for agent-directed decisions—and when a predictable workflow is the better choice. Next, “Building tool-based agents with LangGraph,” from AI Agents and Applications by Roberto Infante, puts those ideas to work in a travel informa- tion agent. You’ll register and execute tools, maintain conversation state, add conditional routing, inspect messages and tool calls, and compare a custom graph with LangGraph’s prebuilt ReAct agent. By following the agent’s execu- tion, you’ll gain a clearer understanding of how tools, state, and control flow fit together—knowledge you can apply when building and debugging your own LangGraph agents. As workflows become more complex, “Errand: Breaking complex workflows into units,” from Agent Design Patterns by Peter Belcak, offers a way to divide them into focused, testable routines. You’ll explore how errands support

Page 7

vintroduction maintenance, reuse, and parallel execution while limiting what enters the wider context. The chapter’s persona, reflection, compaction, structured-extraction, and editing errands will give you practical patterns for making complex agent behavior easier to test, manage, and extend. Finally, “Building robust agents with evaluation and feedback,” from AI Agents in Action, Second Edition by Micheal Lanham, focuses on finding weak- nesses and improving behavior. Using a RAG knowledge agent, you’ll work with test-driven development, benchmarks, critic and grounding agents, evaluation rubrics, and guardrails. You’ll also see how Arize Phoenix supports tracing, experiments, evaluators, session tracking, and feedback annotations—giving you concrete methods for inspecting agent behavior, identifying problems, and measuring the effects of changes. Together, these combined chapters are a practical powerhouse, helping you progress from deciding when to use an agent to building, structuring, testing, and refining one. If this sampler sparks your interest and you’re ready to con- tinue exploring the concepts and tools introduced here, we invite you to pur- chase the complete books. Thank you, The Manning Team

Page 8

Start with the pieces that turn an LLM into an agent: tools and a loop for choosing and carrying out actions. In this chapter, you’ll compare agents with predefined workflows, consider when each approach fits, and explore the GAIA benchmark for measuring agent performance and context-engineering strategies for helping agents use information effectively.

Page 9

2 What is an AI agent? This chapter covers ¡ The landscape of AI agents today ¡ LLMs as the decision-making core of agents ¡ Workflows versus agents and when to use each ¡ GAIA benchmark for measuring agent performance ¡ Context engineering for building effective agents You may have heard of agent-building frameworks such as LangGraph, CrewAI, AutoGen, and OpenAI Agents. These frameworks make it easy to build agents quickly, but they also hide what’s happening inside. This book takes a different approach: we’ll build agents from scratch, understanding every component before relying on any framework. Why build from scratch? Agent development is fundamentally about debugging failures. When your agent gives a wrong answer or gets stuck in a loop, you have to understand exactly what went wrong. Did the large language model (LLM) misin- terpret the context? Did a tool return unexpected results? Was crucial information Chapter 1 from Build an AI Agent (From Scratch) by Jungjun Hur and Younghee Song

Page 10

3The age of AI agents missing? Without understanding how agents work internally, diagnosing these prob- lems is difficult regardless of which tools you use. By building each component yourself, you’ll develop the mental model you need to troubleshoot any agent system, whether you built or inherited it. This chapter establishes the foundation for everything that follows. We’ll start by sur- veying the landscape of AI agents, from personal assistants to specialized coding tools, to understand what we’re building toward. Then we’ll examine how LLMs serve as the brain of an agent and what distinguishes a true agent from a simple workflow. We’ll introduce General AI Assistants (GAIA), the benchmark we’ll use throughout this book to measure progress, and explore context engineering, the discipline that determines whether an agent succeeds or fails. By the end, you’ll have a clear mental model of what agents are, when to use them, and the principles that will guide your implementation journey. 1.1 The age of AI agents AI agents are rapidly transforming how we interact with technology. From personal assistants that help with everyday tasks to sophisticated systems that handle complex professional work, agents are becoming integral parts of both consumer products and enterprise solutions. Before diving into how to build them, let’s survey the landscape of AI agents to understand what we’re working toward: ¡ Personal AI agents—These agents are the most familiar form of AI agents today. Services such as ChatGPT, Claude, and Gemini started as conversational chat- bots; users asked them a question, and they provided an answer. But these systems have evolved dramatically. Modern personal agents can analyze uploaded docu- ments, search the web for current information, generate images, write and exe- cute code, and even help with shopping decisions. They serve as general- purpose assistants that adapt to whatever task you bring them, learning your preferences and communication style over time. ¡ Customer-facing agents—These agents operate on behalf of businesses, interacting directly with customers in real time. Going beyond simple FAQ chatbots, they can access company policies, retrieve customer data, process transactions, and make decisions based on business rules. When you interact with a support agent that checks your order status, processes a refund, or helps troubleshoot a product issue, you’re increasingly likely to be communicating with an AI agent. These sys- tems must balance helpfulness with strict adherence to company guidelines and regulatory requirements. ¡ Specialized agents—These agents tackle domain-specific tasks that require deep expertise or extended processing time. Coding agents like Claude Code, Cursor, and Codex CLI can navigate codebases, implement features, fix bugs, and refac- tor code, which are tasks that previously required hours of developer time. Deep Research agents can conduct comprehensive investigations across hundreds of sources, synthesizing findings into detailed reports. Because these tasks often

Page 11

4 chapter 1 What is an AI agent? take significant time to complete, specialized agents frequently operate asyn- chronously, notifying users when results are ready. When connected to propri- etary enterprise data, these agents become vertical solutions tailored to specific industries or use cases. Despite their different applications and interfaces, these agents have a common foundation: they’re powered by LLMs. Understanding how LLMs work and how they enable agent capabilities is essential for anyone who wants to build effective agents. In this book, we use AI agent and LLM agent interchangeably because virtually all modern AI agents are powered by LLMs. 1.2 Understanding LLM agents All the agents we explored share something remarkable: they rely on LLMs as their decision- making core. But how does a system designed to predict the next word become capable of completing multistep tasks autonomously? First, let’s unpack what makes LLMs uniquely suited to power agents; then we’ll examine the three essential components that transform an LLM into an agent. This material helps you understand the agent loop that forms the foundation for everything you’ll build in this book. 1.2.1 What is an LLM? An LLM is a language model trained on nearly all publicly available text from the internet. Although modern LLMs have evolved into multimodal models that process images, audio, and video, this book uses the term LLM to refer to all these variants because they share a foundational architecture. The core principle of LLMs is simple: learning to predict the next word by process- ing massive amounts of text data. Given the sentence “Because it’s summer vacation, I didn’t go to ______,” most people would expect the blank to be filled with the word school. Through large-scale training on this simple principle, LLMs develop an under- standing of language structure, context, and meaning. This ability to apply learned pat- terns to new examples is a form of generalization, as shown in figure 1.1. LLMs process text by breaking it into tokens, which are typically words or word fragments. The model predicts the next token based on all preceding tokens. Recently, reasoning models have emerged that analyze tasks or formulate plans before produc- ing results. The reasoning model doesn’t answer immediately; first, it generates thinking tokens starting Few-shot learning Zero-shot learning Translate English to Korean. <examples/> Apple -> 사과 Orange -> 오렌지 </examples> Banana Answer: 바나나 Translate English to Korean. Banana Answer: 바나나 Figure 1.1 Example of a language model’s generalization capability

Page 12

5Understanding LLM agents with <think>. This advance enables LLM agents to tackle complex and unfamiliar prob- lems effectively. 1.2.2 What is an LLM agent? A representative example of an LLM agent is a research agent. When a user asks, “Sum- marize the 2024 Nobel Physics Prize winners’ research,” the agent gathers information from multiple sources, analyzes findings, and produces a comprehensive report. As shown in figure 1.2, the research agent doesn’t simply generate an answer from its training data; it also searches for information about the 2024 Nobel Prize, identifies the winners (Geoffrey Hinton and John J. Hopfield), explores academic papers, and con- sults Wikipedia. Then it synthesizes all this information into a coherent report detailing the winners’ achievements and their effect on AI. What’s worth noticing is who decided each step: which source to check first, whether the search results were sufficient, what to look for next, and when to start writing the report. These decisions weren’t hardcoded; the LLM made them on the fly, based on the context at each moment. User: "Summarize 2024 Nobel Physics winners' research" Research agent [Search] 2024 Nobel Physics [Search] Hinton, Hopfield [Paper] Hinton paper [Wikipedia] Deep learning 1 2 3 4 Synthesis and analysis Final report • Winners and achievements • Impact on AI field core definition: autonomous control flow This example reveals the essence of an agent. An LLM agent is a program that hands part of its control flow to an LLM, letting the model decide what actions to take and when to stop based on its current context and goals. By control flow, we mean the sequence of “what to do next.” In traditional software, this flow is spelled out in the code. if statements, for loops, and the order of function calls Figure 1.2 User requests flow through the research agent, which branches into multiple searches and synthesis.

Page 13

6 chapter 1 What is an AI agent? were all decided by the developer in advance. In an LLM agent, part of this decision- making authority is handed over to the LLM. elements that realize autonomy: llm + tool + loop For autonomous control flow to work in practice, three structural elements come together. The LLM serves as the agent’s brain. It understands the current situation and decides what to do next. Tools extend the agent’s action space into the external world, including web search, code execution, and database access. The loop is the execution structure that lets these decisions happen repeatedly until the goal is achieved. To sum- marize, the LLM makes autonomous decisions possible, tools expand the action space, and the loop unfolds it over time. how the agent loop works The loop is necessary because it’s difficult to know in advance which tools will be needed or how many steps will be required to complete a task. Figure 1.3 shows the research task on the 2024 Nobel Prize winners from figure 1.2, viewed through the lens of the agent loop. User: 2024 Nobel Physics LLM Need tool? Tool Execute. Tool result 1 2 [Search] 2024 Nobel physics [Search] Hinton, Hopfield [Paper] Hinton paper [Wikipedia] Deep learning ... More searches possible 3 Enough information? -> Decide to stop. Final report 4 Add information to LLM context. Decide to use tool. Figure 1.3 The LLM agent’s decision loop is an iterative process of LLM decision-making and tool use. The process follows a continuous loop: 1 The LLM evaluates the current context and determines whether a tool is needed. If more information is required, it decides which tool to use based on what’s missing. 2 The selected tool is executed, whether it’s searching for “2024 Nobel Physics,” looking up “Hinton, Hopfield,” finding academic papers, or accessing Wikipedia for background knowledge. 3 The tool’s results are added back to the LLM’s context, enriching its under- standing of the topic. This accumulated context allows the LLM to make more- informed decisions in subsequent iterations.

Page 14

7Workflow vs. agent 4 The LLM decides whether to continue or stop. If it determines that sufficient information has been gathered to answer the user’s question comprehensively, it generates the final report; otherwise, it returns to step 1 for another iteration. This iterative process of reasoning, acting, and observing enables the agent to handle tasks of unpredictable complexity, from simple queries that require a single search to complex research that requires dozens of information sources. 1.3 Workflow vs. agent When integrating LLMs into applications, developers face a fundamental architectural choice: how much control of the execution flow the LLM should have. This decision shapes everything from system reliability to operational costs. Figure 1.4 illustrates seven distinct levels of agency, progressing from left to right with increasing autonomy. The three rows in the figure represent different aspects of control: who produces the current output (current step), who decides what happens next (next step), and who defines the available options (available steps). As we move rightward, the LLM gains more control of each dimension. Current step output Next step Available steps More agentic Code LLM call Chain Router Tool call Multistep Tool creation ModelDeveloper DeveloperDeveloper Developer DeveloperDeveloper Developer Model Model Model Model Model Model Developer DeveloperDeveloper Model Model Model Model Figure 1.4 Progression of agency levels in LLM applications These seven levels can be broadly categorized into two architectural approaches: work- flows and agents. In workflows, developers predefine the execution flow and use LLMs to perform specific steps within that structure. In agents, LLMs determine their own processes dynamically, deciding which actions to take and when to stop. Understand- ing this distinction is essential in choosing the right approach for your use case.

Page 15

8 chapter 1 What is an AI agent? 1.3.1 Workflow: Developer-defined flow A workflow is a system in which developers explicitly design the sequence of operations, with LLMs executing specific steps within that predefined structure. The key charac- teristic is predictability: given the same input, the system follows the same path through the workflow. llm call The most basic pattern is a single LLM call in which one prompt goes in and one response comes out. This pattern delegates a specific task to the model while keeping everything else under developer control. Examples include text classification, summa- rization, and answering a straightforward question. Despite its simplicity, a well-crafted single call can handle surprisingly complex tasks. chain A chain connects multiple LLM calls in a predefined sequence, with the output of one step becoming the input for the next. The developer designs this flow in advance, and the LLM executes each step sequentially. This pattern follows a fundamental principle: LLMs perform best with focused, well-defined tasks. Rather than ask a model to “analyze this document and create a pre- sentation,” a chain might break this request into discrete steps: extract key points, orga- nize by theme, generate slide content, and refine for clarity. Each step is simpler and more likely to succeed than an attempt to do everything at the same time. router A router introduces conditional logic that the LLM uses to decide which predefined path to take next. Given a user query, the model might classify it and route it to the appropriate handler. Billing questions go to one workflow; questions about technical support go to another. Although the LLM makes the decision, it’s choosing among a fixed set of options that the developer explicitly defined. The available paths and what happens along each path remain under developer control, making routers a workflow pattern rather than an agent pattern. The LLM influences the route but doesn’t control the journey. 1.3.2 Agent: LLM-directed flow An agent is a system in which the LLM dynamically directs its own processes and tool use, maintaining control of how it accomplishes tasks. Rather than follow a predeter- mined path, the agent decides what to do next based on the current context and its progress toward the goal. tool use with a multistep loop The defining pattern of an agent combines two capabilities: the ability to use exter- nal tools and the autonomy to continue operating until the task is complete. Because LLMs can only generate text, they can’t interact directly with external systems, access real-time information, or perform precise calculations. Tools bridge this gap by expos- ing external functionalities (such as web search, code execution, database queries, and

Page 16

9Workflow vs. agent API calls) as callable functions. The LLM examines the available tools and decides whether one would help; if so, it specifies which tool to call with what parameters. What transforms tool use into an agent is the loop. Rather than making a single tool call and stopping, the agent operates in a cycle: assess the current state, decide on an action, observe the result, and repeat until the task is complete. The LLM itself deter- mines when to continue and when to stop. This multistep loop is the core mechanism that enables agents to handle tasks of unpredictable complexity. A simple query might resolve in one step; a complex research task might require dozens of iterations across multiple tools. tool creation At the highest level of autonomy, the agent doesn’t just choose among available tools; it creates new ones. This typically involves generating code to implement functionality that doesn’t exist in the predefined toolkit. If, for example, an agent has to process data in a specific format that no existing tool handles, it might write a custom parsing function, execute it, and use the results to continue its task. This capability allows agents to extend their own abilities dynamically, adapting to requirements that weren’t anticipated when the system was designed. 1.3.3 Combining workflows and agents in practice The distinction between workflows and agents isn’t always binary in production sys- tems. In practice, the most effective architectures often combine both approaches, using workflows to provide structure and predictability, and embedding agents at spe- cific points where flexibility is needed. agents as nodes in a workflow A common pattern is to design an overall workflow with well-defined stages but implement one or more of those stages as an agent. Consider a document-processing pipeline: 1 Document intake (workflow)—Validate format and extract metadata. 2 Content analysis (agent)—Research context, gather related information, and syn- thesize findings. 3 Quality review (workflow)—Check against compliance rules and format output. 4 Delivery (workflow)—Route to the appropriate destination. The agent operates freely within its designated stage, making multiple tool calls, fol- lowing chains of inquiry, and adapting to what it discovers. But the overall process maintains the predictability of a workflow. If the agent fails or produces unexpected results, the workflow can catch them at the quality-review stage. why combine? This hybrid approach offers several advantages: ¡ It contains complexity. Agent behavior is inherently less predictable, so limiting where agents operate makes the overall system easier to reason about and debug.

Page 17

10 chapter 1 What is an AI agent? ¡ It optimizes costs. Agent loops can be expensive due to multiple LLM calls, so using them only where their flexibility is genuinely needed keeps operational costs manageable. ¡ It enables graceful degradation. If an agent component fails, the workflow structure allows for fallback behaviors or human escalation at defined points. The architectural choice isn’t workflow versus agent but where in this system agent behavior provides enough value to justify its costs. This pragmatic framing will serve you well as you design your own LLM-powered applications. 1.4 Tasks that require agents Before jumping into agent development, we have to ask the right questions in the right order. Two key decisions shape whether an agent is the appropriate solution: ¡ Does this task require an LLM? ¡ If an LLM is needed, does it require an agent, or will a workflow suffice? Answering no at either stage means choosing a simpler, more cost-effective approach. Let’s examine the criteria for each decision and then look at a benchmark that cap- tures tasks at which agents truly excel. 1.4.1 Tasks that require an LLM Before considering the use of an LLM agent, you must determine whether the task itself requires an LLM. This foundational question matters because LLMs introduce significant overhead in terms of computational cost and potential for errors. This overhead may be unnecessary if the task can be solved with simpler, more determin- istic approaches. If your task involves basic data processing with predictable inputs and outputs, for example, traditional programming logic will be faster, cheaper, and more reliable than any LLM-based solution. You use two main criteria to make this decision: ¡ Tasks involving unstructured data—If a task requires analyzing unstructured data such as text, images, or audio, it’s a strong candidate for LLM use. Traditional programming excels at analyzing structured data with clear schemas but strug- gles with the ambiguity inherent in natural language or visual content. Although models that handle diverse data formats are generally called large multimodal models (LMMs), they’re fundamentally based on LLMs, so this book collectively refers to them as LLMs. ¡ Input diversity—If user input is limited or the requested tasks are narrowly scoped, using a specialized machine language (ML) model or rule-based system might be more cost effective. Similarly, if the number of potential tasks is small and predefined, it’s more efficient in terms of both cost and latency for a developer to hardcode the logic than to use an LLM. LLMs shine when inputs are unpredict- able and varied, requiring flexible interpretation that can’t be anticipated.

Page 18

11Tasks that require agents 1.4.2 Conditions for using agents When you’ve determined that an LLM is necessary, the next question is how to use it. Will a single LLM call suffice, or does the task require the iterative reasoning and tool use that define an agent? Consider the following question: If Eliud Kipchoge could maintain his marathon world record pace indefinitely, how long would it take him to run from Earth to the Moon? Answering this question requires multiple steps. First, you have to search to find out who Kipchoge is and what his marathon world record is. Next, you check Wikipedia for the distance between Earth and the Moon. Finally, you perform calculations to divide the distance by the speed and derive the final answer. Anyone can perform this task manually. But navigating multiple websites to gather information, convert units, and perform calculations is time consuming and error prone. This kind of multistep research task is exactly the kind at which agents shine. As excitement about agents grows, people fall into a common trap: applying agents to every task simply because they’re new and powerful. But you don’t need a complex web framework to build a simple static website. Choosing the right tool is key to build- ing an efficient system. Agents come with inherent tradeoffs: ¡ Costs are higher. Multiple LLM calls multiply API expenses compared with single requests. ¡ Latency increases. Response time accumulates with each reasoning step. ¡ Errors propagate. Mistakes in early steps cascade through the entire process. Consider a customer service query that could be handled by a single LLM call that costs 1 cent. Processing the same query with an agent that requires 10 calls would cost 10 cents—a 10× difference. When you’re handling thousands of requests daily, this decision isn’t just technical but also business. When should you use an agent? You can decide based on these three criteria: ¡ Task complexity—If it’s difficult to predict how many steps a task will require, agents have the advantage. “Find the population of region A” is straightforward. By contrast, “Analyze how LLM agents will shape the future” requires gathering diverse information and synthesizing complex relationships. This task is hard to predefine with static logic. ¡ Task value—Because agents require multiple calls and dynamic decision-making, they cost more and respond more slowly. Therefore, the value of completing the task should outweigh the additional cost and latency. ¡ Error cost and detectability—LLMs are prone to make incorrect decisions, and the risk increases with more calls. If errors lead to critical consequences, an agent may not be the best choice. Detectability also matters. In domains that require specialized knowledge, users or developers may not even realize that an agent has produced faulty results.

Page 19

12 chapter 1 What is an AI agent? 1.4.3 GAIA: An agent gym The Kipchoge question we examined earlier is from a benchmark called GAIA, a data- set that systematically collects tasks requiring agents. GAIA consists of question-and-an- swer pairs. Each question requires multistep reasoning, web searches, calculations, and more. The questions are difficult to solve with a single LLM call, and it’s not easy to pre- define a clear workflow, either. These problems naturally require an agentic approach. Meta and Hugging Face released GAIA in 2023. Since then, model capabilities have improved significantly, with more challenging benchmarks emerging. Still, GAIA’s prob- lems range from straightforward to genuinely difficult even for current models, making it well suited to learn agent development. This book uses GAIA for three reasons: ¡ Clear answers enable fast feedback. The core of agent development is identifying and reducing failures. When answers are clear, you can verify immediately whether you’re heading in the right direction. Then you can iterate rapidly through cycles of experimentation and improvement without using ambiguous evalua- tion criteria. ¡ It’s ideal for practicing the agent development cycle. The process of building an agent goes like this: ¡ Determine whether an agent would help solve a specific problem. If there’s potential, develop a prototype. ¡ Observe when the agent fails, analyze the cause, and improve it. ¡ Repeat this observe-analyze-improve cycle. GAIA provides an appropriate level of difficulty for experiencing this cycle from start to finish. ¡ No domain knowledge is required. Most GAIA problems center on web search and information synthesis. We all perform tasks in daily life that involve searching, gathering information, and organizing it. You can intuitively understand the prob- lems and the solution processes without specialized knowledge in medicine or law. Throughout this book, we’ll use GAIA to track changes in agent performance as we add new techniques. You’ll see firsthand how each component, such as tool use, mem- ory, and planning, improves the agent’s problem-solving capabilities. GAIA gives us a way to measure agent performance. But measurement alone doesn’t improve an agent. What makes an agent better at solving these problems? The answer lies in how we engineer the context that the LLM receives. 1.5 Context engineering Before diving into agent development, we’ll clarify two terms that are often used inter- changeably. A prompt is the input text that a user sends to an LLM. It can be further divided into system prompts, which specify how the model should respond and behave (such as “You are a friendly travel guide”) and user prompts (or messages), which con- tain the user’s actual requests. Context is a broader concept that encompasses all the information an LLM refer- ences when generating a response. This information includes the system prompt, user

Page 20

13Context engineering messages, conversation history, tool execution results, and retrieved documents. In this sense, prompts are one component of context. Think of context as the LLM’s working memory. Just as you can’t solve a math prob- lem if someone gives you only half the equation, an LLM can’t complete a task if crucial information is missing from its context. An LLM predicts the next token based solely on the information within its context window. The model generates text by using patterns learned during training to employ that information. This simple fact is why context engineering is so critical in agent development. 1.5.1 Why agents fail Agent failures can be attributed to two main causes: ¡ Insufficient model intelligence—This includes cases in which the model fails to solve complex mathematical problems or makes errors in logical reasoning. Address- ing this issue requires using a more powerful model or waiting for advances in model capabilities. ¡ Lack of necessary information—Even if a model is intelligent enough, it can’t pro- duce correct answers if the information needed to complete the task isn’t in the context. If someone asks about the company’s vacation policy and the company policy document isn’t in the context, even the most capable model can’t provide an accurate answer. Interestingly, a significant portion of real-world agent failures stems from information deficiency. An LLM can be only as smart as the context it’s given. This is why what infor- mation to include in the context is a key factor in determining agent performance. 1.5.2 From prompt engineering to context engineering In the early days of LLM adoption, prompt engineering was the primary focus. Practi- tioners concentrated on how to write system prompts that would make models respond more accurately and on which instructions would yield outputs in the desired format. Techniques like “Let’s think step by step” and “You are an expert” emerged during this period. But the landscape changed as agents began using tools. When an agent performs a web search, the results are added to the context. When the agent executes code, the execution results are added. When the agent queries a database, the query results are added. Context began to change dynamically, evolving beyond static system prompts. This shift raised a new question: “How do we maintain only the essential information needed for task completion in the context?” The approach that answers this question is context engineering (figure 1.5). Context engineering is the discipline of providing the information an LLM needs to per- form its tasks at the right time and in the right form. It goes beyond simply writing good prompts to comprehensively designing how tool-execution results should be processed, how much conversation history to retain, and when to retrieve external knowledge.

Tip the Site

Agent Engineering Tools, Workflows, and Feedback (Jungjun Hur, Younghee Son, Roberto Infante etc.)（Z-Library）

Agent Engineering Tools, Workflows, and Feedback (Jungjun Hur, Younghee Son, Roberto Infante etc.)（Z-Library）

Registered users can read the full content for free

AI Reading Assistant

Passage locations

Recommended for You

Tip the Site