Harness Engineering for AI Agents (Nicole Koenigstein) (z-library.sk, 1lib.sk, z-lib.sk)

# Harness Engineering for AI Agents — Reading Guide ## 【One-Line Pitch】 A practical methodology for engineers who want to move beyond prompt tweaking and build the operational infrastructure—context, tools, state, verification, and feedback loops—that turns a raw LLM into a reliable, production-grade agent system. Read this if you already know how to build agents and now need to make them dependable, inspectable, and adaptable without swapping out the model. ## 【Book Arc】 - **Opening (~0%–6%)**: Establishes the core thesis—the model is not the system. Introduces the harness as the engineered layer around AI agents that governs context, tool interaction, state, orchestration, verification, and constraints. Sets up the book's structure across 12 chapters covering identity, action surfaces, coordination, observability, optimization, and governance. - **Early (~6%–24%)**: Walks through the evolution from prompt engineering to context engineering to harness engineering, using a concrete incident-response scenario. Shows how each layer widens the surface you control, and why prompt and context engineering alone leave critical gaps in enforcement and verification. - **Early–Middle (~24%–42%)**: Dives into the mechanics of harness engineering with code examples. Demonstrates the separation between tools the model can see (TOOLS) and tools the harness can actually execute (TOOL_REGISTRY), plus the execution loop that blocks unauthorized actions and feeds refusals back as observations. - **Middle (~42%–52%)**: Covers verification and recovery. Shows how the harness validates model outputs against schemas, checks that evidence cites real observations from the run, and sends failures back to the model as new turns—enforcing boundaries outside the model rather than relying on it to remember instructions. - **Late (~52%+ per TOC)**: The remaining chapters (not covered in available excerpts) extend into agent identity, multi-agent coordination, observability and telemetry, optimization patterns, reinforcement learning over skills and signals, skill internalization, and production governance. ## 【Key Takeaways】 - **The harness, not the model, is the binding constraint on agent reliability** (Early): Even exceptional models depend on the surrounding system—what context they receive, which tools they may call, what state survives, and how outputs are verified. Engineering this layer delivers reliability gains comparable to a model upgrade without touching weights. - **Prompt engineering and context engineering share the same ceiling** (Early): Both only control what goes into a model call. They say nothing about what actually runs, what happens on failure, whether results are good, or what the system should do differently next time. - **Harness engineering moves control outside the model** (Early–Middle): Instead of asking the model to respect policies, the harness enforces them. A tool like `rollback_service` can be exposed to the model without an executor, so the model can propose it but the harness blocks execution and tells the model to recommend it instead. - **Separate what the model can see from what the harness can run** (Middle): The `TOOLS` collection (model-visible schemas) and `TOOL_REGISTRY` (executable implementations) are distinct. Every tool call is checked against the registry before execution—a name that isn't there gets a refusal result fed back into the conversation like any other observation. - **Verification turns final answers into proposals, not conclusions** (Middle): The harness parses model output, validates it against a schema, and checks that every evidence item names a source that actually produced an observation during the run. Failures are sent back to the model as new turns, creating a recovery loop. - **Tool errors are observations, not exceptions** (Middle): When the model supplies wrong arguments, the tool returns an error as its result rather than raising an exception. The model reads the error and corrects itself on the next step—the proposal never becomes an action, and the run continues. - **The same model can produce dramatically different outcomes across engineering layers** (Middle): The incident example runs identically across prompt, context, and harness engineering with nothing about the model changed—only the surrounding system. This demonstrates that harness engineering is a learnable, optimizable layer. ## 【Reading Tips】 - **Deep-read Chapter 1's incident example end-to-end** (~24%–52%): The progression from prompt → context → harness engineering on the same task is the book's clearest teaching device. Follow the code examples in order to see exactly what each layer adds. - **Skim the early framing if you're already convinced** (~6%–15%): The opening chapters establish why the model isn't the system. If you've already hit reliability walls with agents, you can move quickly to the code-heavy sections. - **Pay close attention to the TOOLS vs. TOOL_REGISTRY distinction** (~39%–42%): This separation is the heart of capability scoping. Understanding why the model can see tools it cannot execute is essential for the later chapters on action surfaces and governance. - **Study the verification and recovery loop carefully** (~42%–52%): The pattern of parsing output, validating against schema, checking evidence provenance, and feeding failures back as new turns is directly reusable in your own agent systems. - **Note that most chapters are unavailable in this early release**: The available material covers Chapter 1 thoroughly. Chapters 2–12 are listed in the TOC but not included in these excerpts, so treat this guide as covering the foundational chapter only. ## 【Coverage Limits】 This guide covers only Chapter 1 ("Why Harness Engineering Matters") from the early release. The excerpts do not include material from Chapters 2–12 on stateful substrates, skills, agent identity, multi-agent coordination, observability, optimization patterns, reinforcement learning, or production governance—though the TOC confirms these topics are planned. ##