Spring AI in Action (Craig Walls) (Z-Library)

📄 File Format: PDF

💾 File Size: 14.9 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

M A N N I N G Craig Walls Foreword by Rod Johnson

📄 Page 2

2 EPILOGUE Implementing common Generative AI patterns with Spring AI Essential Generative AI — chapter 3 Retrieval-augmented generation (RAG) — chapter 4 Chat memory — chapter 5 Prompt System message User message LLM Generation response Generation Prompt ChatClient Prompt System message User message LLM Generation response Prompt w/history Generation Prompt and generationChat history ChatClient Prompt System message User message LLM Generation response QuerySimilar docs Augmented prompt Generation Vector store ChatClient Chat memory Licensed to Damian Allen <manning@pixerati.com>

📄 Page 3

Spring AI in ActionLicensed to Damian Allen <manning@pixerati.com>

📄 Page 4

iiLicensed to Damian Allen <manning@pixerati.com>

📄 Page 5

Spring AI in Action CRAIG WALLS FOREWORD BY ROD JOHNSON M A N N I N G SHELTER ISLANDLicensed to Damian Allen <manning@pixerati.com>

📄 Page 6

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2026 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. The author and publisher have made every effort to ensure that the information in this book was correct at press time. The author and publisher do not assume and hereby disclaim any liability to any party for any loss, damage, or disruption caused by errors or omissions, whether such errors or omissions result from negligence, accident, or any other cause, or from any usage of the information herein. Manning Publications Co. Development editor: Doug Rudder 20 Baldwin Road Technical editor: Kenneth Kousen PO Box 761 Review editor: Kishor Rit Shelter Island, NY 11964 Production editor: Keri Hales Copy editor: Alisa Larson Proofreader: Jason Everett Typesetter and cover designer: Marija TudorISBN 9781633436114 Printed in the United States of America Licensed to Damian Allen <manning@pixerati.com>

📄 Page 7

brief contents 1 ■ Getting started with Spring AI 1 2 ■ Evaluating generated responses 20 3 ■ Submitting prompts for generation 28 4 ■ Talking with your documents 59 5 ■ Enabling conversational memory 93 6 ■ Activating tool-driven generation 113 7 ■ Applying Model Context Protocol 133 8 ■ Generating with voice and pictures 165 9 ■ Observing AI operations 192 10 ■ Safeguarding generative AI 221 11 ■ Applying generative AI patterns 243 12 ■ Employing agents 257v Licensed to Damian Allen <manning@pixerati.com>

📄 Page 8

contents foreword x preface xii acknowledgments xiii about this book xv about the author xix about the cover illustration xx 1 Getting started with Spring AI 1 1.1 Hello, Spring AI! 2 Initializing the project 3 ■ Submitting prompts 5 Writing a test 9 ■ Trying it out 12 1.2 Choosing a model 13 Configuring OpenAI models 15 ■ Serving models locally with Ollama 16 1.3 Previewing Spring AI’s capabilities 18 2 Evaluating generated responses 20 2.1 Ensuring relevant answers 21 2.2 Testing for factual accuracy 23 2.3 Applying self-evaluation at runtime 25vi 3 Submitting prompts for generation 28 3.1 Working with prompt templates 29 Licensed to Damian Allen <manning@pixerati.com>

📄 Page 9

CONTENTS vii Defining a prompt template 32 ■ Importing the template as a resource 34 3.2 Stuffing the prompt with context 36 3.3 Assigning prompt roles 40 3.4 Influencing response generation 43 Specifying chat options 44 ■ Formatting response output 48 Streaming the response 52 3.5 Working with response metadata 55 4 Talking with your documents 59 4.1 Understanding RAG 60 4.2 Setting up a vector store 62 4.3 Loading documents 64 Initializing the loader project 65 ■ Defining the loader pipeline 69 ■ Creating the pipeline components 70 Running the pipeline 76 4.4 Implementing RAG 78 Searching for similar documents 78 ■ Updating the service 81 4.5 Implementing RAG with an advisor 83 4.6 Applying modular RAG 86 Rewriting the user’s query 88 ■ Translating user queries 89 ■ Expanding user queries 90 5 Enabling conversational memory 93 5.1 Making memories in AI 94 5.2 Adding conversational memory 95 Enabling an in-memory chat advisor 96 ■ Inspecting the prompt for chat memory 97 ■ Configuring chat memory size 101 5.3 Specifying the conversation ID 101 5.4 Enabling persistent chat memory 104 Persisting chat memory to a database 105 ■ Storing chat memory in a vector store 111 Activating tool-driven generation 1136 6.1 Getting started with AI tools 114 Licensed to Damian Allen <manning@pixerati.com>

📄 Page 10

CONTENTSviii Developing a tools-enabled application 115 ■ Digging deeper 118 6.2 Implementing tools 123 Writing the tool’s foundations 124 ■ Defining the tool 126 Putting the tool to work 128 6.3 Enables functions as tools 129 7 Applying Model Context Protocol 133 7.1 Introducing Model Context Protocol 134 7.2 Working with MCP Clients 136 7.3 Creating your own MCP Server 141 Building the server 141 ■ Setting up the database 142 Creating the MCP Server tools 145 ■ Inspecting the MCP Server 147 ■ Using the server in a client application 149 7.4 Working with the HTTP+SSE transport 151 Configuring an HTTP+SSE in the MCP Server 152 Inspecting the MCP Server 153 ■ Configuring the client to use an HTTP+SSE server 154 7.5 Exposing prompts and resources 156 Declaring prompt and resource-exposing beans 156 Applying annotation-driven prompts and resources 160 8 Generating with voice and pictures 165 8.1 Working with voice 166 Transcribing speech 166 ■ Generating speech from text 170 Applying audio input and output directly 174 8.2 Asking questions about images 177 8.3 Generating images 181 Specifying image options 187 9 Observing AI operations 192 9.1 Enabling Actuator metrics 193 Inspecting vector store operations 195 ■ Examining AI model interaction 197 ■ Counting token usage 199 ■ Observing ChatClient operations 202 9.2 Viewing metrics in Prometheus 205 9.3 Creating AI dashboards 210 9.4 Tracing AI operations 214 Licensed to Damian Allen <manning@pixerati.com>

📄 Page 11

CONTENTS ix 10 Safeguarding generative AI 221 10.1 Controlling document access with RAG 222 Designating premium content 222 ■ Adding security to Board Game Buddy 223 ■ Filtering for premium content 226 Applying per-user conversational memory 227 ■ Trying it out 228 10.2 Securing tools 229 10.3 Safeguarding against adversarial prompting 232 Preventing prompts with sensitive terms 233 ■ Preventing prompt leaks 234 10.4 Moderating user input 237 11 Applying generative AI patterns 243 11.1 Summarizing content 244 11.2 Translating messages 247 Building a simple translator 248 ■ Translating game rule answers 251 11.3 Analyzing sentiment 252 12 Employing agents 257 12.1 Understanding agents 258 12.2 Implementing agentic workflows and patterns 260 Chaining prompts 260 ■ Routing tasks 266 ■ Applying parallelization 272 12.3 Creating self-planning agentic solutions 276 Initializing an Embabel project 277 ■ Defining the agent class 278 ■ Defining an action to get game rules 283 Defining an action to get the rules filename 284 ■ Defining an action to get the game title 285 ■ Running the agent via Embabel’s shell 286 ■ Accessing the agent via MCP 289 index 293 Licensed to Damian Allen <manning@pixerati.com>

📄 Page 12

foreword Generative AI is here to stay, with profound implications for most of us—especially software developers. Despite its proven positive impact on personal productivity, stud- ies consistently show that enterprise Gen AI initiatives usually fail. This book is part of the solution to that problem. Too many people think AI is synonymous with Python. This is a costly misconcep- tion. Python is the language of data science and low-level ML. However, success apply- ing Gen AI in business is about application development, in which Java has long led the way. The JVM has a critically important role to play in reaping the full power of Gen AI. Agents are only as useful as the functionality they can access: Gen AI systems written in Java not only benefit from the maturity of the language ecosystem, but from proximity to valuable domain models and business logic. Most JVM systems are built on Spring. Over the last 22 years, Spring has brought its core principles and engineering rigor to the key challenges faced by Java developers. With the rise of AI it is fitting that Spring AI provides a clear path forward, making it easy and natural to add Gen AI functionality. That path now extends to sophisticated agent workflows with Embabel. Craig Walls has always played a valuable role in the Spring ecosystem. His Spring in Action was probably the first truly good book about Spring. He has a rare gift for mak- ing things easy to understand without glossing over thorny issues. Spring AI in Action finds him in fine form, clearly explaining the key topics with approachable examples. You’ll learn to build chatbots and RAG systems using vectorx databases; to work with MCP tools; to consume and generate audio and images; to introduce guardrails for safety; and to build observability into your Gen AI develop- Licensed to Damian Allen <manning@pixerati.com>

📄 Page 13

FOREWORD xi ment from the start, rather than as an afterthought. I particularly appreciate the cover- age of testing and evaluation—key topics that are too often overlooked. If you’re a Java developer, this book will show what you can achieve with Gen AI. If you’re not a Java developer, you’ll see how easy it is to build Java applications with Gen AI and how competitive the modern Java ecosystem is. Either way, the next step is yours. —ROD JOHNSON FOUNDER, SPRING FRAMEWORK AND EMBABEL Licensed to Damian Allen <manning@pixerati.com>

📄 Page 14

preface The past few years have been beyond exciting. It’s not an overstatement to say that generative AI influences almost everything we do. As developers, you may be wonder- ing how to integrate generative AI in your software projects. But as a Java developer, you may also be afraid that you’ll have to learn Python or some other language that is commonly associated with AI development. The great news is that Spring AI enables integration with generative AI in the lan- guage you already know (i.e., Java or any JVM language) and is built on Spring, the de facto standard framework for the Java platform. In short, no Python? No problem! Mark Pollack, Spring AI project lead, first introduced me to a very early incarna- tion of Spring AI at SpringOne 2023. At first, I didn’t fully grasp how big a deal this project would become. After all, what kind of application would I be writing that would ask simple questions of an LLM? But once the supporting pieces—retrieval-aug- mented generation (RAG), tools, chat memory, etc.—started falling into place, I quickly saw that Spring AI can unlock some very powerful capabilities in any Spring application. With Spring AI in Action, I was given the wonderful opportunity to learn and write about Spring AI as it was being developed and evolving toward a 1.0 release. Naturally, it is a daunting challenge to write about a technology as it is being developed. But it gave me a chance to see it take shape and even help shape the project myself. By the end of the first chapter, you’ll have the foundation of a Spring AI applica- tion. Then, as the book progresses, you’ll build on that foundation, layering on RAG, chat memory, tool use, Model Context Protocol (MCP), multimodal generation, secu- rity, and even agentic workflow patterns. No matter where you are in your generativexii AI journey, you’ll find that Spring AI in Action will be your guidebook to enable gener- ative AI-based capabilities in your Spring Boot projects. Licensed to Damian Allen <manning@pixerati.com>

📄 Page 15

acknowledgments Someday, books like this may be written by generative AI systems, with little or no human involvement. But for this book, the only generative AI applied is in the exam- ples. Many humans were involved in helping me get this book to you. At Manning, several wonderful people worked diligently to ensure that this book is of the highest quality possible. A great many thanks to Doug Rudder, my development editor, who kept me on track and acted as a sounding board as I weighed options for what would go into the book. And many thanks as well to all the members of the Man- ning production team, who helped guide this book into print. As the book was taking shape, several people reviewed the rough drafts along the way and gave feedback that proved invaluable. Particularly, I would like to thank my friend Ken Kousen for serving as the technical editor. Also, I thank the many peer reviewers, including Allen Firstenberg, Amit Basnak, Anver Bogatov, Astha Puri, Barry Kern, Becky Huett, Charly Chávez Ordóñez, Conor Redmond, Daniel Vaughan, Diego Acuña, Erik Weibust, Fernando Bernardino, Giampiero Granatella, Harpal Singh, Hermann Woock, Manas Talukdar, Marco Seguri, Marcus Geselle, Mario Pavlov, Mark Heckler, Mayank Pant, Mikhail Malev, Mohammad Shahnawaz Akhter, Nathan B. Crocker, Nate Schutta, Pablo Sanchidrián Herrera, Ricken Brice Bazolo-Soukoulati, Rohit Saxena, Saravanan Muniraj, Satish Prahalad Gururajan, Sebastien Tardif, Sharath Chandra Parashara, and Vinicius de Albuquerque Campos. Your suggestions helped make this a better book. The entire Spring Engineering team has, in some way or another, contributed to Spring AI. I especially want to give a shout-out to Mark Pollack and Christian Tzolov for their tireless efforts in bringing Spring AI to fruition.xiii For the past couple of years, I’ve also had the privilege of working on a Spring AI–related project with a great team of engineers, including Adib Saikali, Andy Licensed to Damian Allen <manning@pixerati.com>

📄 Page 16

ACKNOWLEDGMENTSxiv Clement, Candice Quates, Gareth Edwards, Greg Meyer, and Paul Warren. Thank you so much for letting me be a part of this journey with you. I’ve also enjoyed the opportunity to speak about Spring AI on the No Fluff-Just Stuff conference tour, alongside some other great speakers, including Daniel Hino- josa, Ken Kousen, Venkat Subramaniam, Michael Carducci, Brent Laster, Brian Slet- ten, Raju Gandhi, and the ringleader of this traveling circus, Jay Zimmerman. Thank you for the camaraderie and discussions (both technical and not-so-technical in nature). As always, I thank the Phoenicians. Without you, I wouldn’t have been able to write this book (or pretty much anything else). I’d also like to thank the artists and designers of the many board games that I’ve played, some of which make an appearance in the examples of this book. Of particu- lar note, Andrew Heath, the creator of Burger Battle, and Travis and Holly Hancock from Facade Games, thank you for so graciously agreeing to let me refer to your games in my examples. And thank you to Jamey Stegmaier of Stonemaier Games for a great discussion about how generative AI can be assistive in learning and playing board games. Finally, to my beautiful and amazing wife, Raymie. You are my constant reminder that the best stories aren’t the ones written but the ones lived with you. Also, to my two wonderful daughters, Maisy and Madi—I am so very proud of you and love you so much. And to Jackson, the newest member of our crew, I couldn’t imagine a better partner in life for Maisy, and I am so happy that you are part of our family now.Licensed to Damian Allen <manning@pixerati.com>

📄 Page 17

about this book Spring AI in Action was written to introduce you to building incredible applications that wield the power of generative AI through Spring AI. It begins with the most basic thing that you can do: submitting a prompt to a large language model (LLM). From there, it shows how to implement techniques such as retrieval-augmented generation (RAG), tool use, and chat memory to augment prompts with context that unlock better responses. Then, you’ll learn how to use Spring AI to add voice and image-based generation to your apps, enable observability and metrics, and work with generative AI securely. Finally, you’ll get a taste of what it takes to build generative AI agents with Spring AI. While Spring AI’s reference documentation is top-notch, this book goes further, providing a hands-on tour through the Spring AI ecosystem, including building a complete generative AI-based application. Who should read this book Spring AI in Action is for Java developers familiar with Spring and Spring Boot who are interested in applying generative AI in their existing Java applications without resort- ing to Python or other languages more commonly associated with AI. How this book is organized: A road map The book is made up of 12 chapters:  Chapter 1 introduces Spring AI, showing how to initialize a new Spring project. In this chapter, you’ll take the first steps toward building a complete Spring AI-xv enabled application that you’ll grow throughout the rest of the book. Licensed to Damian Allen <manning@pixerati.com>

📄 Page 18

ABOUT THIS BOOKxvi  Chapter 2 covers the essentials of testing for quality results from an LLM using Spring AI’s evaluators. Whether applied at test time or at run time, evaluators can ensure that the answers you get are accurate and fact-based.  Chapter 3 delves into the various ways to submit prompts to Spring AI, includ- ing formatting response output, streaming responses, and working with prompt templates. You’ll also see how to augment prompts with additional context, which will set you up for what’s to come in chapter 4.  Chapter 4 shows how to employ retrieval-augmented generation (RAG) using Spring AI advisors. With RAG, you’ll be able to ask questions and get factual answers from sources that the LLM was never trained on, including your own documents.  LLMs have notoriously short memories. Therefore, chapter 5 examines how Spring AI can enable your application to remember conversational context, reminding the LLM of what has been said previously in a conversation and making up for the LLM’s inability to carry on a conversation.  In chapter 6, you’ll learn how to enable an LLM to use tools, making it possible for them to obtain information from APIs and databases. With tools, LLMs can do more than simply provide answers; they’ll be able to take action and per- form tasks as a consequence of a prompt.  Chapter 7 builds on tool use from chapter 6 by showing how to collect tools, resources, and prompts in a Model Context Protocol (MCP) server. You’ll also see how to use MCP servers by building an MCP client in your application.  Chapter 8 shows how Spring AI can add sight and sound to your application. You’ll learn how to employ the transcription capabilities of some LLMs to tran- scribe audio to text and to turn text into audio files with speech. You’ll also see how to ask questions about images and also produce images based on a user prompt.  Observability is an important aspect of any software project, and generative AI is no exception. In chapter 9, you’ll learn how to use Spring Boot’s Actuator and Micrometer to observe metrics and trace the flow of prompts flowing through Spring AI.  Chapter 10 shows how to secure your generative AI interactions, using Spring Security along with Spring AI to ensure that tools aren’t invoked unless the user has permission and that documents aren’t considered for RAG-style context augmentation unless the user is allowed to read those documents.  Chapter 11 explores how to implement a handful of common non-chat use cases for AI using Spring AI, including text summarization, translation, and sen- timent analysis.  Chapter 12 wraps things up with a look toward how you may build agentic AI workflows with Spring AI. The chapter begins by demonstrating how to imple- ment a few common agentic workflow patterns based on Spring AI. Then the Licensed to Damian Allen <manning@pixerati.com>

📄 Page 19

ABOUT THIS BOOK xvii chapter looks at Embabel, an exciting new framework built on top of Spring AI that employs agentic planning to automatically determine the best plan to achieve a goal. Developers new to Spring AI should start with chapter 1 and work through each chap- ter sequentially. Developers who may have worked with Spring AI before may prefer to flip to the chapters that most interest them. That said, most chapters build on the pre- vious ones, so if you dive into the middle of the book, you may find that you are miss- ing some context. About the code This book contains many examples of source code both in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. Sometimes code is also in bold to highlight code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code. In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts. You can get executable snippets of code from the liveBook (online) version of this book at https://livebook.manning.com/book/spring-ai-in-action. The complete code for the examples in the book is available for download from the Manning website at https://www.manning.com/books/spring-ai-in-action, and from GitHub at https:// github.com/habuma/spring-ai-in-action-samples. liveBook discussion forum Purchase of Spring AI in Action includes free access to liveBook, Manning’s online reading platform. Using liveBook’s exclusive discussion features, you can attach com- ments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the author and other users. To access the forum, go to https://livebook.manning .com/book/spring-ai-in-action/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/discussion. Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We sug- gest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print. Licensed to Damian Allen <manning@pixerati.com>

📄 Page 20

ABOUT THIS BOOKxviii Need additional help?  The Spring AI website is a great place to get general information about the Spring AI project, including Spring AI’s reference documentation at https://spring.io/projects/spring-ai.  Spring AI is an open-source project. You can find the source code in the GitHub repository at https://github.com/spring-projects/spring-ai.  The Spring AI team has curated a collection of examples in the Spring AI Examples GitHub repository at https://github.com/spring-projects/ spring-ai-examples.  The author has also made several Spring AI examples available at https://github.com/habuma/spring-ai-examples. Licensed to Damian Allen <manning@pixerati.com>