Spring AI for Your Organization GCP Vertex AI Edition (Muthukumaran Navaneethakrishnan) (z-library.sk, 1lib.sk, z-lib.sk)

Spring AI for Your Organization GCP Vertex AI Edition Muthukumaran Navaneethakrishnan This book is available at https://leanpub.com/springai This version was published on 2025-06-13 This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to get reader feedback, pivot until you have the right book and build traction once you do. © 2025 Muthukumaran Navaneethakrishnan

To Balasundari Navaneethakrishnan.

CONTENTS Contents 1: 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Evolution of AI Frameworks in Java . . . . . . . . . . . . . . . . . . . . 2 1.2 Emergence of Spring AI . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Introduction to Generative AI . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Overview of Spring AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.5 Introduction to Google Cloud’s Vertex AI . . . . . . . . . . . . . . . . 5 1.6 Goals of the Book (What You Will Learn) . . . . . . . . . . . . . . . . 6 1.7 Who Should Read This Book . . . . . . . . . . . . . . . . . . . . . . . . 6 1.8 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.9 What’s Next . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2: Creating ChatBot with Spring AI . . . . . . . . . . . . . . . . . . . . . . . 10 2.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 How to Initialize the project . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Creating a RESTful Chat Endpoint . . . . . . . . . . . . . . . . . . . . . 14 2.4 Connecting to Vertex AI Gemini . . . . . . . . . . . . . . . . . . . . . . 16 2.5 Implementing the Chat Endpoint . . . . . . . . . . . . . . . . . . . . . 16 2.6 How to Test the Application . . . . . . . . . . . . . . . . . . . . . . . . 17 2.7 Configuring Chat Options for Vertex AI . . . . . . . . . . . . . . . . . 18 2.8 How to Test the Application . . . . . . . . . . . . . . . . . . . . . . . . 22 2.9 Understanding API Interaction with Vertex AI . . . . . . . . . . . . . 23 2.10 How to Deploy in GCP Cloud Run . . . . . . . . . . . . . . . . . . . . 26 2.11 Review Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.12 Answers to Review Questions . . . . . . . . . . . . . . . . . . . . . . . 30 2.13 What We Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 © 2024 Muthukumaran Navaneethakrishnan

CONTENTS 3: Building Context-Aware Chatbots with Spring AI: Memory Man- agement and Conversational Continuity . . . . . . . . . . . . . . . . . . 32 3.1 Chapter Highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.2 The Challenge of Contextual Conversation . . . . . . . . . . . . . . . 33 3.3 Understanding LLMs and Contextual Conversations . . . . . . . . . 34 3.4 Implementing Conversation History in Spring AI . . . . . . . . . . . 36 3.5 Managing ChatBot Conversations with Session IDs . . . . . . . . . . 37 3.6 Updating the ChatBot Endpoint for Conversational Context . . . . 40 3.7 Testing the Enhanced ChatBot with Session History . . . . . . . . . 43 3.8 Simplifying Chat History Management with Advisors . . . . . . . . 44 3.9 Creating a DELETE Endpoint to Reset Chat Session History . . . . 48 3.10 Understanding System Prompts in Chatbot Responses . . . . . . . 50 3.11 What We Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 4: Creating Structural Data Bots . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.2 Make Inventory to be queried by Natural Language . . . . . . . . . . 58 4.3 Update Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.4 Setting up an SQL database in GCP Cloud SQL . . . . . . . . . . . . . 60 4.6 Getting Ready for Natural Language Processing . . . . . . . . . . . . 65 4.7 From Natural Queries to SQL . . . . . . . . . . . . . . . . . . . . . . . . 65 4.8 JSON Conversion of SQL Results . . . . . . . . . . . . . . . . . . . . . . 71 4.9 SQL to Natural Language Response . . . . . . . . . . . . . . . . . . . . 73 4.10 Bringing It All Together: The Endpoint . . . . . . . . . . . . . . . . . 75 4.11 Testing the Endpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.12 Exercise: Implement it using CSV instead of JSON . . . . . . . . . 78 4.13 Improving SQL Query Flexibility for User Queries . . . . . . . . . . 80 4.14 Handling multiple questions . . . . . . . . . . . . . . . . . . . . . . . 83 4.15 What We Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5: LLM Tool Calling with Spring AI . . . . . . . . . . . . . . . . . . . . . . . 91 5.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 © 2024 Muthukumaran Navaneethakrishnan

CONTENTS 5.2 Understanding LLM Tool Calls . . . . . . . . . . . . . . . . . . . . . . . 92 5.3 Implementing Tool Calling with REST API . . . . . . . . . . . . . . . . 97 5.4 Java Implementation with Direct API Calls . . . . . . . . . . . . . . . 103 5.5 Handling Multiple Tool Calls . . . . . . . . . . . . . . . . . . . . . . . . 108 5.6 Simplifying with Spring AI . . . . . . . . . . . . . . . . . . . . . . . . . 109 5.7 Advanced Tool Calling: Sequential Tool Routing . . . . . . . . . . . . 116 5.8 FewWords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 5.9 What We Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 6: Building Chatbots with Text and PDF Files . . . . . . . . . . . . . . . . 128 6.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.2 Use Case: Answering Questions About VacuumCleaners based on text file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 6.3 Use Case: Answering Questions About Laptop based on pdf manual135 7: Building Chatbots with Multimedia Capabilities . . . . . . . . . . . . . 141 7.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 7.2 Use Case: Answering Questions About Coupons from an Image . . 142 7.3 Invoking the Chatbot with Image Data . . . . . . . . . . . . . . . . . . 144 7.4 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 7.5 Use Case: Answering Questions based on a Customer Care Audio . 149 7.6 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 7.7 Use Case: Answering Questions from a Vaccum Cleaner Adver- tisement video . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 7.8 Invoking the Chatbot with Video Data . . . . . . . . . . . . . . . . . . 154 7.9 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 8: Using RAG to Make LLMs Smarter with Internal Data . . . . . . . . . 159 8.1 Integrating Internal Data with LLM . . . . . . . . . . . . . . . . . . . . 159 8.2 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 8.3 How to Create Embeddings . . . . . . . . . . . . . . . . . . . . . . . . . 161 8.4 Vector Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 © 2024 Muthukumaran Navaneethakrishnan

8.5 Making Similar Documents as Conversational . . . . . . . . . . . . . 197 8.6 Exercise: Modify Chatbot to Search All Records When Depart- ment Is Not Specified . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 9: Building Internal Knowledge based assistant . . . . . . . . . . . . . . . 204 9.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 9.2 Integrating Google Cloud Storage Bucket . . . . . . . . . . . . . . . . 204 9.3 Building a Chatbot with Internal Documents . . . . . . . . . . . . . . 210 9.4 Adding File References on Chat . . . . . . . . . . . . . . . . . . . . . . 228 10: PDF Documents & Image Embeddings . . . . . . . . . . . . . . . . . . . 235 10.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 10.1 Saving PDF Document Pages as Image Embeddings . . . . . . . . . 235 10.2 Updating Chat Endpoint to use Image embeddings . . . . . . . . . 244 11: Accessing other models in GCP Model Garden . . . . . . . . . . . . . . 251 11.1 Chapter Highlights: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 11.1 Setting Up GCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 11.2 Setting Up Gradle and Configuration . . . . . . . . . . . . . . . . . . 252 11.3 Setting Up Authentication Tokens for Llama Requests . . . . . . . 253 12: Whats Next . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 © 2024 Muthukumaran Navaneethakrishnan

i Acknowledgments Thank you to ShaamaM formaking this new revision possible, Especially the way you helped me with the migration from the old version to the new one. Your support has been invaluable. Thank you toMadhanaGopalaRamachandran andAgalya for their invaluable support in testing code, fixing bugs, and editing. Special thanks to Peter Uballed for encouragingme to complete this project. Without that push, this draft might have remained unfinished for another year. I am also deeply grateful tomymentors, Prabagar Ganapathy and Senthilku- mar Sundararajan, who have always been there when I needed guidance and have helped me take the next step. Lastly, heartfelt thanks to my family and friends for their unwavering sup- port throughout this journey. © 2024 Muthukumaran Navaneethakrishnan

ii Preface I began exploring Spring AI and Vertex AI with the simple goal of gaining a deeper understanding. Along the way, I realized I had valuable insights to share with the Spring community, which led me to write this book. This isn’t an “I’m an expert, look atme” kind of project—it’s a practical, dev-to- dev guide. In the middle of writing, I even found myself contributing to the Spring AI framework. If you’re familiar with Spring and eager to add AI-driven features like chat- bots, natural language data querying, multimedia capabilities, RAG, text embeddings, multimedia embeddings, and using Model Garden—all specif- ically with GCP Vertex AI—this Spring AI handbook walks you through each step. With hands-on examples, we’ll cover everything from setting up your environment to building intelligent, context-aware systems. Each chapter builds on the last, aiming tomake your Spring applications not just smarter, but more intuitive and user-focused. I hope this book becomes your go-to guide for integrating AI into your Spring projects using GCP Vertex AI, helping you build products quicker and more efficiently. Let’s dive in together and explore the exciting possibilities! © 2024 Muthukumaran Navaneethakrishnan

iii Trademark Notice This document and the associated project incorporate technologies devel- oped by various organizations and are used solely for educational and in- structional purposes. Trademark acknowledgments are as follows: • Spring is a trademark of Broadcom Inc. and/or its subsidiaries. • Java and all Java-based trademarks and logos are trademarks or regis- tered trademarks of Oracle Corporation and/or its affiliates. • Google Cloud Platform (GCP) and Vertex AI are trademarks of Google LLC. No affiliation with or endorsement by Google LLC is implied. • PostgreSQL and the Slonik Logo are trademarks or registered trademarks of the PostgreSQL Community Association of Canada. • Any other trademarks mentioned are the property of their respective owners. This project and its documentation are independent of the companies and entities listed and do not imply endorsement. The use of the above trade- marks is strictly for identification purposes and does not signify any asso- ciation with Oracle Corporation, Broadcom Inc., Google LLC, the PostgreSQL Community Association of Canada, or any other mentioned affiliates. © 2024 Muthukumaran Navaneethakrishnan

Chapter 1: 1 Introduction Artificial Intelligence (AI) has evolved significantly, impacting every domain and reshapinghowsoftware is built anddeployed. Java, a language renowned for its stability and enterprise focus, has been a key player in AI’s history, contributing through frameworks like AIML for rule-based chatbots and Deeplearning4j for deep learning. However, with the rise of Python and its dominance in AI development, Java developers faced a gap in leveraging AI’s latest advancements, especially with generative AI models like GPT. This book bridges that gap by introducing Spring AI, a framework that seamlessly integrates AI into the Spring ecosystem. Spring AI simplifies complex tasks such as maintaining conversation history, implementing Retrieval-Augmented Generation (RAG), and using LLM functions, enabling developers to focus on building robust, scalable, and intelligent applications. By leveraging tools like Google Cloud’s Vertex AI, this book offers practical guidance on how Java developers can embrace the capabilities of generative AI without abandoning their existing ecosystem. From building chatbots to creating enterprise-ready AI applications, this book provides a comprehen- sive approach tomastering AI integration with Spring, transforming howwe approach modern software development.

2 1.1 Evolution of AI Frameworks in Java Java has been a key player in the development of artificial intelligence (AI) applications, offering a robust ecosystem of libraries and frameworks. Early AI tools in Java included: • AIML (Artificial Intelligence Markup Language) (1995): AIML’s first widely recognized implementation was in Java. The ALICE project (Ar- tificial Linguistic Internet Computer Entity), which introduced AIML, heavily relied on Java for building its chatbot infrastructure. Java’s platform independence and robust ecosystem made it a popular choice for early AI-related projects, including ALICE. The original interpreter forAIML,ProgramD, was alsowritten in Java andbecame the foundation for many AIML-based chatbots. Java’s simplicity and scalability at the time contributed to the adoption of AIML for creating rule-based conversational agents. • Apache OpenNLP (2010): A library for natural language processing tasks, including entity recognition, tokenization, and text classification. • Stanford CoreNLP (2014): A Java-based framework for natural language processing, offering robust tools for part-of-speech tagging, named entity recognition, dependency parsing, and sentiment analysis. • Deeplearning4j (2014): A suite for buildingdeepneural networks in Java, with support for distributed training and GPU acceleration. 1.1.1 Rise of Python in AI The landscape of AI development shifted with the introduction of PyTorch (2016). PyTorch’s dynamic computation graph, native C bindings, and strong © 2024 Muthukumaran Navaneethakrishnan

3 GPU support made it a favorite for building and training deep learningmod- els. Python’s simplicity and vast ecosystem of AI libraries like TensorFlow, Scikit-learn, and PyTorch propelled it to the forefront of AI development, particularly with the emergence of generative AI. 1.2 Emergence of Spring AI Generative AI and large language models (LLMs) have introduced transfor- mative opportunities for developers. While Python has dominated due to its rich AI libraries, the introduction of generativemodels throughAPIs brought challenges such as: • Managing conversation history. • Implementing Retrieval-Augmented Generation (RAG) workflows. • Leveraging LLM functions for complex tasks. Manually handling these complexities can lead to inefficiencies. Recogniz- ing this gap, the Spring community launched Spring AI. Spring AI extends Spring’s core principles—portability, modularity, and the use of Plain Old Java Objects (POJOs)—to simplify AI integration. Features include: • Built-in support for conversation history and RAG workflows. • Unified API formultiple AI providers likeOpenAI,Google Vertex AI, and Microsoft Azure. • Seamless integrationwith the Spring ecosystem, allowing developers to leverage familiar tools and methodologies. By abstracting the complexities of AI, Spring AI empowers developers to create scalable, enterprise-ready AI applications efficiently. © 2024 Muthukumaran Navaneethakrishnan

4 1.3 Introduction to Generative AI Generative AI focuses on creating new content, such as text, images, or audio, based on patterns learned from training data. Unlike traditional AI models that predict or classify, generative AI learns underlying structures and generates creative outputs. For example: • Traditional AI: Predicts the next word in a sentence or identifies an image as a cat or dog. • Generative AI: Composes entire paragraphs, generates realistic images, or creates music. Generative AI opens up vast possibilities for innovation, such as: • Chatbots producing human-like responses. • Art and music creation. • Summarizing and analyzing complex datasets. This shift in capabilities pushes AI from prediction to creation, transform- ing applications in industries like education, entertainment, and customer service. 1.4 Overview of Spring AI Spring AI is a Java framework that integrates artificial intelligence capabili- ties into applications. Key features include: © 2024 Muthukumaran Navaneethakrishnan

5 • A unified API supporting chat, text-to-image, and embedding models from providers like OpenAI, Google Vertex AI, andMicrosoft Azure. • Seamless compatibility with the Spring ecosystem, enabling modular and scalable application design. • Easy switching between AI components with minimal code changes, promoting flexibility. Spring AI is designed for enterprise-level applications, offering: 1. Portability: Avoid vendor lock-in with support for multiple AI services. 2. Scalability: Grow AI capabilities alongside business needs. 3. Efficiency: Leverage Spring’s existing infrastructure to save develop- ment time. With Spring AI, Java developers can incorporate AI into applications without needing to switch to Python or learn new frameworks. 1.5 Introduction to Google Cloud’s Vertex AI Vertex AI on Google Cloud provides powerful tools to integrate AI into applications, such as: • Chatbots with memory for conversational continuity. • Natural language queries for SQL-like interactions. • Handling diverse inputs like text, PDFs, and images. Vertex AI works seamlessly with other Google Cloud services, including: • Cloud Run: For scalable, serverless deployments. © 2024 Muthukumaran Navaneethakrishnan

6 • Cloud SQL: For database integration. • Cloud Storage: For managing multimedia files. This integrationmakes Vertex AI an ideal platform for deploying intelligent, data-driven applications. 1.6 Goals of the Book (What YouWill Learn) This book is a practical guide for Spring developers to integrate Generative AI into their applications. By the end of this book, you will learn to: • Integrate AI into Java Applications: Discover how Spring AI bridges traditional Java development with AI. • Build AI-Powered Chatbots: Create chatbots capable of understanding and responding naturally. • Handle Complex Data Formats: Process text, PDFs, and multimedia inputs. • Leverage Retrieval-Augmented Generation (RAG): Enhance chatbot intelligence using internal data. • Utilize LLM Functions: Streamline workflows with advanced AI func- tionalities. • Optimize and Deploy AI Applications: Deploy enterprise-ready appli- cations on Google Cloud Platform (GCP) using services like Cloud Run. This combination of conceptual knowledge, practical implementation, and deployment strategies makes this book a comprehensive resource for inte- grating AI with Spring. © 2024 Muthukumaran Navaneethakrishnan

7 1.7 Who Should Read This Book This book caters to: 1. Backend Developers: Whether seasoned or new, this book enhances your skills for building AI-driven applications. 2. Managers Returning to Technical Roles: Gain practical insights into integrating AI into projects. 3. Java Developers Exploring AI: Transition to AI development without switching to Python. 4. Engineers Building Scalable AI Solutions: Access hands-on examples and real-world use cases. Whether you’re diving into AI for the first time or looking to enhance your expertise, this book equips you to navigate the evolving landscape of Generative AI. © 2024 Muthukumaran Navaneethakrishnan

8 1.8 Prerequisites 1.8.1 Technical Requirements To get started, youneed the JavaDevelopmentKit (JDK) version 17 or higher. You can verify the installation with the following command: java -version The preferred build tool for this book is Gradle, which can be checked using: gradle -version You also need the Google Cloud SDK (gcloud CLI) installed. Verify its installation with: gcloud --version For development, an IDE like IntelliJ IDEA or Eclipse is recommended. Lastly, ensure you have an active Google Cloud Platform (GCP) account. You can sign up here. 1.8.2 Knowledge Requirements • Familiarity with Java & Spring Boot. • Understanding of generative AI concepts and cloud platforms. © 2024 Muthukumaran Navaneethakrishnan

9 1.8.3 GCP Setup To set upyourGoogleCloudPlatform (GCP) environment, start by visiting the GCP Console and creating a new project named myspringai. Once the project is created, navigate to the API Library and enable the Vertex AI API. This ensures the project is ready to integrate AI functionalities. Next, configure the gcloud CLI on your system. Begin by authenticating your GCP account using the following command: gcloud auth login This will open a browser for authentication. After successful authentication, set the newly created project as the default for your gcloud CLI: gcloud config set project myspringai Finally, configure the default region for Cloud Run to us-central1: gcloud config set run/region us-central1 This setup ensures that your environment is ready to deploy AI-enabled applications using Vertex AI and other GCP services. 1.9 What’s Next This chapter covered settingup Java, Gradle, and theGoogleCloudSDK, along with creating a GCP project, enabling the Vertex AI API, and configuring the gcloud CLI. With these essentials in place, the next chapter will guide you through building a Spring AI chatbot, starting with project setup and designing a RESTful API for chatbot interactions, while integrating Vertex AI with Spring Boot. © 2024 Muthukumaran Navaneethakrishnan

Chapter 2: Creating ChatBot with Spring AI We begin our journey into building a Spring AI chatbot by establishing the project’s foundation. This section will guide you through the initial setup, design considerations, and integration steps. We’ll utilize Spring Boot to develop an application that interacts with Google Cloud Platform (GCP) and leverages its powerful AI capabilities. Through this process, we’ll construct a chatbot that can engage users in conversation and respond to their inquiries. 2.1 Chapter Highlights: 1. Project Initialization with Spring AI: • We start by initializing our project using Spring Initializr, selecting essential dependencies such as Spring Web, Vertex AI Gemini, and GoogleCloudSupport. This lays the groundwork for building a robust Spring Boot application. 2. Setting Up the Gradle Build File: • Next, we configure the Gradle build file, adding necessary reposito- ries and properties. This step ensures that our project is equipped with the right tools and dependencies needed for seamless integra- tion with GCP and Vertex AI. 3. Designing the ChatBot Endpoint: