Why Distributed SQL Is the Modern Foundation for AI (Steve Suehring) (z-library.sk, 1lib.sk, z-lib.sk)

Yugabyte

Why Distributed SQL Is the Modern Foundation for AI Simplifying and Accelerating the Development of Resilient AI Applications Steve Suehring

Why Distributed SQL Is the Modern Foundation for AI by Steve Suehring Copyright © 2026 O’Reilly Media, Inc. All rights reserved. Published by O’Reilly Media, Inc., 141 Stony Circle, Suite 195, Santa Rosa, CA 95401. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (https://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Nicole Butterfield Development Editor: Gary O’Brien Production Editor: Katherine Tozer Copyeditor: Piper Content Partners Proofreader: O’Reilly Media, Inc. Cover Designer: Susan Brown Cover Illustrator: Susan Brown Interior Designer: David Futato Interior Illustrator: Kate Dullea July 2026: First Edition Revision History for the First Edition 2026-07-14: First Release

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Why Distributed SQL Is the Modern Foundation for AI, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and Yugabyte. See our statement of editorial independence. 979-8-341-67373-1 [LSI]

Chapter 1. The AI Application Revolution Artificial intelligence (AI) is no longer confined to experimentation at the edges of the enterprise. It has moved into core workflows, decision systems, and customer-facing applications. This integration has reshaped expectations about speed, insight, and automation. As AI capabilities accelerate, the demands placed on underlying systems, particularly databases, increase just as rapidly. This chapter explores how the exponential growth of AI is influencing enterprise adoption patterns and exposing structural gaps in traditional architectures. The chapter examines how organizations interpret change acceleration and how AI adoption reshapes expectations around build-to- last architectures. The chapter concludes with a look at why modernization efforts must extend beyond the application layer and into the database and infrastructure as well. Comprehending the Exponential Growth of AI Organizations struggle with AI adoption because they plan for linear improvement while AI capabilities evolve nonlinearly. Traditional systems improve in predictable increments, which supports steady planning, budgeting, and scaling assumptions. AI does not follow this pattern. Exponential growth appears gradual at first but accelerates rapidly as each improvement compounds on the last. Early progress may seem incremental, but once capability reaches a threshold, each subsequent doubling produces significantly greater impact. Figure 1-1 illustrates this concept.

Figure 1-1. Exponential growth versus linear growth; comparing AI systems with traditional systems The Rapidly Evolving State of Enterprise AI Adoption AI development reflects this pattern. Advances in data, model size, architecture, and training techniques build on one another, transforming early novelty into practical utility. As a result, enterprise adoption often follows a similar trajectory where limited experimentation gives way to rapid expansion when performance crosses a threshold of real-world value. For enterprise adoption, the difficulty is not recognizing the potential of AI, but identifying the inflection point where incremental progress becomes exponential change. Organizations that continue to plan for linear growth risk underestimating future demand and designing systems that cannot support the pace of AI-driven transformation. Managing the Reality Gap AI-driven progress raises expectations as quickly as it improves capability. Tasks that once took days are now expected to happen in hours, and manual workflows are assumed to be automated. The problem is that processes like planning, governance, and procurement do not accelerate at the same pace. This creates a gap between what AI enables and what organizations can operationalize. The gap is not caused by a lack of awareness, but by a mismatch in speed. As new use cases emerge and scope expands, static planning models struggle to keep pace. AI adoption is not a one-time initiative but an ongoing acceleration, and organizations that treat it as a discrete risk are falling behind. Closing the gap requires aligning expectations with systems and structures that can continuously adapt. Why Build-to-Last Fails for AI The build-to-last (B2L) mindset assumes stable requirements and gradual change. That model no longer works. AI introduces continuous and

compounding improvements in capability and demand, making static architectures more of a liability than a strength. B2L Now Means Designing for Change Designing for change means treating evolution as a default condition. The goal is not to freeze requirements, but instead to build systems that can absorb new use cases and new patterns of demand without repeated redesign. Iteration becomes a property of the system itself. Modularity is favored over rigidity, automation over manual intervention, and adaptive scaling and resilience as part of the core architecture. In the context of AI, this also means accounting for model updates, evolving data pipelines, and unpredictable workload patterns driven by agents and real-time inference. Systems must be able to incorporate new capabilities without disrupting existing operations, while adhering to performance criteria and compliance requirements. Designing for change is now about building and sustaining momentum in an environment where advancements push forward rapidly. Going Beyond Code Changes AI transformation is frequently treated as an application-layer initiative. Organizations focus on models, APIs, and user-facing functionality. However, intelligent operational applications often depend on real-time analytic processing within the data platform itself, rather than relying on batch-oriented data movement. As operational analytical workloads converge, the database becomes a central component of AI execution rather than a passive store of record. This shift exposes the limits of legacy architectures. Systems designed for predictable transaction processing and incremental growth often rely on centralized scaling models and manual operational controls. While migration may relocate these systems to new environments, it does not alter their fundamental characteristics. Structural constraints around scalability, availability, and performance still remain in effect.

Modernization addresses the constraints, but not without introducing new considerations. Distributed architectures provide horizontal scalability by adding nodes rather than vertically expanding a single system. Built-in replication and automated failover support high availability, while geo- distribution brings data closer to users and applications to enable consistent performance across regions. Observability and operational control also become embedded capabilities rather than external add-ons. However, many organizations encounter a new form of complexity as they modernize. In practice, supporting diverse workloads such as transactional processing, analytics, and AI often requires assembling multiple specialized systems. This results in database sprawl, duplicated data pipelines, and increased operational overhead. Instead of eliminating constraints, modernization efforts can shift them into integration challenges across fragmented data platforms. Reducing this complexity requires a data foundation capable of supporting multiple access patterns and workloads within a unified architecture. This challenge becomes even more pronounced in the context of AI-driven applications. Modern AI systems depend on heterogeneous data, including structured application data, unstructured documents, embeddings, and vector representations. In many environments, these data types are managed across separate systems, requiring preprocessing pipelines to transform and synchronize data between them. This fragmentation increases latency, complicates governance, and makes it difficult for AI agents to maintain consistent context and memory. Figure 1-2 shows this problem. The reason most multi-agent systems fail isn’t due to logic but rather a lack of shared memory and knowledge.

Figure 1-2. Data fragmentation becomes a significant challenge that prevents AI systems from using the latest data Shared State Inconsistency A study on multi-agent systems found that 36.9% of failures in agentic AI environments originate with inconsistent shared state between agents. These failures occur when one agent updates the system but that change is not reliably visible to others, leading later agents to act on stale or invalid

assumptions. As a result, decisions that were correct at one point in time quickly become misaligned with the current state of the system (Cemri et al., 2025).1 True modernization is not only about scalability and resiliency but also about unifying how data is stored, accessed, and processed. Platforms that can support relational, analytical, and vector workloads reduce the need for complex data movement and enable more efficient AI execution. With this type of platform and architecture, the database evolves from a system of record into a system of intelligence, where agents can access and update context in real time. Additional important characteristics such as scalability, replication, availability, and regional distribution are not optional enhancements but are core to the functionality needed to drive the real-time value gains from AI. These capabilities are the operational foundation that enables intelligent applications to scale reliably. Without modernization at the data and infrastructure layer, AI adoption outpaces the systems required to sustain it. 1 Cemri, M., et al. (2025). “Why Do Multi-Agent LLM Systems Fail?” arXiv: 2503.13657, https://doi.org/10.48550/arxiv.2503.13657.

Chapter 2. Architectural Principles for AI Applications When architected with intent, AI initiatives bring significant value to the business. Data is the foundation on which successful architectures are created. This chapter looks at the architectural decisions that unlock business value from a well-designed AI-enabled application. Outcomes Drive Results To succeed with AI, organizations need to understand their existing systems and data. Architectural decisions made early based on the desired outcome help to ensure success. When incorporating data into early decisions, architects also ensure scalability and enable rapid response, increasing agility for the business. Business-Driven and Data-Oriented Architecture Business outcomes determine the real-world constraints of the AI-enabled system, such as response times, fault tolerance, scaling, and other operational considerations of the system. An AI system that needs to support real-time decision making has different architectural requirements than a system designed for asynchronous analysis or offline recommendations. Important decisions like data placement, system boundaries, and deployment characteristics need to be made early. A data-oriented perspective is essential when architecting AI systems for production. Data provides the true foundational value for AI systems, making decisions around data essential for the overall success of the project. Postponing decisions about data creates risks such as needing to

rework the architecture due to performance or reliability problems. Rearchitecting becomes exceedingly difficult for mission-critical systems. Reducing user burden Reducing user burden is not simply about automation. From an architectural perspective, reducing user burden means reducing operational friction and cognitive load. An AI system that behaves inconsistently or requires constant verification increases user burden rather than reducing it. This inconsistency is caused not only by slow or unstable system behavior but also by fragmented data that exists across operational, vector, and unstructured systems. From an architectural perspective, reducing user burden requires explicit design choices that limit the surface area of unpredictability: Predictable behavior boundaries AI systems must clearly signal when outputs are reliable and when they are uncertain. Likewise, architectural design choices should help to detect and mitigate partial failures such as latency and capacity pressure. The architecture should be able to surface these conditions and steer around the unhealthy components. Users should never be required to infer system state. Deterministic system integration Distributed data architectures create a level of abstraction, hiding things like shard placement, replication, and underlying failover mechanics. This abstraction enables application logic to remain deterministic, even as the underlying system adapts to failure and scaling events. The overall result is a reduction in the operational burden placed on users, developers, and systems staff. Data pipeline simplification

AI systems often rely on complex preprocessing pipelines to prepare operational, unstructured, and vector data for use in models. When these pipelines are manual, fragmented, or inconsistent, they introduce errors and increase the burden on both developers and operators. Architectural design should reduce this complexity by minimizing the need for custom preprocessing steps and enabling more integrated, consistent data handling across the system. Failure containment Ultra-resilience explicitly prioritizes containment of failures within a given fault domain. This prevents outages from becoming user-facing service degradations. By maintaining service continuity through replication and automated failover, systems avoid forcing users to diagnose or work around infrastructure-level issues. Enabling rapid response Rapid response in AI systems is not determined by inference speed alone. End-to-end response time is heavily influenced by data locality, coordination overhead, and consistency guarantees. This is especially important in globally distributed environments. Architecturally, enabling rapid response requires latency budgets, intentional AI logic placement, and time-bounded execution: Explicit latency budgets The full request path and all layers, including data access, coordination, and replication, need to have predictable low- latency architectures that minimize coordination delays. Systems that lack explicit constraints around latency often degrade quietly under load, creating a cascade of delays. Intentional placement of AI logic

Placing applications close to data replicas helps to ensure that time-sensitive functions execute with minimal latency. Distributed SQL systems support locality-aware data placement, which then guards against latency-related issues and cross-region penalties. Time-bounded execution An architecture should be built to fail fast, detecting potential issues and then quickly routing around those issues. Degraded regions or nodes should not affect overall application availability, thereby preventing slow components from dominating response times. Additionally, architectures that support persistent agent memory and reuse of prior context can reduce redundant computation, improving response time, increasing accuracy, and lowering token consumption over repeated interactions. Simplifying tasks AI systems are often introduced to help simplify complex tasks. But simplification at the interface level can conceal significant architectural complexity. A modern distributed architecture helps to reduce operational complexity by centralizing intelligence in the platform rather than the application. Three key characteristics are common in architectural simplification efforts: Clear separation of responsibilities Application logic and infrastructure are clearly separate. Application developers should not create or add logic around resiliency but rather should be able to rely on the underlying infrastructure to work around any potential issues. Modular boundaries

AI behavior should evolve separately from core business workflows. Too much integration leads to fragility and difficulty innovating. Observable decision paths Built-in metrics and monitoring provide visibility into latency, load distribution, and failure modes. Performance and compliance are then verifiable by operators at the system level and in a repeatable manner. Ensuring scalability For AI systems, consider whether the AI component needs to operate synchronously in response to user-facing events or if the AI responds to background workflows. Asynchronous workflows can often be predicted or controlled to work around other demand-related access. However, architects need to define how the system should behave in response to burst traffic and uneven access patterns. Distributed SQL systems address these issues through automatic sharding, replication, and load balancing. However, architects should be aware that early decisions about these elements and early assumptions about application demand cycles can make it difficult to rearchitect for scaling later. Reducing costs Intentional design and architectural decisions reduce cost by avoiding manual intervention and reducing operational overhead. Making these decisions requires visibility into cost drivers, avoidance of duplicated effort and infrastructure, and predictability around scaling behaviors. This is true even if demand cannot always be predicted. Assessing Business Requirements

A business requirements assessment for AI systems is not a traditional requirements gathering exercise. The purpose is not solely tied to enumerating features but rather to surfacing large scale architectural constraints early, before assumptions create difficult implementations. The goal of the requirements gathering exercise is to determine what kind of system is being built, what guarantees it must provide, and where uncertainty is acceptable. Four dimensions are particularly helpful: the role of AI within the application, the interaction models that the system needs to support, the way in which data is generated and updated, and how users and consumers are distributed. Is AI the App? One of the most consequential decisions that needs to be made is whether AI is the app itself or is a capability that is embedded into an existing (or newly built) system. This distinction determines where authority resides and how much variability the system can tolerate. Conversational systems, AI-native analytics tools, and planning agents are prime examples of AI being the application itself. When created for internally facing systems, probabilistic behavior is often acceptable as long as interactions remain coherent and recoverable within boundaries. Users expect iteration, approximation, and refinement in these systems, and priority is often placed on availability and responsiveness over strict determinism. By contrast, when AI is added to an existing application or workflow, the AI portion operates within constraints and expectations around correctness and consistency. The AI-produced output cannot become the system of record or redefine correctness for the system in such cases. Distributed data platforms are explicitly designed to preserve transactional integrity and predictable behavior, even as their dependent systems have failures and service degradations. The distributed data platform provides the foundation for deterministic outcomes around which the AI portion can be built.

Architecturally, this portion of the requirements assessment determines whether AI outputs are: Advisory Influencing decisions that are validated elsewhere, such as other portions of the app or external to the app Authoritative Providing direct triggers for state changes or actions Failing to make this architectural distinction often results in systems where AI components silently accumulate authority that they were never designed to have. Understanding User Interaction Model Options AI that interacts through a traditional user interface (UI) might show AI- generated recommendations, summaries, and predictions rather than acting autonomously. Chatbots extend the UI interaction into a more conversational format. Like familiar generative AI, chatbots can recall context throughout the conversation. AI chatbots utilize stateful interaction to maintain persistence of context and conversation history. This increases the importance of observability, as chatbots are required to respond even when confidence is low, making it more difficult to identify and diagnose unreliable outputs. Model Context Protocol (MCP) represents a different type of user interaction. Instead of being human-oriented like a chatbot, MCP is machine-oriented. MCP enables external tools, services, and agents to provide context to AI models. MCP interactions are programmatic rather than conversational. Agentic behavior is typically the most demanding interaction model. Autonomous agents may invoke AI continuously, chaining actions across multiple systems. From an architectural perspective, autonomous or agentic