Enterprise Architecture for Digital BusinessTransforming IT (Lin, GengMacVittie, Lori A.) (Z-Library)

GENG LIN & LORI A. MACVITTIE Enterprise Architecture for Digital Business Transforming IT

978-1-098-12145-7 [LSI] Enterprise Architecture for Digital Business by Geng Lin and Lori A. MacVittie Copyright © 2022 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisition Editor: Melissa Duffield Development Editor: Gary O’Brien Production Editor: Gregory Hyman Copyeditor: Sharon Wilkey Proofreader: Piper Editorial Consulting, LLC Indexer: Judith McConville Interior Designer: Monica Kamsvaag Cover Designer: Susan Thompson Illustrator: Kate Dullea July 2022: First Edition Revision History for the First Edition 2022-07-14: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098121457 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Enterprise Architecture for Digital Business, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and F5. See our statement of editorial independence.

Contents | Introduction v 1 | Form Follows Function 1 2 | An Infrastructure Renaissance 17 3 | From Marathon to Messaging 37 4 | Operational Data Is the New Oil 51 5 | Moving Beyond “Fight or Flight” 65 6 | Observability and Automation 87 7 | The Need for Speed 107 | Afterword: Digital Changes Everything 127 | Index 137 iii

Introduction —Geng Lin, F5 CTO, and Lori MacVittie, F5 principal technical evangelist The destination of the digital transformation journey is a digital business. During the first two decades of the 21st century, a digital transformation trend has driven the global economy. Over just two years—from 2019 to 2020— the pace of that transformation accelerated nearly tenfold. The forces behind this acceleration span a broad spectrum of technological and societal changes that are not likely to slow for decades: wireless access for billions of people, the smartphone as an app platform, the increased speed and size of applications built on the cloud, the constrained number of engineers and developers, and customer demand for better, simpler experiences that empower them. The common theme of all these forces is the increased need and demand for innovation. The outlines of this digital transformation journey are starting to take shape. What is emerging is a transformation to fully digital and automated businesses, resulting in the adaptability needed to respond to changes in the ecosystem, soci- ety, technology, and customer needs with a laser focus on customer experience, new ways of creating value, and a reshaping of the technical foundation of the businesses. At the heart of this transformation is the need to optimize business and technology for innovation. The Innovation Equation Today, most companies have a consistent theme of being stable at their core and innovating at the edge. If a business has a profitable product line or service, it won’t gamble with the core capabilities required to support it on a whim. However, company leaders might try new ideas on a small number of customers v

1 Khalid Kark, “Maximizing the Impact of Technology Investments in the New Normal,” Deloitte Insights, February 23, 2021, https://oreil.ly/elI2k. 2 Spiceworks Ziff Davis, “The 2022 State of IT,” https://oreil.ly/D795y. to discover whether the new product or service has traction without risking the whole business. This is often reflected in the amount of money spent on maintaining core capabilities, which has consumed the bulk of corporate technology budgets for decades. Even after the explosive acceleration of digital transformation due to the global pandemic, organizations still allocated significantly more budget to their core than to innovation, on average, investing only 15% of budgets on business innovation initiatives and 59% on day-to-day business operations.1 In a normal environment, this is not necessarily bad. But we are not in a nor- mal environment. We have shifted into a business cycle in which organizations need to innovate. Budget alone does not adequately express the ability of a business to adapt. A better measure is based on the capacity to engage in innovation, and that measure ultimately relies on people. The good news is that despite the additional burdens placed on IT by recent moves to a largely hybrid (remote/office) work- force, nearly half (40%) of organizations plan to expand their IT staff to meet demand.2 The bad news is that more people maintaining the core—keeping the lights on—does not necessarily contribute to the business’s ability to innovate. For that, technology needs to be used to enable staff to focus on adding value through innovation, rather than maintaining value by sustaining the status quo. Consider an example: a company has been in business for 40 years. It has multiple product lines that are all profitable and can launch one or two new products or services a year. Maintaining its core ties up most of the business’s resources, and only a small percentage of the staff and resources are focused on developing new products or services. Most of its staff and budget are sustaining operations. This means that the company’s ability to adapt is low. The business can’t take advantage of shifts in the market, putting it at a significant competitive disadvantage. With the advent of the public cloud and subsequent adoption of its operating model, much of the core infrastructure needed to build products or run services became available as an operational cost and required minimal personnel to use. VI | INTRODUCTION

3 Jason Perlow et al., “The 2021 Open Source Jobs Report” The Linux Foundation, https://oreil.ly/xg3zu. 4 Vivian Chan, “New Tech Leader Survey Reveals Why the Time for Real-Time Operations Is Now,” PagerDuty, November 10, 2021, https://oreil.ly/ltEok. 5 Tara Balakrishnan et al., “The State of AI in 2020,” McKinsey and Company, November 17, 2020, https://oreil.ly/gORVN. This shift was largely due to the ability of the public cloud to bear the cost of managing the infrastructure needed to deliver applications. All the budget and people dedicated to provisioning, scaling, monitoring, and operating infrastruc- ture can be freed to focus on innovation. But even with a jump in adaptability, we still have a significant shortage of skilled people to address market demand. A recent survey found that although half (50%) of employers are increasing hires, nearly all (92%) report difficulty finding talent with the right skills. Exacerbating the challenge is their struggle to retain existing talent in a fiercely competitive market.3 Organizations need to find another way to realize an order-of-magnitude jump in their capacity to innovate—not just to survive, but also thrive, in a digital economy. Accelerating Adaptability The goal of every business should be to achieve a budget and staff that increases the capacity to innovate. This requires more staff working on innovation than those maintaining the core. This is simply not possible for most organizations because of the following four key issues: An operating model and culture intolerant of failure, leading to resistance to change A majority of digital leaders (62%) see the traditional, ticket-based approach to ITOps as a waste of time because IT spends too much time figuring out how to respond to a digital incident.4 A security strategy based on control that requires significant resources rather than one based on adaptable risk management An average of 46% of organizations report reductions in managing risk after adopting AI for security operations, with nearly half (44%) reporting cost reductions of up to 20% across all functions.5 INTRODUCTION | VII

6 Chan, “New Tech Leader Survey Reveals Why the Time for Real-Time Operations is Now.” 7 “The State of Application Strategy in 2022,” F5, April 12, 2022, https://oreil.ly/FRlOx. Reliance on human intervention to operate applications and the infrastructure that delivers them A plurality of tech leaders (70%) believe they need a new way to address digital incidents if they are expected to innovate.6 A lack of visibility into every part of the enterprise ecosystem, which hampers the ability to leverage technology that can reduce the budgetary burden of maintaining the core The lack of insights is nearly universal. About 98 percent of IT staff and leaders report missing data critical to sustaining operations.7 As digital transformation drives businesses toward the goal of becoming adaptive, it must solve for these four key areas to enable adaptability and unlock the ability to innovate. People turn to technology when economies of scale are needed. The technol- ogy changes, evolving from stone to steel to silicon, but the maxim remains intact: technology improves the efficacy and speed of business, allowing it to scale innovation. The Role of Technology in Digital Transformation Digital transformation is not a new phenomenon brought about by the global pandemic or the adoption of the internet, but the rate of transformation is accelerating because of the rapid advance of technology and the impact of macro- economic and societal changes that the pandemic introduced. Today we see every industry engaged in digital transformation. From banking to retail, from media and entertainment to education, to manufacturing—virtually every industry is on a trajectory to become a digital business. This process does not—and cannot—occur overnight. It is a journey that mirrors transformations in nature and that of a human life. A monarch butterfly, with an average lifespan of two to six weeks, can spend up to half that time in its transformational form as a chrysalis. Human beings spend one-fifth of their life growing from infancy to adulthood, with several significant transfor- mations taking place within that period. Transformation takes time, and busi- nesses should expect a similar experience. Human lives are often described as progressing through six distinct phases: fetus, baby, child, adolescent, adult, and VIII | INTRODUCTION

elder. Business transformation is also delineated by phases of development, each marked by distinct characteristics and activities: Phase 1: Task automation In this stage, digitization leads businesses to turn human-oriented busi- ness tasks to various forms of automation, which means more applications are introduced or created as part of the business flow. This began with automating well-defined, individual tasks to improve efficiencies. A com- mon example is interactive voice response (IVR) systems that answer com- mon questions about a product or service but may need to hand them off to a human representative. In this phase, individual tasks are automated but not consistently integrated. Phase 2: Digital expansion As businesses start taking advantage of cloud-native infrastructures and driving automation through their own software development, a new gener- ation of applications supports the scaling and further expansion of their digital model. The drivers behind this phase are business leaders who become involved in application decisions designed to differentiate or pro- vide unique customer engagement. For example, healthcare providers are increasingly integrating patient records and billing with admission, dis- charge, and scheduling systems. Automated appointment reminders can then eliminate manual processes. Focusing on end-to-end business process improvement is the common theme in this phase. Phase 3: AI-assisted business As businesses further advance on their digital journey and leverage more advanced capabilities in application platforms, business telemetry and data analytics, and machine learning (ML) and AI technologies, businesses will become AI assisted. This phase opens new areas of business productivity gains that were previously unavailable. For example, a retailer found that 10% to 20% of its failed login attempts were legitimate users struggling with the validation process. The combination of consumer tendency to abandon a brand after a single bad experience with research that finds “on average, loyal customers are worth up to 10× as much as their first purchase” means that denying access by default represents a potentially INTRODUCTION | IX

8 Douglas Karr, “Customer Retention: Statistics, Strategies, and Calculations,” Martech Zone, May 19, 2021, https://oreil.ly/CvvXA. 9 F5, “The State of Application Strategy in 2022.” significant revenue loss.8 Behavioral analysis can be used to distinguish legitimate users from bots attempting to gain access. Technology and ana- lytics have enabled AI-assisted identification of those users to let them in, boosting revenue and improving customer retention. The inevitableness of digitization means every business will make this jour- ney. As with human journeys, each business will experience this transformation at a different pace. At times, external forces will accelerate or decelerate this journey, as we saw during the global pandemic. Our research, confirmed by the industry at large, indicates that most organi- zations today are in the second phase of their journey.9 This phase is marked by a focus on application and operational modernization, with an increasing tendency to adopt cloud and edge technologies. This is the phase in which many will find their progress decelerated and, for some, blocked by seemingly insurmountable obstacles. What stands in the way of completing this journey is an existing, rigid framework that governs how applications are developed, delivered, secured, and even integrated. It defines how data should be stored, accessed, and governed. It constrains infrastructure to aging standards. It makes assumptions about appli- cations and their interactions, and about the nature of their users. Existing infor- mation architecture frameworks have existed since before the broad adoption of the internet, and well before the era of digitization we find ourselves in today. For CIOs and IT leaders to successfully navigate the second phase of digital transformation, they must first identify key technologies and capabilities critical to enabling businesses to progress into the third and final phase of digital trans- formation. These capabilities include the following: • Using infrastructure as efficiently as possible by delivering applications in a distributed model that includes private and public clouds, data centers, and edge computing • Expanding and scaling digital operations by adopting site reliability engi- neering (SRE) operational practices to align technology with business outcomes X | INTRODUCTION

• Taking advantage of AI and analytics in both IT and lines of business by reimagining data architectures and governance to adapt to the convergence of operational technology (OT) and IT • Operating securely at scale by incorporating security as a key component in every aspect of a digital business and embracing app delivery as a core disciplinary domain Then, with a critical eye, technology leaders must reevaluate their enterprise architecture and determine how best to insert and leverage these technologies and capabilities. It is for this purpose—providing a framework for the transfor- mation of the enterprise architecture—that we have taken on the task of writing this book. Purpose and Scope The purpose of this book is to explore the architecture required to successfully navigate the second phase of digital transformation. That transformation evolves the enterprise architecture into one more suited to support an increasingly data- driven and data-dependent digital business. We’ve written this book for the CIO and the architect, for the IT director and the network engineer. We offer an architecture framework for transitioning IT to operate as a digital business. We do not dive into the details or offer prescriptive advice on how—or what—to implement. Our goal is to provide a clear picture of the architectural transformation needed to enable a digital business to thrive. That transformation is determined not by us, or by any other expert, but by the technological shifts occurring in every industry and at every layer of the IT stack, which we also discuss. In Chapter 1, we discuss the changes to existing enterprise architecture needed to infuse the capabilities required by a digital business. Each follow- ing chapter explores the trends and technologies driving changes in a specific domain. In Chapter 2, we explore the capabilities enabled by the adoption of cloud and edge technologies and the ability to adapt the deployment location of applications. We then look at the need for application delivery as an IT discipline in Chapter 3, driven by the requirement for digital business to operate safely at scale. The expansion of the data domain to embrace operational data (telemetry) and practices required to scale analytics in order to enable a digital business is the focus of Chapter 4. The rapid evolution of security is the focus of Chapter 5, INTRODUCTION | XI

in which we lay out the foundations for a modern security governance and architecture framework that infuses a security-first approach to digital business. Chapter 6 covers the emerging need for observability and the expansion of automation from a productivity tool to an innovation accelerator. Finally, in Chapter 7, we dive into SRE as a catalyst for scaling operations in a modern, digital business. Why We Wrote This Book Why, indeed. As a group of leaders in a company most often identified simply with load balancing, it may surprise you to learn that F5 has been in the business of helping enterprises design, implement, optimize, and secure enterprise archi- tecture for 25 years. From its earliest role as a load balancer to securing, delivering, and distribut- ing applications today, F5 has always been an integral partner with business and IT on the topic of architecture. From securing infrastructure against volumetric attacks, to defending against application attacks, to protecting a business against fraud and abuse, F5 is intimately familiar with the inner workings of enterprise architectures in every industry across the globe. In its lengthy history, F5 has also had the privilege of partnering closely with many application providers. These partnerships have been more than strategic, sales-oriented engagements. F5 has spent considerable effort to understand at a deep, technical level how these applications are deployed, delivered, and integrated with other applications and with the business itself. Even more relevant than its technology portfolio are the leaders, technolo- gists, architects, and strategists who have come together at F5. Hailing from wide-ranging industries—financial services, social networks, transportation, insurance, and other technology firms—the authors have expertise in every layer of the IT and digital business stack. Together, with our deep understanding of technology, we believe that we are uniquely positioned to analyze today’s trends and deliver the insight necessary to identify and articulate the sweeping changes they will have on enterprise architecture. XII | INTRODUCTION

Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, envi- ronment variables, statements, and keywords. O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided tech- nology and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/enterprise- architecture-for-digital-business. INTRODUCTION | XIII

Email bookquestions@oreilly.com to comment or ask technical questions about this book. For news and information about our books and courses, visit https://oreilly.com. Find us on LinkedIn: https://linkedin.com/company/oreilly-media Follow us on Twitter: https://twitter.com/oreillymedia Watch us on YouTube: https://www.youtube.com/oreillymedia XIV | INTRODUCTION

Form Follows Function —Lori MacVittie, F5 principal technical evangelist The way businesses operate has changed over the past 20 years, with even more dramatic changes occurring in just the past 5 years. As businesses continue their rapid transformation from physical to digital, enterprise architecture must also transform to support it. As a technology leader, you will be directly or indirectly involved with guiding and executing this transformation. This is no small task. Enterprise architecture as a discipline was established and standardized in the late 20th century, before the wide adoption of the inter- net and digital business models. The organization you work for likely has an enterprise architecture in place—one that was developed decades ago and has remained largely unchanged. The 20th-century architect Louis Sullivan, mentor of the still celebrated Frank Lloyd Wright, coined the maxim, “Form follows function.” This principle is often applied to software engineering, in which the “function” is the business process, and the “form” is the enterprise architecture. The premise is that if the architecture prescribes how a business operates, the business will be constrained and unable to adapt to changing conditions. Therefore, as the function of busi- ness transforms from physical to digital, the form of the enterprise architecture must also transform. In this chapter, we discuss key technology trends driving new functions into business and the changes needed in enterprise architecture to enable a suc- cessful transformation into a digital business. Grady Booch, codeveloper of the Unified Modeling Language and creator of the Booch object-oriented software development method, says this: 1 | 1

1 “Grady Booch,” Wikiquote, last updated May 14, 2019, https://oreil.ly/U6U2g. Architecture represents the significant decisions, where significance is measured by cost to change.1 That transformation must be strategic, not only tactical, and encompass the whole of the enterprise architecture. A digital business has no component, no cog in the wheel, that does not contribute to or play a significant role in the success of the architecture. As has been true throughout history, standardiza- tion serves as a powerful transformational force, forever changing the course of industries and nations. It should be no surprise that standardization has a significant role in enterprise architecture transformation. Standardization Spurs Innovation Three thousand years ago, Hammurabi carved in stone a set of 282 rules stand- ardizing commercial and judicial practices that today is remembered as the Code of Hammurabi. It is recognized as the first set of codified laws and standards. Hammurabi could not have known the impact this invention would eventually have on history. Since that time, standards have been used in every era to guide, enforce, and encourage humanity in all its endeavors. From ensuring fairness of trade through gold and weight standards and protecting our well-being through safety standards, to ensuring the interoperability of devices that make the internet pos- sible, standards have a significant impact on society, technologies, and business. Figure 1-1 highlights several notable moments in standardization. What isn’t shown is the incredible innovation that occurred as a result of that standardiza- tion. The Gutenberg printing press did more than standardize the process of printing. That standardization significantly reduced reliance on manual methods of production, which spurred greater production of books—and literacy—across societies. This led to the Protestant revolution and the Age of Enlightenment, from which the foundations for modern science developed and drove the estab- lishment of universal education. The impact of the printing press is seen in the innovation of the “digital press,” which continues to rely on principles developed by Johannes Gutenberg and innovated on to facilitate modern communication. The innovation resulting from the invention of and standardization on Guten- berg’s printing press was dramatic and long-lasting. 2 | ENTERPRISE ARCHITECTURE FOR DIGITAL BUSINESS

Figure 1-1. Every period of momentous innovation was preceded by a significant standardization Standardization has served as the catalyst for the transformation of indus- tries and societies for thousands of years. From Gutenberg to TCP/IP, from Henry Ford to the anticipated standardization of 5G, standardization has focused on efficiencies that ultimately increase the velocity of innovation. Henry Ford’s assembly line—the mechanical manifestation of standardiza- tion—is heralded as marking the start of the Industrial Revolution. While some might view his famous quote, “Any customer can have a car painted any color that he wants, so long as it is black,” in a negative light, it reflects the reality that standardization of the processes automated by the assembly line required what is often viewed as compromise—but you will recognize as significant architectural decisions. Henry Ford chose efficiency and improved quality over more color options. The efficiency gained led to industry expansion that ultimately resulted in innovation as a competitive advantage. Those innovations ultimately gave us color choices and more options than Henry Ford could ever have imagined. When businesses ran headlong into similar barriers to scale in the mid-1950s—namely, manual, tedious, human-executed processes—they turned to the modern equivalent of the assembly line: digitization. Business tasks and processes were turned into applications. For example, corporate payroll activities have long been serviced by applications. It was an early form of digitization, but digitization, nonetheless. FORM FOLLOWS FUNCTION | 3

While this initial effort resulted in the desired growth of business, the lack of standardization into the 1980s in software, architectures, and even networking resulted in a new set of inefficiencies. Every project started from scratch, with no blueprint or guidance as to how to proceed. There were no best practices, textbooks, or reference architectures. The Emergence of Architectural Standards Enterprise architecture has a long history, reaching back to the 1960s when Professor Dewey Walker penned multiple manuscripts on Business Systems Planning. Perhaps inspired by the success of Henry Ford in manufacturing to standardize components and processes, one of Walker’s students, John Zach- man, would formulate a standardized version in the late 1980s. The principles and framework Zachman laid out would grow into the discipline we know today as enterprise architecture. The Role of an Enterprise Architecture Framework An enterprise architecture framework defines how to create and use an enterprise architecture. An enterprise architecture describes the business capabilities, processes, and elements needed to operate the business and then maps them to tools, technologies, and practices. An architecture framework provides principles and practices for creating and using the architecture description of a system. It structures archi- tects’ thinking by dividing the architecture description into domains, layers, or views, and offers models—typically, matrices and diagrams— for documenting each view. This allows for making systemic design decisions on all the components of the system and making long-term decisions around new design requirements, sustainability, and support. While the Zachman Framework, as it is now called, is still one of the most broadly used frameworks, it is not the only one. Another effort, begun in the 1960s, would culminate in a more technical approach: The Open Group Architec- ture Framework (TOGAF). TOGAF remains the guide by which an estimated 80% of Global 50 companies design and implement the enterprise architectures on 4 | ENTERPRISE ARCHITECTURE FOR DIGITAL BUSINESS

2 Bob Reselman, “TOGAF and the History of Enterprise Architecture,” Red Hat, September 14, 2020, https://oreil.ly/TjRo0. 3 Nina Anggraini et al., “Cloud Computing Adoption Strategic Planning Using ROCCA and TOGAF 9.2: A Study in Government Agency,” Procedia Computer Science 161, no. 3 (January 2019): 1316–1324, https://oreil.ly/13H44. which business relies.2 First developed in the mid-1990s and published as an official standard by the Open Group in 1995, TOGAF remains a powerful force in the industry. As recently as 2019, it has been recommended as a foundational component to cloud adoption.3 A Traditional Enterprise Architecture Framework The traditional enterprise architecture framework has served organizations well into the internet age. As a technology leader, you may have been tasked with incorporating any number of emerging technologies into one of the four distinct domains that define the foundation of a traditional enterprise architecture, as shown in Figure 1-2. Figure 1-2. A traditional enterprise architecture framework The four domains are as follows: Business Combines key business processes, governance, company structure, and strategy into a holistic view with the goal of describing a vision of business capabilities and value to be delivered. Examples include renting a car, approving a mortgage, or scheduling an appointment. Application Provides an overarching view of the applications needed to support the business vision. Includes describing services and interfaces. FORM FOLLOWS FUNCTION | 5