Page 1

(This page has no text content)

Page 2

DevOps Tools for Java Developers Best Practices from Source Code to Production Containers With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles. Stephen Chin, Melissa McKay, Ixchel Ruiz, and Baruch Sadogursky

Page 3

DevOps Tools for Java Developers by Stephen Chin, Melissa McKay, Ixchel Ruiz, and Baruch Sadogursky Copyright © 2022 Stephen Chin, Baruch Sadogursky, Melissa McKay, and Ixchel Ruiz. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998- 9938 or corporate@oreilly.com. Acquisitions Editor: Suzanne McQuade Development Editor: Corbin Collins Production Editor: Kate Galloway Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: O’Reilly Media February 2022: First Edition Revision History for the Early Release 2020-12-18: First Release 2020-01-07: Second Release 2021-06-11: Third Release 2021-09-01: Fourth Release See http://oreilly.com/catalog/errata.csp?isbn=9781492084020 for release

Page 4

details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. DevOps Tools for Java Developers, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 978-1-492-08395-5

Page 5

Chapter 1. DevOps for (or Possibly Against) Developers Baruch Sadogursky A NOTE FOR EARLY RELEASE READERS With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles. This will be the 1st chapter of the final book. If there is a GitHub repo associated with the book, it will be made active after final publication. If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at ccollins@oreilly.com. “While you here do snoring lie, Open-eyed conspiracy His time doth take. If of life you keep a care, Shake off slumber, and beware: Awake, awake!” —William Shakespeare, The Tempest Some might ask if the DevOps movement is simply an Ops-inspired plot against developers? Most (if not all) who’d do so wouldn’t expect a serious response, not least because they intend the question as tongue-in-cheek teasing. It’s also because – and regardless if your origins are on the Dev or the Ops side of the equation – when anyone strikes up a conversation about DevOps, it will take approximately 60 seconds before someone inquires: “But what DevOps really is?” And you’d think, ten years after the coining of the term (a decade within which industry professionals have spoken, debated, and shouted about it), that we’d all have arrived at a standard, no-nonsense, commonly-understood definition. But this simply isn’t the case. In fact, despite an exponentially-increasing corporate demand for DevOps personnel, it’s highly doubtful that any five DevOps-titled

Page 6

employees, chosen at random, could tell you precisely what DevOps is. So, don’t be embarrassed if you still find yourself scratching your head when the subject comes up. Conceptually, DevOps may not be easy to grok, but it’s not impossible either. But regardless of how we discuss the term or what definition(s) we might agree upon, there’s one thing, above all else that’s critical to bear in mind: DevOps is an entirely invented concept, and the inventors came from the Ops side of the equation That may be provocative, but it’s provable, too. Let’s start with two exhibits. Exhibit #1: The Phoenix Project Co-written by three info-tech professionals, The Phoenix Project, A Novel About IT, DevOps, and Helping Your Business Win was released in 2013. It’s not a how-to manual (not in the traditional sense, anyway). It’s a novel that tells the story of a highly problematic company and its IT manager who is suddenly assigned the task of implementing a make-or-break corporate initiative that’s already way over budget and months behind schedule. If you dwell in the realms of software, the rest of the book’s central characters will be very familiar to you. For the moment, though, let’s have a look at their professional titles: Director, IT Service Support Director, Distributed Technology Manager, Retail Sales Lead Systems Administrator Chief Information Security Officer Chief Financial Officer Chief Executive Officer 1

Page 7

Notice the connective tissue between them? They’re the protagonists of one the most important books about DevOps ever written and not one of them is a developer. Even when developers do figure into the plotline, well…let’s just say they’re not spoken of in particularly glowing terms. When victory comes, it’s the hero of the story (together with a supportive board member) who invents DevOps, pulls the project’s fat out of the fire, turns his company’s fortunes around, and gets rewarded with a promotion to CIO of the enterprise. And everyone lives happily – if not ever after, then for at least the two or three years such successes tend to buy you in this business before it’s time to prove your worth all over again. Exhibit #2: The DevOps Handbook It’s better to read The Phoenix Project before The DevOps Handbook, How to Create World-Class Agility, Reliability, & Security in Technology Organizations because the former places you within a highly believable, human scenario. It’s not difficult to immerse yourself in the personality types, the professional predicaments, and the interpersonal relationships of the characters. The hows and whys of DevOps unfold as inevitable and rational responses to a set of circumstances, which could have just as easily led to business collapse. The stakes, the characters, and the choices they make all seem quite plausible. Parallels to your own experience may not be too hard to draw. The DevOps Handbook allows you to explore the component conceptual parts of DevOps principles and practices in greater depth. As its subtitle suggests, the book goes a long way toward explaining “How to Create World-Class Agility, Reliability, and Security in Technology Organizations.” But shouldn’t that be about development? Whether it should or shouldn’t may be open to debate. What’s incontrovertible is the fact that the book’s authors are bright, super- talented professionals who are, arguably, the fathers of DevOps. However, Exhibit #2 isn’t included here to praise them so much as to take a close look at their backgrounds. Let’s start with Gene Kim. He founded the software security and data integrity firm, Tripwire, for which he served as its Chief Technology Officer for over a decade. As a researcher, he’s devoted his professional attentions to examining and understanding the technological transformations that have and are occurring 2

Page 8

within large, complex businesses and institutions. He also co-authored The Phoenix Project and, in 2019, The Unicorn Project (about which, more later). Everything about his career is steeped in Ops. Even when Unicorn says it’s “about Developers,” it’s still developers as seen through the eyes of an Ops guy! As for the other three authors of the Handbook: Jez Humble has held positions including Site Reliability Engineer (SRE), CTO, Deputy Director of Delivery Architecture and Infrastructure Services, and Developer Relations. An Ops guy! Even where the last of his titles references development, the job isn’t about that. It’s about relations with developers. It’s about narrowing the divide between Dev and Ops, about which he has written, taught, and lectured extensively. Patrick Debois has served as a CTO, Director of Market Strategy, and Director of Dev♥Ops Relations (the heart is his addition). He self- describes himself as a professional who is “bridging the gap between projects and operations by using Agile techniques in development, project management, and system administration.” That sure sounds like an Ops guy. John Willis, as of this writing, holds the title of VP of DevOps and Digital Practices. Previously, he’s been a Director of Ecosystem Development, a VP of Solutions, and notably a VP of Training and Services at Opscode (now known as Chef). And while John’s career has been a bit more deeply involved with development, most of his work has been about Ops, particularly to the degree that he has focused his attentions on tearing down the walls that had once kept developers and operations personnel in very distinct and separate camps. As you can see, all the authors have an Ops background. Coincidence? I think NOT. Still not convinced that DevOps is Ops driven? How about we have a look at who are the leaders trying to sell us on DevOps today. Google It In mid-2020, if you entered the term “What is DevOps?” in a Google search just

Page 9

In mid-2020, if you entered the term “What is DevOps?” in a Google search just to see what would come up, it’s likely that your first page of results would have included the following: Agile Admin, a system administration company Atlassian, whose products include project and issue tracking, list making, and team collaboration platforms AWS, Microsoft Azure, and Rackspace, all of which sell Cloud Ops infrastructure Logz.io, which sells log management and analysis services New Relic, whose specialty is application monitoring All of these are very Ops-focused. Yes, that first page contained one firm that was a bit more on the development side and one other which really wasn’t directly related to the search. The point here is that when you try to look for DevOps, most of what you’ll find tends to skew towards Ops. What Does It Do? DevOps is a thing! It’s in big demand. And with this, there are many who will want to know, concretely, what DevOps does, what it substantively produces. Rather than go down that route, let’s look at it structurally, conceptualizing it as you would the sideways, figure 8-shaped infinity symbol. In this light, we see a loop of processes that goes from coding to building to testing to release to deployment to operations to monitoring, and then back again to begin planning for new features.

Page 10

(This page has no text content)

Page 11

Figure 1-1. DevOps Infinity Loop And if this looks familiar to some readers, it should because it bears a conceptual similarity to the Agile development cycle: Figure 1-2. Agile Development Cycle There’s no profound difference between these two, never-ending stories other than the fact that Ops folks grafted themselves onto the old world of the Agile circle, essentially stretching it across two circles and shoehorning their concerns

Page 12

circle, essentially stretching it across two circles and shoehorning their concerns and pains into the domain that was once considered solely the province of developers. State of the Industry There’s further proof that DevOps is an Ops-driven phenomenon which, since 2014, has come packaged in the form of an easy-to-read, annual digest of data collected, analyzed, and summarized from tens of thousands of industry professionals and organizations worldwide. The Accelerate State of DevOps Report was primarily the work of DevOps Research and Assessment (DORA) and it’s the most important document in the software industry to gauge where DevOps is and where it’s likely going. In the 2018 edition , for example, we can see a serious focus on questions such as: How often do organizations deploy code? How long does it typically take to go from code commits to successfully running in production? When impairments or outages occur, how long does it generally take to restore service? What percentage of deployed changes result in degraded service or require remediation? Notice that all of those are very Ops-centered concerns. What Constitutes Work? Now let’s examine how work is defined by the State of DevOps Report and in The Phoenix Project. Well first, there’s planned work that focuses on business projects and new features, which spans both Ops and Dev. Internal projects, including server migrations, software updates, and changes driven by feedback on deployed projects can be broad-based and may or may not weight more to one side of the DevOps equation than the other. But what about unplanned activities, such as support escalations and emergency outages. Those are very Ops heavy, as is the coding of new features, bug fixes, and refactoring, which are all about how the life of Ops can be made easier by including developers in 3

Page 13

the DevOps story. If We’re Not About Deployment and Operations, Then Just What Is Our Job? So, it’s quite clear that DevOps isn’t anything developers were (or are) demanding. It’s an Ops invention to make everyone else work harder. And assuming this truth, let’s ponder what would happen if developers were to rise up as one and say, “Your Ops concerns are yours, not ours.” Fine. But in that case it would be only right and proper to ask the revolting developers what is their definition of “done.” What standards do they believe they need to achieve to say, “We did our job well and our part is now complete”? This isn’t a flippant question and there are sources we can look to for answers. One, although imperfect and not infrequently criticized, is the Manifesto for Software Craftsmanship , which puts forth four fundamental values that should motivate developers. Let’s consider them: “Well-crafted software.” Yes, indeed, quality is important. “Steadily adding value.” No argument there. Of course we want to provide services and features that people need, want, or would like. “Community of professionals.” In broad strokes, who could oppose this? Cordiality among industry peers is just being professionally neighborly. “Productive partnerships.” Collaboration is certainly the name of the game. Developers aren’t against QA, Ops, or the products themselves. So, in context, this is just about being friendly with everybody (so long as other teams don’t start dictating what their jobs are supposed to be). Just What Does Constitute Done? With all that we’ve established so far, we can safely say that we need to produce code that’s simple, readable, understandable, and easy to deploy. We must be certain that non-functional requirements (e.g., performance, throughput, memory footprint) are satisfied. We should work diligently to avoid incurring any technical baggage and, if we’re lucky, shedding some along the way. We have to 4

Page 14

technical baggage and, if we’re lucky, shedding some along the way. We have to be sure that all tests get passed. And we’re obligated to maintain fruitful relations with QA teams (when they’re happy, we’re happy). With good quality code, plus positive team leader and peer reviews, everything should be fine out of the gate. With a product team that defines standards for value and added value, benchmarks can be firmly established. Through their feedback, product owners help to determine if those benchmarks are (or aren’t) being met and by how much. That’s a pretty good thumbnail definition of a good software developer having “done” what they needed to do. It also demonstrates that well done can never adequately be measured (or even known) without the involvement of and clear communications with Ops personnel. Rivalry? So yes, although it can be proven that DevOps really wasn’t anything for which developers were clamoring, it can equally be shown how its infinite practice benefits everyone. And still there are holdouts; those who imagine that there’s a rivalry, even an antagonism between developers and, for example, QA testers. Developers work hard on their creations and then feel like the QA teams are almost like hackers, out to prove something, just digging and digging until they find a problem. This is where DevOps counseling comes in. Every conscientious developer wants to be proud of what they’re producing. Finding flaws can seem like criticism, when it’s really just conscientious work coming from another direction. Good, clear, open, and continuous conversations between developers and QA personnel helps to reinforce the benefits of DevOps, but it also makes plain that everyone is ultimately working toward the same goal. When QA folks identify bugs, all they’re doing is helping their developer colleagues to write better code, to be better developers. And this example of interplay between those on the Ops side and others on the Dev side demonstrates the useful blurring of the distinctions and separations between the two worlds. Their relationship is necessarily symbiotic and, once again, works along an endless continuum of activities with one informing the other for the mutual benefit of all. More Than Ever Before

Page 15

More Than Ever Before The increasing demand for DevOps is coming as much from external forces as it is from within software companies themselves. And this is because our expectations, all of our expectations, as people living in a 21 century world, continue to change rapidly. The more reliant we become on ever-improving software solutions, the less time there is to waste on information and communication gaps, and delays between developers and operations personnel. Take, for example, banking. A decade ago, most major banks had reasonably adequate websites. You could log in, have a look at your accounts, your statements and recent transactions, maybe you were even beginning to make payments electronically through the e-services your bank was offering. And while it was nice and offered a certain amount of convenience, it’s likely you still needed (or, at least, felt more comfortable) going to your local branch to handle your banking affairs. What didn’t exist is today’s fully digital experience, complete with mobile apps, automated account monitoring and alerts, and enough services that make it more and more common for average account holders to do everything online. You may even be among those who not only don’t care if you ever see the inside of your bricks-and-mortar branch ever again, you don’t even know where that branch is! What’s more, banks are reacting to these rapidly changing banking habits by consolidating and closing physical branches, and offering incentives for their customers to shift their banking to the online sphere. This accelerated even more during the COVID-19 crisis when access to branches was being restricted to appointment-only services, limited walk-in access, and shorter hours. So, whereas ten years ago, if your bank’s website went down for twelve hours of maintenance while they were deploying a better, more secure site, you’d probably have taken that in stride. What’s a dozen hours if it’s going to result in higher quality services? You didn’t need 24/7, online banking and, besides, the local branch was there for you. Today, that’s simply not the case. Half a day’s worth of downtime is unacceptable. In essence, you expect your bank to always be open and available for you. This is because your (and the world’s) definition of quality has changed. And that change requires DevOps more than ever before. Volume and Velocity st

Page 16

Volume and Velocity Another pressure impelling the growth of DevOps is the amount of data that’s being stored and handled. And that’s only logical. If more and more of our daily lives are reliant on software, then there’s obviously going to be a tremendous rise in that realm. In 2020, the entire, global datasphere amounted to nearly 10 zettabytes. A decade prior it was 0.5 zettabytes. By 2025, it’s reasonably estimated that it will balloon exponentially to over 50 zettabytes! This isn’t only about behemoths like Google, Netflix, Twitter, and others getting bigger and better, and therefore needing to process larger amounts of data. This projection affirms that more and more companies will be graduating into the world of Big Data. With that comes the demands of vastly increased data loads, as well as the shift away from traditional staging server environments, which offered exact replicas of given production environments. And this shift is predicated on the fact that maintaining such pair-to-pair schemes is no longer feasible in terms of size or speed. Happy were the days of yore when everything could be tested before going into production, but this isn’t possible anymore. Things are and will increasingly be released into production about which software firms don’t have 100% confidence. Should this cause us to panic? No. The necessity to release fast and remain competitive should inspire innovation and creativity in the ways required to best execute controlled rollovers, test procedures, and more in-production testing – what’s now referred to as progressive delivery. This comes along with feature flags and observability tools, such as distributed tracing. There are some who equate progressive delivery with the blast radius of an explosive device. The idea is that when one deploys to an in-production environment, explosions should be expected. Therefore, to optimize such rollouts, the best one can hope for is to minimize casualties, to keep the size of the blast radius as small as is possible. This is consistently accomplished through improvements in the quality of servers, services, and products. And if we agree that quality is a concern of developers and its achievement is part of a developer’s definition of done, then it means there can be no pause or disconnect between that Dev moment of done and the next, Ops moment of production. Because no sooner than this happens then we’re looping back into development, as bugs are fixed, services restored due to outages, and so on. Done and Done 5

Page 17

Done and Done Maybe it’s becoming clear that it’s the expectations and demands that were and continue to be generated from the Ops environment, which, of necessity, drove the push to DevOps. And as a result, the increased expectations and demands on developers isn’t coming from some festering hatred Ops folks have for their developer colleagues, nor is it part of a plot to deprive them of sleep. Rather, all of this, all of what DevOps represents is a realpolitik business response to our changing world and the changes they’ve forced on the software industry across the board. The fact is that everyone has new responsibilities, some of which require professionals (certainly many departments) to be at the ready to respond whenever duty calls because ours is a non-stop world. Another way of putting this is: Our old definitions of done are done! Our new definition is SRE, or Site Reliability Engineering. It’s a Google-coined term that forever weds Dev to Ops by bridging any lingering, perceived gaps between the two. And while SRE areas of focus may be taken up by personnel on either or both sides of the DevOps equation, these days companies will often have dedicated SRE teams that are specifically responsible for examining issues related to performance, efficiency, emergency responsiveness, monitoring, capacity planning, and more. SRE professionals think like software engineers in devising strategies and solutions for system administration issues. They’re the folks who are increasingly making automated deployments work. When SRE staff are happy, it means builds are becoming ever more reliable, repeatable, and fast, particularly because the landscape is one of scalable, backwards and forwards compatible code operating in stateless environments that are relying on an exploding universe of servers and emitting event streams to allow for real-time observability and alerts when something goes wrong. When there are new builds, they need to launch rapidly (with the full expectation that some will die equally quickly). Services need to return to full functionality as rapidly as possible. When features don’t work, there must be an immediate ability to turn them off programmatically through an API. When new software is released and users update their clients, but then encounter bugs, there must be the ability to execute swift and seamless rollbacks. Old clients and old servers need to be able to communicate with new clients. And while SRE is assessing and monitoring these activities and laying out strategic responses to them, the

Page 18

and monitoring these activities and laying out strategic responses to them, the work in all of these areas is completely that of developers. Therefore, while Dev staff are doing, it’s SRE that is today’s definition of done. Fly Like a Butterfly… In addition to all of the considerations above, there’s a fundamental characteristic that must define code in our modern DevOps (and related SRE) era: lean. And by this we’re talking about saving money. “But what,” you may ask, “does code have to do with saving money?” Well, one illustration might be cloud providers who will charge companies for a plethora of discrete services. Some of this is directly affected by the code being output by those corporate cloud service subscribers. Cost reductions can therefore come from the creation and use of innovative developer tools, as well as writing and deploying better code. The very nature of a global, we-never-close, software-driven society whose constant desire for newer, better software features and services means that DevOps can’t only be concerned with production and deployments. It must also be attentive to the bottom line of business itself. And although this may seem to be yet another burden thrown into the mix, think about it the next time the boss says that costs must be cut. Instead of negative, kneejerk solutions, such as employee layoffs or reductions in salaries and benefits, needed savings may be found in positive, business profile enhancing moves like going serverless and moving to the cloud. Then, no one gets fired and the coffee and donuts in the breakroom are still free! Being lean not only saves money, it gives companies the opportunity to improve their marketplace impact. When firms can achieve efficiencies without staff reductions, they can retain optimal levels of team strength. When teams continue to be well compensated and cared for, they’ll be more motivated to produce their best possible work. When that output is achieving success, it means customers are grinning. So long as customers continue to get well-functioning, new features quickly as a result of faster deployments, well… they’ll keep coming back for more and spreading the word to others. And that’s a virtuous cycle that means money in the bank. Integrity, Authentication, and Availability

Page 19

Integrity, Authentication, and Availability Hand in hand, and shoulder to shoulder with any and all DevOps activities is the perpetual question of security. Of course, some will choose to address this concern by hiring a Chief Information Security Officer. And that’s great because there’ll always be a go-to person to blame when something goes wrong. A better option might be to actually analyze, within a DevOps framework, how individual employees, work teams, and companies-as-a-whole think about security and how it can be strengthened. We are going to talk much more about this in Chapter 11, but for now consider the following: Breaches, bugs, SQL injections, buffer overflows, and more aren’t new. What’s different is the increasing speed of their appearance, their quantity, and the cleverness with which malicious individuals and entities are able to act. It’s not surprising. With more and more code being released, more and more problems will follow, with each type demanding different solutions. With faster deployments it becomes ever more critical to be less reactive to risks and threats. The discovery, in 2018, of the Meltdown and Spectre security vulnerabilities made it clear that some threats are imposible to prevent. We are in a race, and the only thing to do is to deploy fixes as quickly as possible. Fierce Urgency It should be clear to the reader by now that DevOps is not a plot, but a response to evolutionary pressures. It’s a means to ends that: Deliver better quality Produce savings Deploy features faster Strengthen security And it doesn’t matter who likes it or not, it doesn’t matter who came up with the idea first, it doesn’t even matter what was its original intent. What matters is that The software industry has fully embraced DevOps

Page 20

DevOps Bу now, every company is a DevOps company. So, get on board… because you don’t have any other choice. The DevOps of today, what DevOps has evolved into is, as stated earlier, an infinity loop. It doesn’t mean that groups and departments no longer exist. It doesn’t mean that everyone is responsible for their areas of concern along with those of everyone else along this continuum. It does mean that everyone should be working together. It does mean that software professionals within a given enterprise must be aware of and taking into reasonable consideration what all of their other colleagues are doing. They need to care about the issues their peers are confronting, how those matters can and do impact the work they do, the products and services their company offers, and how the totality of this affects their firm’s marketplace reputation. This is why DevOps Engineer is a term that makes no sense because it implies the existence of someone who can comprehensively and competently do (or is, at least, completely versed in) everything that occurs within the DevOps infinity loop. There is no such person. There never will be. In fact, even trying to do so is a mistake because it runs entirely counter to what DevOps is, which is eliminating silos where code developers are walled off from QA testers who are walled off from release personnel, and so on. It’s a coming together of efforts, interests, and feedback in a continuous effort to create, secure, deploy, and perfect code. DevOps is about collaboration. And as collaborations are organic, communicative endeavors, well… just as collaboration engineering isn’t a thing, neither is DevOps engineering (no matter what any institute or university might promise). Making It Manifest Knowing what DevOps is (and isn’t) only establishes a concept. The question is how can it be sensibly and effectively implemented and sustained in software firms far and wide? Best advice? Here goes: You can have DevOps enablers, DevOps evangelists, DevOps consultants and coaches (and I know how Scrum spoiled all those terms for all of us, but there aren’t any better ones). That’s okay. But DevOps 6

Tip the Site

DevOps Tools for Java Developers (Early Release) (Stephen Chin, Melissa McKay, Ixchel Ruiz etc.) (z-library.sk, 1lib.sk, z-lib.sk)

DevOps Tools for Java Developers (Early Release) (Stephen Chin, Melissa McKay, Ixchel Ruiz etc.) (z-library.sk, 1lib.sk, z-lib.sk)

Registered users can read the full content for free

AI Reading Assistant

Passage locations

Recommended for You

Tip the Site