Security as Code DevSecOps Patterns with AWS (BK Sarthak Das, Virginia Chu) (Z-Library)

📄 File Format: PDF

💾 File Size: 3.1 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Security a s C od e Security a s C od e BK Sarthak Das & Virginia Chu Security as Code DevSecOps Patterns with AWS

📄 Page 2

SECURIT Y “An excellent guide. Security as Code takes you from abstract concept to the working technology, people, and processes. If you need to actually do the work of shifting security left, this book is for you.” —Fritz Kunstler Principal, AWS Global Services Security “The ultimate hands-on security guide for DevOps roles, covering tooling and processes.” —Michael Hausenblas Solution Engineering Lead, AWS Security as Code Twitter: @oreillymedia linkedin.com/company/oreilly-media youtube.com/oreillymedia DevOps engineers, developers, and security engineers have ever-changing roles to play in today’s cloud native world. In order to build secure and resilient applications, you have to be equipped with security knowledge. Enter security as code. In this book, authors BK Sarthak Das and Virginia Chu demonstrate how to use this methodology to secure any application and infrastructure you want to deploy. With Security as Code, you’ll learn how to create a secure environment using CI/CD tooling from AWS and open source providers. You’ll also see how a containerized application can be deployed as infrastructure as code (IaC) within AWS. This practical guide also provides common patterns and methods to develop secure and resilient infrastructure. • Learn the tools of the trade using Kubernetes and the AWS Code Suite • Set up IaC and run scans to detect misconfigured resources in your code • Create secure logging patterns with CloudWatch and other tools • Restrict system access to authorized users with role-based access control (RBAC) • Inject faults to test the resiliency of your application with AWS Fault Injection Simulator or open source tooling • Learn how to pull everything together into one deployment BK Sarthak Das works at Google as a security engineer and was previously at AWS as a senior security architect. Virginia Chu is a principal DevSecOps engineer at AWS who began her career as a Linux system administrator and developer. US $55.99 CAN $69.99 ISBN: 978-1-098-12746-6

📄 Page 3

BK Sarthak Das and Virginia Chu Security as Code DevSecOps Patterns with AWS Boston Farnham Sebastopol TokyoBeijing

📄 Page 4

978-1-098-12746-6 [LSI] Security as Code by BK Sarthak Das and Virginia Chu Copyright © 2023 Virginia Chu and BK Sarthak Das. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editors: Jennifer Pollock, Simina Calin Development Editor: Sarah Grey Production Editor: Clare Laylock Copyeditor: Nicole Taché Proofreader: Rachel Head Indexer: Sam Arnold-Boyd Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea January 2023: First Edition Revision History for the First Edition 2023-01-03: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098127466 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Security as Code, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and NGINX. See our statement of editorial independence.

📄 Page 5

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii 1. Introduction to DevSecOps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Before DevOps: The Software Development Life Cycle 3 What Is DevSecOps? 6 Introducing Automatoonz 7 Cloud Infrastructure: Secure by Default 8 Move Fast, Secure Fast: The Importance of Automation 9 DevSecOps Culture 10 Summary 11 2. Setting Up Your Environment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 What You’ll Need 13 Installing and Verifying Your Setup 14 Installing the AWS CLI 14 Installing the Docker Engine 15 Checking Your Python Version 15 Installing Git 16 Installing Kubernetes 16 Creating Your First Bare-Bones Pipeline 17 Summary 18 3. Securing Your Infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 What Makes Infrastructure Secure? 21 Hands Off! Preventing Unwanted Access with IAM Permissions 22 Detecting Misconfigurations 23 Identifying a Standard 23 Threat Modeling 24 iii

📄 Page 6

Security Controls 24 Better Than a Cure: Implementing Preventive Controls 26 Implementation 28 Summary 33 4. Logging and Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 What Are Logging and Monitoring—and Why Do They Matter? 35 Attack Styles 36 Advanced Persistent Threat Attacks 37 Ransomware Strains 38 Passive and Active Attacks 38 Log Types 40 Log Storage 41 Detecting Anomalies 44 Remediation with AWS Config 48 Correlating User Activity with CloudTrail 51 Network Monitoring with an Amazon VPC 53 Summary 55 5. Controlling Access Through Automation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 The Principle of Least Privilege 58 Fine-Tuning Access Controls 60 Use a Tagging System 60 Clarify Team Responsibilities 61 Prevent and Detect 62 The IAM Pipeline 63 Summary 66 6. Fault Injection Test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Distributed Systems 67 Adaptive Security Controls 68 The True Cost of Downtime 69 Methods for Minimizing Downtime 69 Chaos Engineering 70 Basic Principles 71 Advanced Principles 74 Chaos Engineering in AWS Environments 76 Chaos Engineering at Automatoonz 78 AWS Fault Injection Simulator Experiment Examples 78 Kubernetes Pod Stress Testing 79 Throttling EC2 API Calls 80 Stress Testing the CPU on an EC2 Instance 81 iv | Table of Contents

📄 Page 7

Terminating an EC2 Instance 82 Removing Ingress and Egress Rules from a Security Group 83 Detaching an EBS Volume from an EC2 Instance 84 Summary 85 7. People and Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 People: Team Structures and Roles 87 Security Engineers 88 Developers 88 Compliance Team 89 Product Manager 90 Team Structure 90 Processes: Practices and Communication 92 Communicate to the Right People, Consistently 92 Make Product Owners Accountable for Their Security Findings 93 Build Threat Modeling into Your Processes 93 Build Roadmaps to Reach Your DevSecOps Goals 95 What Next? 95 Summary 96 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Table of Contents | v

📄 Page 8

(This page has no text content)

📄 Page 9

Preface The authors of this book work with enterprise AWS customers who have business- critical applications running in the cloud, so we think about security on a daily basis. In recent years, we’ve noticed that the term DevSecOps pops up in nearly every security strategy discussion. Everyone wants it, but not as many people understand it—and it seems like almost nobody knows where to start or what to do. DevSecOps is a relatively new field, and few books are available to guide those who want to learn more about it. We decided to write this book to help fill that gap by showing you how and where to get started on DevSecOps in AWS. This book is not an enterprise-grade solution kit for copying and pasting into pro‐ duction (and since every project and organization has different needs, we sincerely hope you would never do that!). Instead, it’s designed to introduce you to the building blocks of the DevSecOps mindset, and to guide you along the way with practical examples. We use popular open source tools where possible, to show you that it’s not always necessary to buy expensive products to do security the right way. We use a fictitious company called Automatoonz to illustrate some of the real-world issues you’re likely to face in your DevSecOps journey. As we discuss a problem, the Automatoonz team works on it too, giving you a sense of how real teams approach solving the problem at hand. Although the scenarios are fictionalized, these examples come from our extensive personal experience, and we think they’ll resonate with you. The solutions we provide in this book are intended as guidance on the art of the possible. Who Is This Book For? This book is for AWS security engineers, DevOps engineers, security analysts, secu‐ rity engineering managers, and other practitioners and leaders at intermediate and senior levels who want to automate more of their security. We recommend that readers have some practical AWS development knowledge and familiarity with Git vii

📄 Page 10

before starting this book: ideally, enough to do basic coding and debugging within AWS. In Chapter 2, for example, we use CloudFormation, Python, and Kubernetes to demonstrate Infrastructure as Code. You should also be comfortable navigating Git repositories. What Do You Need To Get Started? In practical terms, aside from intermediate knowledge of AWS, to follow the exercises in this book you will need an AWS account where you can deploy. You will also need to install the following, if you do not already have them: • AWS Command Line Interface (AWS CLI) (latest version) • Access to an AWS account • Docker (Community Edition) • Python (version 3.x.x or higher) • Git (latest version) • Kubectl (latest version) • Kubernetes (version 1.21 or higher) Chapter 2 has a detailed walkthrough of setting up all these tools. You will also need access to the book’s GitHub repository, which includes code samples and other supplemental materials. What’s in This Book? We’ve tried to ensure that the seven chapters in this book are as independent as possible from one another, so that you can pick it up at any point. However, we recommend that you start from the beginning. Chapter 1 will introduce you to what DevSecOps is, why it is important, and what kind of mindset you’ll need to get started. Chapter 2 helps you install the software you’ll need for the rest of the book, then walks you through a sample application built with secure configurations to ensure you have your toolkit working. In Chapter 3, you’ll learn how to validate Infrastructure as Code to make your resources secure. Chapter 4 looks at how to set up appropriate logging and monitoring to identify and debug issues with your infrastructure. In Chapter 5, you’ll learn about controlling access through automation, including assessing your organization’s identity and access management (IAM) policies and refining them according to the principle of least privilege. Chapter 6 is all about testing: we’ll introduce you to the practice of Chaos Engineering, show you how viii | Preface

📄 Page 11

to use it to make your infrastructure more resilient, and discuss how to focus on possible points of failure. Finally, in Chapter 7, we wrap up with a look at the roles and processes that should be part of any DevSecOps team. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion. This element signifies a general note. This element indicates a warning or caution. Using Code Examples Code examples are available for download at https://oreil.ly/SaCgh. If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com. Preface | ix

📄 Page 12

This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Security as Code: Dev‐ SecOps Patterns with AWS by BK Sarthak Das and Virginia Chu (O’Reilly). Copyright 2023 Virginia Chu and BK Sarthak Das, 978-1-098-12746-6.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided technol‐ ogy and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/SecurityAsCode. x | Preface

📄 Page 13

Email bookquestions@oreilly.com to comment or ask technical questions about this book. For news and information about our books and courses, visit https://oreilly.com. Find us on LinkedIn: https://linkedin.com/company/oreilly-media. Follow us on Twitter: https://twitter.com/oreillymedia. Watch us on YouTube: https://youtube.com/oreillymedia. Acknowledgments There are numerous people who are responsible for this book. We would like to take the time to thank our amazing and wonderful team at O’Reilly Media (Sarah Grey, development editor; Nicole Taché, freelance editor; and Clare Laylock, production editor) for their professionalism, commitment, and guidance in publishing a book. We learned that writing a book is not easy or trivial. Our grateful and humble thanks go to our technical reviewer Joe Milligan. It has been an honor to collaborate with him. BK would like to thank his close friends and family for being supportive throughout the process of writing this book. Virginia would like to thank her family, four-legged sounding board, and close friends for their support and patience throughout this adventure. Special thanks to her mentor Michael Hausenblas for inspiring her to work on this book. Finally, we would like to thank all the wonderful contributors and developers for the tools we have used in our book. Preface | xi

📄 Page 14

(This page has no text content)

📄 Page 15

CHAPTER 1 Introduction to DevSecOps We did not start our careers in security, but we both know that delivering software is of utmost importance to developers. “Delivering software” in this context means delivering something that does what it is supposed to do. This could refer to the code being stable, or the software meeting the functional requirements (for example, a calculator can add, multiply, subtract, and divide numbers) and performance expect‐ ations without any issues (for example, a chat application allows more than 10 people to send messages to each other simultaneously). Building quality software, however, requires good coding practices, resilient architec‐ tures, and security. It’s common for security to be added into the software after it has been built, but the shift-left approach recommends moving security to much earlier in the development life cycle, building it in from the start. We will discuss that approach in this chapter, focusing on Security as Code (SaC). In this approach, our infrastructure’s security policies—detection, prevention, remediation—are all defined as code. We will also discuss DevSecOps in this chapter, focusing on the three major players in software development: Development Is your code doing what it is meant to do? Operations What is this code running on? Do you have the required skills/time to maintain this going forward? Can the provisioned infrastructure handle the expected workload? Testing Can the code survive unexpected use cases? How does the code respond to something you didn’t account for? 1

📄 Page 16

The primary focus of this book is how to integrate security into your development process through cloud infrastructure. Declaring infrastructure in files is also known as Infrastructure as Code (IaC). Kief Morris provides a helpful definition in his book Infrastructure as Code, 2nd edition (O’Reilly): Infrastructure as Code is an approach to infrastructure automation based on practices from software development. It emphasizes consistent, repeatable routines for provi‐ sioning and changing systems and their configuration. You make changes to code, then use automation to test and apply those changes to your systems. Using IaC and declaring the security aspects of that infrastructure is Security as Code (SaC). SaC is not entirely different from IaC, but rather focuses on enabling security controls using templates. This chapter introduces the basic concepts of SaC, and Chapter 2 provides setup and instructions to get you started. After that, the chapters are organized by security domains. Some chapters are exclusive to a single domain; others address multiple domains in a single stage of the buildout. Each domain has its own unique set of questions. For instance: Data protection (Chapter 3) Is everything encrypted? Does the encryption approach follow data classification policies? Do we have data classification implemented? Infrastructure security (Chapter 3) As we are running our application in the cloud, is all the infrastructure deployed securely? Is our S3 bucket publicly accessible when it should not be? Can we prevent deployment of our resources when something is missing? Application security (Chapter 3) Is the code we’re running secure? How many active vulnerabilities are there in our code? Did we release code with a critical vulnerability? Logging and monitoring (Chapter 4) Do we know what to monitor? When does an observation become an anomaly? Are the right application security indicators in place to inform us of any mishap? Identity and access management (Chapter 5) Who has access to what? Does anyone have any elevated privileges? Can someone elevate their own or others’ privileges? Incident response (Chapter 6) How do we react to incidents? Can we replace part of the offending application when something goes wrong? 2 | Chapter 1: Introduction to DevSecOps

📄 Page 17

Throughout the chapters, we will use Amazon Web Services (AWS) native security services and best practices to baseline the environment we are deploying. When we assume certain operational and team structures, we explicitly call out those assumptions. Before DevOps: The Software Development Life Cycle When you are new in your career, writing a piece of code that does exactly what was asked is a joy of its own. With time, however, we’ve realized writing quality code does not stop at making 2 + 2 = 4. What if the user enters “2 + a”? How should your software behave? Well, that’s the responsibility of the Quality and Testing engineers, isn’t it? Wrong. We’ve seen back-and-forth between developers and testers that was time-consuming and created unhealthy expectations from both teams. Let’s take the example of the calculator input of “2 + a”. If, as a coder, you did not think of this use case because it was not part of the requirements, and your tester or QA team did not record it as a test case, you would be shipping broken code to your customers. This broken code would be your final product. A codebase that doesn’t work as expected is not a joy for the end user to work with. Code needs to be hosted on some infrastructure to be built and deployed. Is it going to run on a server in your datacenter, on a virtual machine in the cloud, on a container, or on someone’s laptop? Depending on your answer, you have another set of responsibilities related to infrastructure. Once you set up your infrastructure, you will need to answer a new series of questions, including: • Are the coders developing this on the exact same platform on which it will be deployed? • Who is setting up all the infrastructure? • Does the infrastructure fail open or fail safe, in the case of an error? That set of questions needs another set of tests to make sure that the application code is being run correctly on the right platform, and that the platform is not misbehaving. In traditional software development, only developers take care of development, which means their prime directive is writing code to a specification. The operations team handles the environment and method of deployment, and the change management procedures. Testers take the output of the developers and the operations team and make sure the near-final product does not break. These three roles are not mutually exclusive, but each team needs to wait for input from the previous team to start its work. Before DevOps: The Software Development Life Cycle | 3

📄 Page 18

A very common representation of this flow is the Software Development Life Cycle (SDLC) model (see Figure 1-1). In practice, the waterfall model of the SDLC might operate something like this: developers create code quickly, based on functional requirements. The code is sent for testing, errors are found, and the code is sent back to developers. The developers fix the code and send it for another round of testing. Once the testing is complete, the code is handed off to the operations team for maintenance. Figure 1-1. Software Development Life Cycle: waterfall model (https://oreil.ly/KtVv0) Siloed teams operating in a hands-off style, similar to the SDLC in Figure 1-1, have their disadvantages. Each team has its own toolset and handles a very specific piece of the SDLC, and is typically unaware of the toolsets of the other teams. This makes it difficult to get quality software out the door on time. The waterfall model leads to a lot of back-and-forth between teams, as we’ve noted. The back-and-forth is made worse when you have not delivered anything because your code has not passed your testing teams. So, a whole lot of time and effort is lost without producing any tangible outcomes. Enter DevOps. In order to reduce time to market and improve the quality of software, the concept of DevOps was introduced. In their book Effective DevOps (O’Reilly), authors Jennifer Davis and Ryn Daniels define DevOps as: 4 | Chapter 1: Introduction to DevSecOps

📄 Page 19

A cultural movement that changes how individuals think about their work, values the diversity of work done, supports intentional processes that accelerate the rate by which businesses realize value, and measures the effect of social and technical change. It is a way of thinking and a way of working that enables individuals and organizations to develop and maintain sustainable work practices. It is a cultural framework for sharing stories and developing empathy, enabling people and teams to practice their crafts in effective and lasting ways. In a DevOps model, the development, testing, and operations teams don’t work in silos, but are the same group. If we were to visualize the DevOps model, it would look like a homogeneous blob of roles. Kief Morris writes in Infrastructure as Code that the goal of DevOps is: To reduce barriers and friction between organizational silos—development, opera‐ tions, and other stakeholders involved in planning, building, and running software. Although technology is the most visible, and in some ways simplest face of DevOps, it’s culture, people, and processes that have the most impact on flow and effectiveness. Technology and engineering practices like Infrastructure as Code should be used to support efforts to bridge gaps and improve collaboration. We want to emphasize this point: DevOps is not solely enabled by technology. It is effective only when people, processes, and technology are working together. There is a common misconception that if you use tools that are used in CI/CD systems, you’re automatically practicing DevOps. This is flawed thinking. What enables DevOps is collaboration. Recommendations for Further Reading The basics of DevOps, modern architectures, and application security are all outside the scope of this book, but we recommend the following references if you want to learn more: DevOps • Effective DevOps by Jennifer Davis and Ryn Daniels (O’Reilly) • Understanding Agile DevOps by Jim Bird (O’Reilly) • DevOps Tools for Java Developers by Stephen Chin, Melissa McKay, Ixchel Ruiz, and Baruch Sadogursky (O’Reilly) Modern architectures • Building Microservices, 2nd edition, by Sam Newman (O’Reilly) • Building Event-Driven Microservices by Adam Bellemare (O’Reilly) • Fundamentals of Software Architecture by Mark Richards and Neal Ford (O’Reilly) Before DevOps: The Software Development Life Cycle | 5

📄 Page 20

Application security • Shifting Left for Application Security by Peter Conrad (O’Reilly) • Agile Application Security by Laura Bell, Michael Brunton-Spall, Rich Smith, and Jim Bird (O’Reilly) What Is DevSecOps? DevSecOps is not a “new version” of DevOps, but rather a conscious effort to add security into your DevOps framework. Like with DevOps, there are numerous defi‐ nitions of and approaches to DevSecOps. For the purposes of this book, we define DevSecOps as the ability to implement and deploy SaC in software. We will be leaning heavily on APIs, cloud services, and other open source projects to implement SaC. When a part of your SDLC becomes “as code,” your team should have the openness to build things. As your organization begins to implement DevSecOps, there are two important things your team will need to outline: tools and security guidelines. First, how will you write your IaC? In other words, what tools will you use? This could be a tool like AWS CloudFormation or Terraform. There are numerous services and products available from vendors and the open source community that you can use to build and integrate SaC into your pipeline. As we mentioned, this book will use AWS and open source projects to demonstrate the why and how of doing DevSecOps, instead of fixating on a particular tool’s licensing or procurement. We chose to focus on AWS in this book since it is currently the most popular cloud infrastructure vendor, controlling 33% of the market. However, the book’s underlying principles will be useful to all readers. Second, what are your company’s security “rules of the road”? What has your security team designated as “definitely don’t do this” rules? Understanding why the security team provides certain guidance helps you understand the concerns underlying the rules. In DevSecOps, you are building security directly into your software development pipeline (see Figure 1-2). In step 1, the developer lints their code locally and makes sure its formatting follows the team’s conventions and standards. They then commit the code to the repository. In step 2, the build system of the pipeline scans for errors and any other vulnerabilities and misconfigurations. The security of the pipeline is built into this stage. Finally, in step 3, the code is (possibly) ready for deployment. The last hurdle is a decision gate—a mechanism that checks for errors. If any errors are found, deployment is canceled and the developer is informed. If the code has no errors, the deployment goes through. 6 | Chapter 1: Introduction to DevSecOps