Docker in Action (Jeff Nickoloff, Stephen Kuenzli) (Z-Library)

📄 File Format: PDF

💾 File Size: 3.2 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

M A N N I N G Jeff Nickoloff Stephen Kuenzli FOREWORD BY Bret Fisher SECOND EDITION IN ACTION

📄 Page 2

Docker running three containers on a Linux system Network interface Memory Operating system User space CPU Persistent storage IO Devices Docker daemon Docker CLICommand line Database Container space C Hello World Web server Container space B Container space A

📄 Page 3

Praise for the first edition “All there is to know about Docker. Clear, complete, and precise.” —Jean-Pol Landrain, Agile Partner Luxembourg “A compelling narrative for real-world Docker solutions. A must-read!” —John Guthrie, Pivotal, Inc. “An indispensable guide to understanding Docker and how it fits into your infrastructure.” —Jeremy Gailor, Gracenote “Will help you transition quickly to effective Docker use in complex real-world situations.” —Peter Sellars, Fraedom “. . . a superlative introduction to, and reference for, the Docker ecosystem.” —Amazon reader

📄 Page 4

(This page has no text content)

📄 Page 5

Docker in Action SECOND EDITION JEFF NICKOLOFF STEPHEN KUENZLI FOREWORD BY BRET FISHER M A N N I N G SHELTER ISLAND

📄 Page 6

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2019 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Development editor: Jennifer StoutManning Publications Co. 20 Baldwin Road Technical development editor: Raphael Villela PO Box 761 Review editor: Aleksandar Dragosavljević Shelter Island, NY 11964 Project editor: Janet Vail Copy editor: Sharon Wilkey Proofreader: Keri Hales Technical proofreader: Niek Palm Typesetter: Dennis Dalinnik Cover designer: Marija Tudor ISBN: 9781617294761 Printed in the United States of America

📄 Page 7

For Jarrod Nickoloff and William Kuenzli

📄 Page 8

(This page has no text content)

📄 Page 9

contents foreword xiii preface xv acknowledgments xviii about this book xx about the authors xxii about the cover illustration xxiii 1 Welcome to Docker 1 1.1 What is Docker? 3 “Hello, World” 3 ■ Containers 5 ■ Containers are not virtualization 5 ■ Running software in containers for isolation 6 ■ Shipping containers 7 1.2 What problems does Docker solve? 8 Getting organized 9 ■ Improving portability 10 Protecting your computer 11 1.3 Why is Docker important? 12 1.4 Where and when to use Docker 13 1.5 Docker in the larger ecosystem 14 1.6 Getting help with the Docker command line 14vii

📄 Page 10

CONTENTSviiiPART P1 ROCESS ISOLATION AND ENVIRONMENT- INDEPENDENT COMPUTING ................................17 2 Running software in containers 19 2.1 Controlling containers: Building a website monitor 20 Creating and starting a new container 21 ■ Running interactive containers 22 ■ Listing, stopping, restarting, and viewing output of containers 23 2.2 Solved problems and the PID namespace 25 2.3 Eliminating metaconflicts: Building a website farm 28 Flexible container identification 28 ■ Container state and dependencies 31 2.4 Building environment-agnostic systems 34 Read-only filesystems 34 ■ Environment variable injection 37 2.5 Building durable containers 40 Automatically restarting containers 41 ■ Using PID 1 and init systems 42 2.6 Cleaning up 44 3 Software installation simplified 47 3.1 Identifying software 48 What is a named repository? 48 ■ Using tags 49 3.2 Finding and installing software 50 Working with Docker registries from the command line 50 Using alternative registries 51 ■ Working with images as files 52 Installing from a Dockerfile 53 ■ Using Docker Hub from the website 54 3.3 Installation files and isolation 56 Image layers in action 57 ■ Layer relationships 58 Container filesystem abstraction and isolation 59 Benefits of this toolset and filesystem structure 60 Weaknesses of union filesystems 60 4 Working with storage and volumes 62 4.1 File trees and mount points 63 4.2 Bind mounts 64 4.3 In-memory storage 67

📄 Page 11

CONTENTS ix4.4 Docker volumes 68 Volumes provide container-independent data management 70 Using volumes with a NoSQL database 71 4.5 Shared mount points and sharing files 73 Anonymous volumes and the volumes-from flag 74 4.6 Cleaning up volumes 77 4.7 Advanced storage with volume plugins 78 5 Single-host networking 80 5.1 Networking background (for beginners) 81 Basics: Protocols, interfaces, and ports 81 ■ Bigger picture: Networks, NAT, and port forwarding 82 5.2 Docker container networking 83 Creating a user-defined bridge network 84 ■ Exploring a bridge network 86 ■ Beyond bridge networks 88 5.3 Special container networks: host and none 89 5.4 Handling inbound traffic with NodePort publishing 91 5.5 Container networking caveats and customizations 93 No firewalls or network policies 93 ■ Custom DNS configuration 93 ■ Externalizing network management 97 6 Limiting risk with resource controls 99 6.1 Setting resource allowances 100 Memory limits 101 ■ CPU 102 ■ Access to devices 105 6.2 Sharing memory 105 Sharing IPC primitives between containers 106 6.3 Understanding users 107 Working with the run-as user 108 ■ Users and volumes 111 Introduction to the Linux user namespace and UID remapping 113 6.4 Adjusting OS feature access with capabilities 114 6.5 Running a container with full privileges 116 6.6 Strengthening containers with enhanced tools 117 Specifying additional security options 118 6.7 Building use-case-appropriate containers 119 Applications 119 ■ High-level system services 120 Low-level system services 120

📄 Page 12

CONTENTSx PART 2 PACKAGING SOFTWARE FOR DISTRIBUTION.......123 7 Packaging software in images 125 7.1 Building Docker images from a container 126 Packaging “Hello, World” 126 ■ Preparing packaging for Git 127 ■ Reviewing filesystem changes 128 ■ Committing a new image 129 ■ Configuring image attributes 130 7.2 Going deep on Docker images and layers 131 Exploring union filesystems 131 ■ Reintroducing images, layers, repositories, and tags 134 ■ Managing image size and layer limits 137 7.3 Exporting and importing flat filesystems 139 7.4 Versioning best practices 141 8 Building images automatically with Dockerfiles 144 8.1 Packaging Git with a Dockerfile 145 8.2 A Dockerfile primer 148 Metadata instructions 149 ■ Filesystem instructions 153 8.3 Injecting downstream build-time behavior 156 8.4 Creating maintainable Dockerfiles 159 8.5 Using startup scripts and multiprocess containers 162 Environmental preconditions validation 163 ■ Initialization processes 164 ■ The purpose and use of health checks 166 8.6 Building hardened application images 167 Content-addressable image identifiers 168 ■ User permissions 169 ■ SUID and SGID permissions 171 9 Public and private software distribution 174 9.1 Choosing a distribution method 175 A distribution spectrum 175 ■ Selection criteria 176 9.2 Publishing with hosted registries 178 Publishing with public repositories: “Hello World!” via Docker Hub 179 ■ Private hosted repositories 181 9.3 Introducing private registries 183 Using the registry image 186 ■ Consuming images from your registry 187 9.4 Manual image publishing and distribution 188 A sample distribution infrastructure using FTP 190 9.5 Image source-distribution workflows 194 Distributing a project with Dockerfile on GitHub 194

📄 Page 13

CONTENTS xi10 Image pipelines 197 10.1 Goals of an image build pipeline 198 10.2 Patterns for building images 199 All-in-one images 200 ■ Separate build and runtime images 201 ■ Variations of runtime image via multi-stage builds 202 10.3 Record metadata at image build time 204 Orchestrating the build with make 205 10.4 Testing images in a build pipeline 209 10.5 Patterns for tagging images 212 Background 212 ■ Continuous delivery with unique tags 213 Configuration image per deployment stage 214 ■ Semantic versioning 215 PART 3 HIGHER-LEVEL ABSTRACTIONS AND ORCHESTRATION......................................217 11 Services with Docker and Compose 219 11.1 A service “Hello World!” 220 Automated resurrection and replication 222 ■ Automated rollout 224 ■ Service health and rollback 226 11.2 Declarative service environments with Compose V3 229 A YAML primer 231 ■ Collections of services with Compose V3 233 11.3 Stateful services and preserving data 237 11.4 Load balancing, service discovery, and networks with Compose 239 12 First-class configuration abstractions 244 12.1 Configuration distribution and management 245 12.2 Separating application and configuration 247 Working with the config resource 249 ■ Deploying the application 250 ■ Managing config resources directly 251 12.3 Secrets—A special kind of configuration 255 Using Docker secrets 257

📄 Page 14

CONTENTSxii13 Orchestrating services on a cluster of Docker hosts with Swarm 264 13.1 Clustering with Docker Swarm 264 Introducing Docker Swarm mode 265 ■ Deploying a Swarm cluster 267 13.2 Deploying an application to a Swarm cluster 267 Introducing Docker Swarm cluster resource types 267 Defining an application and its dependencies by using Docker services 268 ■ Deploying the application 273 13.3 Communicating with services running on a Swarm cluster 278 Routing client requests to services by using the Swarm routing mesh 278 ■ Working with overlay networks 281 ■ Discovering services on an overlay network 282 ■ Isolating service-to-service communication with overlay networks 284 ■ Load balancing 286 13.4 Placing service tasks on the cluster 287 Replicating services 288 ■ Constraining where tasks run 292 Using global services for one task per node 297 ■ Deploying real applications onto real clusters 299 301index

📄 Page 15

foreword Welcome to the container revolution. By reading this book, you’re opening your eyes to a new world of tools that are forever changing the way we build, deploy, and run software. Once I discovered Docker in 2014 (the year after it was open-sourced) I did something I had never done in my 20+ year career: I decided to focus exclusively on this single technology. That’s how much I believed in what Docker was doing to make our ever-increasing IT world easier to manage. Fast forward to today, and what’s still unique about Docker’s way of creating and deploying containers is that it has both developers and operators in mind. You can see this in the user-experience of its command-line tools, and with hundreds of tools in the container ecosystem, I keep coming back to Docker as the easiest and smoothest way to get things done. Jeff and Stephen know this too about Docker’s streamlined approach to contain- ers, which is why this book focuses on the details of the core tools. Docker Engine, Docker Compose, and Docker Swarm are key tools we should all know. They often solve your problems without the need for more complex solutions. This same method- ology is how I teach my students and how I guide my clients. Containers couldn’t have come at a better time, taking features of the Linux ker- nel (and now Windows, ARM, and more) and automating them into accessible one- line commands. Sure, we had container-like features for years in Solaris, FreeBSD, and then Linux, but it was only the bravest sysadmins who got those features to work before Docker.xiii

📄 Page 16

FOREWORDxiv Containers today are now more than the sum of their parts. The workflow speed and agility that a fully Dockerized software lifecycle gives a team cannot be under- stated. I’m glad Jeff and Stephen took their battle-hardened experience and updated this already great book with new details and examples, and I’m confident you’ll gain benefits by putting their recommendations into practice. —BRET FISHER, DOCKER CAPTAIN AND CONTAINER CONSULTANT bretfisher.com twitter.com/bretfisher

📄 Page 17

preface Docker and the container community have come a long way since we started partici- pating in 2013. And Docker has changed in some unexpected ways since 2016, when Jeff released the first edition of this book. Thankfully, most of the user-facing inter- faces and core concepts were maintained in a backward-compatible manner. The first two-thirds of the book needed updates only for additional features or closed issues. As anticipated, part 3 of the previous edition needed a full rewrite. Since publication of the previous book, we’ve seen progress in orchestration, app connectivity, proprietary cloud container offerings, multicontainer app packaging, and function-as-a-service platforms. This edition focuses on the fundamental concepts and practices for using Docker containers and steers clear of rapidly changing technologies that comple- ment Docker. The biggest change is the development and adoption of several container orchestra- tors. The primary purpose of a container orchestrator is to run applications modeled as services across a cluster of hosts. Kubernetes, the most famous of these orchestrators, has seen significant adoption and gained support from every major technology vendor. The Cloud Native Computing Foundation was formed around that project, and if you ask them, a “cloud native” app is one designed for deployment on Kubernetes. But it is important not to get too caught up in the marketing or the specific orchestration tech- nology. This book does not cover Kubernetes for two reasons. While Kubernetes is included with Docker for Desktop, it is massive and in con- stant flux. It could never be covered at any depth in a handful of chapters or even in a book with fewer than 400 pages. A wealth of excellent resources are availablexv

📄 Page 18

PREFACExvionline as well as wonderful published books on Kubernetes. We wanted to focus on the big idea—service orchestration—in this book without getting too lost in the nuances. Second, Docker ships with Swarm clustering and orchestration included. That sys- tem is more than adequate for smaller clusters, or clusters in edge computing environ- ments. A huge number of organizations are happily using Swarm every day. Swarm is great for people getting started with orchestration and containers at the same time. Most of the tooling and ideas carry over from containers to services with ease. Applica- tion developers will likely benefit the most from this approach. System administrators or cluster operations personnel might be disappointed, or might find that Swarm meets their needs. But, we’re not sure they’ll ever find a long-form written resource that will satisfy their needs. The next biggest change is that Docker runs everywhere today. Docker for Desktop is well integrated for use on Apple and Microsoft operating systems. It hides the underlying virtual machine from users. For the most part, this is a success; on macOS, the experience is nearly seamless. On Windows, things seem to go well at least for a few moments. Windows users will deal with an intimidating number of configuration variations from corporate firewalls, aggressive antivirus configuration, shell prefer- ences, and several layers of indirection. That variation makes delivering written con- tent for Windows impossible. Any attempt to do so would age out before the material went to production. For that reason, we’ve again limited the included syntax and system- specific material to Linux and macOS. A reader just might find that all the examples actually run in their environment, but we can’t promise that they will or reasonably help guide troubleshooting efforts. Next, getting an internet-attached virtual machine with Docker installed has become trivial. Every major and minor cloud provider offers as much. For that reason, we’ve removed material pertaining to Docker Machine and installing Docker. We’re confident that our readers will be able to find installation instructions that are most appropriate for the platform of their choice. And today, they might even skip that step and adopt one of the many container-first cloud platforms like AWS ECS. This book won’t cover those platforms. They’re each unique enough to be difficult to discuss in aggregate. And all of them have put significant effort into their adoption stories and documentation. Finally, containers and networking have had a complicated history. In the last few years, that story became just a little bit more complicated with the emergence of service mesh platforms and other complementary technologies. A service mesh is a platform of application-aware smart pipes that provide microservice networking best practices out of the box. They use proxies to provide point-to-point encryption, authentication, authorization, circuit-breakers, and advanced request routing. The container net- working fundamentals presented in this book should prove useful in understanding and evaluating service mesh technologies.

📄 Page 19

PREFACE xvii This book is intended as a deep introduction to the fundamentals of working with Docker. A reader might not learn everything that they need in their daily application of this technology. But they will have the fundamental skillset required to learn advanced topics more quickly and further those pursuits. We wish you the best of luck in those containerized ventures.

📄 Page 20

acknowledgments We would like to thank Manning Publications for the opportunity to write this book; the generous help from our editors, particularly Jennifer Stout; and feedback from all of our reviewers: Andy Wiesendanger, Borko Djurkovic, Carlos Curotto, Casey Burnett, Chris Phillips, Christian Kreutzer-Beck, Christopher Phillips, David Knepprath, Dennis Reil, Des Horsley, Ernesto Cárdenas Cangahuala, Ethan Rivett, Georgios Doumas, Gerd Klevesaat, Giuseppe Caruso, Kelly E. Hair, Paul Brown, Reka Horvath, Richard Lebel, Robert Koch, Tim Gallagher, Wendell Beckwith, and Yan Guo. You all helped make this a better book. Jeff Nickoloff: A second edition is a burden and an opportunity. It is the same burden any SaaS owner feels. People are consuming your work, and, ultimately, you’re in some small part responsible for their success or failure. I took on this work knowing that it needed to be done, but also that I would struggle without a coauthor. It is an opportunity to continue sharing what I know with the world, but more importantly an opportunity to introduce and share Stephen Kuenzli’s knowledge. He and I have had several opportunities to work together in Phoenix, including co-organizing DevOps- Days, running the Docker PHX meetup, and bouncing a constant stream of ideas off each other. Since 2013, I’ve watched and helped countless people and teams work through their container and cloud adoption stories. I learn something new from each encoun- ter, and it is safe to say that I would not be where I am today if it were not for their will- ingness to include me. xviii