Linux in Action (David Clinton) (z-library.sk, 1lib.sk, z-lib.sk)

M A N N I N G David Clinton

Linux in Action topics Chapter Skill domains Tools 1 Welcome to Linux Shells, partitions, and file systems Bash, man 2 Linux virtualization: Building a safe and simple Linux working environment Virtualization, file systems VirtualBox, LXC, apt, yum/dnf 3 Remote connectivity: Safely accessing networked machines Security, remote connectivity ssh, scp, systemctl, ps, grep 4 Archive management: Backing up or copying entire file systems Partitions and file systems, text streams tar, dd, redirects, rsync, locate, split, chmod, chown 5 Automated administration: Configuring automated offsite backups Scripts, system process management, security Scripts, cron, anacron, systemd timers 6 Emergency tools: Building a system recovery device Partitions and file systems, device management parted, GRUB, mount, chroot 7 Web servers: Building a MediaWiki server Databases, networking, package management PHP, MySQL (MariaDB), Apache web server, package dependencies 8 Networked file sharing: Building a Nextcloud file-sharing server Package management, networking, security snapd, file systems, encryption 9 Securing your web server Networking, security, system monitoring Apache, Iptables, /etc/group, SELinux, apt, yum/dnf, chmod, chown, Let’s Encrypt 10 Secure your network connections: Creating a VPN or DMZ Networking, security Firewalls, ssh, Apache, OpenVPN, sysctl, easy rsa 11 System monitoring: Working with log files System monitoring, text streams, security grep, sed, journal ctl, rsyslogd, /var/log/, Tripwire 12 Sharing data over a private network Networking, partitions, file systems nfs, smb, ln, /etc/fstab 13 Troubleshooting system performance issues System monitoring, system process management, networking top, free, nice, nmon, tc, iftop, df, kill, killall, uptime 14 Troubleshooting network issues Networking ip, dhclient, dmesg, ping, nmap, traceroute, netstat, netcat(nc) 15 Troubleshooting peripheral devices Device management lshw, lspci, lsusb, modprobe, CUPS 16 DevOps tools: Deploying a scripted server environment using Ansible Scripts, virtualization Ansible, YAML, apt

Linux in Action DAVID CLINTON M A N N I N G SHELTER ISLAND

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2018 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Manning Publications Co. Development editor: Frances Lefkowitz 20 Baldwin Road Review editor: Ivan Martinović PO Box 761 Technical development editor: John Guthrie Shelter Island, NY 11964 Project manager: Deirdre Hiam Copyeditor: Frances Buran Proofreader: Tiffany Taylor Technical proofreader: Reka Horvath Typesetter: Gordan Salinovic Cover designer: Marija Tudor ISBN 9781617294938 Printed in the United States of America 1 2 3 4 5 6 7 8 9 10 – DP – 23 22 21 20 19 18

iii brief contents 1 ■ Welcome to Linux 1 2 ■ Linux virtualization: Building a Linux working environment 22 3 ■ Remote connectivity: Safely accessing networked machines 49 4 ■ Archive management: Backing up or copying entire file systems 68 5 ■ Automated administration: Configuring automated offsite backups 90 6 ■ Emergency tools: Building a system recovery device 109 7 ■ Web servers: Building a MediaWiki server 130 8 ■ Networked file sharing: Building a Nextcloud file-sharing server 155 9 ■ Securing your web server 174 10 ■ Securing network connections: Creating a VPN or DMZ 203 11 ■ System monitoring: Working with log files 229 12 ■ Sharing data over a private network 251 13 ■ Troubleshooting system performance issues 268

BRIEF CONTENTSiv 14 ■ Troubleshooting network issues 289 15 ■ Troubleshooting peripheral devices 308 16 ■ DevOps tools: Deploying a scripted server environment using Ansible 322

v contents preface xi acknowledgments xii about this book xiv about the author xviii about the cover illustration xix 1 Welcome to Linux 1 1.1 What makes Linux different from other operating systems 2 1.2 Basic survival skills 3 The Linux file system 4 ■ Getting around: Linux navigation tools 5 ■ Getting things done: Linux file management tools 9 Keyboard tricks 13 ■ Pseudo file systems 14 ■ Showing ’em who’s boss: sudo 15 1.3 Getting help 16 Man files 16 ■ Info 16 ■ The internet 17 2 Linux virtualization: Building a Linux working environment 22 2.1 What is virtualization? 23 2.2 Working with VirtualBox 26 Working with Linux package managers 26 ■ Defining a virtual machine (VM) 33 ■ Installing an operating system (OS) 36 Cloning and sharing a VirtualBox VM 39

CONTENTSvi 2.3 Working with Linux containers (LXC) 41 Getting started with LXC 41 ■ Creating your first container 42 3 Remote connectivity: Safely accessing networked machines 49 3.1 The importance of encryption 50 3.2 Getting started with OpenSSH 51 3.3 Logging in to a remote server with SSH 53 3.4 Password-free SSH access 55 Generating a new key pair 56 ■ Copying the public key over a network 57 ■ Working with multiple encryption keys 59 3.5 Safely copying files with SCP 59 3.6 Using remote graphic programs over SSH connections 60 3.7 Linux process management 61 Viewing processes with the ps command 62 ■ Working with systemd 64 4 Archive management: Backing up or copying entire file systems 68 4.1 Why archive? 69 Compression 69 ■ Archives: Some important considerations 70 4.2 What to archive 71 4.3 Where to back up 73 4.4 Archiving files and file systems using tar 74 Simple archive and compression examples 74 ■ Streaming file system archives 76 ■ Aggregating files with find 78 ■ Preserving permissions and ownership…and extracting archives 79 4.5 Archiving partitions with dd 83 dd operations 83 ■ Wiping disks with dd 84 4.6 Synchronizing archives with rsync 85 4.7 Planning considerations 86 5 Automated administration: Configuring automated offsite backups 90 5.1 Scripting with Bash 91 A sample script for backing up system files 91 ■ A sample script for changing filenames 95

CONTENTS vii 5.2 Backing up data to AWS S3 97 Installing the AWS command-line interface (CLI) 97 Configuring your AWS account 98 ■ Creating your first bucket 99 5.3 Scheduling regular backups with cron 100 5.4 Scheduling irregular backups with anacron 103 Running the S3 sync job 103 5.5 Scheduling regular backups with systemd timers 104 6 Emergency tools: Building a system recovery device 109 6.1 Working in recovery/rescue mode 111 The GRUB bootloader 111 ■ Using recovery mode on Ubuntu 112 ■ Using rescue mode on CentOS 113 ■ Finding command-line rescue tools 113 6.2 Building a live-boot recovery drive 114 System rescue images 115 ■ Writing live-boot images to USB drives 116 6.3 Putting your live-boot drive to work 120 Testing system memory 120 ■ Damaged partitions 122 Recovering files from a damaged file system 124 6.4 Password recovery: Mounting a file system using chroot 126 7 Web servers: Building a MediaWiki server 130 7.1 Building a LAMP server 131 7.2 Manually setting up an Apache web server 133 Installing the Apache web server on Ubuntu 133 ■ Populating your website document root 134 7.3 Installing an SQL database 134 Hardening SQL 136 ■ SQL administration 137 7.4 Installing PHP 140 Installing PHP on Ubuntu 140 ■ Testing your PHP installation 140 7.5 Installing and configuring MediaWiki 141 Troubleshooting missing extensions 142 ■ Connecting MediaWiki to the database 145 7.6 Installing the Apache web server on CentOS 146 Understanding network ports 147 ■ Controlling network traffic 148 ■ Installing MariaDB on CentOS 149 Installing PHP on CentOS 149

CONTENTSviii 8 Networked file sharing: Building a Nextcloud file-sharing server 155 8.1 Enterprise file sharing and Nextcloud 156 8.2 Installing Nextcloud using snaps 157 8.3 Installing Nextcloud manually 159 Hardware prerequisites 159 ■ Building a LAMP server 161 Configuring Apache 161 ■ Downloading and unpacking Nextcloud 163 8.4 Nextcloud administration 166 8.5 Using AWS S3 as the primary Nextcloud storage 169 9 Securing your web server 174 9.1 The obvious stuff 175 9.2 Controlling network access 177 Configuring a firewall 177 ■ Using nonstandard ports 183 9.3 Encrypting data in transit 185 Preparing your website domain 187 ■ Generating certificates using Let’s Encrypt 187 9.4 Hardening the authentication process 189 Controlling file system objects with SELinux 189 ■ Installing and activating SELinux 191 ■ Applying SELinux policies 193 System groups and the principle of least privilege 194 Isolating processes within containers 196 ■ Scanning for dangerous user ID values 197 9.5 Auditing system resources 197 Scanning for open ports 198 ■ Scanning for active services 198 Searching for installed software 199 10 Securing network connections: Creating a VPN or DMZ 203 10.1 Building an OpenVPN tunnel 204 Configuring an OpenVPN server 205 ■ Configuring an OpenVPN client 212 ■ Testing your VPN 214 10.2 Building intrusion-resistant networks 215 Demilitarized zones (DMZs) 216 ■ Using iptables 218 Creating a DMZ using iptables 218 ■ Creating a DMZ using Shorewall 221 10.3 Building a virtual network for infrastructure testing 224

CONTENTS ix 11 System monitoring: Working with log files 229 11.1 Working with system logs 230 Logging with journald 231 ■ Logging with syslogd 233 11.2 Managing log files 235 The journald way 235 ■ The syslogd way 236 11.3 Consuming large files 237 Using grep 237 ■ Using awk 238 ■ Using sed 239 11.4 Monitoring with intrusion detection 241 Setting up a mail server 241 ■ Installing Tripwire 242 Configuring Tripwire 244 ■ Generating a test Tripwire report 247 12 Sharing data over a private network 251 12.1 Sharing files through Network File System (NFS) 252 Setting up the NFS server 253 ■ Setting up the client 255 Mounting an NFS share at boot time 256 ■ NFS security 257 12.2 Sharing files with Windows users using Samba 259 Testing your Samba configuration 261 ■ Accessing a Samba server from Windows 262 12.3 Sharing files with yourself using symbolic links 262 13 Troubleshooting system performance issues 268 13.1 CPU load problems 269 Measuring CPU load 269 ■ Managing CPU load 270 Making trouble (simulating CPU load) 274 13.2 Memory problems 274 Assessing memory status 274 ■ Assessing swap status 275 13.3 Storage availability problems 275 Inode limits 276 ■ The solution 278 13.4 Network load problems 279 Measuring bandwidth 279 ■ Solutions 280 ■ Shaping network traffic with tc 281 13.5 Monitoring tools 282 Aggregating monitoring data 283 ■ Visualizing your data 284 14 Troubleshooting network issues 289 14.1 Understanding TCP/IP addressing 290 What’s NAT addressing? 290 ■ Working with NAT addressing 291

CONTENTSx 14.2 Establishing network connectivity 293 14.3 Troubleshooting outbound connectivity 295 Tracking down the status of your network 295 ■ Assigning IP addresses 297 ■ Configuring DNS service 300 Plumbing 302 14.4 Troubleshooting inbound connectivity 302 Internal connection scanning: netstat 303 ■ External connection scanning: netcat 303 15 Troubleshooting peripheral devices 308 15.1 Identifying attached devices 309 15.2 Managing peripherals with Linux kernel modules 311 Finding kernel modules 311 ■ Manually loading kernel modules 313 15.3 Manually managing kernel parameters at boot time 315 Passing parameters at boot time 315 ■ Passing parameters via the file system 317 15.4 Managing printers 317 Basics of lp 317 ■ Managing printers using CUPS 318 16 DevOps tools: Deploying a scripted server environment using Ansible 322 16.1 What deployment orchestrators can do for you 324 16.2 Ansible: Installation and setup 326 Setting up passwordless access to hosts 326 ■ Organizing Ansible hosts 327 ■ Testing connectivity 328 16.3 Authentication 328 16.4 Ansible playbooks 330 Writing a simple playbook 330 ■ Creating multi-tiered, role- powered playbooks 332 ■ Managing passwords in Ansible 334 Conclusion 339 appendix A chapter-by-chapter, command-line review 343 index 351

xi preface No matter what you do or how long you’ve been doing it in the IT or programming world, if you’re not learning new stuff, you’re probably not doing it right. It’s not that the platforms and paradigms are constantly changing. Nor is it that new business demands require fresh thinking. Or that the bad guys are constantly coming up with new ways to attack your servers. It’s all of those things and more. You can’t afford to stop learning. The trick is finding a way to learn the high-priority skills without turn- ing the experience into a major detour. It’s my intention and desire that you should be able to read even a single chapter from this book, Linux in Action, and walk away feeling confident enough to take on something challenging and productive—something you wouldn’t previously have even considered. If you hang around until the bitter end, you’ll learn to work with critical and current technologies powering virtualization, disaster recovery, infrastruc- ture security, data backups, web servers, DevOps, and system troubleshooting. But why Linux? Because Linux powers most of the internet, most scientific research, and most commerce—in fact, most of the world’s servers. Those servers need to be pro- visioned, launched, secured, and managed effectively by smart and well-trained people. Smart is what you bring to the table, and I think I can help with well trained. Not sure you know enough about Linux to embark on such an ambitious project? Chapter 1 will quickly fill in the holes. After that, fasten your seat belt and prepare for a serious learning curve.

xii acknowledgments It’s impossible to reach the end of a book’s long and sometimes tortured production cycle without reflecting on what it took to make it through. In the case of Linux in Action—as with my Learn Amazon Web Services in a Month of Lunches—survival required the talent and dedication of every part of the Manning team’s deep bench. Once again Frances Lefkowitz, as development editor, added significant clarity and purpose to each chapter, relentlessly keeping me focused and on track. Both Reka Horvath and John Guthrie patiently tested all the book’s projects and added valuable operational insights along the way. The copy editor, Frances Buran, seems never to have met an adverb she approves—at least not when used by me. But the accuracy and grace of the text in its current form clearly indicate the quality of her judgment. In her role as project manager, Deirdre Hiam effectively shepherded us through the last mile, successfully keeping all the many moving parts in sync. Each of the book’s peer reviewers has left an important mark. They may not realize it, but all of their valu- able observations were carefully noted, weighed, and, where possible, applied. Many thanks, therefore, to Angelo Costo, Christopher Phillips, Dario Victor Duran, Flayol Frederic, Foster Haines, George L. Gaines, Gustavo Patino, Javier Collado, Jens Chris- tian B. Madsen, Jonas Medina de los Reyes, Maciej Jurkowski, Mayer Patil, Mohsen Mostafa Jokar, and Tim Kane. This book is about more than just Linux administration skills. It also tries to impart the larger sense of responsibility successful administrators have for the servers and sys- tems under their care. I was lucky to have benefited from a great mentor at the start of my career as a Linux system administrator. Peter Fedorow’s attention to both fine

ACKNOWLEDGMENTS xiii operational details and the big global picture make him an especially effective admin. His dragging me kicking and screaming into the world of Linux virtualization hooked me on containers long before containers were cool. When everything’s said and done, at least some of Peter’s guidance is, no doubt, reflected here. And finally, none of my professional (or private) projects would get off the blocks without the cheerful and helpful participation of my dear wife. We fully share the hard work, but the successes are mostly her doing.

xiv about this book Looking to learn to administer Linux computers? Excellent choice. While it can hold its own in the consumer desktop space, where Linux absolutely dominates is the world of servers, especially virtual and cloud servers. Because most serious server administra- tion these days takes place remotely, working through a GUI interface of one sort or another just adds unnecessary overhead. If you want to manage the servers and net- work architectures that are currently attracting all the attention, you’re going to have to learn your way around the Linux command line. The good news is that the core Linux command set is going to work for you across geographic and corporate lines, just about anywhere computers and business inter- sect. The better news is that, relatively speaking, Linux skills have staying power. Because it’s such a mature and stable operating system, most of the tools used a quar- ter century ago are still just as effective as ever, and most of the tools used today will probably still be actively used after another quarter century. Learning Linux, in other words, is a lifelong investment. But you’re busy and you’ve got deadlines. Well, I can’t promise you that mastering Linux will be as simple as learning to tie your shoes. But I can help you focus like a laser so you can leave all the stuff you don’t need lying on the highway, choking on your exhaust fumes (assuming you’re not driving a Tesla, of course). How am I going to pull that one off? Linux in Action turns technology training side- ways. That is, while other books, courses, and online resources organize their content around categories (“Alright boys and girls, everyone take out your slide rules and

ABOUT THIS BOOK xv charcoal pencils. Today we’re going to learn about Linux file systems.”), I’m going to use real-world projects to teach. So, for example, I could have built an entire chapter (or two) on Linux file sys- tems. But instead, you’ll learn how to build enterprise file servers, system recovery drives, and scripts to replicate archives of critical data. In the process, you’ll pick up the file system knowledge as a free bonus. Don’t think I’m going to cover every Linux administration tool. That’s impossible: there are literally thousands of them out there. But don’t worry. The core skills and functionality needed through the first years of a career in Linux administration will be covered, and covered well, but only when needed for a practical, mission-critical proj- ect. When you’re done, you’ll have learned no less than what you would have from a traditional source, but you’ll also know how to complete more than a dozen major administrative projects, and be comfortable tackling dozens more. Are you in? I thought so. Who should read this book This book is designed to help you acquire a solid range of Linux administration skills. Perhaps you’re a developer who wants to work more directly with the server environ- ment within which your applications will live. Or maybe you’re ready to make your move in the server administration or DevOps worlds. Either way, you belong with us. What should you already know? At the very least, you should be comfortable work- ing with the files, networks, and basic resources of a modern operating system. Experi- ence with system administration, network management, and programming languages definitely won’t hurt, but are not required. Most of all, you should be unafraid of exploring new environments and enthusiastic about experimenting with new tools. One more thing: you’re expected to know how to perform a simple and straightfor- ward installation of a Linux operating system. How this book is organized: A roadmap Just a few words about the way the book is built. Each chapter of Linux in Action covers one or two practical projects—except chapter 1. Chapter 1, because it’s designed to fill in any very basic gaps that might exist in your Linux knowledge, will be different from all the others. Don’t need the basics? I’m absolutely sure you’ll find lots of fun new toys to play with in chapter 2. Along with the book’s projects, I’ll also introduce you to the individual skills and tools that you’ll need. In addition, each chapter’s projects usually build on the skills you’ve learned previously in the book. Just to show you that I mean business, here’s a fairly complete list of the main projects (under the Chapter heading), skill domains, and tools you’ll meet through the course of the book:

ABOUT THIS BOOKxvi About the code This book contains many examples of source code both in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts. Chapter Skill domains Tools 1. Welcome to Linux Shells, partitions, and file systems Bash, man 2. Linux virtualization: Building a simple Linux working environment Virtualization, file systems VirtualBox, LXC, apt, yum/dnf 3. Remote connectivity: Safely access networked machines Security, remote connectivity ssh, scp, systemctl, ps, grep 4. Archive management: Backing up or copying entire file systems Partitions and file systems, text streams tar, dd, redirects, rsync, locate, split, chmod, chown 5. Automated administration: Configur- ing automated offsite backups Scripts, system process management, security scripts, cron, anacron, systemd timers 6. Emergency tools: Building a system recovery device Partitions and file systems, device management parted, GRUB, mount, chroot 7. Web servers: Building a MediaWiki server Databases, networking, package management PHP, MySQL (MariaDB), Apache web server, package dependencies 8. Networked file sharing: Building a Nextcloud file-sharing server Package management, networking, security snapd, file systems, encryption 9. Securing your web server Networking, security, system monitoring Apache, iptables, /etc/group, SELinux, apt, yum/dnf, chmod, chown, Let’s Encrypt 10. Securing network connections: Creating a VPN or DMZ Networking, security firewalls, ssh, Apache, OpenVPN, sysctl, easy-rsa 11. System monitoring: Working with log files System monitoring, text streams, security grep, sed, journalctl, rsyslogd, /var/log/, Tripwire 12. Sharing data over a private network Networking, partitions, file systems nfs, smb, ln, /etc/fstab 13. Troubleshooting system performance issues System monitoring, system process management, networking top, free, nice, nmon, tc, iftop, df, kill, killall, uptime 14. Troubleshooting network issues Networking ip, dhclient, dmesg, ping, nmap, traceroute, netstat, netcat (nc) 15. Troubleshooting peripheral devices Device management lshw, lspci, lsusb, modprobe, CUPS 16. DevOps tools: Deploying a scripted server environment using Ansible Scripts, virtualization Ansible, YAML, apt

ABOUT THIS BOOK xvii Linux distributions There are currently dozens of actively maintained Linux distributions. Even though most of the Linux basics are common to all distros (distributions), there’ll always be little things that’ll work “here” but not “there.” For practicality’s sake, I’m going to concentrate mostly on two distributions: Ubuntu and CentOS. Why those two? Because each represents an entire family of distributions. Ubuntu shares its roots with Debian, Mint, Kali Linux, and others, while CentOS enjoys the company of Red Hat Enterprise Linux (RHEL) and Fedora. That’s not to say I don’t value other distros like Arch Linux, SUSE, and Gentoo, or that what you’ll learn in this book won’t help you work with those environments. But fully covering the Ubuntu and CentOS families means grabbing the largest single slice of the Linux pie that I could reach using just two distributions. Book forum Purchase of Linux in Action includes free access to a private web forum run by Man- ning Publications where you can make comments about the book, ask technical ques- tions, and receive help from the author and from other users. To access the forum, go to https://forums.manning.com/forums/linux-in-action. You can also learn more about Manning’s forums and the rules of conduct at https://forums.manning.com/ forums/about. Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We sug- gest you try asking the author some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print. Other online resources Stuck? Web search is your best friend, as it can quickly connect you with a wealth of existing Linux guides and troubleshooting expertise. But you shouldn’t forget the StackExchange family of sites and, in particular, serverfault.com. If something’s gone wrong with some system configuration or the network has disappeared, then the odds are high that someone else has experienced the same thing, asked about it on Server- Fault, and received an answer already. Nothing yet? Then ask the question yourself. LinuxQuestions.org and ubuntuforums.org can also be helpful. And those who enjoy video training will find a good range of Linux courses on Pluralsight.com, including more than a dozen of my own courses.

xviii about the author DAVID CLINTON is a system administrator, teacher, and writer. He has administered, written about, and created training material for many important technology subjects including Linux systems, cloud comput- ing (AWS in particular), and container technologies like Docker. He’s the author of Learn Amazon Web Services in a Month of Lunches (Manning, 2017). Many of his video training courses can be found on https://www.pluralsight.com/, and links to his other books (on Linux administration and server virtualization) can be found at https://bootstrap-it.com.