Container Security Fundamental Technology Concepts That Protect Containerized Applications Second Edition (First Early… (Liz Rice) (z-library.sk, 1lib.sk, z-lib.sk)

Container Security SECOND EDITION Fundamental Technology Concepts that Protect Containerized Applications With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles. Liz Rice

Container Security by Liz Rice Copyright © 2026 Vertical Shift Ltd. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Megan Laddusaw Developmental Editor: Rita Fernando Production Editor: Elizabeth Faerm Copyeditor: TO COME Proofreader: TO COME Indexer: TO COME Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea October 2025: Second Edition Revision History for the Early Release

2025-06-27: First release See http://oreilly.com/catalog/errata.csp?isbn=9781492056706 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Container Security, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 979-8-341-62765-9 [LSI]

Brief Table of Contents (Not Yet Final) Chapter 1: Container Security Threats (unavailable) Chapter 2: Linux System Calls, Permissions, and Capabilities (available) Chapter 3: Control Groups (available) Chapter 4: Container Isolation (available) Chapter 5: Virtual Machines (available) Chapter 6: Container Images (unavailable) Chapter 7: Software Vulnerabilities in Images (unavailable) Chapter 8: Strengthening Container Isolation (available) Chapter 9: Breaking Container Isolation (unavailable) Chapter 10: Container Network Security (unavailable) Chapter 11: Securely Connecting Components with TLS (unavailable) Chapter 12: Passing Secrets to Containers (available) Chapter 13: Container Runtime Protection (unavailable) Chapter 14: Containers and the OWASP Top 10 (unavailable) Conclusions (unavailable) Security Checklist (unavailable)

Chapter 1. Linux System Calls, Permissions, and Capabilities A NOTE FOR EARLY RELEASE READERS With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles. This will be the 2nd chapter of the final book. Please note that the GitHub repo will be made active later on. If you’d like to be actively involved in reviewing and commenting on this draft, please reach out to the editor at rfernando@oreilly.com. In most cases, containers run within a computer running a Linux operating system, and it’s going to be helpful to understand some of the fundamental features of Linux so that you can see how they affect security, and in particular how they apply to containers. I’ll cover system calls, file-based permissions, and capabilities and conclude with a discussion of privilege escalation. If you’re familiar with these concepts, feel free to skip to the next chapter. This is all important because containers run Linux processes that are visible from the host. A containerized process uses system calls and needs permissions and privileges in just the same way that a regular process does. But containers give us some new ways to control how these permissions are assigned at runtime or during the container image build process, which will have a significant impact on security.

System Calls Applications run in what’s called user space, which has a lower level of privilege than the operating system kernel. If an application wants to do something like access a file, communicate using a network, or even find the time of day, it has to ask the kernel to do it on the application’s behalf. The programmatic interface that the user space code uses to make these requests of the kernel is known as the system call or syscall interface. There are some 400+ different system calls, with the number varying according to the version of Linux kernel. Here are a few examples: read read data from a file write write data to a file open open a file for subsequent reading or writing execve run an executable program chown change the owner of a file clone create a new process Application developers rarely if ever need to worry about system calls directly, as they are usually wrapped in higher-level programming abstractions. The lowest-level abstraction you’re likely to come across as an app developer is the glibc or musl libraries, or the Golang syscall

package. In practice these are usually wrapped by higher layers of abstractions as well. NOTE If you would like to learn more about system calls, check out my talk “A Beginner’s Guide to Syscalls”, available on O’Reilly’s learning platform. Application code uses system calls in exactly the same way whether it’s running in a container or not, but as you will see later in this book, there are security implications to the fact that all the containers on a single host share —that is, they are making system calls to—the same kernel. Not all applications need all system calls, so—following the principle of least privilege—there are Linux security features that allow users to limit the set of system calls that different programs can access. You’ll see how these can be applied to containers in Chapter 5. I’ll return to the subject of user space and kernel-level privileges in Chapter 4. For now let’s turn to the question of how Linux controls permissions on files. File Permissions On any Linux system, whether you are running containers or not, file permissions are the cornerstone of security. There is a saying that in Linux, everything is a file. Application code, data, configuration information, logs, and so on—it’s all held in files. Even physical devices like screens and printers are represented as files. Permissions on files determine which users are allowed to access those files and what actions they can perform on the files. These permissions are sometimes referred to as discretionary access control, or DAC. Let’s examine this a little more closely.

If you have spent much time in a Linux terminal, you will likely have run the ls -l command to retrieve information about files and their attributes. Figure 1-1. Linux file permissions example In the example in Figure 1-1, you can see a file called myapp that is owned by a user called “liz” and is associated with the group “staff.” The permission attributes tell you what actions users can perform on this file, depending on their identity. There are nine characters in this output that represent the permissions attributes, and you should think of these in groups of three: The first group of three characters describes permissions for the user who owns the file (“liz” in this example). The second group gives permissions for members of the file’s group (here, “staff”). The final set shows what any other user (who isn’t “liz” or a member of “staff”) is permitted to do. There are three actions that users might be able to perform on this file: read, write, or execute, depending on whether the r, w, and x bits are set. The three characters in each group represent bits that are either on or off, showing which of these three actions are permitted—a dash means that the bit isn’t set. In this example, only the owner of the file can write to it, because the w bit is set only in the first group, representing the owner permissions. The owner

can execute the file, as can any member of the group “staff.” Any user is allowed to read the file, because the r bit is set in all three groups. NOTE If you’d like more detail on Linux permissions, there is a good article at https://www.linuxjournal.com/content/mastering-linux-file-permissions-and-ownership. There’s a good chance that you were already familiar with these r, w, and x bits, but that’s not the end of the story. Permissions can be affected by the use of setuid, setgid, and sticky bits. The first two are important from a security perspective because they can allow a process to obtain additional permissions, which an attacker might use for malevolent purposes. setuid and setgid Normally, when you execute a file, the process that gets started inherits your user ID. If the file has the setuid bit set, the process will have the user ID of the file’s owner. The following example uses a copy of the sleep executable owned by a non-root user: liz@vm:~$ ls -l `which sleep` -rwxr-xr-x 1 root root 35336 Apr 5 2024 /usr/bin/sleep liz@vm:~$ cp /usr/bin/sleep ./mysleep liz@vm:~$ ls -l mysleep -rwxr-xr-x 1 liz liz 35336 May 7 10:50 mysleep The ls output shows that the copy is owned by the user called liz. Run this by executing ./mysleep 100, and in a second terminal you can take a look at the running process—the 100 means you’ll have 100 seconds to do this before the process terminates (I have removed some lines from this output for clarity): liz@vm:~$ ps ajf PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 1351 1352 1352 1352 pts/1 1376 Ss 1001 0:00 -bash

1352 1376 1376 1352 pts/1 1376 S+ 1001 0:00 \_ ./mysleep 10 This is running under user ID 1001, which corresponds to the user liz on my VM. Now let’s run the same program under root by executing sudo ./mysleep 100. In a second terminal the process looks slightly different. PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 1351 1352 1352 1352 pts/1 2127 Ss 1001 0:00 -bash 1352 2127 2127 1352 pts/1 2127 S+ 0 0:00 \_ sudo ./mysleep 10 2127 2128 2128 2128 pts/2 2129 Ss 0 0:00 \_ sudo ./mysleep 10 2128 2129 2129 2128 pts/2 2129 S+ 0 0:00 \_ ./mysleep 10 The UID of 0 shows that both the sudo process and the mysleep process are running under the root UID. NOTE If your version of sudo is 1.9.14 or above, as mine is, you’ll see two sudo processes, but for older versions there will be only one. The man page for sudo tells us that in newer versions, it forks one process in a new pseudo-terminal (pts/2 in my output above) to act as a monitor process, before forking a second time to run the command. Now let’s try turning on the setuid bit: liz@vm:~$ chmod +s mysleep liz@vvm:~$ ls -l mysleep -rwsr-sr-x 1 liz liz 35336 May 7 10:50 mysleep Run sudo ./mysleep 100 again, and look at the running processes again from the second terminal: PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 1351 1352 1352 1352 pts/1 29543 Ss 1001 0:00 -bash

1352 29543 29543 1352 pts/1 29543 S+ 0 0:00 \_ sudo ./mysleep 10 29543 29544 29544 29544 pts/2 29545 Ss 0 0:00 \_ sudo ./mysleep 10 29544 29545 29545 29544 pts/2 29545 S+ 1001 0:00 \_ ./mysleep 10 The sudo processes are still running as root, but this time mysleep has taken its user ID from the owner of the file. This setuid bit can be used to give a program privileges that it needs but that are not usually extended to regular users. NOTE Perhaps the canonical example of the setuid bit used to be the executable ping, which needed permission to open raw network sockets in order to send its ping message. An administrator might be happy for their users to run ping, but that doesn’t mean they are comfortable letting users open raw network sockets for any other purpose they might think of. Instead, the ping executable was installed with the setuid bit set and owned by the root user so that ping can use privileges normally associated with root. This is no longer needed since the addition of ICMP sockets in kernel version 5.6, which are designed to allow non-privileged processes to open a socket purely for ICMP protocol messages, as used by ping. At the time of writing, most distributions don’t yet use this ICMP sockets mechanism for ping. Instead, they give the ping executable permission to access raw sockets using a capability called CAP_NET_RAW. We’ll look into this in more detail shortly, in “Linux Capabilities”. We’ve already seen setuid in action, because it’s used by sudo, which is an executable owned by root. liz@vm:~$ ls -l `which sudo` -rwsr-xr-x 1 root root 277936 Apr 8 2024 /usr/bin/sudo The setuid bit on sudo means that the executable runs as the root user, which matches what we saw in the output from ps earlier.

NOTE Once it’s running, sudo goes on to check that the real user that invoked it actually has permissions to run sudo, by checking the sudoers file or some other configured security policy mechanism. If you want to explore this in more detail, man sudo has a good explanation. As you saw when copying sleep, when you copy a file, its ownership attributes are set according to the user ID you’re operating as, and the setuid bit is not carried over. If you want the setuid bit you can run chmod +s on the file. Let’s explore setuid further by taking a copy of bash. liz@vm:~$ cp `which bash` ./mybash liz@vm:~$ ls -l mybash -rwxr-xr-x 1 liz liz 1446024 May 7 15:33 mybash This file is owned by my regular user and doesn’t have the setuid bit set. Now let’s change both those things. liz@vm:~$ sudo chown root ./mybash liz@vm:~$ sudo chmod +s ./mybash liz@vm:~$ ls -l mybash -rwsr-sr-x 1 root liz 1446024 May 7 15:33 mybash Since this executable is owned by root and has setuid, it seems reasonable to imagine that when you run it, the process will be running as root. And yet, look what happens when you try it. liz@vm:~$ ./mybash mybash-5.2$ whoami liz As you can see, the process is not running as root, even though the setuid bit is on and the file is owned by root. What’s happening here? The answer is that in modern versions of bash (and several other interpreters like

python, node and ruby) the executable might start off running as root, but it explicitly resets its user ID to be that of the original user to avoid potential privilege escalations. To explore this for yourself in more detail, you can use strace to see the system calls that the bash (or mybash) executable makes. Find the process ID of your shell, and then in a second terminal run the following command: liz@vm:~$ sudo strace -f -p <shell process ID> This will trace out all the system calls from within that shell, including any executables running within it. Look for the setresuid() or setuid() system calls being used to reset the user ID.. Not all executables are written to reset the user ID in this way. You can use the copy of sleep from earlier in this chapter to see more normal setuid behavior. Change the ownership to root, set the setuid bit (this gets reset when you change ownership), and then run it as a non-root user: liz@vm:~$ sudo chown root mysleep liz@vm:~$ sudo chmod +s mysleep liz@vm:~$ ls -l ./mysleep -rwsr-sr-x 1 root liz 35336 May 7 10:50 mysleep liz@vm:~$ ./mysleep 100 In another terminal you can use ps to see that this process is running under root’s user ID: liz@vm:~$ ps ajf 6646 0.0 0.0 7468 764 pts/2 S+ 00:38 0:00 ./mysleep 100 PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 35834 35835 35835 35835 pts/0 42509 Ss 1001 0:00 -bash 35835 42509 42509 35835 pts/0 42509 S+ 0 0:00 \_ ./mysleep 100 Now that you have experimented with the setuid bit, you are in a good position to consider its security implications.

Security implications of setuid The setuid bit allows someone to act as if they were a different user, which could give them access to different files, executables, and privileges that they are not supposed to have. As you’ve seen, modern versions of bash and most shells and interpreters reset their user ID to avoid being used for trivial privilege escalations. Because setuid provides a dangerous pathway to privilege escalation, some container image scanners (covered in Chapter 7) will report on the presence of files with the setuid bit set. You can also prevent setuid from being used within a container using the - -security-opt no-new-privileges option on a docker run command - I’ll come back to this in Chapter 3. However, that won’t stop an attacker from writing a setuid executable owned by root onto mounted directory on the host. You’ll find an example of this in the chapter2/setuid directory of the GitHub repo that accompanies this book. Host volume mounts can lead to all sorts of attacks, and we’ll discuss this more in Chapter X. The setuid bit dates from a time when privileges were much simpler—either your process had root privileges or it didn’t. The setuid bit provided a mechanism for granting extra privileges to non-root users. Version 2.2 of the Linux kernel introduced more granular control over these extra privileges through capabilities. Linux Capabilities There are over 30 different capabilities in today’s Linux kernel. Capabilities can be assigned to a thread to determine whether that thread can perform certain actions. For example, a thread needs the CAP_NET_BIND_SERVICE capability in order to bind to a low-numbered (below 1024) port. CAP_SYS_BOOT exists so that arbitrary executables don’t have permission to reboot the system. CAP_SYS_MODULE is needed to load or unload kernel modules, and CAP_BPF is needed to load eBPF programs.

NOTE Consult man capabilities on a Linux machine for detailed information on each individual capabilities. I mentioned earlier that the ping tool uses the CAP_NET_RAW capability so that it can open a raw network socket. T Capabilities can be assigned to both files and processes. You can see the capabilities for a file using getcap, like this: liz@vm:~$ getcap which ping /usr/bin/ping cap_net_raw=ep You can see the capabilities assigned to a process by using the getpcaps command. Many processes typically won’t have capabilities: liz@vm:~$ ps PID TTY TIME CMD 22355 pts/0 00:00:00 bash 25058 pts/0 00:00:00 ps liz@vm:~$ getpcaps 22355 22355: = In the past, getpcaps assumed that if a process was running as root, it had all capabilities, so would report the whole list. These days, getpcaps and other tools have been updated not to make this assumption, so processes running as root will typically appear with no capabilities. We’ve seen that the executable file for ping has CAP_NET_RAW associated with it, so let’s see the capabilities assigned to the process when we run it. Leave ping running in one terminal: liz@vm:~$ ping 127.0.0.1 PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data. 64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.162 ms 64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.188 ms …

In a second terminal, get the process ID and check its capabilities. liz@vm:~$ getpcaps 50394 50394: = How is ping successfully opening a socket if it doesn’t have the CAP_NET_RAW capability required? And why doesn’t it have it, if the executable file has it? The answer can be found by using a second terminal to trace the system calls, exactly the same as described earlier when examining how bash behaves. You’ll see that ping is now capabilities-aware, and deliberately discards CAP_NET_RAW once the socket is open and it has no further use for the capability. (A few irrelevant calls and details have been omitted for clarity.) capget({version=_LINUX_CAPABILITY_VERSION_3, pid=0}, {effective=0, permitted=1<<CAP_NET_RAW, inheritable=0}) = 0 capset({version=_LINUX_CAPABILITY_VERSION_3, pid=0}, {effective=1<<CAP_NET_RAW, permitted=1<<CAP_NET_RAW, inheritable=0}) = 0 socket(AF_INET, SOCK_RAW, IPPROTO_ICMP) = 3 socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6) = 4 capget({version=_LINUX_CAPABILITY_VERSION_3, pid=0}, {effective=1<<CAP_NET_RAW, permitted=1<<CAP_NET_RAW, inheritable=0}) = 0 capset({version=_LINUX_CAPABILITY_VERSION_3, pid=0}, {effective=0, permitted=1<<CAP_NET_RAW, inheritable=0}) The process checks that CAP_NET_RAW is in the set of permissions that it is permitted to use. The return code 0 tells it that it is. The process uses capset to make that capability effective. It opens sockets for IPv4 and IPv6. Now that the sockets are open, it uses capset to remove the capability from its effective set.

By the time getpcaps inspected the process’s capabilities, it was no longer in effect for the process. NOTE For a more in-depth discussion of the ways that file and process permissions interact, see Adrian Mouat’s post on Linux capabilities in practice. Following the principle of least privilege, it’s a good idea to grant only the capabilities that are needed for a process to do its job. When you run a container, you get the option to control the capabilities that are permitted, as you’ll see in Chapter 5. Now that you are familiar with the basic concepts of permissions and privileges in Linux, I’d like to turn to the idea of escalating privileges. Privilege Escalation The term “privilege escalation” means extending beyond the privileges you were supposed to have so that you can take actions that you shouldn’t be permitted to take. To escalate their privileges, an attacker takes advantage of a system vulnerability or poor configuration to grant themselves extra permissions. Oftentimes, the attacker starts as a non-privileged user and wants to gain root privileges on the machine. A common method of escalating privileges is to look for software that’s already running as root and then take advantage of known vulnerabilities in the software. For example, web server software might include a vulnerability that allows an attacker to remotely execute code, such as the Struts vulnerabilities1. If the web server is running as root, anything that is remotely executed by an attacker will run with root privileges. For this reason, it is a good idea to run software as a non-privileged user whenever possible.

As you’ll learn later in this book, by default containers run as root. This means that compared with a traditional Linux machine, applications running in containers are far more likely to be running as root. An attacker who can take control of a process inside a container still has to somehow escape the container, but once they achieve that, they will be root on the host, and there is no need for any further privilege escalation. Chapter 9 discusses this in more detail. Even if a container is running as a non-root user, there is potential for privilege escalation based on the Linux permissions mechanisms you have seen earlier in this chapter: Container images including executable files with the setuid bit Additional capabilities granted to a container running as a non-root user You’ll learn about approaches for mitigating these issues later in the book. Summary In this chapter you have learned (or revised) some fundamental Linux mechanisms that will be essential to understanding later chapters of this book. They also come into play in security in numerous ways; the container security controls that you will encounter are all built on top of these fundamentals. Now that you have some basic Linux security controls under your belt, it’s time to start looking at the mechanisms that make up containers so that you can understand for yourself how root on the host and in the container are one and the same thing. 1 In the first edition of this book, I was referring to a 2018 critical remote code execution vulnerability that had received a lot of press coverage. It turns out there have been other serious Struts vulnerabilities since then, which is a good case study in how vulnerabilities

continue to be found in widely-used software packages, and why it’s important to keep updating your dependencies!