PB158 — Containerization

<h1>Containerization</h1>

<p class="lede">A container is not a lightweight virtual machine. It is an ordinary process that Linux kernel mechanisms isolate.</p>

<div class="author-block">
  <span class="name">Ondřej (Ondra) Krajíček</span>
  <a href="mailto:me@ondrejkrajicek.com">me@ondrejkrajicek.com</a> ·
  <a href="https://www.linkedin.com/in/OndrejKrajicek">www.linkedin.com/in/OndrejKrajicek</a>
  <span class="built">v5 · 37b395f62 · built 2026-09-30</span>
  <span class="attribution">Generated by the agentic vault — an autonomous research and authoring harness. Every factual claim was gated against a cited source registry before this document was produced.</span>
</div>

Note:

<ul>
  <li>A container is an ordinary process. Unrelated kernel interfaces restrict what it sees and what it consumes [V1].</li>
  <li>A container is "not a virtual machine at all, it's just processes running in the same Linux kernel" [S18].</li>
  <li>The shared kernel makes the isolation weaker than a virtual machine's separation [V7].</li>
  <li>The engine's own teaching documentation says the same thing: containers are isolated processes [S75].</li>
</ul>

---

<h2>Two commands, one flag apart</h2>

<p class="lede">One flag is the only difference between these two commands. Only the second one works.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker rm -f web 2>/dev/null
docker run --rm -d --name web nginx:alpine
docker exec web sh -c \
  'apk add -q strace; strace -p 1'
# refused

docker rm -f web
docker run --rm -d --name web \
  --cap-add SYS_PTRACE nginx:alpine
docker exec web sh -c \
  'apk add -q strace; strace -p 1'
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman rm -f web 2>/dev/null
podman run --rm -d --name web nginx:alpine
podman exec web sh -c \
  'apk add -q strace; strace -p 1'
# refused

podman rm -f web
podman run --rm -d --name web \
  --cap-add SYS_PTRACE nginx:alpine
podman exec web sh -c \
  'apk add -q strace; strace -p 1'
```

</div>
</div>

<ul>
  <li>Without the flag, <code>strace</code> is refused inside the container. With the flag it works.</li>
  <li>Both engines answer the same way. You run both commands in Part II.</li>
</ul>

Note:

<ul>
  <li>Under the default setting, <code>strace</code> attaches only with <code>CAP_SYS_PTRACE</code>, or when the target is a descendant of the tracer [S67].</li>
  <li>Neither engine grants <code>CAP_SYS_PTRACE</code> by default [S42] [S43].</li>
  <li>The refusal is the capability machinery working as documented [V10].</li>
  <li>A container can be taken apart with <code>ls</code>, <code>cat</code> and <code>strace</code> [V12].</li>
  <li>The image in this example is built in Part II, step 4.</li>
</ul>

Collect the room's guesses.

---

<h2>Roadmap</h2>

<p class="lede">Seven parts. Part II runs containers. Part III shows how they work.</p>

<div class="kv">
<div class="pair">
<div class="k">I · The need</div>
<div class="v">Three problems, all older than the word.</div>
</div>
<div class="pair">
<div class="k">II · Run your first container</div>
<div class="v">Install an engine, run it, keep the data, repeat the run.</div>
</div>
<div class="pair">
<div class="k">III · What is underneath</div>
<div class="v">Five kernel mechanisms, five answers.</div>
</div>
<div class="pair">
<div class="k">IV · Two architectures</div>
<div class="v">The same mechanisms, assembled differently.</div>
</div>
<div class="pair">
<div class="k">V · What follows</div>
<div class="v">Why macOS and Windows need Linux inside.</div>
</div>
<div class="pair">
<div class="k">VI · Where it breaks</div>
<div class="v">The boundary, the escapes, the leaks.</div>
</div>
<div class="pair">
<div class="k">VII · Where it came from</div>
<div class="v">Optional. Timelines.</div>
</div>
</div>

Note:

The room runs the hands-on work on its own laptops in Part II. Say why the order is this way. Each kernel mechanism in Part III answers a question the room met in Part II. Part VII is optional. Skip it without loss.

---

<h2>Takeaways</h2>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">No kernel object</div>
<div class="desc">Unrelated kernel interfaces restrict an ordinary process.</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">Not a small machine</div>
<div class="desc">A container shares the host kernel. A virtual machine runs its own.</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">Same parts, two shapes</div>
<div class="desc">Same mechanisms, different owner of the container process.</div>
</div>
<div class="row">
<div class="num">4</div>
<div class="term">Linux, or a Linux inside</div>
<div class="desc">A Linux container on macOS or Windows runs inside a Linux virtual machine.</div>
</div>
<div class="row">
<div class="num">5</div>
<div class="term">Configuration, not magic</div>
<div class="desc">What it may do is written in settings you can read and change.</div>
</div>
</div>

Note:

<ul>
  <li>A container is not a kernel object [T1] [V1].</li>
  <li>A container is not a lightweight virtual machine [T2] [V7].</li>
  <li>A container does not boot another copy of the operating system, and a virtual machine does [S63].</li>
  <li>Docker and Podman differ in architecture, not in kernel usage [T3] [V15].</li>
  <li>A Linux container on macOS or Windows runs inside a Linux virtual machine [T4] [V9].</li>
  <li>What a process may do is readable configuration [T5] [V10].</li>
</ul>

---

<h2>In scope, and out of scope</h2>

<div class="compare">
<div>
<p class="kicker">Covered</p>
<ul>
<li>The five kernel mechanisms, each observed with a command.</li>
<li>Running one engine on your own laptop, from a first run to a repeatable setup.</li>
<li>Docker against Podman, by architecture.</li>
<li>Where the abstraction leaks, and the escapes recorded through it.</li>
</ul>
</div>
<div class="side-b">
<p class="kicker">Not covered</p>
<ul>
<li>Orchestration across several machines.</li>
<li>Container networking beyond one published port.</li>
<li>Image hardening, image size and the build cache.</li>
<li>Supply-chain practice.</li>
</ul>
</div>
</div>

Note:

<ul>
  <li>A student observes each of the five kernel mechanisms with a command.</li>
  <li>One engine runs from an image to a repeatable setup.</li>
  <li>Image bloat and supply-chain risk get one mention each, and no exercise.</li>
  <li>A one-container Compose file is in scope. Orchestration across machines is not.</li>
</ul>

---

<p class="lede">Three problems, all older than the word container.</p>

Note:

Part I states the problems. No mechanism yet.

---

<h2>One machine, many workloads</h2>

<p class="lede">One server runs several workloads, and three conditions must hold at once.</p>

<ul>
  <li>No application reads another application's data.</li>
  <li>No application changes how another one runs.</li>
  <li>No application takes so much of the machine that the others stop.</li>
</ul>

<div class="kv">
<div class="pair">
<div class="k">Partial answer</div>
<div class="v">An old call named <code>chroot</code> shows a process one subtree of the file tree and hides the rest.</div>
</div>
<div class="pair">
<div class="k">Why it fails</div>
<div class="v">A process inside <code>chroot</code> still lists the other processes and reaches the network.</div>
</div>
</div>

<p class="closing">Isolation for workloads that cooperate, and one defence among several. Every container uses the host kernel, so it is not a wall.</p>

Note:

<ul>
  <li>The Solaris Zones designers state the need as isolation plus resource control [S58].</li>
  <li><code>chroot</code> does not compartmentalise the process or networking spaces [S19].</li>
  <li>A process inside <code>chroot</code> therefore still reaches the other processes and the network [S19].</li>
  <li>The author of FreeBSD jails states that <code>chroot</code> was never intended as a security mechanism [S20].</li>
  <li>The shared kernel makes the isolation weaker than a virtual machine's separation [V7].</li>
  <li>Kubernetes states that containers "offer a weaker isolation boundary than virtual machines that utilize hardware-based virtualization" [S73].</li>
  <li>Part VI shows where that boundary has failed.</li>
</ul>

---

<h2>One machine, two applications with different needs</h2>

<p class="lede">Two applications on one host need different versions of the same thing.</p>

<ul>
  <li>The Solaris Zones paper records two applications at one customer site, both hard-coded to listen on port 80. The customer bought two servers.</li>
  <li>Hard-coded paths and log locations can make two versions of one application hard to install side by side.</li>
  <li>Boettiger cites a University of Arizona study: less than half of the software published with computer-systems papers could be built or installed.</li>
</ul>

<p class="closing">The need is a package that carries the whole user-space environment with it.</p>

Note:

<ul>
  <li>The paper records "two poorly written applications" contending for port 80, resolved "by buying two servers" [S58].</li>
  <li>The same paper names hard-coded pathnames as a barrier, including the place a log file is written [S58].</li>
  <li>They can make two versions of one application difficult to deploy on one node [S58].</li>
  <li>Boettiger reports a University of Arizona study finding that "less than 50% of software could even be successfully built or installed" [S62].</li>
  <li>The literature calls this barrier "Dependency Hell" [S62].</li>
  <li>The need is packaging, and it is separate from the isolation need.</li>
</ul>

---

<h2>A virtual machine solves both, and what it costs</h2>

<p class="lede">A virtual machine is a simulated computer that runs its own kernel, so the guest kernel is not the host kernel.</p>

<div class="stats">
<div>
<div class="stat">11 s</div>
<div class="stat-cap">to start a virtual machine, in a measurement Felter's paper cites</div>
</div>
<div>
<div class="stat">
<span class="pos">under 1 s</span>
</div>
<div class="stat-cap">to start a container, on the same hardware</div>
</div>
<div>
<div class="stat">1</div>
<div class="stat-cap">kernel per virtual machine, booted and held in memory</div>
</div>
</div>

<ul>
  <li>Felter's own measurements found containers equal to or better than virtual machines on almost every workload, and both needed tuning for heavy input and output.</li>
  <li>Speed is not the argument. Start-up time and density are.</li>
</ul>

Note:

<ul>
  <li>Felter's paper reports that "other measurements have shown" under one second against eleven seconds on that hardware [S63].</li>
  <li>The reason given is that a container does not boot another copy of the operating system [S63].</li>
  <li>Felter's own results report containers "equal or better" in almost all cases, and add that both need tuning for input-heavy work [S63].</li>
  <li>The Solaris Zones paper names administering many operating system instances as the other cost [S58].</li>
</ul>

---

<h2>A third answer, between the two</h2>

<p class="lede">A microVM is a virtual machine with almost every simulated device removed. It keeps its own kernel and gives up the rest.</p>

| | Container | MicroVM | Full virtual machine |
|---|---|---|---|
| Kernel | the host's | its own | its own |
| Boundary | the system-call interface | hardware, plus the monitor | hardware, plus the monitor |
| Start | fastest | fast | slowest |

<ul>
  <li>Firecracker is one microVM monitor, and it runs a minimal guest kernel of its own.</li>
  <li>Its authors report a boot to application code in under 125 ms, and under 5 MB of added memory for each container.</li>
  <li>AWS runs Lambda and Fargate on it, and states it built Firecracker rather than choose between isolation and compatibility.</li>
</ul>

Note:

<ul>
  <li>A microVM gives each sandbox "its own virtual hardware, page tables, and operating system kernel" [S38].</li>
  <li>The paper's claim is that this "moves the security-critical interface from the OS boundary to a boundary supported in hardware and comparatively simpler software", which is the monitor [S38].</li>
  <li>With its minimal guest-kernel configuration Firecracker "boots to application code in less than 125ms" and "offers memory overhead of less than 5MB per container" [S38].</li>
  <li>AWS runs Firecracker in Lambda and Fargate, "where it supports millions of production workloads" [S38].</li>
  <li>Reliance on a single kernel means "a fundamental tradeoff between security and code compatibility" [S38].</li>
  <li>AWS states "We built Firecracker because we didn't want to choose" [S38].</li>
  <li>A container does not boot another copy of the operating system, so it starts fastest of the three [S63].</li>
</ul>

---

<h2>What a container is</h2>

<p class="lede">A container is a process, or a tree of processes, that runs with its own view of the machine on the host's kernel.</p>

<div class="kv">
<div class="pair">
<div class="k">Its own view</div>
<div class="v">Its own files, its own process list, its own network.</div>
</div>
<div class="pair">
<div class="k">Its own share</div>
<div class="v">A set amount of memory and of processor time.</div>
</div>
<div class="pair">
<div class="k">The host's kernel</div>
<div class="v">No second kernel boots. It starts as fast as an ordinary program.</div>
</div>
</div>

<ul>
  <li>This course's own Linux toolchain is an image. You start a container from it in Part II.</li>
</ul>

Note:

<ul>
  <li>A container is an ordinary process. Unrelated kernel interfaces restrict what it sees and what it consumes [V1].</li>
  <li>The engine's own teaching documentation describes containers as isolated processes [S75].</li>
  <li>A container does not boot another copy of the operating system, and a virtual machine does [S63].</li>
  <li>This course's own Linux toolchain is a Debian-family image, which the room has not run before this deck [V17].</li>
  <li>No mechanism name is given yet. Part III names them.</li>
</ul>

---

<h1>Run your first container</h1>

<p class="lede">Install an engine, run a container, keep its data, and make the run repeatable.</p>

Note:

Part II uses the tool. It explains nothing about the kernel yet. Every step closes with one question that Part III answers.

---

<p class="kicker">Part II · Setting up</p>

<h2>The engine is the program you install</h2>

<p class="lede">A container engine takes a command, finds the files the application needs, and starts the container.</p>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">Resolve</div>
<div class="desc">Turn the name <code>python:3.12-slim</code> into a list of files to fetch from a server.</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">Unpack</div>
<div class="desc">Fetch the missing files and unpack them.</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">Configure</div>
<div class="desc">Write a configuration naming what the process may see and use.</div>
</div>
<div class="row">
<div class="num">4</div>
<div class="term">Start</div>
<div class="desc">Start the process with that configuration.</div>
</div>
</div>

<ul>
  <li>Docker and Podman are container engines, and both take the same verbs from the command line.</li>
</ul>

Note:

<ul>
  <li>Docker's engine is a client and a daemon that builds, runs and distributes containers [S10].</li>
  <li>Podman is "a fully featured container engine that is a simple daemonless tool" [S13].</li>
  <li>Docker's licence lists education among the uses for which Docker Desktop is free [S50].</li>
  <li>The command line is enough for every step in this topic.</li>
  <li>Step 3 writes the file the OCI runtime specification defines [S7].</li>
</ul>

---

<p class="kicker">Part II · Setting up</p>

<h2>Linux: install the engine directly</h2>

<p class="lede">Linux runs containers natively. Install one engine, and follow it for the rest of the topic.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
# 1. remove the distribution's packages
sudo apt remove docker.io docker-compose
# 2. run Docker's own install script
curl -fsSL https://get.docker.com \
  -o get-docker.sh
sudo sh get-docker.sh
# 3. let your account reach the daemon
sudo usermod -aG docker $USER
newgrp docker
docker run --rm hello-world
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
sudo apt install podman
# or: sudo dnf install podman
podman --version
podman run --rm hello-world
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">If Podman refuses</div>
<div class="v">Podman 6 needs version 2 of the kernel's resource-limit interface. <code>stat -fc %T /sys/fs/cgroup</code> prints <code>cgroup2fs</code> for version 2.</div>
</div>
<div class="pair">
<div class="k">Version</div>
<div class="v">A distribution package can be older than Podman 6. Read <code>podman --version</code>.</div>
</div>
</div>

Note:

<ul>
  <li>Docker publishes installation steps for each Linux distribution, not one script [S51].</li>
  <li>Docker's instructions start by removing the distribution's <code>docker.io</code> and related packages, "which may conflict with the official packages provided by Docker" [S82].</li>
  <li>Docker publishes the same install steps as a convenience script fetched from <code>get.docker.com</code> [S82].</li>
  <li>Docker documents <code>sudo usermod -aG docker $USER</code> and <code>newgrp docker</code> as the step that lets a non-root account use the daemon socket [S85].</li>
  <li>A distribution package can lag the upstream Podman release, so a version-specific claim needs <code>podman --version</code> first.</li>
  <li>Podman publishes package steps for each major Linux distribution [S52]. It is built in on Fedora CoreOS and Fedora Silverblue [S52].</li>
  <li>Podman 6.0.0 removed "support for running on cgroups v1 systems" [S65].</li>
  <li>Part III explains control groups.</li>
</ul>

---

<p class="kicker">Part II · Setting up</p>

<h2>macOS first: Homebrew installs the command-line tools</h2>

<p class="lede">Homebrew is a package manager for macOS. It installs command-line programs, as <code>apt</code> does on Ubuntu.</p>

```bash
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```

<div class="kv">
<div class="pair">
<div class="k">1 · Run the installer</div>
<div class="v">One command, taken from brew.sh. It asks for your password once.</div>
</div>
<div class="pair">
<div class="k">2 · Next steps</div>
<div class="v">The installer prints lines under "Next steps". Run them. They put <code>brew</code> on your PATH.</div>
</div>
<div class="pair">
<div class="k">3 · Where it installs</div>
<div class="v"><code>/opt/homebrew</code> on Apple Silicon, <code>/usr/local</code> on an Intel Mac. The PATH lines differ for that reason.</div>
</div>
</div>

<ul>
  <li>Check the installation with <code>brew --version</code>. Only then run <code>brew install lima</code>.</li>
</ul>

Note:

<ul>
  <li>Homebrew "installs the command-line tools and applications you need across macOS, Linux and WSL" [S83].</li>
  <li>The one-line installer is one of the install routes the project publishes [S83].</li>
  <li>Homebrew installs under "/opt/homebrew on Apple Silicon" and "/usr/local on macOS Intel" [S83].</li>
  <li>The installer prints directions for updating the shell's configuration, and Homebrew "will not work" until they are followed [S86].</li>
</ul>

---

<p class="kicker">Part II · Setting up</p>

<h2>macOS: a Linux virtual machine, driven from the shell</h2>

<p class="lede">macOS has no Linux kernel. The engine runs inside a Linux virtual machine that Lima creates.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
brew install lima
limactl create template:docker
limactl start docker
limactl shell docker \
  docker run --rm hello-world
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
brew install lima
limactl create template:podman
limactl start podman
limactl shell podman \
  podman run --rm hello-world
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Template syntax</div>
<div class="v">Current Lima takes <code>template:docker</code>, not the older <code>template://docker</code>.</div>
</div>
<div class="pair">
<div class="k">Read-only home</div>
<div class="v">Lima mounts your home directory read-only. A write there fails by design.</div>
</div>
<div class="pair">
<div class="k">Intel Mac</div>
<div class="v">Podman 6 dropped Intel Macs. Use the Lima <code>docker</code> template there.</div>
</div>
</div>

Note:

<ul>
  <li>Lima documents the template-URL change from the 1.x form [S53].</li>
  <li>The Lima defaults are four cores, 4 GiB of memory and a read-only home mount [S54].</li>
  <li><code>podman machine</code> instead mounts the home directory read-write by default [S55].</li>
  <li>Podman 6.0.0 removed "support for running on Intel Macs" [S65].</li>
  <li>Why a Linux virtual machine is needed at all is Part V.</li>
</ul>

---

<p class="kicker">Part II · Setting up</p>

<h2>Windows: WSL 2, and two settings to check</h2>

<p class="lede">Windows Subsystem for Linux runs a real Linux kernel in a small virtual machine.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
wsl --install -d Ubuntu
# then, in the Ubuntu shell:
curl -fsSL https://get.docker.com \
  -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker $USER
newgrp docker
docker run --rm hello-world
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
wsl --install -d Ubuntu
# then, in the Ubuntu shell
sudo apt install podman && \
  podman run --rm hello-world
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Virtualisation</div>
<div class="v">Error 0x80370102: turn on Virtual Machine Platform, then BIOS virtualisation.</div>
</div>
<div class="pair">
<div class="k">Systemd, for Podman</div>
<div class="v">Podman without root needs the Linux service manager. Set <code>systemd=true</code> under <code>[boot]</code>, then run <code>wsl --shutdown</code>.</div>
</div>
</div>

Note:

<ul>
  <li>WSL 2 uses "a Linux kernel inside of a lightweight utility virtual machine" [S46].</li>
  <li>The WSL virtual machine's memory default is 50 percent of host memory [S71].</li>
  <li>Podman without root needs systemd, and WSL 2 does not enable it by default [V38].</li>
  <li>Systemd is now the default for the Ubuntu that <code>wsl --install</code> installs, so check it with <code>systemctl status</code> first [S47].</li>
  <li>The <code>[boot]</code> edit needs WSL version 0.67.6 or newer [S47].</li>
  <li>Podman 6.0.0 removed control groups version 1 support, so <code>v1</code> here is a hard stop [S65].</li>
  <li>For error 0x80370102 Microsoft asks the reader to enable the Virtual Machine Platform feature and to "ensure virtualization is enabled in the BIOS" [S79].</li>
</ul>

---

<p class="kicker">Part II · Setting up</p>

<h2>macOS and Windows needed a virtual machine</h2>

<p class="lede">A container on macOS or Windows needs a Linux virtual machine, and that machine costs a boot only once.</p>

<div class="boxes">
<div class="box">
<div class="box-label">Why</div>
<div class="box-head">The mechanisms are Linux features</div>
<ul>
<li>The kernel features a container needs exist in the Linux kernel.</li>
<li>The macOS and Windows kernels do not have them.</li>
<li>So Lima and WSL 2 supply a Linux kernel, inside a virtual machine.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">What it buys back</div>
<div class="box-head">One machine, many containers</div>
<ul>
<li>The virtual machine boots once.</li>
<li>Each container after that costs one process start, not one boot.</li>
</ul>
</div>
</div>

Note:

<ul>
  <li>Every Docker on a Mac is Docker inside a Linux virtual machine [V9].</li>
  <li>Lima creates that Linux virtual machine on macOS [S53].</li>
  <li>WSL 2 supplies the Linux kernel on Windows in the same way [S46].</li>
  <li>The cost is paid once for the virtual machine, not once per container.</li>
  <li>Part V gives the mechanism and the exception.</li>
</ul>

---

<p class="kicker">Part II · Setting up</p>

<h2>One shell, and the same commands after it</h2>

<p class="lede">Every command from here runs in a Linux shell. Open that shell once, and stay in it.</p>

<div class="kv">
<div class="pair">
<div class="k">Linux</div>
<div class="v">Your own terminal. Nothing to open.</div>
</div>
<div class="pair">
<div class="k">macOS</div>
<div class="v"><code>limactl shell docker</code>, or <code>limactl shell podman</code>, opens a shell inside the virtual machine.</div>
</div>
<div class="pair">
<div class="k">Windows</div>
<div class="v"><code>wsl</code> opens a shell inside the Linux distribution.</div>
</div>
</div>

<ul>
  <li>Make your working files in that shell. A program reads a file on the macOS or Windows side more slowly than one inside the Linux filesystem, and Part VI gives the measurement.</li>
  <li>On Windows the subsystem supplies the ownership of files under <code>/mnt/c</code>.</li>
</ul>

Note:

<ul>
  <li>Lima runs the engine inside a virtual machine, reached with <code>limactl shell</code> [S53].</li>
  <li>WSL 2 runs a Linux kernel in a small virtual machine, reached with <code>wsl</code> [S46].</li>
  <li>Podman keeps a deliberately compatible command set, so most commands differ by one word [S13].</li>
  <li>Microsoft advises keeping files on the Linux drive for speed [S72].</li>
  <li>WSL adds no metadata to Windows files by default, so Linux permissions under <code>/mnt/c</code> are supplied rather than stored [S71].</li>
</ul>

---

<p class="kicker">Part II · Step 1 · An image and a container</p>

<h2>An image is a stack of layers</h2>

<p class="lede">An image is the set of files an application runs in. Those files are stored as layers.</p>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">Layer</div>
<div class="desc">One set of file changes: files added, replaced or removed.</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">Stack</div>
<div class="desc">A file in a higher layer hides the file of the same name below it.</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">Manifest</div>
<div class="desc">The list that names each layer by a hash of its content.</div>
</div>
</div>

<ul>
  <li>Every layer is read-only. The files in an image never change after the build.</li>
</ul>

Note:

<ul>
  <li>The image specification defines a manifest naming content-addressed layer archives [S8].</li>
  <li>Each layer is one set of filesystem changes [S23].</li>
  <li>The image's layers are read-only, and a container adds its own writable layer above them [S64].</li>
  <li>A file in a higher layer hides the file of the same name in a lower one [S6].</li>
</ul>

---

<p class="kicker">Part II · Step 1 · An image and a container</p>

<h2>An image is not a container</h2>

<p class="lede">Two words, and every later sentence depends on telling them apart.</p>

<div class="boxes">
<div class="box">
<div class="box-label">Image</div>
<div class="box-head">Inert data</div>
<ul>
<li>A stack of read-only layers, plus the manifest that names them.</li>
<li>Nothing about it runs. It can sit in a file or on a server.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">Container</div>
<div class="box-head">A running process</div>
<ul>
<li>A process, or a tree of processes, started from an image.</li>
<li>It has a process identifier on the host.</li>
<li>It ends when its command ends.</li>
</ul>
</div>
</div>

<p class="closing">You cannot download a container. You download an image and start one.</p>

Note:

<ul>
  <li>The image specification defines a manifest naming content-addressed layer archives [S8].</li>
  <li>A container is a process or process tree started from those layers [V1].</li>
  <li>The image holds no writable layer, so a running container does not change it.</li>
  <li>One image starts many containers, and each one gets its own writable layer [S64].</li>
</ul>

---

<p class="kicker">Part II · Step 1 · An image and a container</p>

<h2>A registry is where images come from</h2>

<p class="lede">A registry stores images under a name and a hash, and serves them over a documented protocol.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker pull python:3.12-slim
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman pull python:3.12-slim
```

</div>
</div>

<ul>
  <li>The engine turns the name into a manifest, then fetches the layers the manifest names.</li>
  <li>The name has two halves. <code>python</code> is the repository, and <code>3.12-slim</code> is the tag.</li>
  <li>Layers already on disk are not fetched again.</li>
  <li>Docker Hub allows an unauthenticated address 100 pulls in six hours, and a room behind one shared address shares that budget.</li>
  <li>Pull the images before you need them. Pull 101 returns an error that looks like broken Wi-Fi.</li>
</ul>

Note:

<ul>
  <li>Docker Hub meters "Unauthenticated users | 100 per IPv4 address or IPv6 /64 subnet" per six hours [S49].</li>
  <li>The manifest names the layers by the hash of their content [S8].</li>
  <li>Many machines behind one shared address draw on one budget [S49].</li>
</ul>

Pre-pull before the session.

---

<p class="kicker">Part II · Step 1 · An image and a container</p>

<h2>Predict: where does the container appear?</h2>

<p class="lede">Write your answer down before anything runs.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker rm -f web 2>/dev/null
docker run --rm -d --name web \
  -p 8080:80 nginx:alpine
curl -s localhost:8080 | head -2
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman rm -f web 2>/dev/null
podman run --rm -d --name web \
  -p 8080:80 nginx:alpine
curl -s localhost:8080 | head -2
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Question 1</div>
<div class="v">Does the web server appear in the host process list? Does it survive closing the terminal?</div>
</div>
<div class="pair">
<div class="k">Question 2</div>
<div class="v">Run <code>ps</code> inside the container. How many processes does it see?</div>
</div>
</div>

<ul>
  <li><code>--rm</code> deletes the container when it stops. <code>-d</code> returns your shell at once.</li>
  <li><code>--name</code> labels it. <code>-p 8080:80</code> maps host port 8080 to port 80 inside.</li>
</ul>

Note:

<ul>
  <li><code>--rm</code> removes the container when it exits, <code>-d</code> detaches it, and <code>-p</code> publishes a container port on a host port [S61].</li>
  <li>A written prediction before the observation gives the room a stake in the result [S57].</li>
  <li>The container is a process, or a tree of processes, on the host [V1].</li>
  <li>In a PID namespace the first process becomes PID 1, and the others disappear from view [S18].</li>
</ul>

Collect the answers first.

---

<p class="kicker">Part II · Step 1 · An image and a container</p>

<h2>Observe: the same question, three platforms</h2>

<div class="matrix">
<div class="m-row">
<div class="src">Linux</div>
<div class="ref">host shell</div>
<div class="claim">The web server is in the host process list.</div>
<div class="kind"><code>ps -ef</code> on your own machine</div>
</div>
<div class="m-row">
<div class="src">macOS</div>
<div class="ref">Lima shell</div>
<div class="claim">The host list shows one virtual machine. The server is inside it.</div>
<div class="kind"><code>limactl shell</code> then <code>ps -ef</code></div>
</div>
<div class="m-row">
<div class="src">Windows</div>
<div class="ref">WSL shell</div>
<div class="claim">The same shape as macOS. Run the command inside the Linux shell.</div>
<div class="kind"><code>wsl</code> then <code>ps -ef</code></div>
</div>
</div>

<ul>
  <li>Inside the container, <code>ps</code> shows one or two processes and nothing else. On the host the same processes are there, under different numbers.</li>
</ul>

Note:

<ul>
  <li>The container process is visible on the Linux host running the engine [V1].</li>
  <li>On macOS and Windows the host is the Linux virtual machine, not the laptop [V9].</li>
  <li>A process in a PID namespace sees itself as PID 1 and cannot see the rest [S18].</li>
  <li>Parentage differs by engine, and Part IV takes that apart.</li>
</ul>

---

<p class="kicker">Part II · Step 1 · An image and a container</p>

<h2>Explain: the engine started a process</h2>

<p class="lede">The engine created nothing new. It started a process with an unusual view of the machine.</p>

<div class="boxes">
<div class="box">
<div class="box-label">On the host</div>
<div class="box-head">An ordinary process</div>
<ul>
<li>It has a process identifier.</li>
<li>It can be listed, signalled and traced.</li>
<li>The engine started it.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">Inside</div>
<div class="box-head">A very short process list</div>
<ul>
<li>The command is process number 1.</li>
<li>Every other process on the machine is missing.</li>
<li>The hostname is different too.</li>
</ul>
</div>
</div>

<p class="closing">Open question 1: one process, two process lists. What makes the second one short?</p>

Note:

<ul>
  <li>The container is an ordinary process. Unrelated kernel interfaces restrict what it sees and what it consumes [V1].</li>
  <li>In a PID namespace the process becomes PID 1 and the others are gone [S18].</li>
  <li>The answer is the PID namespace. Part III teaches it.</li>
</ul>

---

<p class="kicker">Part II · Step 2 · Storing data</p>

<h2>Predict: what survives stop, start and remove?</h2>

<p class="lede">The application writes a file. Write down what happens to that file.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker rm -f web          # clear step 1's container
docker run -d --name web \
  nginx:alpine       # no --rm: it survives a stop
docker exec web sh -c \
  'echo hello > /data.txt'
docker stop web
docker start web
docker exec web cat /data.txt
docker rm -f web
docker run --rm nginx:alpine \
  cat /data.txt
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman rm -f web          # clear step 1's container
podman run -d --name web \
  nginx:alpine       # no --rm: it survives a stop
podman exec web sh -c \
  'echo hello > /data.txt'
podman stop web
podman start web
podman exec web cat /data.txt
podman rm -f web
podman run --rm nginx:alpine \
  cat /data.txt
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Three questions</div>
<div class="v">Is <code>/data.txt</code> there after stop and start? Is it there after remove and a new run? Did the image change?</div>
</div>
</div>

Note:

<ul>
  <li>A writable layer holds the files created inside a container [S64].</li>
  <li>Data on that layer "doesn't persist when the container is destroyed" [S64].</li>
  <li>A written prediction first is the method this course uses [S57].</li>
</ul>

---

<p class="kicker">Part II · Step 2 · Storing data</p>

<h2>Observe: it survives one of the two</h2>

<div class="stats">
<div>
<div class="stat">
<span class="pos">kept</span>
</div>
<div class="stat-cap">after stop and start, same container</div>
</div>
<div>
<div class="stat">
<span class="neg">gone</span>
</div>
<div class="stat-cap">after remove, in a new container</div>
</div>
<div>
<div class="stat">0</div>
<div class="stat-cap">changes to the image, either way</div>
</div>
</div>

<ul>
  <li>Stopping a container does not delete it. Removing it does.</li>
  <li>The new container started from the same image, and the image never held the file.</li>
</ul>

Note:

<ul>
  <li>The writable layer holds files created inside the container [S64].</li>
  <li>That layer does not persist when the container is destroyed [S64].</li>
  <li>An image is a stack of read-only layers, so a running container does not modify it [S8].</li>
</ul>

---

<p class="kicker">Part II · Step 2 · Storing data</p>

<h2>The writable layer, and the image below it</h2>

<p class="lede">An image is a stack of read-only layers. Starting a container puts one writable layer on top of that stack.</p>

```text
   writable layer   <- everything the container writes, deleted with it
   ------------
   layer 3          |
   layer 2          |  read-only, shared by every container from this image
   layer 1          |
```

<ul>
  <li>The topmost layer that holds the file answers a read.</li>
  <li>A write always lands in the writable layer.</li>
  <li>Remove the container and the writable layer goes with it. The image is untouched.</li>
</ul>

<p class="closing">Open question 2: layer is the tool's word. Which kernel mechanism is it?</p>

Note:

<ul>
  <li>The writable layer holds every file created inside a container [S64].</li>
  <li>An image is a stack of read-only layers named by the hash of their content [S8].</li>
  <li>Each layer is one set of filesystem changes [S23].</li>
  <li>Open question 2: a layer is a word from the tool. The kernel mechanism is in Part III.</li>
</ul>

---

<p class="kicker">Part II · Step 3 · Keeping the data</p>

<h2>A bind mount shows a host directory</h2>

<p class="lede">A bind mount makes a directory that already exists on the host appear at a path inside the container. To mount a directory is to make its contents appear at a chosen path.</p>

```bash
# run this in the Linux shell from the setup. /tmp is writable on all three platforms,
# and Lima shares your macOS home read-only, so a directory under ~ would refuse the write
mkdir -p /tmp/work && echo hi > /tmp/work/f.txt
```

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker run --rm -v /tmp/work:/data \
  alpine cat /data/f.txt
docker run --rm -v /tmp/work:/data \
  alpine touch /data/new.txt
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman run --rm -v /tmp/work:/data \
  alpine cat /data/f.txt
podman run --rm -v /tmp/work:/data \
  alpine touch /data/new.txt
```

</div>
</div>

Note:

<ul>
  <li>A volume is engine-managed storage, and a bind mount is an existing host directory [S64].</li>
  <li>The Lima default mounts the macOS home directory read-only, so a bind mount under <code>~</code> refuses the write and <code>/tmp</code> does not [S54].</li>
  <li>On Windows, keep the directory inside the Linux filesystem rather than under <code>/mnt/c</code> [S72].</li>
</ul>

---

<p class="kicker">Part II · Step 3 · Keeping the data</p>

<h2>The pitfall: who owns the files</h2>

<p class="lede">A bind mount can leave files under an owner nobody chose.</p>

<div class="kv">
<div class="pair">
<div class="k">1 · What you see</div>
<div class="v"><code>ls -l /tmp/work</code> shows <code>new.txt</code> under an unexpected owner. Under Docker's root daemon it is root.</div>
</div>
<div class="pair">
<div class="k">2 · Why</div>
<div class="v">Whichever account writes the file decides its owner outside. Rootless Podman maps it into the range reserved in <code>/etc/subuid</code>.</div>
</div>
<div class="pair">
<div class="k">3 · Fix</div>
<div class="v">Run the container under your own number with <code>--user $(id -u)</code>, or add Podman's <code>:U</code> suffix to <code>-v</code>.</div>
</div>
<div class="pair">
<div class="k">4 · Windows</div>
<div class="v">Under <code>/mnt/c</code>, WSL supplies the ownership.</div>
</div>
</div>

Note:

<ul>
  <li>The documented rule is to treat root within a container as root outside of it [S32]. Under Docker's root daemon a container therefore writes host files as root.</li>
  <li>Docker's rootless mode runs the daemon and the containers inside a user namespace [S11].</li>
  <li>It needs at least 65,536 subordinate identifiers for the user [S11].</li>
  <li>Rootless Podman reads <code>/etc/subuid</code> and <code>/etc/subgid</code> and builds the identifier mapping itself [V32].</li>
  <li>A file written by a non-root user inside a rootless container therefore carries a number from the reserved range.</li>
  <li>WSL sets file metadata to <code>disabled</code> by default, so permissions under <code>/mnt/c</code> are supplied rather than stored [S71].</li>
</ul>

---

<p class="kicker">Part II · Step 3 · Keeping the data</p>

<h2>Why a bind mount is not enough</h2>

<p class="lede">A bind mount works. It also ties the container to one machine.</p>

<div class="kv">
<div class="pair">
<div class="k">1 · One machine</div>
<div class="v">The command needs this directory at this path. Another laptop does not have it.</div>
</div>
<div class="pair">
<div class="k">2 · Your real files</div>
<div class="v">The container can change or delete your files on the host.</div>
</div>
<div class="pair">
<div class="k">3 · Across a virtual machine</div>
<div class="v">On macOS and Windows the files cross into the Linux virtual machine. That is slow.</div>
</div>
<div class="pair">
<div class="k">4 · Owners</div>
<div class="v">Files can get an owner nobody chose.</div>
</div>
</div>

<p class="closing">Missing: storage the engine owns, under a name, on any machine.</p>

Note:

<ul>
  <li>Bind mounts "rely on the host machine's filesystem having a specific directory structure available" [S80].</li>
  <li>A process in the container can create, modify or delete files on the host through a bind mount [S80].</li>
  <li>One measurement on one macOS codebase found bind-mounted access to many small files about 3.5 times slower than Linux [S44].</li>
  <li>Moving that directory into a named volume brought the timings back in line with Linux [S44].</li>
  <li>The Lima default shares the home directory read-only [S54].</li>
  <li>On Windows, files under <code>/mnt/c</code> are slower than files inside the Linux filesystem [S72].</li>
  <li>The ownership problem is the previous slide.</li>
</ul>

---

<p class="kicker">Part II · Step 3 · Keeping the data</p>

<h2>A volume outlives the container</h2>

<p class="lede">A volume is storage the engine creates and tracks under a name. Its lifetime is not the container's lifetime, and you name no host path.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker volume create appdata
docker run --rm -v appdata:/data \
  alpine sh -c 'echo hi > /data/f'
docker run --rm -v appdata:/data \
  alpine cat /data/f
docker volume rm appdata
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman volume create appdata
podman run --rm -v appdata:/data \
  alpine sh -c 'echo hi > /data/f'
podman run --rm -v appdata:/data \
  alpine cat /data/f
podman volume rm appdata
```

</div>
</div>

<ul>
  <li>The first container is removed and the file is still there.</li>
  <li>A second container can mount the same volume and read it.</li>
  <li>A volume stays until you delete it.</li>
  <li>The same command works on any machine, because you give the volume a name and not a host path.</li>
</ul>

Note:

<ul>
  <li>Volumes "retain data even after the containers using them are removed" [S64].</li>
  <li>The engine's own wording is that volumes are "completely managed by Docker" [S76].</li>
  <li>Docker calls volumes "the preferred mechanism for persisting data" [S76].</li>
  <li>A named volume needs no directory on the host, so the command does not depend on one machine's layout [S76].</li>
</ul>

---

<p class="kicker">Part II · Step 3 · Keeping the data</p>

<h2>Observe: one write, three destinations</h2>

<div class="matrix">
<div class="m-row">
<div class="src">Writable layer</div>
<div class="ref">no option</div>
<div class="claim">Survives stop and start. Gone after remove.</div>
<div class="kind">Default for every path</div>
</div>
<div class="m-row">
<div class="src">Volume</div>
<div class="ref">-v name:/path</div>
<div class="claim">Survives remove. Readable by the next container.</div>
<div class="kind">Engine creates and tracks it</div>
</div>
<div class="m-row">
<div class="src">Bind mount</div>
<div class="ref">-v /host:/path</div>
<div class="claim">Survives remove. Visible with <code>ls</code> in the shell you started in.</div>
<div class="kind">Host directory you name</div>
</div>
</div>

<p class="closing">Same write, three lifetimes. Open question 3: how does one directory appear inside one process and not the others?</p>

Note:

<ul>
  <li>The writable layer does not persist past the container [S64].</li>
  <li>A volume retains data after the container using it is removed [S64].</li>
  <li>A bind mount reads and writes the host filesystem directly [S64].</li>
  <li>Contrasting the three side by side is what prepares the room for the mechanism [S57].</li>
  <li>The answer to the open question is the mount namespace, in Part III.</li>
</ul>

---

<p class="kicker">Part II · Step 4 · Making it repeatable</p>

<h2>The problem: a run you cannot repeat</h2>

<p class="lede">You typed everything so far. Nothing on your machine records what you typed.</p>

<ul>
  <li>You pulled the image by hand.</li>
  <li>You created the volume by hand.</li>
  <li>You chose the port by hand.</li>
  <li>A second person must repeat every command exactly, and so must you next week.</li>
  <li>Two files fix this. One builds the image. One records the run.</li>
</ul>

<div class="kv">
<div class="pair">
<div class="k">Dockerfile</div>
<div class="v">A recipe that builds an image from a base image and some commands.</div>
</div>
<div class="pair">
<div class="k">Compose file</div>
<div class="v">A record of one run: which image, which volume, which port.</div>
</div>
</div>

Note:

<ul>
  <li>An image is built from a base image plus instructions, one layer per instruction [S23].</li>
  <li>The image configuration's <code>history</code> array records the command that created each layer [S60].</li>
  <li>Making one container repeatable is the goal, and orchestration is out of scope.</li>
</ul>

---

<p class="kicker">Part II · Step 4 · Making it repeatable</p>

<h2>The application: a page that counts its visits</h2>

<p class="lede">Save this as <code>app.py</code> in an empty directory. It is also in the deck's <code>demo/</code> folder.</p>

```python
# app.py: a web page that counts its visits and keeps the count in /data.
from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path

COUNT = Path("/data/count.txt")

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        COUNT.parent.mkdir(parents=True, exist_ok=True)
        n = int(COUNT.read_text()) + 1 if COUNT.exists() else 1
        COUNT.write_text(str(n))
        self.send_response(200)
        self.send_header("Content-Type", "text/plain")
        self.end_headers()
        self.wfile.write(f"visit {n}\n".encode())

HTTPServer(("0.0.0.0", 8080), Handler).serve_forever()
```

<ul>
  <li>With a volume on <code>/data</code>, the count survives removing the container.</li>
</ul>

Note:

<ul>
  <li>The program uses only the Python standard library, so the base image needs nothing else to run it.</li>
  <li>The count lives in <code>/data/count.txt</code>, the same path the volume and the Compose file mount, which ties step 3 to step 4.</li>
  <li>The file ships in the talk's <code>demo/</code> bundle, so a student can download it rather than copy it from a slide.</li>
</ul>

---

<p class="kicker">Part II · Step 4 · Making it repeatable</p>

<h2>A minimal Dockerfile</h2>

<p class="lede">Save this as <code>Dockerfile</code> next to <code>app.py</code>. Each instruction that changes the filesystem adds one layer.</p>

```dockerfile
FROM python:3.12-slim
RUN apt-get update && apt-get install -y --no-install-recommends strace \
    && rm -rf /var/lib/apt/lists/*
COPY app.py /app/app.py
CMD ["python", "/app/app.py"]
```

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker build -t pb158-app .
docker run --rm -d --name app \
  -p 8080:8080 -v appdata:/data pb158-app
curl -s localhost:8080
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman build -t pb158-app .
podman run --rm -d --name app \
  -p 8080:8080 -v appdata:/data pb158-app
curl -s localhost:8080
```

</div>
</div>

Note:

<ul>
  <li>Image layers form a stack, each adding content on top of the ones before it [S23].</li>
  <li>The image configuration marks an instruction that created no layer with <code>empty_layer</code> [S60].</li>
  <li>The build cache is not taught here.</li>
</ul>

---

<p class="kicker">Part II · Step 4 · Making it repeatable</p>

<h2>Observe: one row for each instruction</h2>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker image history pb158-app
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman image history pb158-app
```

</div>
</div>

```text
IMAGE   CREATED         CREATED BY                                 SIZE
<id>    2 minutes ago   CMD ["python" "/app/app.py"]               0B
<id>    2 minutes ago   COPY app.py /app/app.py                    412B
<id>    2 minutes ago   RUN apt-get update && apt-get install ...  4.1MB
...     3 weeks ago     the rows of the python:3.12-slim base
```

<ul>
  <li>The rows read bottom to top, and the base image is at the bottom.</li>
  <li>The <code>CMD</code> row is 0 bytes. It changed no file, so it added no layer.</li>
</ul>

Note:

<ul>
  <li>The <code>history</code> field "Describes the history of each layer", and <code>created_by</code> is the command that created it [S60].</li>
  <li>A history entry with no filesystem change carries <code>empty_layer</code> [S60].</li>
  <li>The rows read bottom to top, and the base image sits at the bottom.</li>
</ul>

---

<p class="kicker">Part II · Step 4 · Making it repeatable</p>

<h2>One file brings the same setup up</h2>

```yaml
services:
  app:
    build: .
    ports: ["8080:8080"]
    volumes: ["appdata:/data"]
volumes:
  appdata:
```

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker rm -f app 2>/dev/null
docker compose up -d
docker compose down
```

<p>One command builds the image, creates the volume and publishes the port.</p>
</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman rm -f app 2>/dev/null
sudo apt install podman-compose
podman compose up -d
podman compose down
```

<p>Without <code>podman-compose</code> the command reports no provider.</p>
</div>
</div>

<ul>
  <li>Compose is the one place where the engines differ in substance.</li>
  <li>It names its volume after the project, so the count starts again.</li>
</ul>

Note:

<ul>
  <li>"podman compose is a thin wrapper around an external compose provider such as docker-compose or podman-compose" [S66].</li>
  <li>Docker ships the compose implementation with the engine, and Podman ships only the wrapper [S66].</li>
  <li>The file records one container's image, volume and port.</li>
  <li>Compose names its volume after the project, so the run does not reuse the <code>appdata</code> volume.</li>
  <li>Several services in one file is orchestration and is out of scope.</li>
</ul>

---

<h2>Predict: how much memory will it report?</h2>

<p class="lede">You start the container with a hard memory limit. Write down what the tool inside will print.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker run --rm --memory 256m \
  alpine sh -c 'free -h; nproc'
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman run --rm --memory 256m \
  alpine sh -c 'free -h; nproc'
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Question 1</div>
<div class="v">Does <code>free -h</code> inside print 256 MB, or the whole machine? Does <code>nproc</code> count the host processors?</div>
</div>
<div class="pair">
<div class="k">Question 2</div>
<div class="v">Does the limit actually hold if the program allocates 512 MB?</div>
</div>
</div>

<ul>
  <li>Keep the figure small. On Windows the virtual machine already took half of the memory.</li>
</ul>

Note:

<ul>
  <li><code>--memory</code> sets a memory limit on the container [S61], and a control group is what enforces it [S2].</li>
  <li>Docker states that <code>free</code> inside a container reports "the host's available swap" [S68].</li>
  <li><code>/proc/meminfo</code> and its neighbours "will always display memory numbers from the host system" [S41].</li>
  <li>The WSL virtual machine defaults to 50 percent of Windows memory [S71].</li>
</ul>

---

<h2>Observe: the limit holds, the report does not</h2>

<div class="boxes">
<div class="box">
<div class="box-label">The limit</div>
<div class="box-head">Enforced</div>
<ul>
<li>Allocate 512 MB under a 256 MB limit, and the kernel kills the process.</li>
<li>The kernel enforces the limit. The engine does not.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">The report</div>
<div class="box-head">The host total</div>
<ul>
<li><code>free -h</code> prints the whole machine's memory, and so does <code>/proc/meminfo</code>.</li>
<li><code>nproc</code> counts the host processors. The memory limit does not change that number.</li>
</ul>
</div>
</div>

<p class="closing">Open question 4: the limit is real and the report ignores it. Two different mechanisms are at work.</p>

Note:

<ul>
  <li><code>--memory</code> sets the limit the container runs under [S61], and a control group is what holds it [S2].</li>
  <li>An allocation above that limit fails inside the kernel, so the kernel stops the process and the engine does not.</li>
  <li>Docker states that <code>free</code> inside a container reports the host's available swap [S68].</li>
  <li><code>/proc/meminfo</code> and its neighbours "will always display memory numbers from the host system" [S41].</li>
  <li>No namespace covers the processor count [S1].</li>
  <li>A control group limits and measures consumption, and it does not change what a process reads [S2].</li>
  <li>So <code>nproc</code> inside the container counts the host processors.</li>
  <li>Part III names the two mechanisms and why only one of them hides anything.</li>
</ul>

---

<h2>Predict: two tracing questions, not one</h2>

<p class="lede">The same tool, two targets. Write down whether each one works.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
# A: the tracer starts the target
docker run --rm alpine sh -c \
  'apk add -q strace; strace -c /bin/true'

# B: attach to a running process
docker rm -f web 2>/dev/null
docker run --rm -d --name web nginx:alpine
docker exec web sh -c \
  'apk add -q strace; strace -p 1'
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
# A: the tracer starts the target
podman run --rm alpine sh -c \
  'apk add -q strace; strace -c /bin/true'

# B: attach to a running process
podman rm -f web 2>/dev/null
podman run --rm -d --name web nginx:alpine
podman exec web sh -c \
  'apk add -q strace; strace -p 1'
```

</div>
</div>

<ul>
  <li>Case A starts its own target.</li>
  <li>Case B attaches to a process that is already running.</li>
</ul>

Note:

<ul>
  <li><code>strace</code> in case A traces its own child [S67].</li>
  <li><code>strace</code> in case B attaches to a process it did not start [S67].</li>
  <li>The default capability set of both engines excludes <code>CAP_SYS_PTRACE</code> [S42] [S43].</li>
</ul>

---

<h2>Observe: one works, one refuses</h2>

<div class="matrix">
<div class="m-row">
<div class="src">Case A</div>
<div class="ref">own child</div>
<div class="claim">It runs, and prints the calls it saw. No extra option was given.</div>
<div class="kind">Usually works on a current kernel</div>
</div>
<div class="m-row">
<div class="src">Case B</div>
<div class="ref">attach</div>
<div class="claim"><code>strace: attach: ptrace(PTRACE_SEIZE, 1): Operation not permitted</code></div>
<div class="kind">Refused</div>
</div>
<div class="m-row">
<div class="src">Case B again</div>
<div class="ref">--cap-add SYS_PTRACE</div>
<div class="claim">Add the option to the <code>run</code> above, and the same attach succeeds.</div>
<div class="kind">One option changed the answer</div>
</div>
</div>

```bash
docker rm -f web 2>/dev/null                      # podman: the same, with podman
docker run --rm -d --name web --cap-add SYS_PTRACE nginx:alpine
docker exec web sh -c 'apk add -q strace; strace -p 1'
```

<p class="closing">Open question 5: one option, and the refused command was allowed to run. What did it switch on?</p>

Note:

<ul>
  <li>Attaching needs <code>CAP_SYS_PTRACE</code>, or a target that is a descendant of the tracer [S67].</li>
  <li>Neither engine grants <code>CAP_SYS_PTRACE</code> by default [S42] [S43].</li>
  <li>Adding the option also allowlists the related calls in the filter [S17].</li>
  <li>The refusal is the machinery working as documented, and is not a broken tool [V10].</li>
</ul>

---

<h2>Observe: what the option switched on</h2>

<p class="lede">A capability is one named piece of root's power, such as the right to trace another process. The kernel lists a process's capabilities in <code>/proc/self/status</code>.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker run --rm alpine sh -c '
  apk add -q libcap
  grep Cap /proc/self/status |
  while read n h; do
    echo "$n $(capsh --decode=$h)"
  done'
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman run --rm alpine sh -c '
  apk add -q libcap
  grep Cap /proc/self/status |
  while read n h; do
    echo "$n $(capsh --decode=$h)"
  done'
```

</div>
</div>

<ul>
  <li>Run it twice, once with <code>--cap-add SYS_PTRACE</code> after <code>run</code>.</li>
  <li>Without the option, the <code>CapEff</code> line has no <code>cap_sys_ptrace</code>. With it, <code>cap_sys_ptrace</code> joins the list.</li>
  <li>The two engines start from different lists. Docker keeps <code>cap_net_raw</code>, <code>cap_mknod</code> and <code>cap_audit_write</code>, and Podman drops them.</li>
</ul>

Note:

<ul>
  <li>A capability is a distinct unit of the privileges traditionally associated with the superuser, which the kernel enables and disables one by one [S3].</li>
  <li><code>/proc/<pid>/status</code> shows a process's capability sets, and <code>CapEff</code> is the effective set that the kernel checks [S3].</li>
  <li><code>CAP_SYS_PTRACE</code> is the capability that allows tracing a process the tracer did not start [S3] [S67].</li>
  <li>Verified 2026-09-29 on kernel 6.8 with Podman 4.9.3 and Docker 29.7.2: the default <code>CapEff</code> lacks <code>cap_sys_ptrace</code> on both engines, and <code>--cap-add SYS_PTRACE</code> adds it.</li>
  <li>Docker's default set holds three capabilities that Podman's does not, <code>cap_net_raw</code>, <code>cap_mknod</code> and <code>cap_audit_write</code> [S42] [S43].</li>
</ul>

---

<p class="kicker">Part II · Five open questions</p>

<h2>Five questions the commands left open</h2>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">Two process lists</div>
<div class="desc">One process, two views. What makes the second one short?</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">The vanishing file</div>
<div class="desc">What is a layer, and why is the image unchanged?</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">The mounted directory</div>
<div class="desc">How does a directory appear inside one process only?</div>
</div>
<div class="row">
<div class="num">4</div>
<div class="term">The honest limit</div>
<div class="desc">Why does the limit hold while the report shows the host?</div>
</div>
<div class="row">
<div class="num">5</div>
<div class="term">The one option</div>
<div class="desc">What did the one option change?</div>
</div>
</div>

<p class="closing">Five questions, five kernel mechanisms, in this order.</p>

Note:

<ul>
  <li>The five mechanisms together are what the word container means [V1].</li>
</ul>

---

<h1>What is underneath</h1>

<p class="lede">Part II raised five questions that running the commands could not answer. Part III answers them with five kernel mechanisms, in the same order.</p>

Note:

Part III answers Part II. Same application, same commands, new vocabulary.

---

<p class="kicker">Part III · Question 1</p>

<h2>Nothing was hidden from the host</h2>

<p class="lede">The container process was in the host list all along. What changed was the list the process itself can read.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker rm -f web 2>/dev/null
docker run --rm -d --name web nginx:alpine
docker inspect -f '{{.State.Pid}}' web
docker exec web ps -o pid,comm
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman rm -f web 2>/dev/null
podman run --rm -d --name web nginx:alpine
podman inspect -f '{{.State.Pid}}' web
podman exec web ps -o pid,comm
```

</div>
</div>

<ul>
  <li>The second command prints a host process identifier. The third prints 1, and almost nothing else.</li>
  <li>The host numbering did not change. A second numbering was added.</li>
  <li>Inside that second numbering the command is process 1, and the host's processes have no number at all.</li>
  <li>A process with no number in your table is a process you cannot signal.</li>
</ul>

Note:

<ul>
  <li>In a PID namespace the process becomes PID 1 and the other programs are gone [S18].</li>
  <li>A namespace "wraps a global system resource in an abstraction" so the processes inside see their own instance [S1].</li>
  <li>The resource wrapped here is the process identifier space [S1].</li>
</ul>

---

<p class="kicker">Part III · Question 1</p>

<h2>A namespace is a kernel object, not a file</h2>

<p class="lede">A namespace is a kernel object that wraps one global resource, so the processes inside it see their own instance. The entries under <code>/proc</code> are handles to it.</p>

```bash
ls -l /proc/self/ns/
# pid -> 'pid:[4026531836]'    uts -> 'uts:[4026531838]'
```

<div class="kv">
<div class="pair">
<div class="k">1 · The link</div>
<div class="v">Each entry is a symbolic link whose target names the type and a number.</div>
</div>
<div class="pair">
<div class="k">2 · The handle</div>
<div class="v">Opening one returns a file handle for that namespace, and keeps it alive while open.</div>
</div>
<div class="pair">
<div class="k">3 · The test</div>
<div class="v">Two processes are in the same namespace when the two numbers match.</div>
</div>
</div>

<p class="closing">Compare the numbers for a container process and for your shell.</p>

Note:

<ul>
  <li>Each <code>/proc/pid/ns/</code> entry is a symbolic link naming the type and an inode number [S1].</li>
  <li>Opening one "returns a file handle for the corresponding namespace" [S1].</li>
  <li><code>/proc</code> exposes the kernel object and does not represent it as a file [V11].</li>
</ul>

---

<p class="kicker">Part III · Question 1</p>

<h2>One idea, applied eight times</h2>

<p class="lede">Linux defines eight namespace types. Each one wraps one global resource.</p>

| Type | Flag | What the process sees its own copy of |
|---|---|---|
| PID | `CLONE_NEWPID` | Process identifiers |
| Mount | `CLONE_NEWNS` | Mount points |
| Network | `CLONE_NEWNET` | Network devices, addresses and ports |
| User | `CLONE_NEWUSER` | User and group identifiers |
| IPC | `CLONE_NEWIPC` | Shared memory and message queues |
| UTS | `CLONE_NEWUTS` | The hostname |
| Cgroup | `CLONE_NEWCGROUP` | Its own position in the kernel's resource-limit tree |
| Time | `CLONE_NEWTIME` | The boot and monotonic clocks |

<ul>
  <li>The eight types are independent, and UTS is the kernel's old name for the hostname.</li>
</ul>

Note:

<ul>
  <li>The kernel documents eight namespace types with these flags [S1].</li>
  <li>The IPC namespace partitions System V IPC and POSIX message queues [V18].</li>
  <li>A container takes a private set of namespaces, and the set is a choice [V1].</li>
</ul>

---

<p class="kicker">Part III · Question 1</p>

<h2>Three system calls, all traceable</h2>

<p class="lede">No special interface creates a namespace. Three ordinary system calls do, and <code>strace</code> shows them.</p>

<div class="kv">
<div class="pair">
<div class="k"><code>clone(2)</code></div>
<div class="v">Creates a process. With one or more <code>CLONE_NEW*</code> flags it creates new namespaces at the same time.</div>
</div>
<div class="pair">
<div class="k"><code>unshare(2)</code></div>
<div class="v">Moves the calling process into a new namespace.</div>
</div>
<div class="pair">
<div class="k"><code>setns(2)</code></div>
<div class="v">Joins an existing namespace, named by a file descriptor from <code>/proc</code>.</div>
</div>
</div>

```bash
sudo unshare --pid --fork --mount-proc ps -ef   # a two-line process list
```

<ul>
  <li><code>--mount-proc</code> gives the new namespace its own <code>/proc</code>, which is what <code>ps</code> reads. No engine ran.</li>
  <li>Part IV traces both engines and names the program that makes the call.</li>
</ul>

Note:

<ul>
  <li><code>clone(2)</code> creates new namespaces when the flags request them [S1].</li>
  <li>Part IV traces <code>docker run</code> and <code>podman run</code> and finds the caller.</li>
  <li><code>unshare(2)</code> moves the caller to a new namespace, and <code>setns(2)</code> joins one by descriptor [S1].</li>
  <li>A short program that calls <code>clone</code> with these flags builds a container without an engine [S56].</li>
</ul>

---

<p class="kicker">Part III · Question 1</p>

<h2>The user namespace is the load-bearing one</h2>

<p class="lede">Only one namespace type changes which account counts as root inside it.</p>

<ul>
  <li>The user namespace remaps user and group numbers, so root inside it is an ordinary account outside.</li>
  <li>Creating any other namespace type needs administrative privilege an ordinary account does not hold.</li>
  <li>Creating a user namespace does not, unless the Linux distribution switches that off. Ubuntu restricts it by default from release 23.10.</li>
  <li>That single exception is what containers without root rest on.</li>
</ul>

Note:

<ul>
  <li><code>CLONE_NEWUSER</code> appeared in 2.6.23 and the final pieces landed in 3.8 [S21].</li>
  <li>Every namespace type other than the user namespace requires <code>CAP_SYS_ADMIN</code> to create [S21].</li>
  <li>The user namespace is the one that isolates user and group identifiers [S1].</li>
  <li>Ubuntu 24.04 "restricts the use of unprivileged user namespaces" by default [S48].</li>
  <li>Rootless containers are a consequence of that single decision [V1].</li>
  <li>Part IV shows what both engines build on top of it.</li>
</ul>

---

<p class="kicker">Part III · Question 2</p>

<h2>Union filesystem: what it sees as root</h2>

<p class="lede">A union filesystem presents several read-only directories plus one writable directory as a single tree.</p>

<p class="lede">On Linux this filesystem is <code>overlayfs</code>, and it takes three directories as mount options.</p>

<div class="kv">
<div class="pair">
<div class="k">lowerdir</div>
<div class="v">The read-only image layers, stacked.</div>
</div>
<div class="pair">
<div class="k">upperdir</div>
<div class="v">The container's writable directory.</div>
</div>
<div class="pair">
<div class="k">merged</div>
<div class="v">The tree the process sees as its root.</div>
</div>
</div>

<ul>
  <li>The kernel implements this. The engine's storage driver names which filesystem does the work.</li>
  <li>The union filesystem serves a read from the topmost layer that holds the file. A write goes to the writable directory.</li>
</ul>

<p class="closing">A layer is one directory in this stack.</p>

Note:

<ul>
  <li>The kernel describes an overlay filesystem as presenting the result of overlaying one filesystem on another [S74].</li>
  <li>Docker's driver names the three directories <code>lowerdir</code>, <code>upperdir</code> and <code>merged</code> [S6].</li>
  <li>The storage driver is a name for a kernel filesystem, not for a vendor invention.</li>
</ul>

---

<p class="kicker">Part III · Question 2</p>

<h2>Layering is copy-on-write, one instruction at a time</h2>

<p class="lede">The first write to an existing file copies it up from the image layer into the writable layer.</p>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">Share</div>
<div class="desc">Every container from one image reads the same layer directories.</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">Protect</div>
<div class="desc">The kernel mounts those directories read-only.</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">Copy on write</div>
<div class="desc">The first write to a file copies that file into the writable layer, and the write lands on the copy.</div>
</div>
</div>

<ul>
  <li>The copy happens on the first write to each file, not on the first write overall.</li>
  <li><code>fork(2)</code> runs this same protocol on memory pages instead of files.</li>
</ul>

Note:

<ul>
  <li>The first write to an existing file triggers a <code>copy_up</code> from the image layer [S6].</li>
  <li>The trap fires on the first write to each unit, not the first write overall [V19].</li>
  <li><code>fork(2)</code> runs the same protocol at page granularity [V19].</li>
  <li>One build instruction produces one layer in the stack [S23].</li>
</ul>

---

<p class="kicker">Part III · Question 2</p>

<h2>The unit of sharing is a whole file</h2>

<p class="lede">A union filesystem works at file level, not block level. The copy is never partial.</p>

<div class="stats">
<div>
<div class="stat">
<span class="neg">2 GB</span>
</div>
<div class="stat-cap">the overlay driver copies when a process appends one byte to a 2 GB file</div>
</div>
<div>
<div class="stat">128</div>
<div class="stat-cap">read-only layers the default overlay driver supports at once</div>
</div>
<div>
<div class="stat">
<span class="pos">1</span>
</div>
<div class="stat-cap">copy of a file held in memory, shared by every container reading it</div>
</div>
</div>

<ul>
  <li>A database writing small records into an image layer therefore copies each file whole before its first change.</li>
  <li>It is also why fifty containers from one image cost one copy of that image's files in memory.</li>
</ul>

Note:

<ul>
  <li>All <code>copy_up</code> operations copy the entire file, however small the change [S6].</li>
  <li>The overlay2 driver supports up to 128 lower layers [S6].</li>
  <li>"Multiple containers accessing the same file share a single page cache entry for that file" [S6].</li>
  <li>Modifying a large file in an image layer copies all of it first [V20].</li>
</ul>

---

<p class="kicker">Part III · Question 3</p>

<h2>The mount namespace, and what a volume really is</h2>

<p class="lede">A mount namespace gives the process its own table of mount points.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker run --rm -v appdata:/data alpine \
  cat /proc/self/mounts | grep /data
# an entry the host's own table does not have
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman run --rm -v appdata:/data alpine \
  cat /proc/self/mounts | grep /data
# an entry the host's own table does not have
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">1 · The kernel part</div>
<div class="v">The mount namespace. Mount points added inside it are invisible outside.</div>
</div>
<div class="pair">
<div class="k">2 · The engine part</div>
<div class="v">Creating the directory, remembering its name, and asking for the mount.</div>
</div>
<div class="pair">
<div class="k">3 · The consequence</div>
<div class="v">A volume is a host directory placed into that table, so it survives the container.</div>
</div>
</div>

Note:

<ul>
  <li>The mount namespace isolates mount points [S1].</li>
  <li>The OCI runtime bind-mounts volumes while setting the container up [V29].</li>
  <li>The engine says a volume is "completely managed by Docker" [S76].</li>
  <li>That wording describes the engine's bookkeeping and names no kernel object.</li>
  <li>Volumes retain data after the containers using them are removed [S64].</li>
</ul>

---

<p class="kicker">Part III · Question 4</p>

<h2>Control groups: what it can consume</h2>

<p class="lede">A control group organises processes into a hierarchy whose use of a resource can be limited and measured.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker run --rm --memory 256m alpine \
  cat /sys/fs/cgroup/memory.max
# 268435456
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman run --rm --memory 256m alpine \
  cat /sys/fs/cgroup/memory.max
# 268435456
```

</div>
</div>

<ul>
  <li>Version 2 collects every resource type in one tree, and only the tree's outermost groups may hold processes.</li>
  <li>The resource types that matter here are memory, processor time, process count and input or output.</li>
  <li>A control group constrains what a process may consume. It changes nothing a process reads.</li>
</ul>

Note:

<ul>
  <li>Control groups organise processes into hierarchical groups whose resource use is limited and monitored [S2].</li>
  <li>Under version 2 "all mounted controllers reside in a single unified hierarchy" [S2].</li>
  <li>Version 2 requires that processes reside only in leaf nodes [S2].</li>
  <li>Resource limits on a container without root require version 2 [S61].</li>
</ul>

---

<p class="kicker">Part III · Question 4</p>

<h2>Namespaces hide, control groups constrain</h2>

<p class="lede">Two mechanisms, two different verbs. The memory result needs both words to explain it.</p>

<div class="boxes">
<div class="box">
<div class="box-label">Namespace</div>
<div class="box-head">Changes what is visible</div>
<ul>
<li>Process identifiers, mounts, hostname, network.</li>
<li>Nothing about consumption.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">Control group</div>
<div class="box-head">Changes what is allowed</div>
<ul>
<li>Memory, processor time, process count.</li>
<li>Nothing about visibility.</li>
</ul>
</div>
</div>

<p class="closing">The limit held because a control group enforced it. The report showed the host because no namespace covers that file.</p>

Note:

<ul>
  <li>A namespace makes processes inside see their own instance of one resource [S1].</li>
  <li>A control group limits and measures resource usage [S2].</li>
  <li>The two are configured separately and do different work [V1].</li>
</ul>

---

<p class="kicker">Part III · Question 4</p>

<h2>Nobody gave <code>/proc/meminfo</code> a namespace</h2>

<p class="lede">No kernel change fixes the memory report. A userspace workaround covers those paths instead.</p>

<ul>
  <li><code>/proc/meminfo</code> and its neighbours always display numbers from the host system.</li>
  <li>Docker's own documentation states that <code>free</code> inside a container reports the host's swap.</li>
  <li>A separate project, LXCFS, mounts a small userspace filesystem over those paths so the numbers match the limit.</li>
  <li>It is a workaround in userspace, and no kernel change replaces it.</li>
</ul>

Note:

<ul>
  <li><code>/proc/meminfo</code>, <code>/proc/vmstat</code> and friends "are not containerized" and always show the host [S41].</li>
  <li>Docker states that <code>free</code> inside a container reports the host's available swap [S68].</li>
  <li>LXCFS is "a small FUSE filesystem written with the intention of making Linux containers feel more like a virtual machine" [S69].</li>
  <li>A container whose memory numbers match its limit is still one process on the host kernel.</li>
</ul>

---

<p class="kicker">Part III · Question 5</p>

<h2>Capabilities: how much of root is left</h2>

<p class="lede">Linux divides the privilege traditionally held by the superuser into distinct units, each switchable on its own. One such unit is a capability.</p>

<div class="kv">
<div class="pair">
<div class="k"><code>CAP_SYS_ADMIN</code></div>
<div class="v">Mounting filesystems, setting the hostname, and creating namespaces. Close to root on its own.</div>
</div>
<div class="pair">
<div class="k"><code>CAP_SYS_PTRACE</code></div>
<div class="v">Tracing arbitrary processes and reading their memory. Not in either engine's default set.</div>
</div>
<div class="pair">
<div class="k"><code>CAP_NET_RAW</code></div>
<div class="v">Raw and packet sockets. In Docker's default set and not in Podman's.</div>
</div>
</div>

<ul>
  <li>The engine drops most capabilities before it starts the container, and you can print the set.</li>
</ul>

Note:

<ul>
  <li>Linux divides superuser privilege into independently switchable capabilities [S3].</li>
  <li><code>CAP_SYS_ADMIN</code> permits mounting filesystems, setting the hostname and creating namespaces [S3].</li>
  <li><code>CAP_SYS_PTRACE</code> permits tracing arbitrary processes and reading their memory [S3].</li>
  <li><code>CAP_NET_RAW</code> permits raw and packet sockets [S3].</li>
  <li>Docker's default set adds <code>CAP_NET_RAW</code> [S43], and Podman's eleven defaults do not include it [S42].</li>
  <li>Neither default set holds <code>CAP_SYS_PTRACE</code>: Podman's eleven [S42], or Docker's fourteen [S43].</li>
  <li>The manual page warns against assigning new features to <code>CAP_SYS_ADMIN</code>, whose scope is already excessive [S3].</li>
</ul>

---

<p class="kicker">Part III · Question 5</p>

<h2>Read the default set from the engine's own source</h2>

<p class="lede">The default is not folklore. Both engines declare it in one file of their source code.</p>

<div class="kv">
<div class="pair">
<div class="k">Podman</div>
<div class="v">Eleven capabilities, from CHOWN to SYS_CHROOT, in <code>pkg/config/default.go</code>.</div>
</div>
<div class="pair">
<div class="k">Docker</div>
<div class="v">The same eleven plus MKNOD, NET_RAW and AUDIT_WRITE, in <code>daemon/pkg/oci/caps/defaults.go</code>.</div>
</div>
<div class="pair">
<div class="k">Both</div>
<div class="v">Neither list contains SYS_PTRACE.</div>
</div>
</div>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker run --rm alpine sh -c \
  'grep Cap /proc/self/status'
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman run --rm alpine sh -c \
  'grep Cap /proc/self/status'
```

</div>
</div>

Note:

<ul>
  <li>Podman's default set lists eleven capabilities and no <code>CAP_SYS_PTRACE</code> [S42].</li>
  <li>Docker's default set adds <code>CAP_MKNOD</code>, <code>CAP_NET_RAW</code> and <code>CAP_AUDIT_WRITE</code> to those eleven [S43].</li>
  <li>The capability set of a running process can be read from <code>/proc</code> [S3].</li>
</ul>

---

<p class="kicker">Part III · Question 5</p>

<h2>Seccomp: which calls it may issue</h2>

<p class="lede">A seccomp filter is a small program the kernel runs before each system call. It returns allow, an error, a kill, or a notification to a tracer.</p>

<ul>
  <li>Docker's default profile refuses about 44 of more than 300 system calls.</li>
  <li><code>mount</code> is refused, with the stated reason that the capability already gates it.</li>
  <li>Installing a filter needs <code>CAP_SYS_ADMIN</code>, or the <code>no_new_privs</code> setting, which stops the process gaining privilege later.</li>
  <li>The capability set and the filter are coupled in practice. Granting a capability also allowlists the calls that go with it.</li>
</ul>

Note:

<ul>
  <li>The default profile "disables around 44 system calls out of 300+" [S5].</li>
  <li><code>mount</code> carries the reason "Deny mounting, already gated by <code>CAP_SYS_ADMIN</code>" [S5].</li>
  <li>Installing a filter requires <code>CAP_SYS_ADMIN</code> or the <code>no_new_privs</code> bit [S4].</li>
  <li>In containerd, a process that has a capability is also given the system calls that go with it [S17].</li>
</ul>

---

<p class="kicker">Part III · Question 5</p>

<h2>The refusal named a mechanism</h2>

<p class="lede">A tracer attaches in two cases only. The Yama kernel module applies both.</p>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">Capability</div>
<div class="desc">The tracer holds <code>CAP_SYS_PTRACE</code>. The option grants it.</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">Relationship</div>
<div class="desc">Or the target is a descendant of the tracer.</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">Yama</div>
<div class="desc">Yama applies the limit that <code>kernel.yama.ptrace_scope</code> sets.</div>
</div>
<div class="row">
<div class="num">4</div>
<div class="term">The filter</div>
<div class="desc">A seccomp filter refuses separately, on the call itself.</div>
</div>
</div>

<p class="closing">Read the refusal. <code>--privileged</code> switches every restriction off.</p>

Note:

<ul>
  <li>Yama is a Linux Security Module, and <code>kernel.yama.ptrace_scope</code> sets how far a tracer may reach [S84].</li>
  <li>Scope 1 is the default. There a tracer attaches only to its own descendants, unless it holds <code>CAP_SYS_PTRACE</code> or the target declared it a tracer [S84] [S67].</li>
  <li>At scope 2 only a holder of <code>CAP_SYS_PTRACE</code> may attach at all [S84].</li>
  <li>Under Yama's default setting, attaching requires <code>CAP_SYS_PTRACE</code> or a predefined relationship, by default a descendant of the caller [T5] [S67].</li>
  <li>The manual page names Yama as one enabled Linux Security Module, beside SELinux and Smack, and the tracing checks depend on it [S67].</li>
  <li>Adding the capability also allowlists <code>ptrace</code>, <code>process_vm_readv</code>, <code>process_vm_writev</code> and <code>kcmp</code> [S17].</li>
  <li>Docker's profile blocked <code>ptrace</code> only on kernels before 4.8 [S5].</li>
  <li><code>--privileged</code> and a mounted engine socket are the absence of a boundary [V8].</li>
</ul>

---

<p class="kicker">Part III · The five together</p>

<h2>One file holds all five requests</h2>

<p class="lede">One configuration file is written before a container starts. Every mechanism in this part has a field in it.</p>

```json
{ "root":    { "path": "rootfs" },
  "process": { "capabilities": { "bounding": ["CAP_CHOWN", "CAP_KILL"] } },
  "linux": {
    "namespaces":  [ {"type": "pid"}, {"type": "mount"}, {"type": "ipc"},
                     {"type": "uts"}, {"type": "network"} ],
    "cgroupsPath": "/machine.slice/container.scope",
    "resources":   { "memory": { "limit": 268435456 } },
    "seccomp":     { "defaultAction": "SCMP_ACT_ERRNO", "syscalls": [] } } }
```

<ul>
  <li><code>root</code> names the tree the union filesystem assembled. <code>process.capabilities</code> says how much of root is left.</li>
  <li><code>namespaces</code> says what the process can see. <code>cgroupsPath</code> and <code>resources</code> say what it can consume. <code>seccomp</code> says which calls it may issue.</li>
  <li>This is an excerpt. Each mechanism from this part has its own field, and one file describes one container.</li>
</ul>

Note:

<ul>
  <li>The configuration's <code>root</code> object names the container's root filesystem path [S81].</li>
  <li><code>process.capabilities</code> carries the capability sets of the container process [S81].</li>
  <li>The Linux section of the file defines <code>namespaces</code>, <code>cgroupsPath</code>, <code>resources</code> and <code>seccomp</code> [S7].</li>
  <li>The <code>namespaces</code> array names each namespace, and an optional <code>path</code> joins an existing one [S7].</li>
  <li><code>cgroupsPath</code> names the control group [S7].</li>
  <li><code>resources</code> carries the resource limits [S7].</li>
  <li><code>seccomp</code> carries the system-call filter [S7].</li>
  <li>The OCI runtime reads this file and creates the namespaces [V29].</li>
  <li>The OCI runtime then attaches the control group and applies the SELinux labels [V29].</li>
  <li>The OCI runtime loads the seccomp filter and bind-mounts the volumes [V29].</li>
</ul>

---

<p class="kicker">Part III · The five together</p>

<h2>A container is a list of requests</h2>

<p class="lede">There is no <code>struct container</code> in Linux, no call that creates one, and no subsystem that owns the word.</p>

<div class="kv">
<div class="pair">
<div class="k">1 · Namespaces</div>
<div class="v">What the process can see.</div>
</div>
<div class="pair">
<div class="k">2 · Control group</div>
<div class="v">What it can consume.</div>
</div>
<div class="pair">
<div class="k">3 · Capabilities</div>
<div class="v">How much of root it keeps.</div>
</div>
<div class="pair">
<div class="k">4 · Seccomp filter</div>
<div class="v">Which system calls it may issue.</div>
</div>
<div class="pair">
<div class="k">5 · Union filesystem</div>
<div class="v">What it sees as its root directory.</div>
</div>
</div>

<p class="closing">Remove any one and something still runs. You just have less of a container.</p>

Note:

<ul>
  <li>A container is an ordinary process. Unrelated kernel interfaces restrict what it sees and what it consumes [T1] [V1].</li>
  <li>"Remove any one of those five and you still have something that runs" [V2].</li>
  <li>The question "is this a container?" has no answer, and "which of the five did this command request?" does [V2].</li>
  <li>The kernel maintainers declined the word, and renamed process containers to control groups [V6].</li>
</ul>

---

<h1>Two architectures</h1>

<p class="lede">The same five requests. Two different answers to one question: which process owns the container?</p>

Note:

Part IV compares process architectures, not kernel usage.

---

<p class="kicker">Part IV · The tiers</p>

<h2>Engine, OCI runtime, and the work</h2>

<p class="lede">Two programs, two jobs. The engine writes the request. The OCI runtime issues it.</p>

<div class="compare">
<div>
<p class="kicker">The engine</p>
<ul>
<li>Resolves the image name.</li>
<li>Unpacks the layers.</li>
<li>Writes the configuration file.</li>
</ul>
</div>
<div class="side-b">
<p class="kicker">The OCI runtime</p>
<ul>
<li>Reads that file.</li>
<li>Creates the namespaces.</li>
<li>Attaches the control group.</li>
<li>Sets the capability set and installs the seccomp filter.</li>
<li>Mounts the volumes.</li>
<li>Starts the command.</li>
</ul>
</div>
</div>

<ul>
  <li><code>runc</code>, <code>crun</code> and <code>youki</code> are OCI runtimes. They implement one standard, published by the Open Container Initiative.</li>
  <li>Changing the engine does not change this tier. Both engines can use either OCI runtime.</li>
</ul>

Note:

<ul>
  <li>The OCI runtime creates the namespaces and attaches the control group [V29].</li>
  <li>The OCI runtime applies the SELinux labels and loads the seccomp filter [V29].</li>
  <li>The OCI runtime bind-mounts the volumes and forks the container process [V29].</li>
  <li>The three OCI runtimes are interchangeable implementations of one specification [V16].</li>
  <li>The Open Container Initiative publishes that runtime specification [S7] [S9].</li>
  <li>Podman's default OCI runtime is <code>crun</code>, selectable with the <code>--runtime</code> option [S13].</li>
</ul>

---

<p class="kicker">Part IV · Docker</p>

<h2>Docker: a client, a daemon, and two tiers</h2>

<p class="lede">A daemon is a program that keeps running after the command that started it exits, and owns resources on the user's behalf.</p>

<g id="node1" class="node">
<title>docker (client)</title>
<polygon fill="none" stroke="black" points="93,-36 0,-36 0,0 93,0 93,-36"/>
<text text-anchor="middle" x="46.5" y="-15.2" font-family="Inter" font-size="11.00">docker (client)</text>
</g>

<g id="node2" class="node">
<title>dockerd (daemon)</title>
<polygon fill="none" stroke="black" points="312,-36 199,-36 199,0 312,0 312,-36"/>
<text text-anchor="middle" x="255.5" y="-15.2" font-family="Inter" font-size="11.00">dockerd (daemon)</text>
</g>

<g id="edge1" class="edge">
<title>docker (client)->dockerd (daemon)</title>
<path fill="none" stroke="black" d="M93.18,-18C121.21,-18 157.56,-18 188.66,-18"/>
<polygon fill="black" stroke="black" points="188.99,-21.5 198.99,-18 188.99,-14.5 188.99,-21.5"/>
<text text-anchor="middle" x="146" y="-21.8" font-family="Times,serif" font-size="14.00">HTTP API</text>
</g>

<g id="node3" class="node">
<title>containerd</title>
<polygon fill="none" stroke="black" points="422,-36 349,-36 349,0 422,0 422,-36"/>
<text text-anchor="middle" x="385.5" y="-15.2" font-family="Inter" font-size="11.00">containerd</text>
</g>

<g id="edge2" class="edge">
<title>dockerd (daemon)->containerd</title>
<path fill="none" stroke="black" d="M312.27,-18C321.08,-18 330.11,-18 338.66,-18"/>
<polygon fill="black" stroke="black" points="338.81,-21.5 348.81,-18 338.81,-14.5 338.81,-21.5"/>
</g>

<g id="edge3" class="edge">
<title>containerd->shim</title>
<path fill="none" stroke="black" d="M422.06,-18C430.68,-18 439.91,-18 448.56,-18"/>
<polygon fill="black" stroke="black" points="448.77,-21.5 458.77,-18 448.77,-14.5 448.77,-21.5"/>
</g>

<g id="node6" class="node">
<title>your process</title>
<polygon fill="none" stroke="black" points="727,-36 641,-36 641,0 727,0 727,-36"/>
<text text-anchor="middle" x="684" y="-15.2" font-family="Inter" font-size="11.00">your process</text>
</g>

<g id="edge5" class="edge">
<title>runc->your process</title>
<path fill="none" stroke="black" d="M604.23,-18C612.26,-18 621.4,-18 630.53,-18"/>
<polygon fill="black" stroke="black" points="630.79,-21.5 640.79,-18 630.79,-14.5 630.79,-21.5"/>
</g>
</g>
</svg></figure>
<ul>
  <li>The client and the daemon talk over an HTTP interface on a local socket file.</li>
  <li>The daemon builds, runs and distributes containers, and owns the images and volumes.</li>
  <li>A shim is a small process that containerd leaves in place to own one container.</li>
  <li><code>containerd</code> was split out of the daemon to separate low-level container management.</li>
</ul>

Note:

<ul>
  <li>"Docker uses a client-server architecture", talking over a REST API on UNIX sockets [S10].</li>
  <li>The daemon manages images, containers, networks and volumes [S10].</li>
  <li>Low-level container management was split into containerd in Docker Engine 1.11 [S22].</li>
  <li>Each shim instance "communicates with containerd as a daemon, while parenting containers via invoking independent runtimes" [S70].</li>
</ul>

---

<p class="kicker">Part IV · Docker</p>

<h2>What the host process table actually shows</h2>

<p class="lede">The container is not a child of the daemon on a current installation. A small shim owns it.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
ps -ef --forest | grep -A2 containerd-shim
# containerd-shim-runc-v2, then the container
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
ps -ef --forest | grep -B2 nginx
# no daemon above it, only your login session
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Why</div>
<div class="v">The daemon does not launch containers itself. A shim instance parents the container and invokes the OCI runtime.</div>
</div>
<div class="pair">
<div class="k">What survives</div>
<div class="v">Under Docker the container outlives your terminal, because nothing in your session owns it.</div>
</div>
</div>

<p class="closing">Read the process tree before trusting any drawing of it.</p>

Note:

<ul>
  <li>"containerd, the daemon, does not directly launch containers" [S70].</li>
  <li>Each shim instance "communicates with containerd as a daemon, while parenting containers via invoking independent runtimes" [S70].</li>
  <li>The container outlives the terminal that started it.</li>
</ul>

---

<p class="kicker">Part IV · Docker</p>

<h2>Stop the daemon, and its containers stop</h2>

<p class="lede">The shim is the container's parent, but the daemon decides its lifecycle. When the daemon stops, by default it stops every container it started.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker run -d --name web nginx:alpine
systemctl --user stop docker   # rootful: sudo systemctl stop docker
systemctl --user start docker
docker inspect -f '{{.State.Status}}' web
# exited
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman run -d --name web nginx:alpine
# there is no daemon to stop
podman inspect -f '{{.State.Status}}' web
# running
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Why</div>
<div class="v">One daemon owns the lifecycle of every container. If it stops or crashes, its containers stop with it.</div>
</div>
<div class="pair">
<div class="k">Live restore</div>
<div class="v">An optional daemon setting that keeps containers running while the daemon is down. It is off by default.</div>
</div>
</div>

Note:

<ul>
  <li>"By default, when the Docker daemon terminates, it shuts down running containers" [S12].</li>
  <li>The shim parents the container [S70], so the process tree and the lifecycle are separate facts.</li>
  <li>Live restore is the setting <code>"live-restore": true</code>, made for "daemon crashes, planned outages, or upgrades" [S12].</li>
  <li>Across daemon updates it works "only" for patch releases (<code>YY.MM.x</code>), "not for major (<code>YY.MM</code>) daemon upgrades" [S12].</li>
  <li>Verified 2026-09-30 in the Lima VM, rootless Docker 29.7.2 and Podman 4.9.3.</li>
  <li>Stopping the daemon ended the Docker container's process, and it came back as <code>exited</code>.</li>
  <li>The Podman container kept running.</li>
  <li>This is takeaway 3 made visible: no single daemon whose stop takes every Podman container with it.</li>
</ul>

---

<p class="kicker">Part IV · Podman</p>

<h2>Podman: fork and exec, with a monitor left behind</h2>

<p class="lede">Podman has no daemon. Each command is an ordinary program that sets the container up and starts the OCI runtime.</p>

```bash
podman rm -f web 2>/dev/null
podman run -d --name web nginx:alpine
ps -ef --forest | grep -B1 nginx   # conmon, then nginx, under your session
```

<ul>
  <li>The <code>podman</code> process itself exits once the container is running.</li>
  <li>A small program named <code>conmon</code> stays behind. It started the OCI runtime, and it sits between your session and the container.</li>
  <li>The container is therefore a child of your session or of a systemd unit.</li>
</ul>

Note:

<ul>
  <li>Podman is "a fully featured container engine that is a simple daemonless tool" [S13].</li>
  <li>Most Podman commands run as a regular user without extra privileges [S13].</li>
  <li><code>conmon</code> execs <code>crun</code> or <code>runc</code>, which sets the container up and forks the container process [V29].</li>
  <li><code>conmon</code> therefore remains between the user's session and the container process.</li>
</ul>

---

<p class="kicker">Part IV · Podman</p>

<h2>What a start without root actually does</h2>

<p class="lede">Removing the daemon moves work into userspace. Podman does it in a fixed order.</p>

<div class="kv">
<div class="pair">
<div class="k">1 · Pause process</div>
<div class="v">Keeps one user namespace alive for all your containers.</div>
</div>
<div class="pair">
<div class="k">2 · Identifier map</div>
<div class="v">Reads your account's reserved user numbers and maps them.</div>
</div>
<div class="pair">
<div class="k">3 · Storage</div>
<div class="v">Makes a mount namespace and assembles the layers.</div>
</div>
<div class="pair">
<div class="k">4 · Network</div>
<div class="v">Starts a userspace network stack. The kernel refuses a full one.</div>
</div>
<div class="pair">
<div class="k">5 · OCI runtime</div>
<div class="v">Execs <code>conmon</code>, then the OCI runtime.</div>
</div>
</div>

Note:

<ul>
  <li>Podman joins the pause process namespace "because a multithreaded program cannot change its user namespace" [S15].</li>
  <li>Without that process Podman reads <code>/etc/subuid</code> and <code>/etc/subgid</code>, builds the mapping, and forks the pause process [V32].</li>
  <li>Podman then creates a mount namespace and assembles the image layers [V33].</li>
  <li>Full separate networking is not allowed for non-root users, so a userspace stack emulates it [S15].</li>
  <li>The OCI runtime then creates the namespaces and attaches the control group [V29].</li>
  <li>The OCI runtime applies the SELinux labels and loads the seccomp filter [V29].</li>
  <li>The OCI runtime bind-mounts the volumes last [V29].</li>
</ul>

---

<p class="kicker">Part IV · Which program asks the kernel</p>

<h2>Predict: whose trace shows the namespace call?</h2>

<p class="lede">Write your answer down. Run these in the Linux shell from the setup, as your own user.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
strace -f -Y \
  -e trace=clone,clone3,unshare,setns \
  docker run --rm alpine true
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
strace -f -Y \
  -e trace=clone,clone3,unshare,setns \
  podman run --rm alpine true
```

</div>
</div>

<div class="kv">
<div class="pair">
<div class="k">Question 1</div>
<div class="v">In which of the two traces do you see <code>unshare</code>?</div>
</div>
<div class="pair">
<div class="k">Question 2</div>
<div class="v">Which program makes the call: the one you typed, or a descendant of it?</div>
</div>
</div>

<ul>
  <li><code>-Y</code> prints the program name beside each process number. The <code>-e trace=</code> list keeps the trace short.</li>
  <li>Install the tracer inside the Linux shell first: <code>sudo apt install strace</code>, or <code>sudo dnf install strace</code>.</li>
</ul>

Note:

<ul>
  <li><code>unshare(2)</code> moves the caller into a new namespace, and <code>setns(2)</code> joins one by descriptor [S1].</li>
  <li>A written prediction before the observation is the method this course uses [S57].</li>
  <li>Podman execs <code>conmon</code>, and <code>conmon</code> execs the OCI runtime [S15].</li>
  <li>Docker's client talks to a daemon over a socket, so the work happens in another process tree [S10].</li>
</ul>

Collect the answers first.

---

<p class="kicker">Part IV · Which program asks the kernel</p>

<h2>Observe: only one of the two traces holds the call</h2>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```text
connect(3, ".../docker.sock") = 0
# and no namespace call at all
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```text
<podman> setns(5, 0) = 0
<runc:[1:CHILD]> unshare(
  CLONE_NEWNS|CLONE_NEWCGROUP|
  CLONE_NEWUTS|CLONE_NEWIPC|
  CLONE_NEWPID) = 0
```

</div>
</div>

<ul>
  <li>Under Podman, one call creates five namespaces at once.</li>
  <li>The program that makes it is <code>runc:[1:CHILD]</code>, a descendant of the <code>podman</code> command you typed.</li>
  <li>Under Docker, the client only opens a socket to the daemon.</li>
  <li>The same call happens under the daemon, in a process tree your trace never entered.</li>
</ul>

Note:

<ul>
  <li>The OCI runtime creates the namespaces for the container [V29].</li>
  <li>Podman execs <code>conmon</code>, and <code>conmon</code> execs the OCI runtime [S15], so the runtime is a descendant of the command you typed.</li>
  <li><code>unshare(2)</code> takes one or more <code>CLONE_NEW*</code> flags and moves the caller into new namespaces [S1].</li>
  <li>Docker's client and daemon talk over a REST API on a UNIX socket [S10].</li>
  <li>Nothing in the client's own trace therefore creates a namespace [S10].</li>
</ul>

---

<p class="kicker">Part IV · Which program asks the kernel</p>

<h2>Explain: the call is the same, the parent is not</h2>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
sudo strace -f -Y \
  -e trace=clone,clone3,unshare,setns \
  -p $(pgrep -u $USER -x containerd)
# then, in a second shell:
docker run --rm alpine true
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
# nothing extra: the trace you
# already ran holds the call
```

</div>
</div>

<ul>
  <li>Attached to <code>containerd</code>, the trace shows the same <code>runc:[1:CHILD]</code> calling <code>unshare</code>.</li>
  <li><code>strace -p</code> joins a process that is already running.</li>
  <li>A Linux setting named Yama allows that only with extra privilege, unless <code>strace</code> started the process itself.</li>
  <li>The system started <code>containerd</code>, not your shell. So you need <code>sudo</code> to trace it, even when it runs under your account.</li>
</ul>

<p class="closing">Two architectures, one system call. What differs is whose child makes it.</p>

Note:

<ul>
  <li>Each shim instance parents containers by invoking independent runtimes under <code>containerd</code> [S70].</li>
  <li>The OCI runtime is what creates the namespaces, under either engine [V29].</li>
  <li>Under Yama's default setting a tracer is restricted to its own descendants, unless it holds <code>CAP_SYS_PTRACE</code> [S84] [S67].</li>
  <li><code>containerd</code> is a long-lived service, so its tree is not a descendant of the user's shell and the attach needs privilege [S70] [S84].</li>
  <li>The flag set differs between engine and kernel versions, so a trace is read and not memorised.</li>
  <li>Say it aloud: flag lists differ between versions, so read your own trace rather than this one.</li>
</ul>

---

<p class="kicker">Part IV · The OCI runtimes</p>

<h2>Two OCI runtimes, one specification</h2>

<p class="lede"><code>runc</code> is written in Go. <code>crun</code> is written in C. They implement the same standard and are interchangeable.</p>

<div class="stats">
<div>
<div class="stat">3.34 s</div>
<div class="stat-cap">100 sequential containers under runc, in the crun project's own benchmark</div>
</div>
<div>
<div class="stat">
<span class="pos">1.69 s</span>
</div>
<div class="stat-cap">the same 100 under crun, same benchmark</div>
</div>
<div>
<div class="stat">512 KB</div>
<div class="stat-cap">memory limit crun still ran under. runc failed even with 4 MB</div>
</div>
</div>

<ul>
  <li>These are the project's own figures on its own benchmark. Read them as directional.</li>
  <li><code>youki</code> is a third OCI runtime built to the same standard.</li>
</ul>

Note:

<ul>
  <li><code>crun</code> is "a fast and low-memory footprint OCI Container Runtime fully written in C" [S16].</li>
  <li>The 1.69-second and 3.34-second figures, and the 512 KB result, are the project's own [S16].</li>
  <li><code>runc</code>, <code>crun</code> and <code>youki</code> are interchangeable implementations of one specification [V16].</li>
</ul>

---

<p class="kicker">Part IV · Without root</p>

<h2>Rootless: two different concessions</h2>

<p class="lede">Rootless means both the engine and the container run as an unprivileged user, on top of a user namespace.</p>

<div class="boxes">
<div class="box">
<div class="box-label">Docker</div>
<div class="box-head">Keeps the daemon, moves it</div>
<ul>
<li>The daemon and the containers both run inside a user namespace.</li>
<li>The daemon itself runs unprivileged too.</li>
<li>It needs at least 65,536 reserved user numbers for your account.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">Podman</div>
<div class="box-head">Has no daemon to move</div>
<ul>
<li>Rootless is the ordinary way to run it.</li>
<li>Podman states it holds no more privilege than the account that started it.</li>
<li>Part VI shows a kernel flaw that no identifier mapping stops.</li>
</ul>
</div>
</div>

Note:

<ul>
  <li>Rootless mode runs "the Docker daemon and containers inside a user namespace" [S11].</li>
  <li>In rootless mode "both the daemon and the container are running without root privileges" [S11].</li>
  <li>It requires at least 65,536 subordinate identifiers for the user [S11].</li>
  <li>Rootless Podman builds its identifier mapping from <code>/etc/subuid</code> and <code>/etc/subgid</code> [V32].</li>
  <li>Podman states that rootless containers "will never have more privileges than the user that launched them" [S14].</li>
  <li>The sentence is a design statement, not a measurement.</li>
  <li>Dirty Pipe let an unprivileged process write to any file it could read [S30].</li>
  <li>The published proof of concept escaped a container that was granted no extra privilege [S30].</li>
  <li>An identifier map does not change a kernel code path, so it does not stop that flaw.</li>
</ul>

---

<p class="kicker">Part IV · Without root</p>

<h2>On Ubuntu, a security setting can stop rootless containers</h2>

<p class="lede">Rootless containers need a user namespace. Ubuntu 24.04 gives a user namespace its full use only to programs with an AppArmor profile.</p>

```bash
sysctl kernel.apparmor_restrict_unprivileged_userns   # 1 means the setting is on
unshare -Ur id -u                                     # a program with no profile
# unshare: write failed /proc/self/uid_map: Operation not permitted
```

<ul>
  <li>AppArmor is a Linux security module. It gives each program a profile of what the program may do.</li>
  <li>Ubuntu ships profiles for <code>podman</code>, <code>runc</code>, <code>crun</code> and <code>rootlesskit</code>. So the engines from the Ubuntu or vendor packages work.</li>
  <li>If an engine stops with the error above, it runs without a profile. Install it from the packages instead.</li>
</ul>

<p class="closing">The setting belongs to the host. No engine option changes it.</p>

Note:

<ul>
  <li>Ubuntu announced the restriction on unprivileged user namespaces with release 23.10 [S77].</li>
  <li>Ubuntu 24.04 "restricts the use of unprivileged user namespaces" by default [S48].</li>
  <li>Programs whose AppArmor profile carries the <code>userns</code> rule keep the full use of user namespaces [S77].</li>
  <li>Verified 2026-09-30 on Ubuntu 24.04.4: the sysctl reads 1, and <code>unshare -Ur id -u</code> fails at the user map.</li>
  <li>On the same system rootless Podman runs, and <code>/etc/apparmor.d</code> holds profiles for <code>podman</code>, <code>crun</code>, <code>runc</code>, <code>rootlesskit</code>, <code>slirp4netns</code> and <code>buildah</code>.</li>
  <li>Whether the restriction applies inside WSL depends on whether the WSL kernel runs AppArmor, which this deck does not claim.</li>
</ul>

---

<p class="kicker">Part IV · Without root</p>

<h2>The network path without root is a userspace stack</h2>

<p class="lede">An unprivileged user cannot create the full network path. A program in userspace copies packets instead.</p>

<div class="stats">
<div>
<div class="stat">17.9 Gbps</div>
<div class="stat-cap">one iperf3 stream, container running as root</div>
</div>
<div>
<div class="stat">
<span class="neg">570 Mbps</span>
</div>
<div class="stat-cap">the same stream, container without root, in one study</div>
</div>
</div>

<ul>
  <li>The study's authors attributed the gap to per-packet handling in <code>slirp4netns</code>.</li>
  <li>Podman 6 removed that stack. The shipped default is now <code>pasta</code>.</li>
  <li>Publishing a port without root is a userspace listener, not a firewall rule.</li>
</ul>

Note:

<ul>
  <li>One study measured 570 Mbps against 17.9 Gbps and attributed the gap to <code>slirp4netns</code> [V14].</li>
  <li>Podman 6.0.0 removed slirp4netns and directs users to Pasta [S65].</li>
  <li>The shipped configuration now sets the rootless network command to <code>pasta</code> [S78].</li>
  <li>Both engines use a userspace stack for this path [V28].</li>
</ul>

---

<p class="kicker">Part IV · One format</p>

<h2>One image, either engine</h2>

<p class="lede">The image format and the OCI runtime specification are written standards, not one vendor's implementation detail.</p>

<div class="compare">
<div>
<p class="kicker">Docker</p>

```bash
docker pull \
  docker.io/library/python:3.12-slim
```

</div>
<div class="side-b">
<p class="kicker">Podman</p>

```bash
podman pull \
  docker.io/library/python:3.12-slim
```

</div>
</div>

<ul>
  <li>The same registry, the same manifest, the same layer archives for both.</li>
</ul>

<ul>
  <li>The manifest names each layer archive by the hash of its content, and any engine can unpack them.</li>
  <li>A registry serves the same image to both engines over the same protocol.</li>
  <li>This is why a second engine can be a replacement rather than a copy.</li>
</ul>

Note:

<ul>
  <li>The image manifest names "the content-addressable identity of one or more filesystem layer changeset archives" [S8].</li>
  <li>The runtime specification defines the configuration every OCI runtime reads [S7].</li>
  <li>Podman keeps the OCI image format and a deliberately compatible command set [S13].</li>
</ul>

---

<p class="kicker">Part IV · The summary</p>

<h2>The disagreement is about ownership</h2>

<p class="lede">Both engines make the same five requests to the same kernel. They disagree about which process holds the container.</p>

| | Docker | Podman |
|---|---|---|
| Path | client to daemon to containerd to shim to `runc` | one command, `conmon`, then `crun` |
| Owner | a long-lived privileged program | your own session, or a systemd unit |
| Default OCI runtime | `runc` | `crun` |
| Without root | supported, daemon moved into a user namespace | the ordinary mode |
| Default capabilities | eleven, plus `MKNOD`, `NET_RAW` and `AUDIT_WRITE` | eleven |
| Buys | one interface always listening, an integrated build system | least privilege by default, no idle program |
| Costs | a privileged program on the path of every container | userspace shims where the kernel refuses |

Note:

<ul>
  <li>The disagreement is about who owns the container process [T3] [V15].</li>
  <li>Docker's path runs through a daemon and containerd [S10] [S22].</li>
  <li>Podman execs <code>conmon</code>, which execs the OCI runtime and stays behind as the monitor [S15].</li>
  <li>Neither engine asks the kernel for a different mechanism [V1].</li>
  <li>Their default capability sets do differ: Docker's fourteen against Podman's eleven [S42] [S43].</li>
  <li>Podman's default OCI runtime is <code>crun</code> [S13]. Docker reaches <code>runc</code> through the shim named <code>containerd-shim-runc-v2</code> [S70].</li>
  <li>Docker runs without root by moving the daemon and the containers into a user namespace [S11].</li>
  <li>Most Podman commands run as a regular user without extra privilege, so this is its ordinary mode [S13].</li>
  <li>The Docker daemon listens for API requests on a socket, and it also builds and distributes images [S10].</li>
  <li>Rootless mode is what runs Docker's daemon without root privileges, and <code>userns-remap</code> mode keeps it privileged [S11].</li>
  <li>The daemon therefore sits on the path of every Docker container, and it holds root privileges unless rootless mode runs it [S10] [S11].</li>
  <li>Podman runs no daemon, so no long-lived program waits between commands [S13].</li>
  <li>Podman's default set holds three fewer capabilities than Docker's, so its container starts with less privilege [S42] [S43].</li>
  <li>Without root the kernel refuses full networking to Podman, so a userspace program carries the packets instead [S15].</li>
</ul>

---

<h1>What follows from that</h1>

<p class="lede">Two consequences of Part IV: the commands are portable, and macOS and Windows needed a Linux virtual machine.</p>

Note:

Part V draws consequences. No new mechanism.

---

<p class="kicker">Part V · Consequence 1</p>

<h2>Same verbs, different engine</h2>

<p class="lede">Part II showed the commands for one engine. Almost all of them run on the other unchanged.</p>

| Task | Docker | Podman |
|---|---|---|
| Run | `docker run` | `podman run` |
| List | `docker ps` | `podman ps` |
| Build | `docker build` | `podman build` |
| Compose | built in | needs an external provider installed |
| Trace | `--cap-add SYS_PTRACE` | `--cap-add SYS_PTRACE` |

<ul>
  <li>The verbs match because the image format and the OCI runtime specification are standards.</li>
  <li>Compose is the exception. Podman ships a wrapper and no implementation.</li>
</ul>

Note:

<ul>
  <li>Podman keeps the OCI image format and a deliberately compatible command set [S13].</li>
  <li>"podman compose is a thin wrapper around an external compose provider" [S66].</li>
  <li>Podman states that a container without root never holds more privilege than the user who launched it [S14].</li>
</ul>

---

<p class="kicker">Part V · Consequence 2</p>

<h2>Why macOS and Windows need a Linux virtual machine</h2>

<p class="lede">A virtual machine runs its own kernel, so the guest kernel is not the host kernel. That is what macOS and Windows are missing.</p>

<div class="kv">
<div class="pair">
<div class="k">1 · The mechanisms</div>
<div class="v">All five are Linux kernel features.</div>
</div>
<div class="pair">
<div class="k">2 · The gap</div>
<div class="v">The macOS and Windows kernels lack them.</div>
</div>
<div class="pair">
<div class="k">3 · The answer</div>
<div class="v">Supply a Linux kernel in a virtual machine.</div>
</div>
<div class="pair">
<div class="k">4 · Your own case</div>
<div class="v">Lima and WSL 2 are instances of this, not workarounds.</div>
</div>
</div>

<p class="closing">Microsoft says so plainly: WSL 2 is a Linux kernel in a small virtual machine.</p>

Note:

<ul>
  <li>Every Docker on a Mac is Docker inside a Linux virtual machine [T4] [V9].</li>
  <li>WSL 2 uses "a Linux kernel inside of a lightweight utility virtual machine (VM)" [S46].</li>
  <li>WSL 2 also "runs Linux distributions as isolated containers inside the managed VM" [S46].</li>
  <li>The mechanisms named are Linux kernel interfaces [S1] [S2] [S3].</li>
</ul>

---

<p class="kicker">Part V · Consequence 2</p>

<h2>Where "a Linux virtual machine" stops being the answer</h2>

<p class="lede">The claim holds for Linux containers. Windows has containers of its own, and they need a Windows kernel.</p>

<div class="boxes">
<div class="box">
<div class="box-label">Linux container on Windows</div>
<div class="box-head">Needs a Linux kernel</div>
<ul>
<li>Supplied by WSL 2, inside a virtual machine.</li>
<li>Everything in this topic applies.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">Windows container on Windows</div>
<div class="box-head">Needs the Windows kernel</div>
<ul>
<li>It shares the Windows kernel directly. Microsoft describes that as roughly how Linux containers run.</li>
<li>Adding a Linux virtual machine does not help here.</li>
</ul>
</div>
</div>

Note:

<ul>
  <li>Process-isolated Windows containers "share the same kernel with the host as well as each other" [S35].</li>
  <li>Microsoft calls that "approximately the same as how Linux containers run" [S35].</li>
  <li>The takeaway is scoped to Linux containers for this reason.</li>
</ul>

---

<p class="kicker">Part V · Consequence 2</p>

<h2>One shared machine, or one for each container</h2>

<p class="lede">How many virtual machines the engine builds is a design choice, and the answers differ.</p>

<div class="matrix">
<div class="m-row">
<div class="src">Lima and WSL 2</div>
<div class="ref">Lima, WSL 2</div>
<div class="claim">One virtual machine. Every container is a process inside it.</div>
<div class="kind">One kernel, many containers</div>
</div>
<div class="m-row">
<div class="src">Apple</div>
<div class="ref">its Containerization project</div>
<div class="claim">Each Linux container runs inside its own small virtual machine.</div>
<div class="kind">Stronger separation, more machines</div>
</div>
</div>

<ul>
  <li>On the second shape two containers share no kernel, so the shared-kernel claim describes the usual shape and not a law.</li>
</ul>

Note:

<ul>
  <li>Apple states that "Containerization executes each Linux container inside of its own lightweight virtual machine" [S37].</li>
  <li>The Lima and WSL 2 path runs one virtual machine for many containers [V9].</li>
  <li>Part VI states the shared-kernel argument for the usual shape and names this exception.</li>
</ul>

---

<h1>Where it breaks</h1>

<p class="lede">The boundary, the escapes through it, and the leaks you can measure on your own laptop.</p>

Note:

Part VI is the diagnostic part. Evidence first, judgement after.

---

<p class="kicker">Part VI · The boundary</p>

<h2>The boundary is the system-call interface</h2>

<p class="lede">A virtual machine gives a guest its own kernel behind a hardware boundary. A container shares the host kernel, so its boundary is that kernel's system-call interface.</p>

<div class="stats">
<div>
<div class="stat">1</div>
<div class="stat-cap">kernel shared by every container on the host</div>
</div>
<div>
<div class="stat">300+</div>
<div class="stat-cap">system calls the boundary is drawn on</div>
</div>
<div>
<div class="stat">
<span class="neu">~44</span>
</div>
<div class="stat-cap">of them refused by Docker's default profile</div>
</div>
</div>

<ul>
  <li>Linux publishes a stable, versioned, documented system-call interface.</li>
  <li>Stability makes it dependable. Width makes it large to defend.</li>
  <li>A kernel bug is therefore a possible container escape.</li>
</ul>

Note:

<ul>
  <li>Namespaces separate a container more weakly than a hardware-backed virtual machine separates a guest [T2] [V7] [S73].</li>
  <li>The attack surface takes in the kernel's whole system-call interface [V7].</li>
  <li>The kernel's stable, versioned, documented user-space interface is the numbered system-call interface [V21].</li>
  <li>The default profile refuses around 44 of more than 300 calls [S5].</li>
  <li>Kubernetes states that containers "offer a weaker isolation boundary than virtual machines that utilize hardware-based virtualization" [S73].</li>
</ul>

---

<p class="kicker">Part VI · The escapes</p>

<h2>A container escape, in four steps</h2>

<p class="lede">CVE-2019-5736 overwrote the host OCI runtime binary from inside a container.</p>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">Bait</div>
<div class="desc">A malicious image points its entry at <code>/proc/self/exe</code>.</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">Re-entry</div>
<div class="desc">The OCI runtime re-executes itself inside it.</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">Handle</div>
<div class="desc">A library the attacker loaded opens the host's OCI runtime binary for writing.</div>
</div>
<div class="row">
<div class="num">4</div>
<div class="term">Overwrite</div>
<div class="desc">The next start runs attacker code as root.</div>
</div>
</div>

<ul>
  <li>The attack needed only the default options.</li>
  <li>It reached Docker, Podman, CRI-O and containerd alike.</li>
</ul>

Note:

<ul>
  <li>The entry point is a symlink to <code>/proc/self/exe</code>, so the OCI runtime re-executes itself inside the container [S25].</li>
  <li>An <code>LD_PRELOAD</code>ed library constructor "can obtain a writable descriptor to the host binary" [V34].</li>
  <li>The flaw "let a malicious image overwrite the host <code>runc</code> binary" [V34].</li>
  <li>The engine executes that same host <code>runc</code> binary again at every later container start [V34].</li>
  <li>Rootless mode is what runs Docker's daemon without root privileges, and <code>userns-remap</code> mode keeps it privileged [S11].</li>
  <li>Docker's daemon binds a UNIX socket owned by root, so a default install runs it with root privileges [S85].</li>
  <li>The OCI runtime that this privileged path executes therefore runs as root too.</li>
  <li>The next start therefore executes the attacker's code as root.</li>
  <li>The escape needed only the default options and a standard configuration [V37].</li>
  <li>The documented response is to treat the kernel as belonging to a single tenant [S25].</li>
  <li>The flaw reached Docker, Podman, Kubernetes, CRI-O and containerd alike [S24].</li>
</ul>

---

<p class="kicker">Part VI · The escapes</p>

<h2>One class, disclosed four times</h2>

<p class="lede">One class of bug returned four times in <code>runc</code>. The container controls a path, and the OCI runtime follows it.</p>

<div class="kv">
<div class="pair">
<div class="k">Nov 2025</div>
<div class="v">Three flaws of one class. Each turned on a path the container controlled.</div>
</div>
<div class="pair">
<div class="k">One mechanism</div>
<div class="v">A link replaces <code>/dev/null</code>, so the OCI runtime mounts a host path read-write.</div>
</div>
<div class="pair">
<div class="k">Another</div>
<div class="v">The kernel runs a helper on the host, and that helper ignores namespaces.</div>
</div>
<div class="pair">
<div class="k">2026</div>
<div class="v">CVE-2026-41579. Again a <code>/dev</code> link, again limited host writes.</div>
</div>
</div>

<ul>
  <li>An untrusted image or build file is the most likely way in. The <code>runc</code> 1.1.x branch stayed unpatched.</li>
</ul>

Note:

<ul>
  <li>November 2025 disclosed three CVEs of one class: CVE-2025-31133, CVE-2025-52565 and CVE-2025-52881 [V36].</li>
  <li>Those three abused masked paths and <code>/dev/console</code> mount races [V36].</li>
  <li>Each of the four turned on a path the container itself controlled, which is what makes them one class.</li>
  <li><code>runc</code> is the OCI runtime Docker, Podman, Kubernetes, CRI-O and containerd all use [S24].</li>
  <li>The fixes shipped in runc 1.2.8, 1.3.3 and 1.4.0-rc.3 [S26].</li>
  <li>The runc 1.1.x branch and everything earlier were left unpatched [S26].</li>
  <li>CVE-2026-41579 is the fourth disclosure of the same class [S28] [V36].</li>
  <li>In CVE-2025-31133, replacing <code>/dev/null</code> with a symlink makes runc bind-mount an attacker-chosen host path read-write [S26].</li>
  <li>"As kernel upcalls are not namespaced", the configured binary runs with full host privileges [S26].</li>
  <li>CVE-2026-41579 "allowed a malicious image with a <code>/dev</code> symlink to have limited write access to the host filesystem" [S28].</li>
  <li>Untrusted images and build files are the "most likely delivery method" [S27]. Youki and crun "are understood to have similar flaws" [S27].</li>
</ul>

---

<p class="kicker">Part VI · The escapes</p>

<h2>When the kernel is the bug</h2>

<p class="lede">Dirty Pipe was a kernel flaw. No fix in an OCI runtime could have prevented it.</p>

<ul>
  <li>CVE-2022-0847 let an unprivileged process write to any file it could read.</li>
  <li>A public proof of concept escaped a container that was granted no extra privilege. It overwrote the cached copy of the OCI runtime binary.</li>
  <li>Patching the engine would not have helped. Changing the engine would not have helped.</li>
  <li>This is what "the boundary is the kernel" means in practice.</li>
</ul>

<p class="closing">Every process in every container on that host was behind the same one bug.</p>

Note:

<ul>
  <li>Dirty Pipe allowed an unprivileged process to write to any file it could read [S30].</li>
  <li>The published proof of concept escaped a container that was granted no extra privilege, through the page-cache copy of <code>runc</code> [S30].</li>
  <li>No fix in the OCI runtime could have prevented a kernel bug.</li>
</ul>

---

<p class="kicker">Part VI · The evidence</p>

<h2>One systematic study, and what it counted</h2>

<p class="lede">One published survey counted the flaws across eleven implementations instead of estimating them.</p>

<div class="stats">
<div>
<div class="stat">59</div>
<div class="stat-cap">published flaws across the 11 "container runtimes" one survey examined</div>
</div>
<div>
<div class="stat">46%</div>
<div class="stat-cap">of the 28 that had a working proof of concept enabled an escape</div>
</div>
<div>
<div class="stat">
<span class="pos">7 of 9</span>
</div>
<div class="stat-cap">tested escapes prevented by user namespaces</div>
</div>
</div>

<ul>
  <li>The survey's authors put most cases in one category: a host component leaked into the container.</li>
  <li>A user namespace prevented seven of the nine exploits the survey tested. Two still worked.</li>
  <li>The two that survived involved the host OCI runtime still running as root against objects the container controls.</li>
</ul>

Note:

<ul>
  <li>The study covers 59 flaws across the 11 "container runtimes" it examined, 28 of them with public proof-of-concept exploits [S29].</li>
  <li>46 percent of those enabled container escape, and the dominant cause was a leaked host component [S29].</li>
  <li>User namespaces prevented 7 of 9 tested escape exploits, so two of the nine still worked [S29].</li>
  <li>The two exploits that survived did so because the host OCI runtime still executes as root against container-controlled objects [V35].</li>
  <li>An LXC maintainer who worked on the runc fix states that privileged containers "are simply never going to be truly secure" [S24].</li>
</ul>

---

<p class="kicker">Part VI · The comparison</p>

<h2>The other boundary is narrower, not absolute</h2>

<p class="lede">A virtual machine's interface is smaller than a system-call interface. Smaller is not zero.</p>

<ul>
  <li>CVE-2025-22225 is a published flaw in VMware ESXi. An attacker with enough privilege could reach the host through the process that runs the virtual machine.</li>
  <li>The comparison is about the width of the interface, not about a guarantee on either side.</li>
</ul>

<div class="kv">
<div class="pair">
<div class="k">Container</div>
<div class="v">The whole system-call interface of a shared kernel.</div>
</div>
<div class="pair">
<div class="k">Virtual machine</div>
<div class="v">A much smaller device and instruction interface. An attacker crossed it too.</div>
</div>
</div>

Note:

<ul>
  <li>For CVE-2025-22225 "an attacker with requisite privileges could exploit this vulnerability through the VMX process to escape the sandbox" [S36].</li>
  <li>Kubernetes states the container boundary is weaker than hardware-based virtualization [S73].</li>
  <li>The claim is comparative width, not a guarantee on either side.</li>
</ul>

---

<p class="kicker">Part VI · The comparison</p>

<h2>Containers went back inside virtual machines</h2>

<p class="lede">AWS replaced Linux containers with microVMs to isolate the functions its on-demand service runs.</p>

<ul>
  <li>AWS first isolated its on-demand function service with Linux containers, and reported itself unsatisfied.</li>
  <li>Their stated reason: reliance on a single kernel forces a trade-off between security and compatibility.</li>
  <li>They built Firecracker instead of choosing between the two.</li>
  <li>Kubernetes says sandboxing is often recommended for untrusted code, where workloads are assumed to be malicious.</li>
</ul>

<p class="closing">Packaging is where containers replaced the alternative. Untrusted tenants are where they did not.</p>

Note:

<ul>
  <li>AWS replaced Linux containers with Firecracker microVMs for the functions Lambda runs [S38].</li>
  <li>"When we first built AWS Lambda, we chose to use Linux containers to isolate functions… We were unsatisfied with this approach" [S38].</li>
  <li>Reliance on a single kernel means "a fundamental tradeoff between security and code compatibility" [S38].</li>
  <li>AWS states "We built Firecracker because we didn't want to choose" [S38].</li>
  <li>Kubernetes states that sandboxing "is often recommended when you are running untrusted code, where workloads are assumed to be malicious" [S73].</li>
  <li>Part I named multi-tenancy as the first need, and this is where it landed.</li>
</ul>

---

<p class="kicker">Part VI · The scope</p>

<h2>Which containers are not virtual machines</h2>

<p class="lede">The shared-kernel claim covers the usual shape. Two documented cases fall outside it.</p>

<div class="matrix">
<div class="m-row">
<div class="src">Kata Containers</div>
<div class="ref">project page</div>
<div class="claim">An OCI runtime that builds a small virtual machine for each container.</div>
<div class="kind">An engine runs it instead of <code>runc</code></div>
</div>
<div class="m-row">
<div class="src">The standard</div>
<div class="ref">the OCI runtime specification</div>
<div class="claim">The runtime specification carries a virtual-machine section of its own.</div>
<div class="kind">Not a deviation</div>
</div>
</div>

<ul>
  <li>So the claim is scoped. A namespace-based Linux container shares the host kernel, and an OCI runtime that builds a virtual machine still conforms to the same standard.</li>
</ul>

Note:

<ul>
  <li>Kata Containers is "An open source container runtime, building lightweight virtual machines" [S33].</li>
  <li>An engine's <code>--runtime</code> option takes any OCI-compatible binary, so a conforming runtime replaces the default one [S13].</li>
  <li>The runtime specification "describes the schema for the virtual-machine-specific section of the container configuration" [S34].</li>
  <li>The claim is therefore scoped to namespace-based Linux containers.</li>
</ul>

---

<p class="kicker">Part VI · The leaks</p>

<h2>The leak you can measure: files across the boundary</h2>

<p class="lede">On macOS a bind mount crosses out of the virtual machine on every access. A volume does not.</p>

<div class="stats">
<div>
<div class="stat">3.5x</div>
<div class="stat-cap">slower than Linux with VirtioFS, many small files through a bind mount, in one measurement</div>
</div>
<div>
<div class="stat">
<span class="neg">10x</span>
</div>
<div class="stat-cap">slower on the same files with gRPC-FUSE</div>
</div>
<div>
<div class="stat">
<span class="pos">~1x</span>
</div>
<div class="stat-cap">in line with Linux, inside a named volume</div>
</div>
</div>

<ul>
  <li>One author measured these ratios on one codebase. They show direction only.</li>
  <li>Many small files dominate the cost, not total bytes.</li>
  <li>On Windows, keep working files in the Linux filesystem.</li>
  <li>Two machines on one engine version differ if one uses the slower file-sharing driver.</li>
</ul>

Note:

<ul>
  <li>Every container on macOS runs inside a Linux virtual machine, so a bind mount crosses that boundary [V9].</li>
  <li>One measurement puts macOS at roughly 3.5 times slower with VirtioFS and 10 times with gRPC-FUSE [S44].</li>
  <li>Moving the directory into a named volume brought timings back in line with Linux [S44].</li>
  <li>Microsoft advises storing files on the Linux drive for speed [S72].</li>
  <li>Storage carrying a <code>.has-mount-program</code> flag file keeps using FUSE whatever the kernel [V13].</li>
  <li>The program leaves the virtual machine on every access to a bind-mounted path, and the measurement tracks that crossing [V9] [S44].</li>
</ul>

---

<p class="kicker">Part VI · The leaks</p>

<h2>The leak you can trip over: the wrong architecture</h2>

<p class="lede">An image built for one processor architecture on a machine with another runs under translation, or not at all.</p>

<div class="kv">
<div class="pair">
<div class="k">Clear failure</div>
<div class="v"><code>exec format error</code>. The image was built for another architecture and nothing translates it here.</div>
</div>
<div class="pair">
<div class="k">Unclear failure</div>
<div class="v"><code>Illegal instruction</code>. The translator does not implement an instruction the program uses.</div>
</div>
<div class="pair">
<div class="k">Slow success</div>
<div class="v">It runs. Each system call is caught, converted and forwarded, so system-call-heavy work suffers most.</div>
</div>
<div class="pair">
<div class="k">Fix</div>
<div class="v">Name your own architecture with <code>--platform linux/arm64</code> or <code>--platform linux/amd64</code>. Prefer images built for that architecture.</div>
</div>
</div>

Note:

<ul>
  <li><code>exec format error</code> is the canonical symptom of an image built for the wrong architecture [S59].</li>
  <li>Rosetta does not implement AVX, so such images fail with <code>Illegal instruction</code> [S45].</li>
  <li>Under translation each system call is caught, converted and forwarded to the host kernel [S59].</li>
  <li><code>--platform=OS/ARCH</code> selects the image for a given platform [S61].</li>
</ul>

---

<p class="kicker">Part VI · The leaks</p>

<h2>What is inside an image nobody opened</h2>

<p class="lede">An image freezes a package set on the day it was built, and nothing refreshes it afterwards.</p>

<ul>
  <li>One scanning study of images built for neuroscience data analysis found they contain hundreds of known vulnerabilities.</li>
  <li>Rebuilding the same images against current packages removed about two thirds of those vulnerabilities.</li>
  <li>The same paper reports, from earlier work rather than its own, that over 30 percent of official images on one registry carry high-priority vulnerabilities.</li>
  <li>The images pulled in Part II are somebody else's filesystem, running under your engine.</li>
  <li>Rebuild on a schedule, because nothing else refreshes the package set.</li>
</ul>

Note:

<ul>
  <li>The study measured images for neuroscience data analysis and found they "contain hundreds of vulnerabilities" [S31].</li>
  <li>Software updates removed "roughly two-thirds of these" [S31]. The 30-percent figure for official images is quoted by that paper from earlier work [S31].</li>
  <li>The 2025 advisories name untrusted images as the most likely delivery method [S27].</li>
  <li>Image layers are read-only below the container's writable layer [S6], so only a new build applies those updates.</li>
</ul>

---

<p class="kicker">Part VI · Honest limits</p>

<h2>Three things the evidence leaves open</h2>

<div class="gaps">
<div class="gap">
<div class="dot">•</div>
<div class="gap-term">A namespace can outlive its processes</div>
<div class="ref">namespaces(7)</div>
<div class="gap-desc">Bind-mounting an entry in <code>/proc</code> keeps a namespace alive after every process in it exits.</div>
</div>
<div class="gap">
<div class="dot">•</div>
<div class="gap-term">The memory report has no kernel fix</div>
<div class="ref">Docker docs</div>
<div class="gap-desc">Nothing namespaced those files. The workaround is a userspace filesystem.</div>
</div>
<div class="gap">
<div class="dot">•</div>
<div class="gap-term">Your own installation has its own defaults</div>
<div class="ref">containers.conf</div>
<div class="gap-desc">Read your network backend and storage driver from <code>containers.conf</code>.</div>
</div>
</div>

<p class="closing">Read the real limit from <code>/sys/fs/cgroup/memory.max</code>, not <code>free</code>. Check <code>/proc</code> for a bind mount before trusting that a namespace is gone.</p>

Note:

<ul>
  <li>Bind mounting a file in <code>/proc/pid/ns</code> "keeps the corresponding namespace… alive even if all processes currently in the namespace terminate" [S1].</li>
  <li><code>/proc/meminfo</code> and its neighbours always display host numbers [S41].</li>
  <li>The control group holds the limit, and the files <code>free</code> reads do not [S2] [S41].</li>
  <li>The remaining default is read from <code>containers.conf</code> and the release notes for the installed version [V27].</li>
  <li>Where an inside number and an outside number disagree, count the boundary crossings between them [V23].</li>
</ul>

---

<p class="kicker">Part VII · Optional</p>

<h1>Where it came from</h1>

<p class="lede">Four pictures. Optional.</p>

Note:

Part VII is optional. Skip it without loss.

---

<p class="kicker">Part VII · Optional</p>

<g id="node1" class="node">
<title>a</title>
<polygon fill="none" stroke="black" points="54,-38.5 0,-38.5 0,-2.5 54,-2.5 54,-38.5"/>
<text text-anchor="middle" x="27" y="-23.5" font-family="Inter" font-size="10.00">1982</text>
<text text-anchor="middle" x="27" y="-12.5" font-family="Inter" font-size="10.00">chroot</text>
</g>

<g id="node2" class="node">
<title>b</title>
<polygon fill="none" stroke="black" points="169,-38.5 90,-38.5 90,-2.5 169,-2.5 169,-38.5"/>
<text text-anchor="middle" x="129.5" y="-23.5" font-family="Inter" font-size="10.00">2000</text>
<text text-anchor="middle" x="129.5" y="-12.5" font-family="Inter" font-size="10.00">FreeBSD jails</text>
</g>

<g id="node3" class="node">
<title>c</title>
<polygon fill="none" stroke="black" points="309,-38.5 205,-38.5 205,-2.5 309,-2.5 309,-38.5"/>
<text text-anchor="middle" x="257" y="-23.5" font-family="Inter" font-size="10.00">2002</text>
<text text-anchor="middle" x="257" y="-12.5" font-family="Inter" font-size="10.00">mount namespace</text>
</g>

<g id="node4" class="node">
<title>d</title>
<polygon fill="none" stroke="black" points="425,-38.5 345,-38.5 345,-2.5 425,-2.5 425,-38.5"/>
<text text-anchor="middle" x="385" y="-23.5" font-family="Inter" font-size="10.00">2004</text>
<text text-anchor="middle" x="385" y="-12.5" font-family="Inter" font-size="10.00">Solaris Zones</text>
</g>

<g id="node5" class="node">
<title>e</title>
<polygon fill="none" stroke="black" points="547,-41 461,-41 461,0 547,0 547,-41"/>
<text text-anchor="middle" x="504" y="-29" font-family="Inter" font-size="10.00">2008</text>
<text text-anchor="middle" x="504" y="-18" font-family="Inter" font-size="10.00">control groups</text>
<text text-anchor="middle" x="504" y="-7" font-family="Inter" font-size="10.00">Linux 2.6.24</text>
</g>

<g id="node6" class="node">
<title>f</title>
<polygon fill="none" stroke="black" points="666,-41 583,-41 583,0 666,0 666,-41"/>
<text text-anchor="middle" x="624.5" y="-29" font-family="Inter" font-size="10.00">2013</text>
<text text-anchor="middle" x="624.5" y="-18" font-family="Inter" font-size="10.00">Docker image</text>
<text text-anchor="middle" x="624.5" y="-7" font-family="Inter" font-size="10.00">and registry</text>
</g>

<g id="edge6" class="edge">
<title>f->g</title>
<path fill="none" stroke="#83838f" d="M666.01,-20.5C678.1,-20.5 691.02,-20.5 701.92,-20.5"/>
</g>
</g>
</svg></figure>
<ul>
  <li><code>chroot</code> shows a process one subtree of the file tree and hides the rest.</li>
  <li>FreeBSD jails, in 2000, answered the same need with a compartment that reached past the file tree.</li>
  <li>Solaris Zones, in 2004, answered the same need on Solaris, and limited what each compartment consumed.</li>
  <li>The <code>chroot</code> date comes from a reading of the source-control logs, not from folklore.</li>
  <li>Popular timelines print 1979. The two accounts genuinely disagree, and the log date is better evidenced.</li>
  <li>Control groups are dated by the kernel release that carried them, not by a press date.</li>
</ul>

Note:

<ul>
  <li><code>chroot(2)</code> was added to BSD by Bill Joy on 18 March 1982, per Kirk McKusick's reading of the SCCS logs [V30].</li>
  <li><code>chroot</code> limits a process to a single subtree of the filesystem name space [S19].</li>
  <li>The compartment of <code>chroot</code> "does not extend to the process or networking spaces", and a jail does [S19].</li>
  <li>The jails paper states that a jail "effectively amounts to a virtual machine environment" [S19].</li>
  <li>The Solaris Zones paper states the need as isolation plus a limit on resource use, which is what a zone provides [S58].</li>
  <li>The author of FreeBSD jails states that <code>chroot</code> was never intended as a security mechanism [S20].</li>
  <li>Popular timelines date the call to 1979 and to Version 7 Unix, and the two accounts genuinely disagree [V31].</li>
  <li>The mount namespace arrived in kernel 2.4.19 in 2002 [S21].</li>
  <li>Control groups version 1 appeared in Linux 2.6.24 [S2].</li>
  <li>The 2013 date for the first Docker image and registry is the one every account cited here gives.</li>
  <li>The Open Container Initiative was launched on 22 June 2015 [S9].</li>
</ul>

---

<p class="kicker">Part VII · Optional</p>

<h2>Three lanes that met</h2>

<figure class="diagram"><svg width="420pt" height="250pt"
 viewBox="0.00 0.00 420.00 250.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 246)">
<title>%3</title>
<polygon fill="white" stroke="transparent" points="-4,4 -4,-246 416,-246 416,4 -4,4"/>
<g id="clust1" class="cluster">
<title>cluster_0</title>
<polygon fill="none" stroke="#dcdce4" points="35.5,-8 35.5,-78 394.5,-78 394.5,-8 35.5,-8"/>
<text text-anchor="middle" x="215" y="-66.8" font-family="Inter" font-size="9.00">Isolating one workload from another</text>
</g>
<g id="clust2" class="cluster">
<title>cluster_1</title>
<polygon fill="none" stroke="#dcdce4" points="8,-86 8,-156 404,-156 404,-86 8,-86"/>
<text text-anchor="middle" x="206" y="-144.8" font-family="Inter" font-size="9.00">Linux kernel mechanisms</text>
</g>
<g id="clust3" class="cluster">
<title>cluster_2</title>
<polygon fill="none" stroke="#dcdce4" points="19,-164 19,-234 258,-234 258,-164 19,-164"/>
<text text-anchor="middle" x="138.5" y="-222.8" font-family="Inter" font-size="9.00">Packaging and transport</text>
</g>

<g id="node1" class="node">
<title>x1</title>
<polygon fill="none" stroke="black" points="108.5,-52 43.5,-52 43.5,-16 108.5,-16 108.5,-52"/>
<text text-anchor="middle" x="76" y="-31.8" font-family="Inter" font-size="9.00">chroot 1982</text>
</g>

<g id="node2" class="node">
<title>x2</title>
<polygon fill="none" stroke="black" points="252,-52 193,-52 193,-16 252,-16 252,-52"/>
<text text-anchor="middle" x="222.5" y="-31.8" font-family="Inter" font-size="9.00">jails 2000</text>
</g>

<g id="node3" class="node">
<title>x3</title>
<polygon fill="none" stroke="black" points="386.5,-52 318.5,-52 318.5,-16 386.5,-16 386.5,-52"/>
<text text-anchor="middle" x="352.5" y="-31.8" font-family="Inter" font-size="9.00">Zones 2004</text>
</g>

<g id="node4" class="node">
<title>y1</title>
<polygon fill="none" stroke="black" points="136,-130 16,-130 16,-94 136,-94 136,-130"/>
<text text-anchor="middle" x="76" y="-109.8" font-family="Inter" font-size="9.00">mount namespace 2002</text>
</g>

<g id="node5" class="node">
<title>y2</title>
<polygon fill="none" stroke="black" points="273,-130 172,-130 172,-94 273,-94 273,-130"/>
<text text-anchor="middle" x="222.5" y="-109.8" font-family="Inter" font-size="9.00">control groups 2008</text>
</g>

<g id="node6" class="node">
<title>y3</title>
<polygon fill="none" stroke="black" points="396,-130 309,-130 309,-94 396,-94 396,-130"/>
<text text-anchor="middle" x="352.5" y="-114.8" font-family="Inter" font-size="9.00">user namespace</text>
<text text-anchor="middle" x="352.5" y="-104.8" font-family="Inter" font-size="9.00">Linux 3.8, 2013</text>
</g>

<g id="node7" class="node">
<title>z1</title>
<polygon fill="none" stroke="black" points="125,-208 27,-208 27,-172 125,-172 125,-208"/>
<text text-anchor="middle" x="76" y="-187.8" font-family="Inter" font-size="9.00">Docker image 2013</text>
</g>

<g id="edge5" class="edge">
<title>z1->z2</title>
<path fill="none" stroke="#83838f" d="M125.12,-190C148.27,-190 175.16,-190 194.64,-190"/>
</g>
</g>
</svg></figure>
<ul>
  <li>The lanes are parallel answers to one pressure, not a line of descent. Accounts written at the time attribute the idea of namespacing to Plan 9, not to BSD.</li>
  <li>The kernel maintainers refused the word. Google's "process containers" became control groups.</li>
</ul>

Note:

<ul>
  <li>Jails, Zones and namespaces are three parallel responses rather than one inherited design [V5].</li>
  <li>Contemporaneous accounts attribute the idea of namespacing to Bell Labs' Plan 9 [V5].</li>
  <li>Google engineers began "process containers" in 2006, and the name changed to control groups to avoid overloading the word container [V6].</li>
  <li>Jails were presented as amounting to a virtual machine environment, which is where the reading began [S19].</li>
</ul>

---

<p class="kicker">Part VII · Optional</p>

<h2>What 2013 added, and what it did not</h2>

<p class="lede">Isolation was available for a decade before the first Docker release. What was missing was a way to move an application between machines.</p>

<div class="boxes">
<div class="box">
<div class="box-label">Not new in 2013</div>
<div class="box-head">The isolation</div>
<ul>
<li>Namespaces from 2002, control groups from 2008.</li>
<li>LXC had already combined them, and Docker began as a command-line front end to LXC.</li>
</ul>
</div>
<div class="box filled">
<div class="box-label">New in 2013</div>
<div class="box-head">The image and the transport</div>
<ul>
<li>A layered image named by the hash of its content.</li>
<li>A protocol for moving it between machines.</li>
<li>Deployment becomes one command against a named hash.</li>
</ul>
</div>
</div>

Note:

<ul>
  <li>The problem containers were built to solve is distribution, not isolation [V3].</li>
  <li>The technically interesting part was the least technically novel part [V4].</li>
  <li>Docker began as a Python command-line front end to LXC and AUFS [S22].</li>
  <li>Low-level container management later moved out of Docker into containerd [S22].</li>
</ul>

---

<p class="kicker">Part VII · Optional</p>

<h2>Kubernetes dropped Docker, and the images kept working</h2>

<p class="lede">Since 2015 the image format and the runtime contract are standards. One consequence arrived seven years later.</p>

<ul>
  <li>Kubernetes removed <code>dockershim</code>, its adapter for Docker, in release 1.24.</li>
  <li>Images built by Docker kept working everywhere, because they are standard OCI images.</li>
  <li>Docker was replaced underneath the format. The format did not move.</li>
</ul>

<p class="closing">Docker no longer controls the implementation, and the images it builds still run under every conforming engine.</p>

Note:

<ul>
  <li>The v1.24 release of Kubernetes "actually removed the dockershim", the adapter for Docker [S39].</li>
  <li>"The image that Docker produces isn't really a Docker-specific image—it's an OCI image" [S40].</li>
  <li>A standard image therefore keeps working under every conforming engine [S40].</li>
  <li>The standard was launched in June 2015 by Docker, CoreOS and others [S9].</li>
</ul>

---

<h2>Takeaways</h2>

<div class="rows">
<div class="row">
<div class="num">1</div>
<div class="term">No kernel object</div>
<div class="desc">Five mechanisms in one configuration file. Remove one and something still runs.</div>
</div>
<div class="row">
<div class="num">2</div>
<div class="term">Not a small machine</div>
<div class="desc">The boundary is a shared kernel. The record has real escapes through it.</div>
</div>
<div class="row">
<div class="num">3</div>
<div class="term">Same parts, two shapes</div>
<div class="desc">They differ over who owns the container, not over what is asked of the kernel.</div>
</div>
<div class="row">
<div class="num">4</div>
<div class="term">Linux, or a Linux inside</div>
<div class="desc">Lima and WSL 2 supply that kernel. Windows containers are the exception.</div>
</div>
<div class="row">
<div class="num">5</div>
<div class="term">Configuration, not magic</div>
<div class="desc">Read the capability set before reaching for an option that removes it.</div>
</div>
</div>

Note:

<ul>
  <li>A container is a list of kernel requests, not a kernel object [T1] [V2].</li>
  <li>The boundary is the system-call interface of a shared kernel [T2] [V7].</li>
  <li>The two engines disagree about ownership of the container process [T3] [V15].</li>
  <li>A Linux container on macOS or Windows runs inside a Linux virtual machine [T4] [V9].</li>
  <li>What a process may do is a readable capability set and a readable filter [T5] [V10].</li>
</ul>

---

<h2>What follows</h2>

<p class="lede">Four things to do differently once a container stops being a black box.</p>

<div class="kv">
<div class="pair">
<div class="k">1 · Read the refusal</div>
<div class="v">Find which capability or filter rule refused. Then grant that one.</div>
</div>
<div class="pair">
<div class="k">2 · Choose where data lives</div>
<div class="v">Writable layer, volume or bind mount. Decide before the first run.</div>
</div>
<div class="pair">
<div class="k">3 · Check the host, not the flag</div>
<div class="v">Control group version, namespace policy and architecture set the limits. No option overrides them.</div>
</div>
<div class="pair">
<div class="k">4 · Keep the habit</div>
<div class="v">Predict, observe, explain.</div>
</div>
</div>

<p class="closing">Three of these read a value the machine already holds: the capability set, the storage destination, and the control group version.</p>

Note:

<ul>
  <li>Never answer a permission error with <code>--privileged</code>, which is the absence of a boundary [V8].</li>
  <li>Treat root within a container as if it is root outside of the container [S32].</li>
  <li>The three storage destinations have three different lifetimes [S64].</li>
  <li>The capability set of a running process is readable from <code>/proc</code> [S3].</li>
  <li>Host policy gates rootless operation, and no engine option reaches it [S48] [S77].</li>
  <li>Podman 6.0.0 removed support for control groups version 1, so the host's version decides whether the engine runs at all [S65].</li>
  <li>Resource limits on a rootless container need control groups version 2 [S61].</li>
  <li><code>exec format error</code> is the symptom of an image built for the host's other architecture [S59].</li>
  <li><code>--platform=OS/ARCH</code> selects the image for a platform, and it cannot change the host's own [S61].</li>
  <li>The same correction applies to another everyday word: a thread is not a kernel concept [V26].</li>
  <li>The course toolchain is one reproducible environment built from these primitives [V24].</li>
</ul>

---

## References · 1/18

- **[S1]** <span class="src-title">namespaces(7) — Linux manual page</span> — https://man7.org/linux/man-pages/man7/namespaces.7.html
- **[S2]** <span class="src-title">cgroups(7) — Linux manual page</span> — https://man7.org/linux/man-pages/man7/cgroups.7.html
- **[S3]** <span class="src-title">capabilities(7) — Linux manual page</span> — https://man7.org/linux/man-pages/man7/capabilities.7.html
- **[S4]** <span class="src-title">seccomp(2) — Linux manual page</span> — https://man7.org/linux/man-pages/man2/seccomp.2.html
- **[S5]** <span class="src-title">Seccomp security profiles for Docker</span> — https://docs.docker.com/engine/security/seccomp/
- **[S6]** <span class="src-title">Use the OverlayFS storage driver</span> — https://docs.docker.com/engine/storage/drivers/overlayfs-driver/

---

## References · 2/18

- **[S7]** <span class="src-title">OCI Runtime Specification — Linux Container Configuration</span> — https://github.com/opencontainers/runtime-spec/blob/main/config-linux.md
- **[S8]** <span class="src-title">OCI Image Format Specification</span> — https://github.com/opencontainers/image-spec/blob/main/spec.md
- **[S9]** <span class="src-title">About the Open Container Initiative</span> — https://opencontainers.org/about/overview/
- **[S10]** <span class="src-title">Docker overview</span> — https://docs.docker.com/get-started/docker-overview/
- **[S11]** <span class="src-title">Rootless mode</span> — https://docs.docker.com/engine/security/rootless/
- **[S12]** <span class="src-title">Live restore</span> — https://docs.docker.com/engine/daemon/live-restore/
- **[S13]** <span class="src-title">podman(1) — Podman manual page</span> — https://docs.podman.io/en/latest/markdown/podman.1.html

---

## References · 3/18

- **[S14]** <span class="src-title">podman — project README</span> — https://github.com/podman-container-tools/podman
- **[S15]** <span class="src-title">What happens behind the scenes of a rootless Podman container?</span> — https://www.redhat.com/en/blog/behind-scenes-podman
- **[S16]** <span class="src-title">containers/crun</span> — https://github.com/containers/crun
- **[S17]** <span class="src-title">Why strace doesn't work in Docker</span> — https://jvns.ca/blog/2020/04/29/why-strace-doesnt-work-in-docker/
- **[S18]** <span class="src-title">What even is a container</span> — https://jvns.ca/blog/2016/10/10/what-even-is-a-container/
- **[S19]** <span class="src-title">Jails: Confining the omnipotent root</span> — http://phk.freebsd.dk/pubs/sane2000-jail.pdf
- **[S20]** <span class="src-title">Jails – High value but shitty Virtualization</span> — http://phk.freebsd.dk/sagas/jails/

---

## References · 4/18

- **[S21]** <span class="src-title">Namespaces</span> — https://k1024.org/papers/namespaces.html
- **[S22]** <span class="src-title">From dotCloud to Docker</span> — https://jpetazzo.github.io/2017/02/24/from-dotcloud-to-docker/
- **[S23]** <span class="src-title">Docker build cache</span> — https://docs.docker.com/build/cache/
- **[S24]** <span class="src-title">A container-confinement breakout</span> — https://lwn.net/Articles/781013/
- **[S25]** <span class="src-title">Anatomy of CVE-2019-5736: A runc container escape</span> — https://aws.amazon.com/blogs/compute/anatomy-of-cve-2019-5736-a-runc-container-escape/
- **[S26]** <span class="src-title">runc security advisory GHSA-9493-h29p-rfm2</span> — https://github.com/opencontainers/runc/security/advisories/GHSA-9493-h29p-rfm2
- **[S27]** <span class="src-title">runc container breakout vulnerabilities: a technical overview</span> — https://www.cncf.io/blog/2025/11/28/runc-container-breakout-vulnerabilities-a-technical-overview/

---

## References · 5/18

- **[S28]** <span class="src-title">runc releases</span> — https://github.com/opencontainers/runc/releases
- **[S29]** <span class="src-title">Towards Improving Container Security by Preventing Runtime Escapes</span> — https://doi.org/10.2172/1890908
- **[S30]** <span class="src-title">Escaping containers using the Dirty Pipe vulnerability</span> — https://securitylabs.datadoghq.com/articles/dirty-pipe-container-escape-poc/
- **[S31]** <span class="src-title">An analysis of security vulnerabilities in container images for scientific data analysis</span> — https://academic.oup.com/gigascience/article/10/6/giab025/6291571
- **[S32]** <span class="src-title">Containers do not contain</span> — https://opensource.com/business/14/7/docker-security-selinux
- **[S33]** <span class="src-title">Kata Containers</span> — https://katacontainers.io/

---

## References · 6/18

- **[S34]** <span class="src-title">OCI Runtime Specification — Virtual Machine Container Configuration</span> — https://github.com/opencontainers/runtime-spec/blob/main/config-vm.md
- **[S35]** <span class="src-title">Isolation modes for Windows containers</span> — https://learn.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container
- **[S36]** <span class="src-title">Zero-Day Vulnerabilities in VMware ESXi</span> — https://www.tenable.com/blog/cve-2025-22224-cve-2025-22225-cve-2025-22226-zero-day-vulnerabilities-in-vmware-esxi
- **[S37]** <span class="src-title">Containerization</span> — https://github.com/apple/containerization
- **[S38]** <span class="src-title">Firecracker: Lightweight Virtualization for Serverless Applications</span> — https://www.usenix.org/system/files/nsdi20-paper-agache.pdf
- **[S39]** <span class="src-title">Dockershim Removal FAQ</span> — https://kubernetes.io/blog/2022/02/17/dockershim-faq/

---

## References · 7/18

- **[S40]** <span class="src-title">Don't Panic: Kubernetes and Docker</span> — https://kubernetes.io/blog/2020/12/02/dont-panic-kubernetes-and-docker/
- **[S41]** <span class="src-title">Memory inside Linux containers</span> — https://fabiokung.com/2014/03/13/memory-inside-linux-containers/
- **[S42]** <span class="src-title">containers/common — pkg/config/default.go</span> — https://github.com/containers/common/blob/main/pkg/config/default.go
- **[S43]** <span class="src-title">moby/moby — daemon/pkg/oci/caps/defaults.go</span> — https://github.com/moby/moby/blob/master/daemon/pkg/oci/caps/defaults.go
- **[S44]** <span class="src-title">Docker on macOS is slow and how to fix it</span> — https://www.cncf.io/blog/2023/02/02/docker-on-macos-is-slow-and-how-to-fix-it/

---

## References · 8/18

- **[S45]** <span class="src-title">Rosetta emulation on arm64 macOS breaks many amd64 images</span> — https://github.com/docker/for-mac/issues/7137
- **[S46]** <span class="src-title">Comparing WSL Versions</span> — https://learn.microsoft.com/en-us/windows/wsl/compare-versions
- **[S47]** <span class="src-title">Systemd support in WSL</span> — https://learn.microsoft.com/en-us/windows/wsl/systemd
- **[S48]** <span class="src-title">Ubuntu 24.04 LTS release notes</span> — https://documentation.ubuntu.com/release-notes/24.04/
- **[S49]** <span class="src-title">Docker Hub usage and rate limits</span> — https://docs.docker.com/docker-hub/usage/
- **[S50]** <span class="src-title">Docker Desktop license agreement</span> — https://docs.docker.com/subscription/desktop-license/
- **[S51]** <span class="src-title">Install Docker Engine</span> — https://docs.docker.com/engine/install/
- **[S52]** <span class="src-title">Podman Installation Instructions</span> — https://podman.io/docs/installation

---

## References · 9/18

- **[S53]** <span class="src-title">Lima templates</span> — https://lima-vm.io/docs/templates/
- **[S54]** <span class="src-title">Lima configuration</span> — https://lima-vm.io/docs/config/
- **[S55]** <span class="src-title">podman-machine-init(1)</span> — https://docs.podman.io/en/latest/markdown/podman-machine-init.1.html
- **[S56]** <span class="src-title">containers-from-scratch</span> — https://github.com/lizrice/containers-from-scratch
- **[S57]** <span class="src-title">Predict-Observe-Explain</span> — https://science-education-research.com/teaching-science/constructivist-pedagogy/predict-observe-explain/
- **[S58]** <span class="src-title">Solaris Zones: Operating System Support for Consolidating Commercial Workloads</span> — https://www.usenix.org/legacy/event/lisa04/tech/full_papers/price/price.pdf

---

## References · 10/18

- **[S59]** <span class="src-title">Moving ARM64 Docker Builds Off QEMU</span> — https://www.warpbuild.com/guides/qemu-vs-native-arm64-builds
- **[S60]** <span class="src-title">OCI Image Format Specification — Image Configuration</span> — https://github.com/opencontainers/image-spec/blob/main/config.md
- **[S61]** <span class="src-title">podman-run(1) — Podman command reference</span> — https://docs.podman.io/en/latest/markdown/podman-run.1.html
- **[S62]** <span class="src-title">An introduction to Docker for reproducible research</span> — https://polaris.imag.fr/arnaud.legrand/research/readings/acm_sigops_si_rsea/p71-boettiger.pdf
- **[S63]** <span class="src-title">An Updated Performance Comparison of Virtual Machines and Linux Containers</span> — https://www.read.seas.harvard.edu/~kohler/class/cs260r-s19/containerperf14.pdf

---

## References · 11/18

- **[S64]** <span class="src-title">Storage overview — Docker Docs</span> — https://docs.docker.com/engine/storage/
- **[S65]** <span class="src-title">Podman v6.0.0 release notes</span> — https://github.com/podman-container-tools/podman/releases/tag/v6.0.0
- **[S66]** <span class="src-title">podman-compose(1) — Podman manual page</span> — https://docs.podman.io/en/latest/markdown/podman-compose.1.html
- **[S67]** <span class="src-title">ptrace(2) — Linux manual page</span> — https://man7.org/linux/man-pages/man2/ptrace.2.html
- **[S68]** <span class="src-title">Resource constraints — Docker Docs</span> — https://docs.docker.com/engine/containers/resource_constraints/
- **[S69]** <span class="src-title">lxc/lxcfs — project README</span> — https://github.com/lxc/lxcfs
- **[S70]** <span class="src-title">containerd Runtime v2 (shim API)</span> — https://containerd.io/docs/2.3/runtime-v2/

---

## References · 12/18

- **[S71]** <span class="src-title">Advanced settings configuration in WSL</span> — https://learn.microsoft.com/en-us/windows/wsl/wsl-config
- **[S72]** <span class="src-title">Working across Windows and Linux file systems (WSL)</span> — https://learn.microsoft.com/en-us/windows/wsl/filesystems
- **[S73]** <span class="src-title">Multi-tenancy — Kubernetes documentation</span> — https://kubernetes.io/docs/concepts/security/multi-tenancy/
- **[S74]** <span class="src-title">Overlay Filesystem — the Linux Kernel documentation</span> — https://docs.kernel.org/filesystems/overlayfs.html
- **[S75]** <span class="src-title">What is a container? — Docker Docs</span> — https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-container/

---

## References · 13/18

- **[S76]** <span class="src-title">Volumes — Docker Docs</span> — https://docs.docker.com/engine/storage/volumes/
- **[S77]** <span class="src-title">Ubuntu 23.10 restricted unprivileged user namespaces</span> — https://ubuntu.com/blog/ubuntu-23-10-restricted-unprivileged-user-namespaces
- **[S78]** <span class="src-title">containers.conf(5) — configuration reference</span> — https://github.com/containers/common/blob/main/docs/containers.conf.5.md
- **[S79]** <span class="src-title">Troubleshooting Windows Subsystem for Linux</span> — https://learn.microsoft.com/en-us/windows/wsl/troubleshooting
- **[S80]** <span class="src-title">Bind mounts — Docker Docs</span> — https://docs.docker.com/engine/storage/bind-mounts/
- **[S81]** <span class="src-title">OCI Runtime Specification — Configuration</span> — https://github.com/opencontainers/runtime-spec/blob/main/config.md

---

## References · 14/18

- **[S82]** <span class="src-title">Install Docker Engine on Ubuntu</span> — https://docs.docker.com/engine/install/ubuntu/
- **[S83]** <span class="src-title">Homebrew: The Package Manager for Everywhere</span> — https://brew.sh
- **[S84]** <span class="src-title">Yama — the Linux Kernel documentation</span> — https://docs.kernel.org/admin-guide/LSM/Yama.html
- **[S85]** <span class="src-title">Linux post-installation steps for Docker Engine</span> — https://docs.docker.com/engine/install/linux-postinstall/
- **[S86]** <span class="src-title">Homebrew — Installation</span> — https://docs.brew.sh/Installation
- **[V1]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V2]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V3]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.

---

## References · 15/18

- **[V4]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V5]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V6]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V7]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V8]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V9]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V10]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V11]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V12]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.

---

## References · 16/18

- **[V13]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V14]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V15]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V16]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V17]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V18]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V19]** <span class="src-title">Copy-on-Write</span> — note in the author's research vault.
- **[V20]** <span class="src-title">Copy-on-Write</span> — note in the author's research vault.
- **[V21]** <span class="src-title">The System Call Boundary</span> — note in the author's research vault.

---

## References · 17/18

- **[V22]** <span class="src-title">Where the Abstraction Leaks</span> — note in the author's research vault.
- **[V23]** <span class="src-title">Where the Abstraction Leaks</span> — note in the author's research vault.
- **[V24]** <span class="src-title">devcontainers</span> — note in the author's research vault.
- **[V26]** <span class="src-title">Processes, Threads and Scheduling</span> — note in the author's research vault.
- **[V27]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V28]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V29]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V30]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V31]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.

---

## References · 18/18

- **[V32]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V33]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V34]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V35]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V36]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V37]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.
- **[V38]** <span class="src-title">Containers with Docker and Podman</span> — note in the author's research vault.

<span class="neu">The references document (PDF and web page) carries the full source list, and a summary of every [V] source with its own external references.</span>

Note:

Source list only.

---

<h2>Everything from this talk</h2>

<div class="downloads">
  <a href="https://ondrasek.github.io/talks/2026-09-24-pb158-containerization/2026-09-24-pb158-containerization-slides.pdf">Slides (PDF)</a>
  <a href="https://ondrasek.github.io/talks/2026-09-24-pb158-containerization/2026-09-24-pb158-containerization-references.pdf">References (PDF)</a>
  <a href="https://ondrasek.github.io/talks/2026-09-24-pb158-containerization/references.html">References (web page)</a>
  <a href="https://ondrasek.github.io/talks/2026-09-24-pb158-containerization/2026-09-24-pb158-containerization-speaker-notes.pdf">Speaker notes (PDF)</a>
  <a href="https://ondrasek.github.io/talks/2026-09-24-pb158-containerization/2026-09-24-pb158-containerization.zip">Everything (.zip) — readable by an AI agent</a>
</div>

<p class="lede"><a href="https://ondrasek.github.io/talks">ondrasek.github.io/talks</a></p>

<p class="closing"><a href="mailto:me@ondrejkrajicek.com">me@ondrejkrajicek.com</a></p>

Note:
Everything is already online — slides, the references and these notes. Say the address once, and let the room photograph the slide rather than writing it down.