sdlc.dev

Linux path · The kernel boundary · Topic 8 of 12

Virtual memory, the page cache and the OOM killer

Not started

Each process sees its own virtual address space. The kernel splits it into pages (4 KiB on most systems; some ARM64 kernels use 16 or 64 KiB; getconf PAGESIZE tells you) and maps each page to physical RAM only when the process first touches it. That is why a process can reserve far more than it uses, why "used memory" means several different things, and why Linux can fill spare RAM with cached file data. When memory truly runs out, the out-of-memory (OOM) killer picks a process and kills it.

Why it matters: Reading free, top and container memory graphs correctly is the difference between a real leak and a healthy cache, and between a crash and an OOM kill.

  1. 1Every process has its own address space

    Addresses are virtual. The kernel maps pages of them to physical memory only when they are first touched.

  2. 2VSZ is promised, RSS is resident

    Virtual size counts everything mapped; resident set size counts what is actually in RAM right now.

  3. 3Free memory becomes cache

    The kernel keeps recently read and written file data in the page cache and gives it back when programs need it. Watch 'available', not 'free'.

  4. 4The OOM killer is the last resort

    When memory (or a cgroup's limit) runs out and nothing can be reclaimed, the kernel kills a process with SIGKILL.

1

How it works

The explanation, with examples you can run.

Reserved is not resident

python3 - <<'EOF'import mmap, os, subprocessdef show(label):    out = subprocess.run(["ps", "-o", "vsz=,rss=", "-p", str(os.getpid())], capture_output=True, text=True)    print(label, out.stdout.strip())m = mmap.mmap(-1, 1 << 30)            # map 1 GiB, touch nothingshow("after mmap 1GiB:")for i in range(0, 200 << 20, 4096):   # touch 200 MiB, one byte per page    m[i] = 1show("after touching 200MiB:")EOF
after mmap 1GiB: 1064824 10592after touching 200MiB: 1064824 215396

Both columns are in KiB. VSZ (virtual size) jumped by 1 GiB at once; RSS (resident set size) grew only when pages were written. The same thing happens when the JVM reserves its maximum heap or Go maps arenas: big VSZ is normal, RSS is what costs RAM. Even RSS overstates things a bit, because shared libraries are counted in every process that maps them; PSS in /proc/<pid>/smaps_rollup shares them out fairly.

The page cache

free -mdd if=/dev/urandom of=/tmp/big bs=1M count=300 status=nonefree -m | head -2
               total        used        free      shared  buff/cache   availableMem:            7835         838        6463          24         729        6997Swap:           1023           0        1023               total        used        free      shared  buff/cache   availableMem:            7835         837        6160          24        1037        6997

Writing a 300 MB file moved about 300 MB from free to buff/cache, while available barely changed. File data you read or write stays in RAM so the next read is fast, and writes are collected as dirty pages and flushed to disk in the background (Dirty: in /proc/meminfo, or call fsync when it must be on disk now). When a program needs memory, the kernel drops clean cached pages first. So on a busy server low free is healthy; low available combined with swapping or OOM kills is the problem.

When memory runs out

If programs need more than RAM plus swap, and the kernel can't reclaim enough cache, it invokes the OOM killer. It scores processes mostly by how much memory they use, adjusted by /proc/<pid>/oom_score_adj (-1000 exempts a process, +1000 makes it the first choice), kills the winner with SIGKILL and logs it to the kernel log (dmesg or journalctl -k shows Out of memory: Killed process ...).

Containers make this local. A memory limit is a cgroup limit, and hitting it triggers the OOM killer inside that cgroup only. This was run on the Mac host against Docker Desktop:

docker run --name oomtest -m 64m --memory-swap 64m ubuntu:24.04 \  bash -c 'head -c 200m /dev/zero | tail > /dev/null'; echo "exit=$?"docker inspect -f '{{.State.OOMKilled}} {{.State.ExitCode}}' oomtest; docker rm oomtest
bash: line 1:     8 Broken pipe             head -c 200m /dev/zero         9 Killed                  | tail > /dev/nullexit=137true 137oomtest

tail buffers its input looking for the last lines of a 200 MB stream with no newlines, passes the 64 MiB limit and is killed. Exit code 137 plus OOMKilled: true is the signature to look for in Kubernetes too (Reason: OOMKilled). The fix is a higher limit or less memory use, not a retry.

On macOS

macOS also uses virtual memory and a file cache, but reports it differently: vm_stat and Activity Monitor show wired, active, inactive and compressed memory (macOS compresses pages before swapping), and there is no OOM killer in the Linux sense; the system asks apps to free memory and may terminate them under pressure. free doesn't exist on macOS.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.free shows very little 'free' memory but a large 'available' figure. Is the machine short of memory?
  2. 2.A process maps 1 GiB but touches only 200 MiB. Roughly what are its VSZ and RSS?
  3. 3.A container exits with code 137 and docker inspect says OOMKilled true. What happened?

0 of 3 right so far. Get all three to finish this topic.

Go deeper