Cap what a container can take

By default a container may use all the memory and all the CPU on the host. One leaking process then takes down every other service on the box. Two flags prevent it, and the failure they produce looks like nothing else.

1. See that there is no limit

docker run --rm alpine:3.20 sh -c 'cat /sys/fs/cgroup/memory.max 2>/dev/null \
  || cat /sys/fs/cgroup/memory/memory.limit_in_bytes'

max — unlimited. The container can request every byte the host has.

2. Set a memory limit and watch the kill

docker run --rm --memory=64m --memory-swap=64m --name labmem alpine:3.20 \
  sh -c 'cat /sys/fs/cgroup/memory.max'
docker run -d --memory=64m --memory-swap=64m --name labmem alpine:3.20 \
  sh -c 'x=""; while :; do x="$x$(head -c 1000000 /dev/zero | tr "\0" "a")"; done'
sleep 8
docker ps -a --filter name=labmem --format '{{.Status}}'
docker inspect labmem \
  --format 'OOMKilled={{.State.OOMKilled}} ExitCode={{.State.ExitCode}}'

OOMKilled=true and exit code 137. That number is worth memorising: 137 is 128 + 9, meaning killed by signal 9. A container that exits 137 with no useful log was almost certainly killed for exceeding its memory limit, and the application's own logs will show nothing because it never got a chance to write any.

Setting --memory-swap equal to --memory disables swap for the container. Without it, the container swaps instead of dying, which is slower and much harder to diagnose.

3. Find the evidence on the host

sudo dmesg -T | grep -i -E "oom|killed process" | tail -n 5
docker events --since 10m --filter event=oom --until now 2>/dev/null | tail -n 3

The kernel logs every OOM kill with the process name and how much it was using. On a host where several containers died overnight, dmesg -T is the record that tells you which and when — the container is gone, but this is not.

4. Limit CPU, and see it throttle rather than die

docker rm -f labmem >/dev/null
nproc
docker run -d --cpus=0.5 --name labcpu alpine:3.20 \
  sh -c 'while :; do :; done'
sleep 5
docker stats --no-stream \
  --format '{{.Name}} CPU={{.CPUPerc}} MEM={{.MemUsage}}' labcpu
docker inspect labcpu --format '{{.HostConfig.NanoCpus}}'
docker rm -f labcpu >/dev/null

CPU is different from memory: exceeding the limit throttles, it does not kill. A container pinned at its CPU ceiling is slow, not dead — which is why a latency problem shows up as high CPU with a healthy process, and why docker stats sitting at exactly your limit is a finding rather than a coincidence.

--cpus=0.5 means half a core's worth of time, whatever the core count. It is a quota, not an affinity — use --cpuset-cpus if you genuinely need to pin to specific cores.

5. Limit the other two things that run out

docker run --rm --pids-limit=20 alpine:3.20 \
  sh -c 'for i in $(seq 1 40); do sleep 60 & done 2>/dev/null; \
         echo "spawned: $(ls /proc | grep -c "^[0-9]")"'

docker run --rm --memory=64m --tmpfs /tmp:size=8m alpine:3.20 \
  sh -c 'dd if=/dev/zero of=/tmp/f bs=1M count=20 2>&1 | tail -n 1'

--pids-limit stops a fork bomb, deliberate or accidental. The --tmpfs size cap matters because a tmpfs is memory: an unbounded one inside a memory-limited container is a way to trigger the OOM kill from a completely different direction.

6. Read what a running container is actually using

docker run -d --memory=128m --cpus=1 --name labapp nginx:alpine >/dev/null
sleep 3
docker stats --no-stream
docker exec labapp cat /sys/fs/cgroup/memory.max
docker exec labapp cat /sys/fs/cgroup/memory.current
docker rm -f labapp >/dev/null

The MEM USAGE / LIMIT column in docker stats is the pair to watch. Sitting near the limit is the state before a 137, and it is visible for minutes or hours beforehand.

One trap worth knowing: many runtimes read the _host's_ memory, not the cgroup limit, and size their heap accordingly — then get OOM-killed at a fraction of what they think they have. Modern JVMs and Node handle this; older ones need the limit passed explicitly.

What to set, always

Compose calls these deploy.resources.limits; Kubernetes calls them resources.limits. Same cgroup underneath.

Clean up

docker rm -f labmem labcpu labapp 2>/dev/null; echo cleaned

Where this goes next

Limits are set. Next: making sure the image you run is the image you tested.