cgroups
Namespaces give processes an isolated view of the system. cgroups (control groups) limit how much they can use. Without cgroups, one container could consume all CPU, RAM, or disk I/O on a host, starving everything else. cgroups are why docker run --memory 512m --cpus 2 actually works.
What Are cgroups?
How do cgroups limit resources?
cgroups organize processes into a hierarchy of groups. Each group has controllers — one for CPU, one for memory, one for I/O. You set limits on a group, and the kernel enforces them for every process in that group. When a container is created, its processes are placed in a cgroup and limits are set. The container can never exceed those limits, no matter what runs inside.
# cgroups are exposed as a filesystem
ls /sys/fs/cgroup/
# cgroup v2 (unified):
# cgroup.controllers cgroup.procs cpu.max memory.max io.max ...
# cgroup v1 (legacy — separate directory per controller):
# /sys/fs/cgroup/cpu/
# /sys/fs/cgroup/memory/
# /sys/fs/cgroup/blkio/
# Check which version you're running:
stat -fc %T /sys/fs/cgroup/
# tmpfs = cgroup v1
# cgroup2fs = cgroup v2 (modern, unified)
cgroup v1 vs cgroup v2
| cgroup v1 | cgroup v2 | |
|---|---|---|
| Structure | Separate hierarchy per controller | Single unified hierarchy |
| Process membership | Process can be in different groups per controller | Process in exactly one group |
| Thread-level control | No | Yes (thread mode) |
| Pressure notifications | Basic | PSI (Pressure Stall Information) |
| Default since | Linux 2.6.24 (2008) | Linux 4.5+ (2016), default ~2021 |
| Docker support | Full | Full (Docker 20.10+) |
Creating and Using cgroups (v2)
# Create a cgroup by making a directory
mkdir /sys/fs/cgroup/myapp
# The kernel auto-populates it:
ls /sys/fs/cgroup/myapp/
# cgroup.procs cpu.max memory.max io.max memory.current cpu.stat ...
# Add a process to the cgroup:
echo $$ > /sys/fs/cgroup/myapp/cgroup.procs
# $$ = current shell PID — now this shell and children are in the cgroup
# Set memory limit (512MB):
echo $((512 * 1024 * 1024)) > /sys/fs/cgroup/myapp/memory.max
# Set CPU limit (50% of one core):
# Format: "quota period" in microseconds
echo "50000 100000" > /sys/fs/cgroup/myapp/cpu.max
# 50000/100000 = 50% of 1 CPU
# Check current memory usage:
cat /sys/fs/cgroup/myapp/memory.current
# 45678592 ← bytes currently used
# Kill cgroup (remove all processes, delete dir):
rmdir /sys/fs/cgroup/myapp
How Docker Uses cgroups
# Start a limited container:
docker run -d --name myapp --memory 512m --cpus 2 nginx
# Docker creates a cgroup automatically:
cat /proc/$(docker inspect myapp --format '{{.State.Pid}}')/cgroup
# 0::/system.slice/docker-abc123.scope ← cgroup v2 path
# Find the cgroup:
ls /sys/fs/cgroup/system.slice/docker-abc123.scope/
# memory.max cpu.max cgroup.procs ...
# Verify limits:
cat /sys/fs/cgroup/system.slice/docker-abc123.scope/memory.max
# 536870912 ← 512MB in bytes
cat /sys/fs/cgroup/system.slice/docker-abc123.scope/cpu.max
# 200000 100000 ← 200ms quota per 100ms period = 2 CPUs
# --memory-swap controls total memory+swap:
docker run --memory 512m --memory-swap 1g myimage
# swap = 1g - 512m = 512m of swap allowed
PSI — Pressure Stall Information
How do you know if a cgroup is actually resource-constrained?
PSI (Pressure Stall Information) measures the fraction of time processes are stalled waiting for a resource. Available in cgroup v2. A CPU pressure of 20% means processes spent 20% of time waiting for CPU. Kubernetes uses PSI to detect and evict memory-pressured pods before they OOM.
# Per-cgroup pressure stats:
cat /sys/fs/cgroup/myapp/memory.pressure
# some avg10=5.00 avg60=2.00 avg300=1.00 total=12345
# full avg10=1.00 avg60=0.50 avg300=0.20 total=5678
# "some" = at least one task stalled
# "full" = all tasks stalled (more severe)
# avg10/60/300 = 10s/60s/5min averages (percentage)
cat /sys/fs/cgroup/myapp/cpu.pressure
cat /sys/fs/cgroup/myapp/io.pressure