DMA

Without DMA (Direct Memory Access), every byte transferred from a disk or network card would require the CPU to copy it. DMA lets devices write directly to RAM — the CPU just tells the device where to put data and gets an interrupt when it's done. This is why modern I/O doesn't peg your CPU at 100%.

How DMA Works

Without DMA (PIO — Programmed I/O):
  CPU: read 512 bytes from disk, one register at a time
  CPU: copy each byte to memory
  CPU: repeat for each sector
  CPU utilization: 100% during entire transfer
  Performance: ~MB/s limited by CPU copy speed

With DMA:
  1. CPU: "DMA controller, copy 4KB from disk to address 0x1234000"
  2. CPU: continues other work (or sleeps)
  3. DMA controller: directly reads from device and writes to RAM
  4. DMA controller: raises interrupt when done
  5. CPU: receives interrupt, processes the data
  CPU utilization: ~1% during transfer
  Performance: limited by bus/device bandwidth, not CPU

Types of DMA Mappings

How does a driver tell the DMA controller where to write? The driver allocates DMA-capable memory using the kernel's DMA API, which returns a DMA address (bus address) the device can use. The DMA address might be a physical address directly, or it might be translated through an IOMMU (I/O Memory Management Unit). The driver passes this address to the device, the device writes there, and the CPU sees the data at the corresponding virtual address.

Type	Use case	Characteristic
Coherent / consistent DMA	Control structures, descriptors	CPU and device always see same data (no cache issues)
Streaming DMA	Data buffers (packets, disk blocks)	One-time transfer, needs explicit cache sync
Scatter-gather DMA	Non-contiguous buffers	Device can write to multiple scattered memory regions

IOMMU — Protecting Against DMA Attacks

What stops a malicious PCIe device from DMA-ing into kernel memory? Without protection, any PCIe device can DMA to any physical address — including kernel code and page tables. This is how "DMA attacks" work (a compromised Thunderbolt device reading RAM). IOMMU (Intel VT-d or AMD-Vi) adds an MMU for device memory accesses. The kernel configures the IOMMU to restrict each device to specific memory ranges — a compromised device can't access other process's or kernel memory.

# Enable IOMMU (add to kernel command line):
# /etc/default/grub:
# GRUB_CMDLINE_LINUX="intel_iommu=on"   (Intel)
# GRUB_CMDLINE_LINUX="amd_iommu=on"    (AMD)

# Check if IOMMU is active:
dmesg | grep -i iommu
# [    0.789] DMAR: IOMMU enabled
# [    0.790] DMAR: Intel(R) Virtualization Technology for Directed I/O

# IOMMU groups (devices that share IOMMU context):
ls /sys/kernel/iommu_groups/
# 0  1  2  3  4  5  ...

# Each group = isolated DMA domain
# Devices in same group share IOMMU protection boundary

# VFIO: pass PCIe device to VM using IOMMU isolation
# The VM gets direct device access via IOMMU, host is protected

DMA Issues and Debugging

# DMA failures often appear as:
dmesg | grep -i "dma\|iommu"
# [  123.456] DMAR: DRHD: handling fault status reg 2
# [  123.457] DMAR: [DMA Write] Request device [02:00.0]
#              fault addr ffff800000000000 [fault reason 02] Present bit clear
# This = device tried to DMA outside its allowed range

# DMA zone limits (older hardware can only DMA to <4GB):
cat /proc/buddyinfo | grep DMA
# Node 0, zone      DMA      1     0     1     0     0 ...
# Node 0, zone    DMA32     ...   (32-bit DMA zone, 0-4GB)
# Node 0, zone   Normal     ...   (full 64-bit addressable)

# Bounce buffers: for devices that can't DMA to high memory
# Kernel copies: device DMA's to low memory, kernel copies to high memory
# Causes extra copy overhead on older hardware

# Check DMA memory stats:
grep -i dma /proc/meminfo
# DirectMap4k:   ...
# DirectMap2M:   ...