journald — systemd Logging
Traditional Linux logging scattered messages across dozens of text files in /var/log/. systemd replaced this with journald — a structured binary logging service that collects everything in one place and makes it searchable.
What journald Does
Why use a binary log format instead of plain text?
Binary logs carry structured metadata with every message: exact timestamp with microsecond precision, service name, PID, UID, priority level, and more. This makes filtering and searching much faster and more reliable. You can't accidentally break parsing with a rogue newline.
journald collects logs from:
- All systemd services (via stdout/stderr)
- The kernel ring buffer (dmesg)
- Traditional syslog via /dev/log socket
- Audit records
Persistent vs Volatile Storage
Where are journal files stored?
By default on many systems:
/run/log/journal/ — this is tmpfs (RAM), so logs disappear on reboot. To persist across reboots: create /var/log/journal/ and journald automatically switches to it.
# Enable persistent logging
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal
# Or set in config
sudo nano /etc/systemd/journald.conf
# Storage=persistent ← change from 'auto' to 'persistent'
sudo systemctl restart systemd-journald
journalctl — Reading the Journal
# View all logs (newest at bottom)
journalctl
# Follow in real-time (like tail -f)
journalctl -f
# Show logs for a specific service
journalctl -u nginx.service
journalctl -u nginx -f # follow nginx logs
# Filter by time
journalctl --since "2024-01-15 09:00:00"
journalctl --since "1 hour ago"
journalctl --since yesterday --until today
# Filter by priority (0=emerg to 7=debug)
journalctl -p err # only errors and above
journalctl -p warning..err # between warning and error
# Show logs from current boot only
journalctl -b
# Show logs from previous boot
journalctl -b -1
# List available boots
journalctl --list-boots
# Show kernel messages only (like dmesg)
journalctl -k
# Format as JSON for parsing
journalctl -o json-pretty -n 5
Log Priority Levels
| Level | Number | Meaning |
|---|---|---|
| emerg | 0 | System unusable |
| alert | 1 | Immediate action required |
| crit | 2 | Critical conditions |
| err | 3 | Error conditions |
| warning | 4 | Warning conditions |
| notice | 5 | Normal but significant |
| info | 6 | Informational |
| debug | 7 | Debug messages |
Log Rotation and Size Limits
# In /etc/systemd/journald.conf:
SystemMaxUse=500M # max disk usage for persistent logs
RuntimeMaxUse=100M # max for volatile (/run) logs
MaxRetentionSec=1month # delete logs older than this
# Check current journal disk usage
journalctl --disk-usage
# Manually vacuum old logs
journalctl --vacuum-size=200M # keep only last 200MB
journalctl --vacuum-time=2weeks # delete older than 2 weeks