A small VM paged me for a full root partition. Not a leaking application, not a runaway core dump, just the journal quietly doing its job a bit too well. journalctl --disk-usage reported the persistent journal had grown to several gigabytes on a box with a 20G root, because nobody had ever told it to stop.
The default behaviour is reasonable in spirit: by default journald caps the persistent store at 10% of the filesystem and keeps 15% free. The trouble is that 10% of a generous partition is a lot, and on a chatty service it fills with noise you'll never read. So set an explicit ceiling. In /etc/systemd/journald.conf:
[Journal]
SystemMaxUse=500M
SystemMaxFileSize=50M
MaxRetentionSec=2week
Then systemctl restart systemd-journald to apply it. The first two cap the total and per-file size, the retention bound throws away anything older than a fortnight regardless of size, which is usually the honest answer to "how far back will I ever actually look".
To reclaim space right now without waiting for rotation, vacuum it:
journalctl --vacuum-size=500M
# or
journalctl --vacuum-time=2weeks
That trims the existing journal in place. The disk freed immediately, the alert cleared, and the box has stayed under half a gig of logs since. None of this is clever. It's the kind of thing you set once in your base image and then never think about again, which is exactly why it's worth writing down so I remember to put it in the base image.