I've spent more evenings than I'd like to admit arguing with myself about swap. Not with other people, with myself, which is worse, because there's no one to concede to. Over the Christmas lull I rebuilt two boxes and decided to just settle it: a written-down policy for my homelab, machine by machine, so I never have to relitigate it at 11pm again.
The short version: swap is not "extra RAM", it never was, and treating it as a tier of slow memory is the source of every bad take. Swap exists so the kernel can evict cold anonymous pages it would otherwise be forced to keep resident. On a box with no swap at all, those cold pages sit in RAM forever, crowding out page cache that would actually speed things up. That is the bit the "just disable swap, RAM is cheap" crowd misses.
The three machines, three answers
My lab isn't homogeneous, so one rule was never going to fit. I split it by what the machine is for.
The hypervisor (64 GB, runs VMs and containers). A small disk-backed swap, 8 GB, and a very low swappiness. I do not want the kernel paging out a busy VM's working set to chase a marginal cache win, but I do want a release valve if something leaks before I notice.
vm.swappiness = 10
vm.vfs_cache_pressure = 50
The little NUC that runs Home Assistant and a few sidecars (8 GB, SSD). This one gets zram and basically nothing else. zram gives you a compressed block device in RAM that the kernel swaps to, so cold pages get squeezed rather than written to flash. On a small box that's a genuine win, and it spares the SSD the write churn.
# /etc/systemd/zram-generator.conf
[zram0]
zram-size = ram / 2
compression-algorithm = zstd
The build/CI box (32 GB, occasionally compiles Rust until it weeps). Here I want real swap, and a decent amount of it, 16 GB on a dedicated partition. Big link steps and the odd parallel cargo build -j can spike memory hard for a few seconds. I would much rather the build go briefly slow than have the OOM killer reap a linker and waste forty minutes. swappiness stays at the default 60 because, frankly, on this box I want the kernel to be free with eviction.
What about the OOM killer?
The thing that finally calmed me down was accepting that swap and the OOM killer solve different problems. Swap handles "I have more cold pages than I'd like resident". The OOM killer handles "I have committed to more memory than exists, full stop". No quantity of swap saves you from a genuine runaway; it just changes how long you thrash before the inevitable. So I stopped sizing swap to "survive a leak" and started sizing it to "absorb a spike".
For the boxes I actually care about, I also wired up systemd-oomd and earlyoom so that pressure gets acted on before the kernel's own OOM killer wades in with a cricket bat. earlyoom watching for low available memory plus low available swap, and killing the fattest offender, has saved me from a hung machine more than once this year. A hung machine you have to walk to. A killed process leaves a log line.
earlyoom -m 5 -s 5 -r 60
The policy, written down
So, settled, and pinned to the wiki so future-me can't wriggle out of it:
- Plenty of RAM, latency-sensitive workloads: small disk swap, swappiness 10.
- Small flash-based box: zram with zstd, no disk swap, save the SSD.
- Memory-spiky batch work: generous real swap, default swappiness, let it breathe.
- Everywhere worth saving: an OOM responder so pressure is handled, not endured.
Is any of this revolutionary? Not remotely. But the value wasn't a clever setting, it was deciding once, per role, and writing it down. The next time I catch myself about to disable swap "because RAM is cheap", the wiki page will be there to tell me to behave. Whether I listen is a separate problem.