Ramblings of an aging IT geek
← Ramblings of an aging IT geek
homelab

i put kubernetes in my spare room and mostly regret it

A short reckoning with running a small Kubernetes cluster at home, what it actually cost in time and patience, and the narrow band of cases where it earned its keep.

A small rack of mini PCs with too many ethernet cables

I run Kubernetes at home, and I would like to gently talk you out of doing the same, while admitting I am not going to stop.

The pitch I made to myself was reasonable. I use it at work, so the practice would be useful, and surely the things I self-host (a couple of web apps, some monitoring, the usual indulgent sprawl) would be tidier as deployments than as a heap of hand-tended docker run lines and a systemd unit I always forget the syntax of. Three small machines, k3s because I am not completely without sense, and off I went.

A k9s dashboard showing more pods than a home really needs

Here is what actually happened. The cluster itself is fine; k3s is genuinely good and I will not hear otherwise. What is not fine is everything around the edges. Storage, mostly. A pod on its own is stateless and happy, but the moment something needs a volume that survives a reschedule you are into persistent volumes, and on a home cluster that means either pinning the pod to one node like it's 2012 or standing up some distributed storage layer that eats more RAM than the apps it serves. I spent an entire wet Sunday on Longhorn and emerged with a working setup and a quiet resentment.

Then there is the day the control plane node decides to update its kernel and not come back cleanly, and you are debugging etcd at home, for a Pi-hole. That is the sentence I keep coming back to. I am debugging a distributed consensus database so that ad-blocking DNS can have a 99.9% SLA that precisely nobody is paying for.

Networking is the other tax. On a home cluster you eventually want to reach a service from your phone on the sofa, which means an ingress controller, which means certificates, which means cert-manager, which means a DNS provider with an API and a token sitting in a secret. Each of those is reasonable on its own. Stacked up to expose one web app to one household, they are faintly absurd, and every one of them is another thing that can quietly break and take a service offline while you're at work and unable to kubectl anything.

Where it has actually earned its keep is narrower than I'd like to admit but real. Rolling updates that don't take a service down while I fiddle. Resource limits that stop one greedy container from starving the box. The fact that a node can die and the workloads reschedule themselves onto the survivors without me waking up, which has happened, and which genuinely impressed me at the time. And the genuine, repeatable practice: I have broken and fixed enough small things here that the work version holds no terror. That last one might be the whole justification, honestly. It's an expensive flight simulator, but the crashes are mine and they cost nothing but a weekend.

If you self-host a handful of things and you want them to just stay up, use Docker Compose and a restart policy and go outside. If you want to learn Kubernetes properly and you accept that your spare room is now a lab and not a utility, do it, eyes open, and keep the important DNS on something dumber that you can fix in your sleep.