Ramblings of an aging IT geek
← Ramblings of an aging IT geek
homelab

kubernetes in the spare room, and why i kept some of it

A frank account of running Kubernetes in my homelab, the parts that were a waste of my evenings, and the surprisingly small core I'd keep.

A server rack with cabling

Running Kubernetes at home was, for the most part, a mistake, and I say that as someone who runs it for a living. The skill transfers beautifully. The justification does not. At work the complexity buys you something: multi-team isolation, real scale, an SRE rota, a reason for all those moving parts. At home it's me, three boxes, and a desire to watch films without the NAS falling over.

I started with the full thing. A proper cluster, ingress controllers, cert-manager, a CNI I'd chosen for reasons I can no longer reconstruct, and a GitOps pipeline so that every change went through a pull request to myself. It was glorious for about a fortnight.

A homelab corner with blinking lights

Then the maintenance tax arrived. Kubernetes upgrades are not gentle. Twice a year there's a release that deprecates an API your charts depend on, and the cluster that ran untouched for months suddenly wants three hours of your Saturday to move from one version to the next without breaking ingress. The etcd database needs backing up, and you find that out the interesting way. A node reboots after a kernel update and the control plane sulks. None of this is hard. All of it is time I was spending on the platform instead of the thing the platform was meant to host.

The breaking point was a dead disk. A single SSD failed in a node, and recovering took me an evening of cordoning, draining, replacing, and persuading a couple of pods that their persistent volumes still existed. For a home media stack, that is an absurd amount of ceremony to watch something fall over and get back up. A plain Docker host with a sensible backup would have shrugged.

So what did I keep? Less than I expected, and more than nothing.

I tore the cluster down and moved most services to a single machine running plain docker compose. Boring, legible, recoverable in minutes from a git repo and a backup. Nine in ten of my workloads have no business pretending they need orchestration.

But I kept one small k3s node for the handful of things where Kubernetes actually pulls its weight at this scale:

# the short list of things i let k3s handle
- name: ingress + tls
  why: one place for cert-manager + a real ingress, not per-app nginx
- name: cron-style jobs
  why: kubernetes CronJob with proper history beats a tangle of host crontabs
- name: the one app with a helm chart i didn't want to unpick
  why: honesty

The lesson, if there is one, is that the question was never "is Kubernetes good". It's clearly good; it's the best answer to a problem I don't have in my spare room. The question is whether the operational burden is repaid by the scale and the team, and at home there's no team and there's no scale. Run it once to learn it properly, by all means. Then run something you can fix at midnight without thinking.