Ramblings of an aging IT geek
← Ramblings of an aging IT geek
homelab

i ran kubernetes in my house and i have regrets (mostly)

An honest account of running a small Kubernetes cluster at home, what it taught me, and why it was mostly the wrong tool.

A server rack with networking gear

I built a three-node Kubernetes cluster in my spare room, ran it for a few months, and I am now in the process of tearing most of it down. Not because it failed. Because it worked, and in working it taught me that I did not need it, which is a more useful and more annoying lesson than outright failure would have been.

Let me be fair to it first. As a learning exercise it was excellent. If you want to actually understand what a Pod, a Service, an Ingress and a PersistentVolume are, nothing beats standing one up yourself on hardware you can see and break. I learned more about Kubernetes networking from one evening of a misbehaving CNI plugin than from a stack of documentation. When the day job involves these clusters, having broken one at home in low stakes is genuinely valuable.

A homelab setup with cables and small servers

But here is the uncomfortable bit. The things I actually run at home are: a media server, a couple of databases, some monitoring, a VPN endpoint, a handful of small web services. None of these need orchestration. None of them autoscale, because the load is me and occasionally my partner. None of them need rolling zero-downtime deployments, because nobody is paged at 3am if my photo gallery is down for ninety seconds during an update. Kubernetes is built to solve problems I do not have.

What it gave me instead was a constant, low-grade tax. A few examples from the actual logbook:

  • Storage was a permanent fight. Stateful workloads on a small home cluster mean wrestling with PersistentVolumes, and every time a node rebooted there was a non-zero chance something would not reattach cleanly. My databases lived in a state of mild anxiety.
  • The control plane is not free. etcd, the API server, the scheduler, the controller manager. On modest home hardware that overhead is a real fraction of your resources, spent managing the thing rather than running your actual workloads.
  • Upgrades were genuinely scary. A minor version bump could and did break my networking. In production a team owns that risk. At home it is me, on a Sunday, with the household asking why Netflix's local equivalent has stopped working.
  • Debugging required keeping a whole mental model resident. To work out why one container could not reach another I had to reason about Pods, Services, kube-dns, network policies and the CNI all at once. For a media server that should be a single docker run.

So where I have landed is this. The cluster is being replaced by plain Docker with docker-compose for the multi-container bits, on a single well-backed-up host, with the occasional systemd unit for things that want to be first-class on the machine. It is boring. It is comprehensible. When something breaks I can usually see why within a minute, because there are about three layers instead of nine.

The "mostly" in the title is doing real work, though. I am keeping a single-node setup around purely as a lab, deliberately ring-fenced from anything I actually rely on, so that I can keep my hand in for work without my home life depending on it. That, I think, is the honest place for Kubernetes in a homelab. A toy you learn on, not the foundation you build your house upon.

If you are reaching for Kubernetes at home, do it with clear eyes. It is a phenomenal tool for the problem it was designed for, which is running fleets of services at a scale you almost certainly do not have in your spare room. For everything else, a single box and Docker will serve you better and let you go to bed earlier.