I run a handful of services at home: a media server, some monitoring, a couple of small web things, the usual stuff that accumulates when you have a spare machine and poor impulse control. For years they lived as plain systemd units on a single box, and they were fine. Boring, but fine. Then I decided I should "properly learn Kubernetes" and put the whole lot on a three-node cluster. This is the honest write-up of how that went.
The short version: mostly a mistake, but a productive one.
what went wrong
The operational tax is real and it does not care that this is a hobby. A single bad box was a single bad box; now I have etcd quorum to think about, a CNI plugin that needs to actually work, and a control plane that can wedge in ways that are genuinely hard to debug at half past eleven on a Tuesday. The first time the cluster went unreachable I spent an hour discovering that one node's clock had drifted enough to upset certificate validation. On the old setup that service would simply have kept running.
Storage was the other swamp. Stateful workloads at home want persistent volumes, and persistent volumes want a storage backend, and suddenly I'm reading about whether to run a distributed filesystem in my spare room to host a service that three people use. That's the moment you should hear the alarm bell. I did eventually, but later than I'd like.
what went right
Here's the bit that stops it being a clean mistake. I learned a colossal amount, and I learned it on infrastructure where the blast radius was my own irritation rather than a customer's outage. When the same patterns came up at work, the failure modes were already familiar in my hands rather than abstract in my head. A liveness probe that's too aggressive and restarts a slow-starting pod into a crash loop, a resource limit set too low so the OOM killer quietly eats a container, the way a misconfigured nodeSelector leaves a pod Pending forever with no obvious shouting. I've made every one of those mistakes at home, cheaply, and I make them far less often where it counts.
So would I tell someone to do this? It depends entirely on why. If you want reliable services at home with the least faff, do not put Kubernetes between you and your media server. A single host with systemd, or docker-compose if you like containers, will serve you better and wake you up less. If the running of services is the point, optimise for boring.
But if your actual goal is to understand Kubernetes the way you only can by living with it, by being the one paged at midnight, then a home cluster is one of the better teachers I've found, precisely because it's allowed to break. The trick is being honest about which of those two things you're doing. I told myself I was doing the first when I was really doing the second, and the cluster ran better the day I admitted it.