I built a four-node Raspberry Pi cluster. It runs Kubernetes. It serves a single static website and a Pi-hole, both of which would be perfectly happy on one Pi, or honestly on a ten-year-old laptop in a cupboard. By any practical measure the cluster was a waste of money and a weekend. I enjoyed it enormously and I'd recommend it to anyone.
Let me be clear about the value proposition up front, because I don't want to oversell this. You do not need a Kubernetes cluster at home. You almost certainly don't need Kubernetes at work either, but that's a different post and a more expensive argument. What a Pi cluster gives you is a real, physical, slightly-too-slow distributed system that you own end to end, and that you can break without anyone paging you.
the hardware
Four Raspberry Pi 4s, the 4GB model, in a stacked acrylic case with little heatsinks and a fan that sounds like an irritated wasp. A cheap 5-port gigabit switch. A USB power supply with enough amps to actually feed four Pis under load, which the first one I bought emphatically did not, leading to an afternoon of mysterious reboots that I initially blamed on the software. It is always the power supply. It is genuinely always the power supply.
The one piece of advice I'd pass on is boot from SSD over USB, not from microSD. The cards are slow and they wear out, and the whole experience is transformed once etcd and the container layers aren't fighting over a sluggish card. A cheap USB-to-SATA adapter and four small SSDs cost more than the romance of "it boots off an SD card" is worth.
the software
I went with k3s, which is Rancher's lightweight Kubernetes distribution and exactly the right choice for this. Full upstream Kubernetes on a Pi is possible and miserable. k3s strips out the cloud provider bits you don't need, bundles a sensible default ingress and storage, and installs with a one-liner. The control plane node:
curl -sfL https://get.k3s.io | sh -
sudo cat /var/lib/rancher/k3s/server/node-token
Then each worker joins with that token and the server's address:
curl -sfL https://get.k3s.io | K3S_URL=https://192.168.1.10:6443 \
K3S_TOKEN=<token> sh -
Within a few minutes kubectl get nodes shows four nodes Ready, and you have a real cluster. The first time all four went green I felt the same daft delight as getting an LED to blink.
what it actually taught me
Here's the honest accounting. It taught me nothing about running Kubernetes at scale, because four underpowered nodes on a home network is not scale, and the failure modes are different. It didn't make me better at the day job in any direct, demonstrable way.
What it did teach me was smaller and more useful than that. I now understand, in my hands rather than in a diagram, what happens when a node drops off the network mid-deployment. I unplugged one to see. The pods rescheduled, eventually, after the node controller decided it was really gone rather than just slow, and watching that timeout play out on hardware I could touch made the eviction timers in the docs mean something. I learned that ARM container images are a real consideration and that "it works on my laptop" hides a multiple-architecture problem until the day it doesn't. I learned, again, that the power supply is the problem.
And I learned that the boring infrastructure I take for granted at work, the load balancer that just works, the storage that's just there, is enormous amounts of someone else's careful engineering. Doing it badly at home, by hand, on four little computers, is the best appreciation course for it I've found.
So: nothing useful, in the sense that I could have spent the money better and the website would render identically. Everything useful, in the sense that I poke a kubectl command now and have a physical picture of what's actually moving. The wasp-fan is still going in the corner of the room. I find I rather like it.