Ramblings of an aging IT geek
← Ramblings of an aging IT geek
networking

the day the house lost the internet, and it was dns, and it was me

A Boxing Day where nothing in the house could resolve a hostname, traced back to a one-character change I made the night before.

Network cables in a patch panel

It's always DNS. You know it's always DNS, I know it's always DNS, and yet on Boxing Day morning I still spent twenty minutes blaming the router, the ISP, the weather, and the cat before I admitted it was DNS. Specifically, it was me, the night before, editing the forwarders on my Pi-hole and not noticing I'd fat-fingered an upstream address.

The symptom was the usual horror show. Everything "worked" in the sense that pings to raw IPs went through fine, but every browser tab span forever and every app claimed it was offline. The kids could not stream the new things they'd been given, which is the homelab equivalent of a Sev 1.

What threw me is that it wasn't a total failure. Cached entries still resolved, so the things I'd used recently were fine and the things I hadn't were dead. That partial behaviour is exactly what makes DNS faults so good at hiding. A quick dig @192.168.1.2 example.com showed the query timing out against my own resolver, and from there it was obvious.

Fixed in ten seconds once I actually looked. I've now put the Pi-hole config in git so the next time I do something daft I can at least git diff my way back to a working house. Lesson relearned, again: when the network is half-broken on a holiday, check the thing you touched last.