Ramblings of an aging IT geek
← Ramblings of an aging IT geek
networking

speaking bgp to my own cupboard

Running BGP inside the homelab with FRR and a private ASN to advertise service routes, mostly to learn it properly but with a real payoff.

Network cables patched into a switch

I now run BGP in my house. Not because anything needed it, exactly, but because every time I touched BGP at work I was borrowing knowledge I didn't really own, and the fastest way to own a protocol is to break it somewhere that only inconveniences you. So: a private ASN, a couple of speakers, and route advertisement between the things in the cupboard. It turned out to be far more useful than "why not" deserves.

Let me explain what problem it actually solves, because "I added BGP to my homelab" sounds like the kind of thing you do to avoid talking to people.

The actual problem

I run services across a few hosts, and some of those services want to be reachable on a stable virtual IP regardless of which host is currently serving them. The naive approach is a static route on the router pointing at a host, but then the host becomes a single point of failure and failover means me, editing config, at the worst possible time. Keepalived with VRRP handles the simple two-node case nicely, and I used it for years. But once you have several services that want to move independently, and a load balancer like MetalLB in the Kubernetes corner that already speaks BGP natively, you start wishing the network just knew where each service IP lived and updated itself when that changed.

That is precisely what BGP does. A host advertises "I can reach 10.80.0.5/32", the router believes it, traffic flows. The host goes away, the advertisement is withdrawn, the route disappears, traffic fails over to whoever else is advertising it. No static config, no manual intervention, no me at 2am.

Patch panel and routing diagram on the wall

The setup

I used FRR (the FRRouting suite) on the hosts, talking to the router. Private ASNs live in the 64512–65534 range, which is the equivalent of 192.168 for autonomous systems: yours to use, never seen on the public internet. I gave the router 65000 and the hosts their own numbers.

A trimmed FRR config on a host looks roughly like this:

router bgp 65010
 bgp router-id 10.0.0.10
 neighbor 10.0.0.1 remote-as 65000
 !
 address-family ipv4 unicast
  network 10.80.0.5/32
  neighbor 10.0.0.1 activate
 exit-address-family

The router side has a matching neighbour statement back, and crucially I set it up so the router only accepts the specific /32s I expect, not whatever a host fancies announcing. That last part matters more than the rest combined.

You bring it up, and vtysh -c "show ip bgp summary" tells you whether the session is Established or sulking in Active/Connect. Most of my early mistakes showed up right there: a firewall eating TCP 179, a mismatched remote-as, a router-id collision. BGP is unglamorous to debug precisely because it tells you exactly what's wrong if you ask it.

The footgun, and the guardrail

BGP will believe what it's told. If a host advertises a route it shouldn't, the router will happily install it and blackhole traffic. In a real network this is how you take down a continent; in a homelab it's how you wonder why DNS broke at teatime. The guardrail is prefix filtering, both ways.

ip prefix-list LAB-SERVICES seq 5 permit 10.80.0.0/24 le 32
route-map IMPORT-LAB permit 10
 match ip address prefix-list LAB-SERVICES

Apply that as an inbound filter on the router's neighbour and it will only accept advertisements from the service range. A host that goes rogue, or a bit of MetalLB misconfiguration, can no longer announce a default route and swallow the lot. I learned to do this after a test announcement of 0.0.0.0/0 made the whole house briefly route through a Raspberry Pi. The Pi did its best. It was not enough.

Was it worth it?

Yes, surprisingly. Service failover is now genuinely hands-off: a host reboots for kernel updates and its service IPs glide to another speaker and back without me noticing. The Kubernetes load balancer slots into the same scheme instead of being a special case. And the work knowledge that I used to borrow is now mine, paid for in a couple of evenings of self-inflicted outages in a place where the only user affected was me.

If you already run a homelab past the single-box stage, and you find yourself reaching for ever more elaborate static routes and floating IPs, this is the point where a real routing protocol stops being overkill and starts being the simple option. Start small: one host, one /32, one session. Get it Established. Add the prefix filter before you add the second speaker, not after. And keep a console route to the router that does not depend on any of it, because the day you fat-finger a route-map you will want a way back in that BGP can't take away from you.