Ramblings of an aging IT geek
← Ramblings of an aging IT geek
networking

qos for a household that streams everything

Setting up sane traffic shaping with CAKE so that a 4K stream and a big upload stop fighting over the same connection at home.

A bundle of network cables

The complaint that started this was simple: every time someone uploaded a video or a backup kicked off, the telly buffered. Classic bufferbloat, the thing where a saturated link fills its buffers and latency goes through the floor for everyone, even the people who only wanted to watch a film. I'd known about it for years and ignored it, because ignoring it was free. Then I started working from home properly, and a call dropping mid-sentence because a phone was syncing photos stopped being funny.

So I sat down and did QoS properly, which in 2025 mostly means CAKE and stopping there.

why CAKE rather than the old approach

For a long time the answer to bufferbloat was HTB plus fq_codel and a lot of fiddling with classes. It works, but you have to think about it, and you have to keep thinking about it. CAKE folds the whole lot into one qdisc: it does the shaping, the fair queueing, and the flow isolation, and it has sensible defaults for almost everything. The only number it really needs from you is your actual line rate, set slightly below what the ISP gives you so the bottleneck lives in your box, not theirs. That last part is the entire game. If the buffer that fills up is one you control, you can manage it. If it's in the ISP's kit, you can't.

On my connection that means setting the shaper a touch under the measured throughput in each direction:

tc qdisc add dev wan root cake bandwidth 220mbit \
    diffserv4 nat dual-srchost
tc qdisc add dev ifb-wan root cake bandwidth 38mbit \
    diffserv4 nat dual-dsthost

The download side runs on an IFB device because you can only really shape egress, so ingress gets redirected through a virtual interface and shaped there. nat tells CAKE to look through the NAT so per-host fairness actually works on the LAN addresses rather than seeing everything as one big flow from the router. dual-srchost and dual-dsthost give you fairness both between hosts and between the flows within a host, which is what you want when one machine has thirty connections open and another has two.

A rack of networking equipment in a small datacentre

the part where you resist the urge to over-classify

The temptation, once you have a shaper, is to start sorting traffic into elaborate priority tiers. Video here, gaming there, this gets the fast lane, that gets throttled. I tried it. It was a waste of an evening. The thing about fair queueing is that it already does the job you were trying to do by hand: a latency-sensitive flow with a few small packets naturally gets served promptly, because it isn't sitting in a queue behind a fat bulk transfer. You don't need to tell CAKE that a video call matters. It works it out from the traffic shape.

The diffserv4 tier in the config above does light touch prioritisation based on DSCP marks, and that's as far as I'd go. If something on your network marks its packets honestly, CAKE will respect it. Most things don't bother, and they're fine in the default tin anyway. The one place I made a deliberate choice was to keep the tiers to four rather than the eight-way diffserv8, because four is plenty for a house and I never want to debug which of eight buckets a stray packet landed in.

did it actually work

Yes, and embarrassingly well for how little config it is. The proof is in the latency under load. Before, saturating the uplink pushed ping times from about 12ms to well over 300ms, which is why calls fell apart. After, a flat-out upload barely moves it, a handful of milliseconds at most. I ran flent against a known server to get the before-and-after, and the difference is the kind you don't need a graph to feel, although the graph is satisfying too.

# rough flent test, runs a bulk transfer and pings throughout
flent rrul -p ping_cdf -H netperf.example.org -t before
flent rrul -p ping_cdf -H netperf.example.org -t after

The streaming complaints have stopped, which is the only metric anyone in the house actually cares about. Nobody has thanked me, because nothing visibly happened, and a thing working quietly is the highest form of praise a home network gets. I'll take it.

The whole setup is maybe a dozen lines, survives a reboot via a oneshot service that sets it up after the WAN comes up, and I've not touched it since. That's the right amount of QoS for a household: enough to fix the actual problem, not so much that it becomes a hobby in its own right.