Ramblings of an aging IT geek
← Ramblings of an aging IT geek
networking

mtu, the silent killer

The classic MTU fault where small requests work and large ones hang forever, why blocking ICMP breaks Path MTU Discovery, and the one-line ping that diagnoses it.

Network cables in a patch panel

The symptom is always the same and always baffling at first: small things work, big things hang. You can SSH in fine. You can run ls. Then you cat a large file or scp something over and the connection just stops, no error, no reset, it sits there until it times out. DNS is fine. Ping is fine. Pages load until one of them doesn't. It feels like a flaky link, and it's almost never a flaky link. It's MTU.

What's happening is that a packet bigger than some hop along the path can carry has to be either fragmented or rejected, and the mechanism that's meant to negotiate this gracefully, Path MTU Discovery, relies on ICMP "fragmentation needed" messages getting back to the sender. If something on the path drops ICMP (a tunnel, a careless firewall, a VPN, an over-zealous "I block ping because security" rule), those messages never arrive. The sender keeps cheerfully firing full-size packets into a hole, and the connection black-holes the moment any transfer needs a big packet. Small interactive traffic stays under the limit, so it works, which is exactly why it's so confusing.

The diagnosis is one command. Send a ping with the don't-fragment bit set and a payload just under your suspected MTU, then walk it up until it fails:

ping -M do -s 1472 8.8.8.8

1472 bytes of payload plus 28 bytes of headers equals 1500, the standard Ethernet MTU. If that works but 1473 fails with "message too long", you're at a clean 1500. If 1472 already fails, something on the path is smaller, very often 1492 on PPPoE lines or less inside a tunnel, and that's your culprit. Lower the interface MTU to match, or fix the device that's eating ICMP, and the hangs vanish like they were never there. They will, of course, come back the next time someone blocks ping for security.