Ramblings of an aging IT geek
← Ramblings of an aging IT geek
networking

small packets fine, large packets gone: an mtu story

A homelab link where SSH worked but file copies hung, all because of a 50-byte MTU mismatch nobody had noticed.

Network cables in a patch panel

The classic MTU symptom is the one that makes you doubt your own eyes: small things work perfectly and large things hang forever. I could SSH into the NAS, run commands, list directories, all instant. Then I'd start an actual file copy and it would transfer a few kilobytes and freeze, no error, no reset, just a dead connection that eventually timed out.

When tiny packets sail through and big ones vanish, it's almost always a path that can't carry the frame size one end is trying to send. SSH handshakes and ls output are small. A bulk copy fills packets to the maximum, those hit a link with a lower MTU somewhere in the middle, and if Path MTU Discovery can't do its job because some box is dropping ICMP "fragmentation needed", the big packets just disappear into the void with nobody told why.

The culprit was a VLAN I'd set up on the new switch that quietly carried a slightly smaller usable MTU than the 1500 the hosts assumed. You can prove it in one command with a do-not-fragment ping and a payload that doesn't leave room for the overhead:

ping -M do -s 1472 nas.lan

1472 bytes of payload plus 28 bytes of headers is exactly 1500. If that succeeds and -s 1473 fails with "message too long" or just silently drops, you've found your wall, and you walk the payload size down until packets get through to learn the real MTU of the path. I either pin the interface MTU to match or, better, fix whatever was eating the ICMP so PMTUD can sort itself out. Either way it stops being a silent killer the moment you can see it.