The symptom was the kind that wastes a whole afternoon: it worked, except when it didn't. A service behind a new WireGuard tunnel was up. ping was fine. curl against a small health endpoint was fine, instant, every time. But fetching anything substantial, a JSON payload of any real size, a page with a body, just hung. No error. No reset. It sat there until the client gave up.
When small things work and large things hang, you should already be thinking about MTU. I was not, because I was thinking about firewalls and TLS and DNS and all the more interesting suspects first, which is exactly how MTU gets you. It is never the first thing you check and it is always the thing it was.
The give-away, once I stopped guessing and started measuring, was this:
$ ping -M do -s 1472 server
PING server: 1472 data bytes
ping: local error: message too long, mtu=1420
The link could not carry a full 1500-byte frame. WireGuard adds its own overhead, so the usable MTU inside the tunnel is smaller, around 1420 here. That on its own is fine, because this is exactly what path MTU discovery exists to handle: the router that cannot forward the oversized packet sends back an ICMP "fragmentation needed" message, and the sender shrinks its packets. The mechanism is decades old and works beautifully, right up until somebody blocks ICMP.
And of course somebody had. A well-meaning firewall rule, on a box I did not own, dropping ICMP wholesale because "ICMP is a security risk". So the large packets left, hit the smaller-MTU link, needed to be told to shrink, and the message telling them to shrink was silently binned. The sender never learned. It just kept retransmitting full-size packets into a link that would not carry them, forever. A black hole, and a textbook one. The small requests fit in a single sub-MTU packet, which is why they sailed through and made the whole thing look intermittent rather than broken.
The proper fix is to stop dropping ICMP type 3, which is not optional plumbing, it is how IP is supposed to work. That rule went, and the tunnel behaved. But the box was not mine to change quickly, so the immediate fix lived on my side: clamp the MSS so TCP never tries to send segments too big for the path in the first place.
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
-j TCPMSS --clamp-mss-to-pmtu
That tells TCP, at the handshake, to negotiate a segment size that fits, so it never relies on the ICMP message that was being eaten. Within seconds, large responses flowed.
So, the recurring lesson, written down again so I believe it next time: when small works and large hangs, suspect MTU before anything clever. And whoever is reflexively dropping all ICMP because it sounds dangerous, please stop. You are not hardening anything. You are just turning a self-correcting network into one that hangs in ways nobody can explain for an entire afternoon. It was the MTU. It is always the MTU.