The disk filled up. Again. du said the log directory was 40GB, but ls -la said the current log was 200MB and the rotated ones were tiny. Classic sign: something is still holding a deleted file open.
logrotate was doing its job. It renamed the file, compressed the old one, and sent the app a SIGHUP via postrotate. The trouble is the app cheerfully ignored SIGHUP. It kept writing to the same file descriptor, which now pointed at a renamed (and later deleted) inode. From the app's point of view nothing changed. From the filesystem's point of view, that inode could never be freed because a process still had it open.
lsof | grep deleted told the whole story in one line: the process holding 38GB of phantom log. The space only comes back when the fd is closed, which here meant a restart.
The fix was copytruncate. Instead of renaming, logrotate copies the file and then truncates the original in place, so the app's fd keeps pointing at the same (now empty) inode. You lose a sliver of lines written between the copy and the truncate, which I can live with for a daemon that treats SIGHUP as a polite suggestion. The proper fix is teaching the app to reopen its logs on signal, but that was someone else's codebase and someone else's sprint.