Ramblings of an aging IT geek
← Ramblings of an aging IT geek
performance

what a syscall actually costs, with numbers

I finally measured the per-call overhead of a trivial syscall on my own box instead of repeating numbers from a blog post, and the result reframed a hot loop.

A line graph on a monitor next to a rack of servers

I've been repeating the phrase "syscalls are expensive" for years without ever having measured one myself, which is a bit embarrassing for someone who's quick to ask others for numbers. So I measured. A tight loop calling getpid() a hundred million times, which is about the cheapest real syscall there is because it barely does anything kernel-side, on an otherwise idle Linux box.

It came out at roughly a couple of hundred nanoseconds per call. That figure includes the mode switch into the kernel and back, and not much else, because getpid itself is trivial. A normal function call on the same box is single-digit nanoseconds. So the syscall boundary is something like one to two orders of magnitude more expensive than a plain call, before the kernel does any actual work.

for (long i = 0; i < 100000000; i++) syscall(SYS_getpid);

Note I used syscall() directly rather than the libc wrapper, because glibc caches the PID and would have optimised the whole thing away, handing me a meaningless near-zero. That subtlety cost me a confused ten minutes before I realised my "syscall benchmark" wasn't making any syscalls.

A couple of hundred nanoseconds sounds like nothing, and per call it is. But it reframes a hot loop instantly. If you're doing a syscall per item across a million items, that's a fifth of a second of pure overhead, and it's why batching, buffering, and reaching for things like sendmmsg exist at all. Worth measuring on your own hardware rather than trusting mine, because the number moves with the CPU and, these days, with whatever mitigations are switched on.