Ramblings of an aging IT geek
← Ramblings of an aging IT geek
linux

the service that wouldn't stay stopped

A systemd unit kept coming back to life after stop, and the culprit was a socket activating it on the next connection.

A terminal on a Linux machine

I stopped a service and it started again. Not slowly, not after a reboot, immediately. systemctl stop foo returned cleanly, systemctl status foo showed it active and running, with an uptime of two seconds. I did it again, watching the status, and the same thing happened. The process I'd just killed had a brand new PID.

My first thought was Restart=always in the unit, the classic. It wasn't there. RemainAfterExit, StartLimit nonsense, a stray override in /etc/systemd/system/foo.service.d/, I checked them all and found nothing. The unit looked entirely ordinary.

The answer was that I'd been looking at the wrong unit. There was a foo.socket alongside foo.service, and the socket was enabled. Socket activation works exactly as advertised: systemd holds the listening port, and the moment a connection arrives it starts the service to handle it. My monitoring was politely health-checking the port every few seconds, so each time I stopped the service the next probe woke it straight back up.

systemctl stop foo.socket foo.service

Stop the socket too, or it'll keep resurrecting the thing the instant anyone knocks on the door. Obvious in hindsight, like most of these. If a unit refuses to stay dead and there's no Restart= to blame, run systemctl list-units '*.socket' and see what's keeping the lights on.