Ramblings of an aging IT geek
← Ramblings of an aging IT geek
linux

the systemd unit that would not stay dead

A service I stopped kept coming back, and the answer was Restart=always doing exactly what I told it to.

A Linux terminal at a prompt

I needed a service stopped so I could swap its config out from under it. systemctl stop reported success. A few seconds later the process was back, same name, new PID. I stopped it again. It came back again. For a moment I genuinely wondered if I had a rootkit, which is the kind of thought that means you should stop and read the unit file.

The unit file had Restart=always. That is the answer to the whole mystery. When I stopped the service, systemd killed it, the process exited, and systemd looked at its policy, saw "always restart", and faithfully started it again. I was not fighting malware. I was fighting a config line I had written myself, working precisely as designed.

The distinction that matters here is between stopping a service and telling systemd to leave it alone. systemctl stop ends the current process. Restart=always is a standing instruction to bring it back. If you actually want the thing to stay down, you stop it and you mask it, or you accept that the supervisor is going to do its job.

systemctl stop myservice
systemctl mask myservice   # now it stays down

mask points the unit at /dev/null so nothing, not a restart policy, not a dependency, not a stray start, can bring it up until you unmask it. That is what I wanted all along: not "stop", but "stay stopped". Once the config swap was done I unmasked it, let Restart=always do its proper job again, and went to find a cup of tea, slightly embarrassed and slightly relieved.