Ramblings of an aging IT geek
← Ramblings of an aging IT geek
tooling

the ssh dropped, and for once i didn't care

A short note on why tmux on the remote host, not the laptop, is the thing that stops a dropped SSH connection from taking a long-running job with it.

A terminal split into several panes over a keyboard

For years my relationship with long-running remote commands was pure superstition. Start a migration over SSH, then sit very still, lid open, hoping the train didn't go through a tunnel and kill the connection halfway through. When it inevitably did, I would reconnect to find the job dead and the database in some half-applied state I now had to reason about. Entirely self-inflicted.

The fix is tmux, and the bit that took me far too long to internalise is where it runs. You start tmux on the remote host, inside your SSH session, not on the laptop. The process you care about lives inside that remote tmux. So when your connection drops, the session detaches and keeps running on the server, blissfully unaware that you have wandered off. You reconnect, run tmux attach, and you are exactly where you were, mid-output, job still going.

The whole loop is three commands:

ssh server
tmux new -s migrate   # do your work in here
# ... connection dies, or you just close the lid ...
ssh server
tmux attach -t migrate

That is genuinely most of the value, on day one, before you learn a single other thing about it. The panes and the windows and the scriptable layouts are lovely and I use them constantly now, but they are gravy. The meat is that a flaky connection stopped being able to kill a job, because the job no longer lived on the flaky side of the link. I should have done this a decade ago.