Ramblings of an aging IT geek
← Ramblings of an aging IT geek
linux

the rollback that turned a disaster into a five minute job

How a btrfs snapshot taken minutes before a botched upgrade let me roll back instead of spending the weekend rebuilding.

A terminal glowing in a dark room

I ran a system upgrade on Friday evening, which was my first mistake, and it left the machine unbootable, which was not my fault but was very much my problem. The thing that turned a ruined weekend into a five minute job was a btrfs snapshot I had taken minutes earlier almost out of habit.

The setup is unremarkable. Root is on btrfs, and snapper takes a snapshot before and after every package transaction. I never normally look at them. They just accumulate quietly, costing almost nothing because btrfs snapshots are copy-on-write, so an unchanged file is shared, not copied.

When the boot dropped me into a recovery shell, I listed the snapshots, found the "pre" one from just before the upgrade, and rolled back:

snapper list
snapper undochange 142..0

Reboot, and I was back to Friday afternoon as if the upgrade had never happened. No reinstall, no restore from backup, no hunting for which library got clobbered. The broken state is still sitting there as a snapshot if I ever want to do the forensics, which I will not, because it is the weekend.

Snapshots are not backups; they live on the same disk and a dead disk takes them with it. But for the specific case of "I changed something and now the machine is unhappy", a filesystem that can travel back in time is the difference between an inconvenience and a write-off. I have been smug about btrfs to colleagues for years on theory. Now I have the receipt.