Ramblings of an aging IT geek
← Ramblings of an aging IT geek
homelab

rebuilding a truenas pool one disk at a time

Growing a TrueNAS ZFS pool by replacing every disk in turn with a larger one, the resilver-and-wait rhythm it takes, and the autoexpand step that finally hands you the new space.

A server rack with disks and cabling

My TrueNAS box was full enough that I had started deleting things I might want, which is the homelab equivalent of a warning light. The pool was a mirror vdev of smaller drives, and I wanted more space without buying a whole new chassis. ZFS lets you do this the slow, safe way: replace each disk in the vdev with a bigger one, one at a time, let it resilver after each swap, and once the smallest disk in the vdev has grown, the pool grows too. It is a patient process and it is the right one.

The rhythm is the whole game. You pull one disk, put in the larger replacement, tell ZFS to replace, and then you wait while it resilvers, copying data onto the new drive and verifying it. Crucially you do not touch the second disk until the first resilver is completely finished and the pool reports healthy again. With a mirror, during a resilver you have no redundancy on the affected data, so a failure of the remaining good disk mid-resilver is the one scenario that ruins your week. Patience is not optional, it is the backup plan.

  pool: tank
 state: ONLINE
  scan: resilvered 1.82T in 04:11:23 with 0 errors
config:
        NAME         STATE     READ WRITE CKSUM
        tank         ONLINE       0     0     0
          mirror-0   ONLINE       0     0     0
            ada0     ONLINE       0     0     0
            ada1     ONLINE       0     0     0

Four hours a disk, two disks, so the better part of a day with a long lunch in the middle. The resilver runs in the background and the pool stays online throughout, which is the genuinely lovely part: shares stayed mounted, the family kept watching things, and nobody but me knew the storage underneath was being rebuilt from the ground up.

A homelab server and storage shelf

The bit that trips people up is the end. You finish the second resilver, the pool is healthy, both disks are the larger size, and the capacity has not changed. This is expected and it is not a bug. ZFS will not automatically claim the new space unless you have told it to, and the setting that controls this is autoexpand. If it was off when you started, the pool stays at its old size until you turn it on and nudge it:

zpool set autoexpand=on tank
zpool online -e tank ada0
zpool online -e tank ada1

The online -e is the part that says "expand to fill the device". Do that and the new capacity appears, no reboot, no export, no drama. I had autoexpand off because that is the default and I had never thought about it, so I spent a slightly anxious ten minutes convinced I had done the whole shuffle for nothing before I remembered this exact step exists.

A couple of things worth saying plainly. Scrub the pool before you start, so you begin the operation knowing the data you are about to copy is good, rather than discovering a latent error halfway through a resilver when you have the least redundancy. And do have a backup that is not this pool, because however safe the one-at-a-time dance is, the period of reduced redundancy is real, and "I have a backup" turns a possible disaster into a mild inconvenience. ZFS resilvers are reliable, but reliable is not the same as guaranteed, and a NAS is not a backup of itself.

The whole exercise cost me a day of waiting, two new drives, and roughly four minutes of actual typing. I got a larger pool with no downtime, no data migration, and no new hardware beyond the disks themselves. For a Saturday job that mostly involves watching a progress percentage tick upward, that is a very good trade.