Nicholas Schmidt

Nicholas Schmidt

CTO, Dad, Husband, and lover of all things 1's and 0's.

Migrating a ZFS Pool with Snapshots

2 minute read

Background

ZFS is one of those technologies that once you are introduced to it, it is challenging to think in the old ways again. One of the features I was always hesitant to embrace was snapshots as a mechanism for backup and recovery. I know that sounds a bit nutty given how powerful they can be, but my old tooling is muscle memory at this time. Bacula, rsync, and tar has been in my mind since I was a child.

This year I finally had to break down and replace two older servers in my basement data centre due to old age and a bathroom water accident the room above. Note to self: Don’t install server rack below a bathroom.

Christmas came early, and I was happy. Santa brought me a Dell r720xd with 14 available drive bays. This allowed me to easily migrate two servers into this new one while also adding a dedicated backup pool!

The new server is running Ubuntu instead of my usual Arch setup due to the fact the Dell server is certified for this distro. Not that I am an “only run certified solutions” type of person, but given I am mixing and matching lots of HCL I wanted to play it safe.

Thanks Sanata!

Old Workflow

Normally I would:

  1. zpool export from old server

  2. zpool create on new server

  3. zpool import $pool on new server

  4. rsync -avhP oldpool newpool

I trust rsync. I have done crazy transfers using rsync and have never experienced data loss from the tool itself. The downside though is that it is unbearably slow even on local syncs of directories.

My pools are 15+ TB a piece, so the rsync dance takes forever. Did I mention the one server that my family uses every day is currently running on a ten-year-old thumb drive? Hell hath no fury like a little girl who can’t watch a Frozen DVD rip before her naptime :)

New Workflow

  1. zpool export from old server

  2. zpool create on new server

  3. zfs snap oldpool@migrate

  4. zfs send -R oldpool@migrate | zfs recv -F newpool

  5. Verify that the snapshot is on the new pool with zfs list -t snapshot

  6. Promote the snapshot to become the newpool’s data with zfs rollback newpool@migrate

Lessons Learned

The time it took to send an 8TB snapshot was about ten times faster. This is non-scientific benchmarking, but also matches findings from Jim Salter @ arstechnica.

I am not sure I will be able to break my rsync habit, but for this type of migration, it definitely will. There is some excellent tooling for incremental and network-based operations that I will be looking into, such as sanoid.

ZFS without the nifty snapshot and send/recv is a great tool I have been using for over a decade when it was introduced on OpenSolaris. Also…holy crap over a decade?!?!

Updated: