Expand ZFS in Linux VM

By Paul Heinlein | May 16, 2017

Ubuntu 16.04 has native support for ZFS, which means that VMs may start to use ZFS for non-root filesystems. Here’s a cookbook for expanding those filesystems. In OpenStack, the ZFS filesystem must be exported before this can be done, but at AWS it can be done without downtime.

AWS

Expanding ZFS in an AWS EC2 instance requires no service interruption.

  1. Add an EBS volume to the target VM. It will show up as, e.g., /dev/xvdf (if the root volume is /dev/xvda).
  2. Install ZFS libraries and utilities.
# or use aptitude, but it's not always available
apt-get install zfs
  1. Create a zpool and zfs filesystem.
# these utilities get installed with the zfs packages.
zpool create -o autoexpand=on zp0 /dev/xvdf
zfs create -o mountpoint=/data zp0/data

Worth noting at this point is that the ZFS on Linux stuff deals with whole-disk virtual volume by initially creating two partitions: partition number 1 contains the filesystem, partition number 9 is a buffering partition.

Time passes. Work goes on until the filesystem needs expansion.

  1. Expand the EBS volume, using the AWS web GUI or the command-line tools. This can take a while.
  2. Expand the ZFS partition, first removing the buffering partition.
# after this command, type ‘Fix’ at prompt to take advantage
# of new space.
parted -l
# remove the buffering partition
parted /dev/xvdf rm 9
# expand partition holding zpool
parted /dev/xvdf resizepart 1 100%
  1. Expand the zpool (and, by extension, the filesystem it contains).
zpool online -e zp0 /dev/xvdf

At that point, the buffering partition is gone. After the next expansion, the resizing can be done without removing any partitions.

OpenStack

Unlike the completely live procedure at Amazon, the OpenStack procedure requires the ZFS filesystem be exported and the virtual drive detached from the VM before the expansion can happen.

  1. Add a cinder volume to the target VM. It will show up as, e.g., /dev/vdb (if the root volume is /dev/vda).
  2. Install zfs virtual package.
# use aptitude if available. :-)
apt-get install zfs
  1. Create a zpool and zfs filesystem:
zpool create -o autoexpand=on zp0 /dev/vdb
zfs create -o mountpoint=/data zp0/data

See AWS note about ZFS creation of two partitions, 1 and 9.

Work goes on until expansion is necessary. Then,

  1. Export the filesystem.
# this is done from within the VM
zpool export zp0
  1. Detach the cinder volume from the VM, grow it, and reattach it.
# this is done from a machine with openstack CLI tools
#
# detach volume
openstack server remove volume $VM_GUID $VOL_GUID
# resize: new size specified in GB
openstack volume set $VOL_GUID --size 16
# attach volume
openstack server add volume $VM_GUID $VOL_GUID
  1. Resize the ZFS partition
# this is done from within the VM
#
# after this command, type ‘Fix’ at prompt to take advantage
# of new space.
parted -l
# remove the buffering partition
parted /dev/vdb rm 9
# expand partition holding zpool
parted /dev/vdb resizepart 1 100%
  1. Import and expand the zpool
# on import, it will show the pre-expansion size
zpool import zp0
# using "online -e" will expand it.
zpool online -e zp0 /dev/vdb

Subsequent expansions won’t have to worry about the buffer partition, which is now gone.

Aws  Howto  Linux