Growing BTRFS in a Logical Volume

As you may know, I run quite a few nspawn containers which are responsible for running my entire home network, entertainment system, home automation, and a few other applications here & there. What you may not be aware of is the fact that I'm running everything on top of BTRFS and systemd-nspawn. Systemd-nspawn expects a volume mounted at /var/lib/machines using BTRFS. While you can run nspawn without BTRFS, doing so would mean missing out on features pre-packaged into systemd-nspawn. While BTRFS, nspawn, and systemd can do amazing things, the server and services still require periodic maintenance and today was one such day.

Overview

I received an alert today that one of my container hosts was filling up; it alerted notifying me that I was under a certain threshold of available storage. After logging into the node and ensuring everything was on the up-and-up I decided to grow the BTRFS partition by ~50GiB giving the server more room to breathe while also fulfilling some nerdy curiosity.

Here's what BTRFS was reporting when I logged into the box.

# btrfs filesystem df -h /var/lib/machines/
Data, single: total=50.00GiB, used=38.52GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=776.00MiB, used=355.91MiB
GlobalReserve, single: total=128.00MiB, used=0.00B

# btrfs filesystem show  /var/lib/machines/
Label: none  uuid: ffa12758-857f-4416-957d-4cfd71f2e42a
        Total devices 1 FS bytes used 38.87GiB
        devid    1 size 50.00GiB used 39.78GiB path /dev/mapper/vg--machines

The container file system /var/lib/machines was by no means full but it was getting close and I didn't want it full. So I decided to extend the logical volume housing my BTRFS partition and grow the underlying file system.

BTRFS in a Logical Volume

As mentioned before, I use logical volumes, like millions of other folks running linux servers, and while some may question the value of LVM and BTRFS, there's no right answer and I find myself in the camp of folks still believing in the power and flexibility of LVM, that is until qgroups are fully flushed out and I have some time to build a bit more trust with them.

A quick overview of the volume I'll be extending.

# lvs
  LV       VG                 Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  machines vg -wi-ao---- 50.00g
[...]

To extend my volume, as a traditional Linux system administrator, I would simply run lvresize -r -L+50G /dev/vg/machines, drink a beer, and be done, however, being BTRFS is in use things are a little different. If you happen to run this command on a logical volume that is also formatted BTRFS the lvresize command will fail when it detects the BTRFS volume (I did this).

fsadm: Filesystem "btrfs" on device "/dev/mapper/vg--machines" is not supported by this tool
  fsadm failed: 1

While the resize command failed, the logical volume detail shows the command partially worked by adding the extents to the volume. This means I need to use the btrfs command to extend the volume instead of just assuming lvresize will do it for me.

# lvdisplay /dev/vg/machines
  --- Logical volume ---
  LV Path                /dev/vg/machines
  LV Name                machines
  VG Name                vg
  LV UUID                ucVj74-t7TN-HV8H-pokI-BYkW-grBY-4ryTn7
  LV Write Access        read/write
  LV Creation host, time carterhouse, 2017-04-23 11:30:31 -0500
  LV Status              available
  # open                 1
  LV Size                100.00 GiB
  Current LE             25600
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:2

In the future, when I need to do this elsewhere, the correct command for extending the volume is to run lvresize as I did before but to omit the -r flag.

BTRFS, a balancing & extension act

The lvresize command grew the logical volume but it didn't resize the underlying file system. This means the BTRFS partition is still only 50GiB even though the surface area of the partition is now larger.

This next step is Not Required but will make sure BTRFS is fully aware of where everything is supposed to exist. To ensure BTRFS is happy, I'm going to balance the file system which will redistribute data and metadata across all available devices, in my case this a logical volume. Normally, this would be used when adding a device to an existing BTRFS volume group or any time we want to ensure there's an equal spread of blocks across the disks but in this instance I'm running the balance command for piece of mind.

Using the btrfs command, balance the file system. While this will work online, the command may take a while, and tax the CPU, be patient.

btrfs balance start -v /var/lib/machines

Once the balancing act has completed, which can take some time, the command will output what was done.

Dumping filters: flags 0x7, state 0x0, force is off
  DATA (flags 0x0): balancing
  METADATA (flags 0x0): balancing
  SYSTEM (flags 0x0): balancing
Done, had to relocate 12 out of 12 chunks

Assuming everything is happy and healthy, resize the partition to the max allowed.

btrfs filesystem resize max /var/lib/machines

Verify that the BTRFS volume is as big as it's supposed to be

# btrfs filesystem show  /var/lib/machines/
Label: none  uuid: ffa12758-857f-4416-957d-4cfd71f2e42a
        Total devices 1 FS bytes used 38.86GiB
        devid    1 size 100.00GiB used 39.78GiB path /dev/mapper/vg--machines

Finally, check the consumption size

# btrfs filesystem df -h /var/lib/machines/
Data, single: total=100.00GiB, used=38.52GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=776.00MiB, used=355.91MiB
GlobalReserve, single: total=128.00MiB, used=0.00B

Log out and time to go get that beer.

That's all folks

With those couple of commands, my logical volume was extended, my file system was balanced, curated, and I've grown the home for my containers. All of which was done while running local workloads without any service interruptions. That was easy!