OpenStack-Ansible Machinectl Storage setup

Kevin Carter -

When using the machinctl container storage backend you've got choices on how to setup your host machines. This article will cover the three very common deployment situations.

  1. Simply let systemd back the /var/lib/machines directory using a loop device file which has been formatted BTRFS; this is all done automatically.
  2. Create a logical volume, format it BTRFS, and mount it at /var/lib/machines.
  3. Create a partition or use a physical device, format it BTRFS, and mount it at /var/lib/machines.

All of these options will work and will serve the needs of an OpenStack cloud however they all have their pros and cons and it will be up to you to decide what makes the most sense for the deployment.

Breakdown of the options

Systemd Machinectl Defaults

If you choose to use the built-in storage option, the loop device file, it will indefinitely grow and is systemd managed. The simplicity of being able to get started and not even think about the underlying storage is great, especially from an ease of use standpoint. While this is the easiest option it may not be the best option especially from a long-term supportability point of view. The loop device file is just that, a file, and there are performance penalties associated with this; for example, effectively all transactions will pass through two file-system layers resulting in more resource consumption and less performance. While modern Linux kernels have improved on the performance using things like Direct and Asynchronous IO, there are still performance taxes and they'll get worse over time.

Creating a Logical Volume For Machinectl

The second option and likely the most forgiving is the use of a Logical volume. Creating an LV, formatting it BTRFS, and mounting it at /var/lib/machines provides the ability to resize underlying volume as needed and has very little, if any, performance impact. While resizing a BTRFS partition within an LV is not as straightforward as it is with other filesystems, like EXT4, it is simple enough and covered at length here.

Using a Physical Device or Partition For Machinectl

Creating a physical partition from a regular block device is by far the most performant and resilient option. While this option has the best performance it does require a bit of planning as you'll need to ensure the partition is large enough for foreseeable future. For this option, a partition of at least 250GiB is recommended. The large initial size gives the deployment a nice area to grow over time and lessens the likelihood of undertaking a major maintenance should the partition ever fill up.

Configuring the device mount

  1. If the decision to use the built-in option has been made, there's nothing more to do.

  2. When using a logical volume create the LV and then create or edit the systemd mount file in the following location /etc/systemd/system/var-lib-machines.mount.

[Unit]
Description=Virtual Machine and Container Storage
ConditionPathExists=/dev/vg01/machines

[Mount]
What=/dev/vg01/machines
Where=/var/lib/machines
Type=btrfs
Options=defaults

NOTICE: In this example, the volume group is vg01 and the name of the mounted volume is machines. This is likely different from your deployment.

  1. When using a physical device or partition, simply format the partition (physical device) BTRFS and edit the systemd mount file in the following location /etc/systemd/system/var-lib-machines.mount.
[Unit]
Description=Virtual Machine and Container Storage
ConditionPathExists=/dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX

[Mount]
What=/dev/disk/by-uuid/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
Where=/mnt/flix
Type=btrfs
Options=defaults

NOTICE: In this example, the device UUID is "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" this will be different on your deployment. Use the blkid command to locate the appropriate UUID information.

Starting the mount

If the var-lib-machines.mount file was edited, reload the systemd daemon (systemctl daemon-reload) and restart the mount (systemctl restart var-lib-machines.mount). Once the mount is ready, set the lxc_container_backing_store option to machinectl in your /etc/openstack_deploy/user_variables.yml file and run the OpenStack-Ansible deployment like normal.