Cloudy NixOS (OpenStack)
Recently, I decided to go down the journey of running things with NixOS. While I started with my laptop, I naturally decided to make my workspace problems part of my cloud adventure. So I figured I'd share my adventure along with my initial process for getting NixOS into the cloud. In this post, I've provided some of the commands I ran and a link to the repository I'm using to setup my OpenStack images on a Genestack cloud.
Why?
There were a couple of places on the internet that highlighted how to generate NixOS images using a generator via nixpkg. While OpenStack was an option, the generator didn't result in an image I could use as a general-purpose, standalone base image and didn't seem to incorporate many of the "features" I've come to expect in the cloud. So, to solve my cloudy wants with my new NixOS addiction, I set off to do a thing.
Image Requirements
After poking around, I decided I needed my NixOS-OpenStack to have the following attributes.
- UEFI boot
- Q35 machine type (compatible)
- QEMU aware
- Virtio support
- Latest stable NixOS 23.11
- Latest Kernel Image (6.8+)
- Cloud-Init enabled
- Standalone system
The Repository
To do what I needed, I put together a repository, which I've iterated on to craft the image I feel is well suited for the cloud and fits my immediate needs.
Git Repository for my OpenStack NixOS image
flakes and nix-command on your system.Running the build within the Repository
To get started, clone the repository locally.
git clone https://github.com/cloudnull/nixos-openstackGit clone for the flake repo
Create an OpenStack-compatible image for NixOS
nix build .#nixosConfigurations.build-qcow2.config.system.build.qcow2Nix build command resulting in a QCOW2
Running the build with a remote repository
There's no need to clone the above repo to run the build; the following command can be used to run the build.
nix build git+https://github.com/cloudnull/nixos-openstack#nixosConfigurations.build-qcow2.config.system.build.qcow2Verifying the Image
After the build, you will have a new qcow2 file in the results directory.
ll result/
total 630988
-r--r--r-- 1 root root 649986048 Dec 31 1969 nixos.qcow2
dr-xr-xr-x 2 root root 4096 Dec 31 1969 nix-supportImage result example
Upload the image and store the result in Glance
openstack --os-cloud default image create \
--progress \
--disk-format qcow2 \
--container-format bare \
--public \
--file result/nixos.qcow2 \
--property hw_scsi_model=virtio-scsi \
--property hw_disk_bus=scsi \
--property hw_vif_multiqueue_enabled=true \
--property hw_qemu_guest_agent=yes \
--property hypervisor_type=kvm \
--property img_config_drive=optional \
--property hw_machine_type=q35 \
--property hw_firmware_type=uefi \
--property os_require_quiesce=yes \
--property os_type=linux \
--property os_admin_user=nixos \
--property os_distro=nixos \
--property os_version=23.11 \
nixos-23.11Glance image upload command
The image output will look similar to this.
+——————+-————————————————————————————————————————————————————————————————————————+
| Field | Value |
+——————+-————————————————————————————————————————————————————————————————————————+
| container_format | bare |
| created_at | 2024-04-06T18:31:55Z |
| disk_format | qcow2 |
| file | /v2/images/f73588c5-468e-4678-9974-33300dce7c39/file |
| id | f73588c5-468e-4678-9974-33300dce7c39 |
| min_disk | 0 |
| min_ram | 0 |
| name | nixos-23.11 |
| owner | 4d04429679c44b9ab3cafd523b9f86fd |
| properties | hw_disk_bus='scsi', hw_firmware_type='uefi', hw_machine_type='q35', hw_qemu_guest_agent='yes', hw_scsi_model='virtio-scsi', hw_vif_multiqueue_enabled='True', hypervisor_type='kvm', img_config_drive='optional', |
| | os_admin_user='nixos', os_distro='nixos', os_hidden='False', os_require_quiesce='True', os_type='linux', os_version='23.11', owner_specified.openstack.md5='', owner_specified.openstack.object='images/nixos-23.11', |
| | owner_specified.openstack.sha256='' |
| protected | False |
| schema | /v2/schemas/image |
| status | queued |
| tags | |
| updated_at | 2024-04-06T18:31:55Z |
| visibility | public |
+------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+Glance Image output
The image we created with all the flags will enable the OpenStack features I need for the cloud: EFI boot, virtio, newer machine type, multi-queue networking, and image quiescing. This is made better by cloud-init, which will handle the systemd-networkd setup, hostnames, host keys, default users, SSH keys, etc. From my Nix noob point of view, the setup is allowing me to succeed in the cloud with NixOS.
Boot your server
Creating the instance is simple and follows the standard OpenStack server create process.
openstack --os-cloud default server create \
--image nixos-23.11 \
--nic net-id=flat \
--flavor m1.small \
--key-name controller-0 \
nixos-23.11OpenStack Server create command example
The server creation will look similar to the following:
+-————————————+-———————————————————+
| Field | Value |
+-------------------------------------+----------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | compute-0.cloud.local |
| OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0.cloud.local |
| OS-EXT-SRV-ATTR:instance_name | instance-0000004d |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | active |
| OS-SRV-USG:launched_at | 2024-04-06T18:32:18.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | flat=172.16.25.163 |
| adminPass | ************ |
| config_drive | |
| created | 2024-04-06T18:32:09Z |
| flavor | m1.small (4ef01fb8-6afa-46f8-b20f-86cf60388791) |
| hostId | afb64fd7a445a42c4dee560085e4dc4db3751f923ecd76f98b21c36e |
| id | d212be0d-3ed0-4096-b763-b08a96fd575e |
| image | nixos-23.11 (f73588c5-468e-4678-9974-33300dce7c39) |
| key_name | controller-0 |
| name | nixos-23.11 |
| progress | 0 |
| project_id | 4d04429679c44b9ab3cafd523b9f86fd |
| properties | |
| security_groups | name='default' |
| status | ACTIVE |
| updated | 2024-04-06T18:32:18Z |
| user_id | 236cbdbe0eb545d68a21c58cb782c924 |
| volumes_attached | |
+-------------------------------------+----------------------------------------------------------+OpenStack CLI Output example
Creating and viewing the instance information in the OpenStack web UI is also possible.
After booting, all of the cloud-init bits will run, and we can log in via SSH with our defined keypair. The instance can also be accessed via the built-in serial console.
Server and system logs are also available for the instance.
Hope this helps someone
While I'm sure I've made some mistakes, and there are ways this could be much better, it works! The following configuration generates a simple, fully featured NixOS virtual machine image (which could probably be used with Baremetal via Ironic, too... soon-tm).
The best part of this setup is that it works inside and outside of OpenStack. The image is general purpose enough to be used on most any KVM hypervisor effectively, which is what I was going after, even if my system of choice happens to be OpenStack 😉
More NixOS to come
Happy Nix'ing! More soon.