↓ Skip to main content
Ifesinachi’s HomeLab

Turning Six Raspberry Pi CM4s Into an OpenStack Cloud

·885 words·5 mins
Ifesinachi Osude
Author
Ifesinachi Osude
I build production infrastructure for a living — and a private cloud in my closet for fun.

I had a six-node Raspberry Pi Compute Module 4 cluster running k3s + Rancher, and I wanted to learn OpenStack. So I did the reasonable thing: wiped it to bare metal and stood up a real OpenStack cloud on 8GB ARM boards. Here’s how that went — the wins, and the three-hour detour into TLS that taught me the most.

Why not just keep k3s?
#

I already run OpenShift as my “real” Kubernetes. A second k8s wasn’t interesting. What I didn’t have was hands-on OpenStack, and the CM4 cluster — 6× 4-core / 8GB / NVMe boards — was sitting there being a slightly redundant k3s. The one thing that made this viable at all: /dev/kvm is present on the CM4. BCM2711 exposes ARM virtualization, so nova-compute can boot real KVM guests, not slow QEMU emulation. That was the make-or-break check, and it passed.

Topology, decided up front: cm1–3 are control-plane and compute, cm4–6 are compute-only. Yes — you can run VMs on the controllers, exactly like schedulable control-plane nodes in OpenShift. Six hypervisors, HA-ish control plane.

The ARM tax: you build your own images
#

Kolla-Ansible is the sane way to deploy OpenStack, but here’s the catch nobody mentions until you hit it: upstream Kolla publishes no arm64 images. Check the manifest for keystone and you’ll find amd64 and nothing else.

So you build them. All of them. kolla-build on one CM4, pushing to a local registry:2, produced 65 aarch64 images over a couple of hours. One-time cost, then cached — but plan for it. A few things bit along the way:

  • packages.treasuredata.com is DNS-dead. The base image fetches the td-agent GPG key from it, and that host no longer resolves — so every image build failed at the base layer. Fix: repoint the key to its S3 origin and drop fluentd (which I didn’t need anyway).
  • kolla-ansible’s git branch install is brittle — stable/2025.1 wasn’t a checkout-able ref by the time I tried. Pin the PyPI release instead (20.4.0).
  • A genuinely weird one: on AlmaLinux 9.8 with OpenSSL 3.5.5, ansible-galaxy and ansible’s URL fetcher trip an ASN1: NOT_ENOUGH_DATA TLS error — while curl and plain Python urllib to the same URL work fine. I ended up installing Ansible collections by pulling the tarballs with urllib and installing them offline, and pre-placing the docker GPG key as a local file.

Deploying: a checklist of small truths
#

bootstrap-servers → prechecks → deploy. Each one taught me something:

  • AlmaLinux isn’t on kolla’s supported-distro allow-list (CentOS/Rocky/ Ubuntu/Debian only), even though it’s RHEL-compatible and everything else keys off os_family == RedHat. One-line override to add it.
  • The registry died mid-deploy. kolla configures docker with iptables: false / bridge: none, which breaks -p port publishing — so my registry:2 container was “Up” but unreachable. Fix: run it with --network host.
  • Namespace mismatch: kolla-build defaults to the kolla namespace, kolla-ansible pulls from openstack.kolla. Align them.
  • proxysql and cinder I disabled — no proxysql image built (haproxy does DB load-balancing fine), and the NVMe is fully consumed by the rootfs so Cinder would need a loopback LVM VG. Both are fast-follows.

The payoff: an ARM64 cirros VM booted BUILD→ACTIVE in ~20 seconds on cm1 (a controller), real KVM, reached its login prompt. Six hypervisors up, OVN healthy.

The part that actually taught me something: PKI
#

I wanted what OpenShift gives you — every component speaking TLS, issued from my homelab CA (mikeOSUDE Root CA), and every container trusting that one root. kolla can do this: kolla_copy_ca_into_containers, internal + external TLS, a cert for openstack.mikeosude.com issued from my step-ca over ACME DNS-01 (Technitium API as the solver — the VIP wasn’t up yet, so HTTP-01 was out).

It half-worked, and the debugging was the best part. Symptoms: Horizon served the right cert, keystone auth worked, but every other service threw 503 Keystone unavailable. Two root causes, both non-obvious:

  1. rabbitmq’s rolling restart races on slow ARM. reconfigure recreates rabbitmq and runs rabbitmqctl wait before the container is back up → “container not running”, aborting the whole run. Patched the wait task to retry (until rc==0, 15×15s).
  2. The real culprit: openstack_cacert. kolla registers services by running the openstack client inside kolla-toolbox. That client uses Python/certifi — not the container’s system trust — so even with the CA copied in, it failed cert verification. The “register services” task aborted before the nova/neutron/glance configs were regenerated to https, leaving them pointing at http:// endpoints that now spoke TLS. Hence the 503s. Setting openstack_cacert to the container CA bundle fixed the whole chain at once.

After that: 6/6 hypervisors up, 12/12 OVN agents, all endpoints https, Horizon signed by my CA, every container trusting one root. Exactly the model I wanted.

Was it worth it?
#

For learning? Absolutely. OpenStack on ARM is a genuinely instructive kind of hard — not because the concepts are exotic, but because every “it just works” assumption from x86 tutorials is a place to actually understand what’s happening. Building the images, fighting the PKI chain end to end, watching a VM schedule onto a Pi — you come out the other side actually knowing how the thing fits together.

Next up: Cinder on a proper disk, floating-IP connectivity, and moving the image builds to GitLab CI on the arm64 Pi 5 so I never hand-build 65 images again.