Use a real block storage backend for Cinder instead of mounting a virtual disk as a loopback device

Proof of Concept (PoC) installations of Openstack using Packstack use a sparse file virtual disk mounted under /var/lib/cinder. New block storage devices to be attached to instances are created under /var/lib/cinder/volumes

This might be OK for a test installation, but it is a terrible idea for production deployments because loopback devices have poor I/O performance under load.

The workaround is to first generate a packstack answer file and then edit the settings for Cinder block storage. To generate an answer file named foo.cfg:

$ packstack --gen-answer-file=foo.cfg

Within the answer file change
CONFIG_CINDER_VOLUMES_CREATE=y to CONFIG_CINDER_VOLUMES_CREATE=n

Packstack will search for a LVM Volume Group named cinder-volumes. If it finds a VG with that name, it will just use it as the backend for Cinder instead of creating a virtual disk file.

Now that you have edited the answer file telling Packstack not to create a Cinder Volume Group, you need to create an LVM Physical Volume and a Volume Group within the PV.

Before you create the PV, first you need to create a new partition. I will do this using gdisk (because my server uses GPT as its partition table):


[fedgro@fx8350no2 ~]$ sudo gdisk /dev/sdb
GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sdb: 1953525168 sectors, 931.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): A2744C5B-BBEE-4FEB-9152-853B5E707FF9
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 1743809901 sectors (831.5 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048       209717247   100.0 GiB   8E00  Linux LVM

Command (? for help): n
Partition number (2-128, default 2): 2
First sector (34-1953525134, default = 209717248) or {+-}size{KMGTP}:
Last sector (209717248-1953525134, default = 1953525134) or {+-}size{KMGTP}: +206G
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): 8e00
Changed type of partition to 'Linux LVM'

Command (? for help): p
Disk /dev/sdb: 1953525168 sectors, 931.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): A2744C5B-BBEE-4FEB-9152-853B5E707FF9
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 1311796589 sectors (625.5 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048       209717247   100.0 GiB   8E00  Linux LVM
   2       209717248       641730559   206.0 GiB   8E00  Linux LVM

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sdb.
Warning: The kernel is still using the old partition table.
The new table will be used at the next reboot or after you
run partprobe(8) or kpartx(8)
The operation has completed successfully.

OK~ now I have created a new partition /dev/sdb2 that is 206 GB in size. Let's check the existing partitions with lsblk:

[fedgro@fx8350no2 ~]$ sudo partprobe /dev/sdb
[fedgro@fx8350no2 ~]$ lsblk -f
NAME                      FSTYPE      LABEL UUID                                   MOUNTPOINT
sda                                                                               
├─sda1                    vfat              EAD9-3A00                              /boot/efi
├─sda2                    ext4              09c9cf88-30d9-4d21-8a5f-5723acec3bff   /boot
└─sda3                    LVM2_member       xT7fD3-vdcc-xgde-oZmD-RCUG-zA1k-Go15vd
  ├─fedora_fx8350no2-root xfs               c4e8d596-c9eb-48ce-a967-212c7f7b037e   /
  ├─fedora_fx8350no2-swap swap              a3c25d7c-d6a4-40db-8b43-488a08785a61   [SWAP]
  ├─fedora_fx8350no2-var  xfs               c244b30d-9f9c-47aa-bd17-3b519dd93c6f   /var
  └─fedora_fx8350no2-home xfs               bd9481ac-b484-495f-b45c-5e78616938df   /home
sdb                                                                               
├─sdb1                    LVM2_member       edPpZO-6Kdz-B46f-3umr-UpzZ-OQ80-jP6Lnc
│ └─fedora_fx8350no2-var  xfs               c244b30d-9f9c-47aa-bd17-3b519dd93c6f   /var
└─sdb2                                                                            


The new partition is highlighted above. 200 GB will be used for block storage while 3% (6 GB) will be used to store LVM metadata.


Next I have to create a LVM PV on /dev/sdb2:

[fedgro@fx8350no2 ~]$ sudo pvcreate /dev/sdb2
  Physical volume "/dev/sdb2" successfully created



Now I will create a the Volume Group cinder-volumes in the PV on /dev/sdb2:

[fedgro@fx8350no2 ~]$ sudo vgcreate cinder-volumes /dev/sdb2
  Volume group "cinder-volumes" successfully created

Now I'm ready to run packstack with the pre-generated answer file foo.cfg:

[fedgro@fx8350no2 ~]$ sudo packstack --answer-file foo.cfg

You might have noticed that I didn't create any LVM Logical Volumes in the VG cinder-volumes. You should not create any LV's inside the VG, nor do you need to format LV's with a file system. This is because every time Cinder creates a volume, it automatically creates its own LV within VG cinder-volumes.