App armor preventing access to volumes

virsh error

virsh # start m-0
error: Failed to start domain 'm-0'
error: internal error: process exited while connecting to monitor: 2024-02-13T00:26:01.090892Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/VM_01/m-l9z6luzxb7t3pti","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/dev/VM_01/m-l9z6luzxb7t3pti': Permission denied

terraform error

│ Error: error creating libvirt domain: internal error: process exited while connecting to monitor: 2024-02-13T00:15:54.995072Z qemu-system-x86_64: -blockdev {"driver":"host_device","filename":"/dev/VM_01/m-l9z6luzxb7t3pti","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}: Could not open '/dev/VM_01/m-l9z6luzxb7t3pti': Permission denied
│
│   with libvirt_domain.m[0],
│   on main.tf line 19, in resource "libvirt_domain" "m":
│   19: resource "libvirt_domain" "m" {

Corresponding dmesg error messages

[16608084.698492] audit: type=1400 audit(1707783354.166:93): apparmor="STATUS" operation="profile_load" profile="unconfined" name="libvirt-0c8ddcf2-dc91-4a2c-b088-d382529755a1" pid=1936661 comm="apparmor_parser"
[12755.809616] audit: type=1400 audit(1761096220.731:1598): apparmor="DENIED" operation="open" class="file" profile="libvirt-1c2a0edb-581c-4eb8-b366-2c678bd384f8" name="/var/lib/libvirt/images/debian-13-base" pid=6973 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055

Root cause is Apparmor denying access to the path, being that path to block device file, or a file on the file system.

Solution is to allow particular paths in the file /etc/apparmor.d/abstractions/libvirt-qemu.d/allow-images on Debian 13 ( older versions use /etc/apparmor.d/local/abstractions/libvirt-qemu ) (replace with a correct paths, below is an LVM example and the corresponding symlinks):

/dev/VM_01/* rwk,
/dev/dm-* rwk,
/var/lib/libvirt/** rwk,

then remove the files specific to the already tried VMs rm -rf /etc/apparmor.d/libvirt/libvirt-*, thern restart apparmor systemctl restart apparmor and libvirtd systemctl restart libvirtd.service

SPICE error

Terraform error:

│ Error: error creating libvirt domain: internal error: qemu unexpectedly closed the monitor: 2024-02-13T00:10:05.568801Z qemu-system-x86_64: warning: Spice: ../server/reds.
cpp:2551:reds_init_socket: getaddrinfo(127.0.0.1,5902): Address family for hostname not supported
│ 2024-02-13T00:10:05.568822Z qemu-system-x86_64: warning: Spice: ../server/reds.cpp:3442:do_spice_init: Failed to open SPICE sockets
│ 2024-02-13T00:10:05.568828Z qemu-system-x86_64: failed to initialize spice server
│
│   with libvirt_domain.m[0],
│   on main.tf line 19, in resource "libvirt_domain" "m":
│   19: resource "libvirt_domain" "m" {

Root cause is that by default SPICE binds to 127.0.0.1. That setup won’t work for some reason if there is no IPv4 configured on the interfaces, even though lo has 120.0.0.1 configured.

Suction is to listen ::1 localhost IPv6:

#spice_listen = "0.0.0.0"
spice_listen = "::1"

then restart libvirtd: systemctl restart libvirtd.service.