zfs unable to mount pool on reboot
Error message
# systemctl status zfs-import-cache.service
* zfs-import-cache.service - Import ZFS pools by cache file
Loaded: loaded (/lib/systemd/system/zfs-import-cache.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2022-05-24 15:10:56 EDT; 2min 41s ago
Docs: man:zpool(8)
Process: 1659 ExecStart=/sbin/zpool import -c /etc/zfs/zpool.cache -aN (code=exited, status=1/FAILURE)
Main PID: 1659 (code=exited, status=1/FAILURE)
CPU: 18ms
May 24 15:10:56 server-uiop.local systemd[1]: Starting Import ZFS pools by cache file...
May 24 15:10:56 server-uiop.local zpool[1659]: cannot import 'tank': one or more devices is currently unavailable
May 24 15:10:56 server-uiop.local systemd[1]: zfs-import-cache.service: Main process exited, code=exited, status=1/FAILURE
May 24 15:10:56 server-uiop.local systemd[1]: zfs-import-cache.service: Failed with result 'exit-code'.
May 24 15:10:56 server-uiop.local systemd[1]: Failed to start Import ZFS pools by cache file.
However all disks were present.
Issue is in the /etc/zfs/zpool.cache file.
Originally tank pool was created as zpool create -f tank raidz /dev/sda4 /dev/sdb4 /dev/sdc4 /dev/sdd4
Linux on some RAID controllers assign letters in a random fashion on reboot, thus the zfs cache points to the wrong drives.
solution
Unmount/export pools zpool expot -a
Remove the zpool.cache file rm /etc/zfs/zpool.cache
Import pool back using wwn ids /sbin/zpool import -a -d /dev/disk/by-id/wwn-0x5000039ab83243c9-part4 -d /dev/disk/by-id/wwn-0x5000039ab832484d-part4 -d /dev/disk/by-id/wwn-0x5000039ab83249ed-part4 -d /dev/disk/by-id/wwn-0x5000039ab833dfb5-part4
Now everything should work after the reboot.