Booting Debian Jessie from an lvmcache

Kernels & Hardware, configuring network, installing services

Booting Debian Jessie from an lvmcache

Postby tvynr » 2014-12-24 23:37

Hey all,

Just wanted to share this little recipe since it took a non-trivial amount of scraping on my part to find it. I am now running a Debian Jessie system using lvmcache. This turned out to be a bit trickier than I'd thought; it doesn't work out of the box. I performed the following steps:

  • Created cached volume via instructions in the "lvmcache" man page
  • Created a single LV from that volume group and used it as the PV for another volume group
  • Moved my root and home volumes onto that volume group
  • Rebooted

Upon reboot, I discovered the dm-cache kernel module had not been placed on the initramfs. After some struggling to find a modern-enough LiveCD that I could mount my system, I rebuilt my initramfs with dm-cache only to discover that a binary, /usr/sbin/check_cache, is necessary when mounting a cached volume. Next, I discovered that the caching policy is actually handled by dm-cache-mq, so I included that. Then everything worked fine.

Ultimately, the solution was twofold. First, install the "thin-provisioning-tools" package (which contains the "check_cache" binary). Then, copy the following simple script into "/etc/initramfs-tools/hooks":

Code: Select all
#!/bin/sh

PREREQ="lvm2"

prereqs()
{
    echo "$PREREQ"
}

case $1 in
prereqs)
    prereqs
    exit 0
    ;;
esac

if [ ! -x /usr/sbin/cache_check ]; then
    exit 0
fi

. /usr/share/initramfs-tools/hook-functions

copy_exec /usr/sbin/cache_check

manual_add_modules dm_cache dm_cache_mq


Finally, run "update-initramfs -u" to update your initramfs image for the current kernel (or add "-k all" to update for all kernels).

This comes with a couple downsides. First, check_cache is a dynamically linked executable and relies on the C++ stdlib; this means that my initramfs image is now 18M or so. (In theory, a statically-linked build could solve this problem... but it's honestly just a lot simpler to do things this way.) Second, check_cache is not fast -- it seems to take several seconds at boot time -- so it doesn't really speed up the boot process. But my whole system is running cached on my SSD, which is really what I wanted. :)
Last edited by tvynr on 2015-01-29 13:51, edited 1 time in total.
tvynr
 
Posts: 24
Joined: 2006-08-22 16:22

Re: Booting Debian Jessie from an lvmcache

Postby zang3tsu » 2015-01-28 05:50

Which live CD did you use? I tried the Debian Jessie's netinst rescue mode but it couldn't see the contents of the root filesystem.
zang3tsu
 
Posts: 3
Joined: 2015-01-28 05:45

Re: Booting Debian Jessie from an lvmcache

Postby tvynr » 2015-01-28 12:51

I seem to remember using the following:

http://www.sysresccd.org/Download

It's called "System Rescue CD" and it's running a fairly new Linux kernel -- one with a new enough kernel with lvmcache support. I tried building my own Jessie LiveCD with a custom kernel package to no avail; I kept getting failures from the build process which didn't make much sense to me. That said, I bought a 32Gb USB3 stick lately and, rather than putting a rescue image on it, I just installed Debian on it using a VM. I can boot my machine with it as if it were an external hard drive. This has the advantage of allowing me to update the kernel (and also the advantage of retaining state such as package installations between boots) without any fancy footwork. :)
tvynr
 
Posts: 24
Joined: 2006-08-22 16:22

Re: Booting Debian Jessie from an lvmcache

Postby zang3tsu » 2015-01-29 10:38

It seems my case is slightly different than yours because my boot was also in the root partition. And I don't think grub supports that configuration at the moment. So what I also did was to create another logical volume and placed a boot partition there.

Another caveat that I found was that the root partition couldn't be resized. I get this error when doing lvextend:

Code: Select all
  Unable to resize logical volumes of cache type.
zang3tsu
 
Posts: 3
Joined: 2015-01-28 05:45

Re: Booting Debian Jessie from an lvmcache

Postby tvynr » 2015-01-29 13:50

Hm; that's good to know. Of course, all you'd have to do is follow the instructions in the lvmcache man page to remove caching on the LV, resize it, and then put the caching back. But that does mean that one would be forced to dump one's cache in order to resize the cached volume.

I take your mention of /boot to mean that GRUB doesn't support booting from lvmcache. That sounds about right to me. I tend to do fun things to my root partition, so keeping /boot on a separate chunk of disk saves me a lot of trouble. :) Sorry that I'd forgotten to mention that detail.
tvynr
 
Posts: 24
Joined: 2006-08-22 16:22

Re: Booting Debian Jessie from an lvmcache

Postby zang3tsu » 2015-01-29 15:44

I checked the lvmcache man page, however, unfortunately, the lvm2 version in Jessie doesn't yet have lvconvert --splitcache or lvconvert --uncache. lvremove, I think, also failed.
zang3tsu
 
Posts: 3
Joined: 2015-01-28 05:45

Re: Booting Debian Jessie from an lvmcache

Postby tvynr » 2015-01-29 16:01

Hm... that's uncomfortable. I'll have to fire up a VM and play with that; I'd like to know if I'm stuck in this regard as well. It's not so much of a problem for me due to the way I'm actually using lvmcache:

Code: Select all
$ sudo lsblk
NAME                             MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                                8:0    0 931.5G  0 disk 
├─sda1                             8:1    0   500M  0 part 
├─sda2                             8:2    0    40M  0 part 
├─sda3                             8:3    0   128M  0 part 
├─sda4                             8:4    0   500M  0 part 
├─sda5                             8:5    0 146.5G  0 part 
├─sda6                             8:6    0  13.5G  0 part 
├─sda7                             8:7    0 770.1G  0 part 
│ └─vg--accelerated-volume_corig 254:2    0 770.1G  0 lvm   
│   └─vg--accelerated-volume     254:3    0 770.1G  0 lvm   
│     └─data_crypt               254:4    0 770.1G  0 crypt
│       ├─vg0-root               254:5    0  43.8G  0 lvm   /
│       ├─vg0-home               254:6    0 542.7G  0 lvm   /var/chroot/sid/mnt/outer-home
│       ├─vg0-backup             254:7    0    48G  0 lvm   /backups
│       └─vg0-config--backup     254:8    0     1G  0 lvm   
├─sda8                             8:8    0   244M  0 part  /boot
└─sda9                             8:9    0  1007K  0 part 
sdb                                8:16   0 119.2G  0 disk 
├─sdb1                             8:17   0  11.2G  0 part 
│ └─swap_crypt                   254:9    0  11.2G  0 crypt [SWAP]
└─sdb2                             8:18   0 108.1G  0 part 
  ├─vg--accelerated-cache_cdata  254:0    0 107.6G  0 lvm   
  │ └─vg--accelerated-volume     254:3    0 770.1G  0 lvm   
  │   └─data_crypt               254:4    0 770.1G  0 crypt
  │     ├─vg0-root               254:5    0  43.8G  0 lvm   /
  │     ├─vg0-home               254:6    0 542.7G  0 lvm   /var/chroot/sid/mnt/outer-home
  │     ├─vg0-backup             254:7    0    48G  0 lvm   /backups
  │     └─vg0-config--backup     254:8    0     1G  0 lvm   
  └─vg--accelerated-cache_cmeta  254:1    0   256M  0 lvm   
    └─vg--accelerated-volume     254:3    0 770.1G  0 lvm   
      └─data_crypt               254:4    0 770.1G  0 crypt
        ├─vg0-root               254:5    0  43.8G  0 lvm   /
        ├─vg0-home               254:6    0 542.7G  0 lvm   /var/chroot/sid/mnt/outer-home
        ├─vg0-backup             254:7    0    48G  0 lvm   /backups
        └─vg0-config--backup     254:8    0     1G  0 lvm   
sdc                                8:32   0  29.8G  0 disk 
sr0                               11:0    1  1024M  0 rom   


The accelerated volume is used as a PV for another VG and I take my root, home, and other volumes from that. As a result, I never need to change the size of my cache-accelerated volume (although, since dm_crypt is sitting between the cache-accelerated LV and the PV of the second group, resizing my cache-accelerated volume would be a bit of a challenge anyway).

If it helps, I'm using the Debian lvm2 package, version 2.02.111-2. I'm reasonably sure that I tried removing the cache at one point ('cause I didn't want to be stuck without resizing in case I needed it), but I might be imagining that.
tvynr
 
Posts: 24
Joined: 2006-08-22 16:22


Return to System configuration

Who is online

Users browsing this forum: No registered users and 8 guests

fashionable