[SOLVED]Grub, LUKS1-partition unlocks LUKS2-part, det. head.

[newsboost]

Hi all,

I'm normally an Arch Linux user, but am now setting up a Debian-server (hence my first post here). My partition setup is:

Code: Select all

/dev/sdb :
  sdb1) 511 MB fat32, flags: boot,esp
  sdb2) 1074 MB - LUKS1-encrypted for boot
  sdb3) 254GB LVM2 inside a LUKS2-encrypted container

I've been running this type of setup for some years on Arch Linux and would like the same for my debian-server. I think (after a loong time) managed to install debian with this setup (wasn't easy: eventually found out that in the "Debian GNU/Linux UEFI Installer menu" choose "Advanced Options..." and "Graphical expert install" - otherwise you won't be able to use "cryptsetup luksOpen" from terminal, which is needed)... I think I can still boot from a USB-installer ("Graphical expert install") and I think I can then unlock the LUKS2-partition and chroot into it! My problem at the moment is GRUB: When I reboot the pc, I get into the GRUB emergency prompt (am using GNU GRUB version 2.02=dfsg1-20).

Code: Select all

grub>

PROBLEM 1 (most urgent): It's wrong - GRUB should be asking for a password automatically, but it isn't asking for password... When the correct password is entered, it should show me the boot-menu with boot-options... I can get a bit further by doing:

Code: Select all

cryptomount (hd3,gpt2)
set prefix=(crypto0)/grub
insmod normal
normal

But it's also wrong (although GRUB starts, sdb3 or gpt3 isn't unlocked, so initrd halts with errors). I'm confused because not many on Debian uses this kind of setup, where GRUB first should unlock a LUKS1-partition, next initrd should take care of unlocking the LUKS2-partition... I'm asking here for feedback, so next time I'll chroot into the existing installation and try what you write I should do + reboot and test if it works...

PROBLEM 2 (please wait with this, until first problem is solved):
I can't boot, because the next step is that the LUKS2-header and keys for both LUKS-partitions should be stored within the initrd-image and then at least on Arch Linux I can make a pre-hook to automatically decrypt sdb3 using a special commandline which deals with the detached header-file. This I think is definately also possible on Debian - anyway, please help with Problem 1first...

[p.H]

sdb1) grub2 : 1007 KiB, flags: bios_grub ; 1MB sdb
2) fat32: 512.00 MiB, flags: "boot" + "esp" ; 536.9 MB ESP-partition <--- LUKS1-encrypted

Nonsense.
A bios_grub (BIOS boot) partition is used for BIOS/legacy boot.
An EFI system partition (ESP) is used for EFI boot. It must be plain FAT and must not be encrypted in order to be readable by the UEFI firmware.

Do you intend to boot in EFI mode or BIOS mode ?

choose "Advanced Options..." and "Graphical expert install" - otherwise you won't be able to use "cryptsetup luksOpen" from terminal

No, you can use the installer shell from any install entry in the installer boot menu.
By the way, what do you need to do in the installer shell ?

I get into the GRUB emergency prompt (am using GNU GRUB version 2.02=dfsg1-20)

Code: Select all

grub>

"grub>" is the normal prompt. The emergency prompt is "grub rescue>". Which one do you get ?

[newsboost]

Nonsense.
A bios_grub (BIOS boot) partition is used for BIOS/legacy boot.
An EFI system partition (ESP) is used for EFI boot. It must be plain FAT and must not be encrypted in order to be readable by the UEFI firmware.

Do you intend to boot in EFI mode or BIOS mode ?

I'm sorry, I had some old notes in a document, that also confused myself - I've updated the original post. I intend to boot in EFI-mode.

choose "Advanced Options..." and "Graphical expert install" - otherwise you won't be able to use "cryptsetup luksOpen" from terminal

No, you can use the installer shell from any install entry in the installer boot menu.
By the way, what do you need to do in the installer shell ?

I don't disagree with you regarding "you can use the installer shell from any install entry in the installer boot menu". But I at least couldn't use cryptsetup for reading/opening/manipulating (incl. formatting) LUKS-encrypted partitions... There's this nice thing about this method, that it allows to "Loading installer components from CD", where a lot of extra things such as dm-crypt-stuff can be loaded. I don't think these are loaded unless they're chosen from the options arising from the "Graphical expert install"-option. I couldn't use cryptsetup other way that this... You're asking what I need the installer for: I really would prefer the large root/LVM-volume to be LUKS2-encrypted with detached header, as I have it on Arch Linux. It sounds as if you're maybe not familiar with this operation - but I create it like this, for instance:

Code: Select all

cryptsetup luksFormat --type luks2 /dev/sdb3 --align-payload 8192 --header cryptLVM_header.img

The graphical installer cannot open the LUKS2-volume. I also did that from the shell. But I've tried many different things. The last time I messed with this is around 3 weeks ago, just before my holiday started. Now I'm back and the Debian net-installer I'm using can't even open the LUKS1-volume anymore. I thought/think it worked before my vacation... The errors I get are "check that kernel supports aes-xts-plain64 cipher". Now I've really confused myself and am almost giving up and maybe I should just make everything LUKS1-containers, that is much easier and simpler... "uname -a" from the within the shell of the installer said 4.19.0.9-amd64 - I think it would be better if that kernel was at least 4.20.6 (seems there have been some dm-crypt changes, i.e. there's a comment and discussion "Try on kernel at LEAST 4.20.6" here: https://bbs.archlinux.org/viewtopic.php?id=243988 )...

I get into the GRUB emergency prompt (am using GNU GRUB version 2.02=dfsg1-20)

Code: Select all

grub>

"grub>" is the normal prompt. The emergency prompt is "grub rescue>". Which one do you get ?

Ok, please bear over with me. I'm not used to seeing that prompt at all, normally I just see the GRUB-menu. So sorry about that - in this case I don't see the rescue prompt, just the normal GRUB prompt... I hope this clarifies some things...

By the way: Is it possible to boot from an Arch Linux ISO / USB and chroot into the unlocked encrypted root-partition, setup everything and run grub-installer from within the chroot'ed debian installation? I've never tried it, but I've never had problems with cryptsetup when using the Arch Linux ISO USB-image...? Just a thought - the debian installer clearly is not happy about my LUKS2-partition, so probably better to completely skip the debian-installer-menu-stuff, if possible - or maybe I'll try it later myself (however it might be that the kernel versions differs too much for chroot to be happy about it?)... Appreciate all help/input/ideas and suggestions and I hope I've explained myself better now. Thanks.

[Head_on_a_Stick]

I can't help directly (I've never used LUKS) but you can install Debian in the same way as Arch: https://www.debian.org/releases/stable/ ... 03.en.html

You can even do this with an Arch live ISO image: https://www.archlinux.org/packages/comm ... bootstrap/

[sickpig]

@OP I have exact same setup as yours. The easiest way is to not bother encrypting /boot during installation but do so after. Let the normal net install process complete and you should end up with a minimum of /boot/efi, /boot and /

Now login and encrypt /boot following https://cryptsetup-team.pages.debian.net/cryptsetup/encrypted-boot.html to the t and you should be good.

my setup after following the earlier mentioned link -

Code: Select all

sda              8:0    0 465.8G  0 disk 
├─sda1           8:1    0   100M  0 part  /boot/efi
├─sda2           8:2    0 288.4G  0 part 
├─sda3           8:3    0     1K  0 part 
├─sda5           8:5    0 125.8G  0 part 
├─sda6           8:6    0    30G  0 part 
│ └─crypt      254:1    0  29.9G  0 crypt /home
├─sda7           8:7    0   492M  0 part 
│ └─boot_crypt 254:2    0   490M  0 crypt /boot
└─sda8           8:8    0    21G  0 part 
  └─sda8_crypt 254:0    0    21G  0 crypt /

Also, note /boot will only support luks1 as grub doesn't work with luks2. Use update-initramfs -c to generate a new initramfs in case of issues.

[newsboost]

@Head_on_a_Stick : Ok, thanks a lot, yes, I've stumbled across these pages. I decided to use an Ubuntu USB, I can mount my LUKS1+2 partitions that way and I could chroot into the system - however I find it strange/concerning that "LANG=C.UTF-8 chroot /mnt/debinst /bin/bash" opens up an ubuntu-command-prompt, I would have expected a debian-command-prompt... Maybe I shouldn't have "mount --rbind" for proc, dev and sys before chroot... I spent 2-3 hours this evening messing up the Grub-installation, which malfunctionend as I forgot "GRUB_ENABLE_CRYPTODISK=y" and that's where the pain started as I couldn't get out of the menu and everything failed, tried CTRL+C, eventually force-closed the whole terminal window and after that I couldn't really re-install grub or had problems and turned off the pc. Have to go to work, will continue tomorrow after work, thanks - might need to start over from fresh... Thanks.

@sickpig : yes, that link seems good... I would however really prefer if I could install grub directly to the decrypted /boot instead of first installing grub to an unencrypted /boot and then convert that to an encrypted partition...Anyway - thanks for the suggestion (if all else fails)... Later, I'm very sure I'll get into trouble, when I also have to "apt-get install cryptsetup-initramfs" as I previously tried to change /etc/cryptsetup-initramfs/conf-hook to include these lines:

Code: Select all

CRYPTSETUP=y
KEYFILE_PATTERN="/root/DECRYPT_KEYS/*"

But from the command ' lsinitramfs /boot/initrd.img-5.4.44-2-pve | grep -i "keyfiles/" ' I remember I had problems, because the DECRYPT_KEYS-files wasn't in the initramfs-image... But I'll hopefully get to that part later, first I need to solve this GRUB-stuff...

Thanks - I'll hopefully come with an update within 1-2 days and hopefully I'll at least install GRUB and make the GRUB-menu come up... Thansk for your help!

[Head_on_a_Stick]

I find it strange/concerning that "LANG=C.UTF-8 chroot /mnt/debinst /bin/bash" opens up an ubuntu-command-prompt, I would have expected a debian-command-prompt... Maybe I shouldn't have "mount --rbind" for proc, dev and sys before chroot

No, you need to mount the API filesystems for the GRUB installation to work from the chroot. Note that Debian (and Ubuntu, presumably) has the arch-choot(8) command in the arch-install-scripts package, which automates that process.

To get the expected prompt try sourcing the system shell profile:

Code: Select all

. /etc/profile

[newsboost]

I find it strange/concerning that "LANG=C.UTF-8 chroot /mnt/debinst /bin/bash" opens up an ubuntu-command-prompt, I would have expected a debian-command-prompt... Maybe I shouldn't have "mount --rbind" for proc, dev and sys before chroot

No, you need to mount the API filesystems for the GRUB installation to work from the chroot. Note that Debian (and Ubuntu, presumably) has the arch-choot(8) command in the arch-install-scripts package, which automates that process.

To get the expected prompt try sourcing the system shell profile:

Code: Select all

. /etc/profile

That didn't change anything... But I suspected it has/had something to do with /etc/hostname - it was "ubuntu" inside the chroot environment (probably because /dev/... and /sys/... is linked to the ubuntu that debian was installed from)... Also "uname -a" showed the same inside the chroot and outside the chroot - again, I'm guessing because /dev/.. and /sys/... etc is the same... Hopefully it hasn't any other consequences... Anyway... It's been hot this weekend so I've been a bit lazy. But I did start all over from scratch again and this is where I am now: I installed arch-chroot and used debootstrap from an Ubuntu-ISO (live USB-stick). This is better than the Debian-installer, I think (no problems with cryptsetup and LUKS2-partitions!)... But I still have GRUB-problems... I ended doing something like this:

Code: Select all

# echo "GRUB_ENABLE_CRYPTODISK=y" >>/etc/default/grub
# apt-get install --reinstall grub-efi
# grub-install
# update-grub


So GRUB is installed... As in the first post, when I reboot I just arrive at the "grub>"-prompt - no menu! This is bad... Second thing - a bit more detailed than I started out with - here I load the linux-kernel from the initramfs-image:

Code: Select all

cryptomount (hd3,gpt2)
set prefix=(crypto0)/grub
insmod normal
normal


I don't think I can decrypt the LUKS2-partition from the GRUB-prompt - so I think I cannot type e.g. "set root=(hd0,gpt2)". I've got 2 unsolved problems:

1) Grub menu doesn't appear (maybe I could live with that)...
2) The LUKS2-partition isn't decrypted, so the initramfs-image loads but quickly fails:

Code: Select all

...
/dev/sdd: open failed: No medium found
volume group "vg" not found...
Cannot process volume group vg
/dev/sdd: open failed: No medium found
volume group "vg" not found...
Cannot process volume group vg
/dev/sdd: open failed: No medium found
volume group "vg" not found...
Cannot process volume group vg
Gave up waiting for root file system device. Common problems:
...
...
ALERT! /dev/mapper/vg-root does not exist. Dropping to a shell!

BusyBox v1.30.1 (Debian 1:1.30.1-4) built-in shell (ash)
(initramfs)

On Arch Linux, when I did something similar as here, I "attached" the LUKS decryption-key-file and LUKS2-header to the initramfs-image (by editing "mkinitcpio.conf", FILES=/root/crypto_keyfile.bin, to include any needed LUKS keyfile(s))... Then because LUKS2 is a bit more complicated than LUKS1, I added some hooks to the initramfs-image.... The result is that (A) Grub asked me to type a password to unlock my /boot-partition (LUKS1-encrypted). Given the correct password, the GRUB menu came up and I choose the linux-kernel to run. Then (B) the initramfs-image loaded the kernel, automatically decrypting the LUKS2-root partition (actually it's an LVM-partition, but by enabling LVM in the initramfs-early boot, that also worked)... I think I need to do something with "apt-get install cryptsetup-initramfs" + "vim /etc/cryptsetup-initramfs/conf-hook" + change 'KEYFILE_PATTERN="/root/BACKUP_KEYS/*"' + "update-initramfs -u -k all" + verify the LUKS-keyfiles are inside the initramfs-image using lsinitramfs /boot/initrd.img-5.4.44-2-pve | grep -i "keyfiles/" - I've partly had success with this... Seemed this method with KEYFILE_PATTERN="/root/BACKUP_KEYS/*"' included one of the files - but for LUKS2 automatic decryption I need both the LUKS2-header and the keyfile + I would also like the BOOT-partitions keyfile to be included so that can be used for /etc/crypttab (I think that's how I've done on Arch Linux)...

The main problem is also to automatically decrypt the LUKS2-volume, but I appreciate any hints/feedback on both problems (A: Grub doesn't ask for password and doesn't show the GRUB-menu + B: The LVM or root-volume-group isn't accessible, so the kernel never really boots up) - and will continue to experiment for a while... Thanks for any ideas!

[sickpig]

if your /boot is now luks1 encrypted then you should just regenerate the initramfs to solve it

[newsboost]

if your /boot is now luks1 encrypted then you should just regenerate the initramfs to solve it

Actually I think I had regenerated it a couple of times... But I think I solved it... I did *MANY* different things - among these booting from an Arch Linux USB-stick a few times and also starting all over from scratch with debootstrap again (so I've now tested, debootstrap on both Ubuntu and Arch and prefer either of these methods, in the future)... I'm not completely sure... But I think basically I installed GRUB incorrectly using BIOS-mode installation (grub-install or maybe grub-install /dev/sdb) instead of the UEFI-method:

Code: Select all

# grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=grub --recheck

In addition - not 100% sure if the LVM-part is needed - but I think so:

Code: Select all

echo "GRUB_ENABLE_CRYPTODISK=y" >>/etc/default/grub
echo "GRUB_PRELOAD_MODULES="lvm" >>/etc/default/grub


It's great, I can now focus on the fun/challenging/interesting part, instead of the boring GRUB-stuff....


Automatic unlock of LUKS2-encrypted root-partition - problem with missing LUKS-keyfiles (should be in the initramfs-image)...
I have done "apt-get install cryptsetup-initramfs" modified /etc/initramfs-tools/initramfs.conf so when I run "update-initramfs -u -k all" then I want to include 2 (or 3) LUKS-key-files from /root/LUKS_KEYS/* into the /boot/iniramfs-image... I modified /etc/cryptsetup-initramfs/conf-hook to include:

Code: Select all

CRYPTSETUP=y
KEYFILE_PATTERN="/root/LUKS_KEYS/*"


My /etc/crypttab looks like (first line is boot - not strictly needed, but the second line is the ROOT-partition, so that is really important):

Code: Select all

cryptBoot  /dev/sdb2  /root/LUKS_KEYS/keyfileBOOT.bin  luks
cryptLVM   /dev/sdb3  /root/LUKS_KEYS/keyfileLVM.bin   luks,header=/root/LUKS_KEYS/cryptLVM_header.img


When I run "update-initramfs -u" I no longer get errors/warnings - except it says "No /etc/pve-efiboot-uuids found, skipping ESP sync" but I don't think it's important... Then I want to verify/test that the initramfs-image in fact DO contain the LUKS-keys... So I type (and repeated with "grep -i header"):

Code: Select all

lsinitramfs /boot/initrd.img-5.4.44-2-pve | grep -i key
cryptroot/keyfiles/cryptLVM.key


And that's not completely what I expected... I wanted *ALL* 3 LUKS-key-files (2 keys+1 header) inside /root/LUKS_KEYS, but it seems only a single LUKS-file is there? Why? I cannot unlock the LUKS2-partition without the header-file in the initramfs-file.... So what am I missing? I'm looking for a "debian"-way of doing what's described here: https://wiki.archlinux.org/index.php/Dm ... UKS_header (namely "Modifying encrypt hook
This method shows how to modify the encrypt hook in order to use a detached LUKS header. Now the encrypt hook has to be modified to let cryptsetup use the separate header (FS#42851; base source and idea for these changes published on the BBS). Make a copy so it is not overwritten on a mkinitcpio update)") - or is it just not possible to use a LUKS2-encrypted root-partition on debian (preferably with automatic unlock)?

Thanks for your support, I hope I'm close to a solution (gotta sleep now and work tomorrow - will continue tomorrow evening after work)!

[newsboost]

Made quite some progress... This is probably completely wrong and there probably is a better solution - but I just couldn't make debian understand that it REALLY SHOULD copy those LUKS-keyfiles to the initramfs... So after struggling with this task, for probably around 5 hours I finally came up with this script, which I currently call /root/usr/share/initramfs-tools/hooks/cryptroot_header - including debug output so I can see what's going on when running "update-initramfs -u":

Code: Select all

#!/bin/sh
PREREQ="cryptroot"

prereqs()
{
    echo "$PREREQ"
}

case "$1" in
    prereqs)
        prereqs
        exit 0
        ;;
esac

. /usr/share/initramfs-tools/hook-functions
. /lib/cryptsetup/functions

#===============================================================

cryptrootDir="$DESTDIR/cryptroot"
headerDir="$cryptrootDir/luks2header"

# Create a new directory to avoid pollution with existing
mkdir -pm0700 -- "$headerDir"
cryptsetup_message "WARNING: TESTING. cryptrootDir:=$(ls -latrh $cryptrootDir)"

# Copy needed files:
copy_file keyfile /root/LUKS_KEYS/keyfileBOOT.bin cryptroot/luks2header
copy_file keyfile /root/LUKS_KEYS/keyfileLVM.bin cryptroot/luks2header
copy_file keyfile /root/LUKS_KEYS/cryptLVM_header.img cryptroot/luks2header

cryptsetup_message "WARNING: TESTING. headerDir:$headerDir=$(ls -latrh $headerDir/)"


It's pretty ugly, but it's the best I can/could come up with now (please let me know if there are better methods, than manually copying the 3 files I want over)...

After rebooting it obviously still can't automatically decrypt the root-partition - that task will have to wait, gotta sleep now... I quickly tested and at the (initramfs)-prompt, I can now "cd cryptroot/luks2header" and see my 3 key-files. I can then "cryptsetup luksOpen --header cryptLVM_header.img --key-file keyfileLVM.bin /dev/sdb3 cryptLVM" and the cryptLVM partition has been decrypted (but my root partition isn't available yet)! Then I typed "vgchange -a y" and it replies with "2 logical volume(s) in volume group "vg" is now active". The command "ls -latrh /dev/mapper" confirms that I now have access to vg-root and vg-swap. I then typed "set root=/dev/mapper/vg-root" followed by "exit" - and HOOORAY - now the kernel boots up, for the first time!

UPDATE:
Final task was to automate all this LUKS2-partition decryption stuff, so I don't have to type in all that stuff manually - the initramfs-image should automatically take care of all this... I solved this by modifying /etc/default/grub (and running update-grub):

Code: Select all

GRUB_CMDLINE_DEFAULT="cryptopts=target=cryptLVM,source=/dev/sdb3,lvm=vg-root,header=/cryptroot/luks2header/cryptLVM_header.img,key=/cryptroot/luks2header/keyfileLVM.bin


In /etc/crypttab, there is only a line for cryptBoot, which has target /boot (this was the LUKS1-partition)... The /etc/fstab contains 3 lines: /dev/mapper/vg-root with / mount point, /dev/mapper/cryptBoot with /boot mountpoint and /dev/sdb1 with /boot/efi-mount point... Basically I think I've nailed it - but please provide feedback, if I did something stupid or there's a better way. I'll mark this as solved within 2-3 days, if nobody has any comments, thanks...