Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

MacMini RAID1 ESP UEFI (SOLVED)

New to Debian (Or Linux in general)? Ask your questions here!
Message
Author
Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

MacMini RAID1 ESP UEFI (SOLVED)

#1 Post by Deekee »

As a beginner wo has never setup a RAID1 I fail miserably :(

I have a Debian 11 installation, where the partitioning and RAID1 configuration where done using the automatic partitioning of the Debian installer.

IMG_1736.JPG
IMG_1736.JPG (131.55 KiB) Viewed 1844 times

But when I try to finish partitioning and write changes to disk I get the following error.

IMG_1735.JPG
IMG_1735.JPG (31.76 KiB) Viewed 1844 times

What have I done wrong?

I already tried to google for a solution but unfortunately I was unable to find one...

Many thanks in advance for any hint :-)
Last edited by Deekee on 2022-09-15 09:41, edited 1 time in total.

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI

#2 Post by p.H »

Setting up UEFI boot with software RAID is a tricky topic.
I see 3 RAID arrays but only 2 RAID partitions on each drive. Did you build the RAID1 device #3 (mounted on /) with the EFI partitions ? This is wrong. EFI partitions cannot be used as normal RAID members nor as the root filesystem. Also 500 MB for the root filesystem is too small.
If I were you I would create only one RAID array, use LVM on top of it and create logical volumes as needed (/, swap, /home...)

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI

#3 Post by Deekee »

Yeah indeed, this seems to be tricky topic indeed!

Actually I want to be able to boot from one or the other device automatically, if one fails, so I think LMV is not suitable or am I wrong here?

So I started again, resized root filesystem and installed Debian 11 on one SSD only and now I'm busy reading ton's of blog post's about mdadm and initramfs :-)

My system has two hard drives, /dev/sda and /dev/sdb which are identical in size. /dev/sda is currently unused, and /dev/sdb has the following partitions:

This is the current situation.

Code: Select all

root@macmini:/home/dirk# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            7.8G     0  7.8G   0% /dev
tmpfs           1.6G  868K  1.6G   1% /run
/dev/sdb2       901G  1.6G  853G   1% /
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sdb1       952M  3.5M  948M   1% /boot/efi
tmpfs           1.6G     0  1.6G   0% /run/user/1000

Code: Select all

root@macmini:/home/dirk# sudo fdisk -l
Disk /dev/sdb: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 5230F3BD-69DD-48A0-918B-A5F124D63E95

Device          Start        End    Sectors   Size Type
/dev/sdb1        2048    1953791    1951744   953M EFI System
/dev/sdb2     1953792 1921875967 1919922176 915.5G Linux filesystem
/dev/sdb3  1921875968 1953523711   31647744  15.1G Linux swap


Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B0F29AA1-B675-4AE8-AC59-4394A64EF7FD

Code: Select all

root@macmini:/# efibootmgr -v
BootCurrent: 0000
Timeout: 5 seconds
BootOrder: 0000,0080
Boot0000* debian	HD(1,GPT,b67c0b2e-3921-45b2-83a6-2f8021c01ecc,0x800,0x1dc800)/File(\EFI\debian\shimx64.efi)
Boot0080* 	PciRoot(0x0)/Pci(0x1f,0x2)/Sata(1,0,0)/HD(1,GPT,fbb8e9c7-869a-4bf6-8dfe-eb56718c1a6c,0x40,0x1fc0)/File(\EFI\BOOT\BOOTX64.efi)
BootFFFF* 	PciRoot(0x0)/Pci(0x1f,0x2)/Sata(1,0,0)/HD(1,GPT,fbb8e9c7-869a-4bf6-8dfe-eb56718c1a6c,0x40,0x1fc0)/File(\EFI\BOOT\BOOTX64.efi)
I'm sure it must be possible to automatically boot from one or the other SSD in case one of them fails and I'm sure that I will get there one day :-)

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI

#4 Post by p.H »

Deekee wrote: 2022-09-14 11:26 Actually I want to be able to boot from one or the other device automatically, if one fails, so I think LMV is not suitable or am I wrong here?
LVM over a single RAID array is more flexible than multiple RAID arrays. It is orthogonal to boot redundancy and disk fault tolerance.

Create an EFI partition on each disk.
Create one (if you will use LVM) or several (if you will not use LVM) RAID partitions on each array.
Create RAID arrays using the RAID partitions.
Use the single RAID array as LVM physical volume or use the multiple RAID arrays as /, swap, /home...
If using LVM, create a volume group with the RAID array and create logical volumes for /, swap, /home...
Finish the installation.

The installer will use only one EFI partition so you will not have boot redundancy. This is when things get tricky. There are basically two different methods.
Method 1: have two independent EFI partitions, manually install GRUB on the second one and create/update EFI boot variables in the NVRAM with efibootmgr.
Method 2: create a RAID1 array with metadata 1.0 (at the end) on the two EFI partitions, format it as FAT, mount it as /boot/efi, reinstall GRUB without updating the NVRAM and manually create/update EFI boot variables in the NVRAM with efibootmgr.

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI

#5 Post by Deekee »

One step further, now I copied the partition table to the empty SSD :-)

Code: Select all

root@macmini:~# sfdisk -d /dev/sdb | sfdisk /dev/sda --force
Checking that no-one is using this disk right now ... OK

Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B0F29AA1-B675-4AE8-AC59-4394A64EF7FD

Old situation:

>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Script header accepted.
>>> Created a new GPT disklabel (GUID: 5230F3BD-69DD-48A0-918B-A5F124D63E95).
/dev/sda1: Created a new partition 1 of type 'EFI System' and of size 953 MiB.
/dev/sda2: Created a new partition 2 of type 'Linux filesystem' and of size 915.5 GiB.
/dev/sda3: Created a new partition 3 of type 'Linux swap' and of size 15.1 GiB.
/dev/sda4: Done.

New situation:
Disklabel type: gpt
Disk identifier: 5230F3BD-69DD-48A0-918B-A5F124D63E95

Device          Start        End    Sectors   Size Type
/dev/sda1        2048    1953791    1951744   953M EFI System
/dev/sda2     1953792 1921875967 1919922176 915.5G Linux filesystem
/dev/sda3  1921875968 1953523711   31647744  15.1G Linux swap

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
... and rebooted successfully, now working on the next step :D

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI

#6 Post by Deekee »

Problem solved I've now successfully installed RAID1 in a blink of an eye :D

Solution:
Using Ubuntu Server 22.04.1 and the following guide:

https://askubuntu.com/questions/1234949 ... wo-devices

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI

#7 Post by p.H »

You could do the same with Debian. This is similar to what I suggested in my previous post.
I'm curious about how Ubuntu handled the boot redundancy though. Did it install GRUB in both EFI partitions and create two boot variables in the UEFI NVRAM ?

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#8 Post by Deekee »

Yeah sure, but not with the debian installer!

Yes, I think it did install GRUB in both EFI Partitions, actually it created two small extra partitions (1GB)

I simply marked the each drive as a boot drive and left it unformatted and then created only one extra partition on each drive with the rest of the remaining capacity, left both unformatted too. Then I setup the RAID1, added and formatted it as ext4 and mounted it on "/", then the installer just took care of the rest.

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#9 Post by p.H »

Deekee wrote: 2022-09-15 17:26 Yes, I think it did install GRUB in both EFI Partitions
If I were you, I'd check
- the contents of both EFI partitions
- the EFI boot variables (with efibootmgr -v)

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#10 Post by Deekee »

Somehow strange I still got an entry for debian in there...

Code: Select all

efibootmgr -v
BootCurrent: 0001
Timeout: 5 seconds
BootOrder: 0000,0001,0002
Boot0000* debian	HD(1,GPT,b67c0b2e-3921-45b2-83a6-2f8021c01ecc,0x800,0x1dc800)/File(\EFI\debian\shimx64.efi)
Boot0001* ubuntu	HD(1,GPT,64190021-1613-40fc-8078-5ee74885e655,0x800,0x219800)/File(\EFI\ubuntu\shimx64.efi)
Boot0002* ubuntu	HD(1,GPT,f5877435-37c6-4081-a98f-62707c7fd6a3,0x800,0x219800)/File(\EFI\ubuntu\shimx64.efi)
Boot0080* 	PciRoot(0x0)/Pci(0x1f,0x2)/Sata(1,0,0)/HD(1,GPT,fbb8e9c7-869a-4bf6-8dfe-eb56718c1a6c,0x40,0x1fc0)/File(\EFI\BOOT\BOOTX64.efi)
BootFFFF* 	PciRoot(0x0)/Pci(0x1f,0x2)/Sata(1,0,0)/HD(1,GPT,fbb8e9c7-869a-4bf6-8dfe-eb56718c1a6c,0x40,0x1fc0)/File(\EFI\BOOT\BOOTX64.efi)
Unfortunately I dont' know how to mount those ubuntu partitions?

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#11 Post by p.H »

Nice, it really looks like Ubuntu installed the boot loader on both drives. Now I'm positively impressed.
I wonder how it is set up. Are both EFI partitions mounted ? Where ? Usually an EFI partition is mounted on /boot/efi, but there is only one /boot/efi.
Also I'm curious about how the grub-efi-amd64 package knows it must install GRUB on both partitions on upgrades. Stored in debconf ?

Code: Select all

debconf-show grub-efi-amd64
Deekee wrote: 2022-09-16 05:48 Somehow strange I still got an entry for debian in there
EFI boot variables are usually not automatically deleted when a boot loader is removed.
Deekee wrote: 2022-09-16 05:48 Unfortunately I dont' know how to mount those ubuntu partitions
You can mount them as any other partitions if they are not already mounted.
blkid will show which partitions have these PARTUUID 64190021-1613-40fc-8078-5ee74885e655 and f5877435-37c6-4081-a98f-62707c7fd6a3.
df or lsblk will show which partitions are mounted where.
Last edited by p.H on 2022-09-16 21:32, edited 1 time in total.

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#12 Post by Deekee »

Well, I the meantime I removed one SSD to test the RAID and it' s runnung nicely after rebooting :-)

Code: Select all

debconf-show grub-efi-amd-64
Returns noting at all?

I''ve now replaced the SSD I removed with a new one and now in the process of figuring out how to restore and rebuild the RAID to see if my recover procedure works...

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#13 Post by p.H »

My mistake, command corrected in previous post.
To rebuild the RAID you need to partition the new SSD and add the RAID partition to the degraded arrays with mdadm.

Code: Select all

mdadm --add /dev/partitionX /dev/mdX
You also need to set up the new EFI partition, but I don't have enough information about your specific configuration to tell how.

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#14 Post by Deekee »

Code: Select all

debconf-show grub-efi-amd64
reports the following:

Code: Select all

$ sudo debconf-show grub-efi-amd64
  grub2/no_efi_extra_removable: false
  grub2/linux_cmdline_default:
  grub2/update_nvram: true
  grub-efi/partition_description:
  grub-efi/install_devices_empty: false
  grub2/unsigned_kernels_title:
  grub-efi/install_devices_disks_changed:
  grub-efi/install_devices_failed: false
* grub-efi/install_devices: /dev/disk/by-id/scsi-3500a0751e62401b8-part1, /dev/disk/by-id/scsi-0ATA_CT1000MX500SSD1_2214E62489DB-part1
  grub2/kfreebsd_cmdline_default: quiet splash
  grub2/linux_cmdline:
  grub2/kfreebsd_cmdline:
  grub2/unsigned_kernels:
Here is the output of lsblk from an earlier state before I removed on SSD:

Code: Select all

$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
loop0         7:0    0    62M  1 loop  /snap/core20/1587
loop1         7:1    0  79.9M  1 loop  /snap/lxd/22923
loop2         7:2    0    47M  1 loop  /snap/snapd/16292
sda           8:0    0 931.5G  0 disk  
├─sda1        8:1    0     1G  0 part  
└─sda2        8:2    0 930.5G  0 part  
  └─md0       9:0    0 930.3G  0 raid1 
    └─md0p1 259:0    0 930.3G  0 part  /
sdb           8:16   0 931.5G  0 disk  
├─sdb1        8:17   0     1G  0 part  /boot/efi
└─sdb2        8:18   0 930.5G  0 part  
  └─md0       9:0    0 930.3G  0 raid1 
    └─md0p1 259:0    0 930.3G  0 part  /
Here was only one EFI partition (sdb1) mounted unfortunately I can't see what's on the other (sda1) anymore as I removed the SSD

I wonder how switching of the EFI boot partition is done when on SSD fails

Now with the removed SSD I got the following output from lsblk:

Code: Select all

$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
loop0         7:0    0    62M  1 loop  /snap/core20/1587
loop1         7:1    0  63.2M  1 loop  /snap/core20/1623
loop2         7:2    0  79.9M  1 loop  /snap/lxd/22923
loop3         7:3    0   103M  1 loop  /snap/lxd/23541
loop4         7:4    0    47M  1 loop  /snap/snapd/16292
loop5         7:5    0    48M  1 loop  /snap/snapd/16778
sda           8:0    0 931.5G  0 disk  
sdb           8:16   0 931.5G  0 disk  
├─sdb1        8:17   0     1G  0 part  /boot/efi
└─sdb2        8:18   0 930.5G  0 part  
  └─md0       9:0    0 930.3G  0 raid1 
    └─md0p1 259:0    0 930.3G  0 part  /
fdisk outputs the following:

Code: Select all

$ sudo fdisk -l
Disk /dev/loop0: 61.96 MiB, 64970752 bytes, 126896 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop1: 63.22 MiB, 66293760 bytes, 129480 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop2: 79.95 MiB, 83832832 bytes, 163736 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop3: 102.98 MiB, 107986944 bytes, 210912 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop4: 46.96 MiB, 49242112 bytes, 96176 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop5: 47.99 MiB, 50323456 bytes, 98288 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdb: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: A271B620-0356-4F13-9A2D-32A289362CA1

Device       Start        End    Sectors   Size Type
/dev/sdb1     2048    2203647    2201600     1G EFI System
/dev/sdb2  2203648 1953521663 1951318016 930.5G Linux filesystem


Disk /dev/md0: 930.33 GiB, 998939557888 bytes, 1951053824 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: BAC0295F-F953-4D32-8461-B51FCC3E874D

Device     Start        End    Sectors   Size Type
/dev/md0p1  2048 1951051775 1951049728 930.3G Linux filesystem
Btw I'm still working on the procedure on how rebuild the RAID, quite a step learning curve for a Unix beginner

Step 1: Check status of RAID

Code: Select all

$ sudo mdadm -D /dev/md0 
/dev/md0:
           Version : 1.2
     Creation Time : Thu Sep 15 09:06:00 2022
        Raid Level : raid1
        Array Size : 975526912 (930.33 GiB 998.94 GB)
     Used Dev Size : 975526912 (930.33 GiB 998.94 GB)
      Raid Devices : 2
     Total Devices : 1
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Sat Sep 17 08:41:28 2022
             State : active, degraded 
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : bitmap

              Name : ubuntu-server:0
              UUID : e3b73fb3:f7179803:669c7bb4:a27c053a
            Events : 3254

    Number   Major   Minor   RaidDevice State
       0       8       18        0      active sync   /dev/sdb2
       -       0        0        1      removed

STEP 2: Check partitions

Code: Select all

$ sudo fdisk -l
Disk /dev/loop0: 61.96 MiB, 64970752 bytes, 126896 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop1: 63.22 MiB, 66293760 bytes, 129480 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/loop2: 79.95 MiB, 83832832 bytes, 163736 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop3: 102.98 MiB, 107986944 bytes, 210912 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop4: 46.96 MiB, 49242112 bytes, 96176 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop5: 47.99 MiB, 50323456 bytes, 98288 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/sdb: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: A271B620-0356-4F13-9A2D-32A289362CA1

Device       Start        End    Sectors   Size Type
/dev/sdb1     2048    2203647    2201600     1G EFI System
/dev/sdb2  2203648 1953521663 1951318016 930.5G Linux filesystem


Disk /dev/md0: 930.33 GiB, 998939557888 bytes, 1951053824 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: BAC0295F-F953-4D32-8461-B51FCC3E874D

Device     Start        End    Sectors   Size Type
/dev/md0p1  2048 1951051775 1951049728 930.3G Linux filesystem
Wondering which files those pseudo devices /dev/loop0-5 make available as a block devices and for what the are used?

Code: Select all

$ sudo lsblk -f
NAME        FSTYPE            FSVER LABEL           UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
loop0       squashfs          4.0                                                              0   100% /snap/core20/1587
loop1       squashfs          4.0                                                              0   100% /snap/core20/1623
loop2       squashfs          4.0                                                              0   100% /snap/lxd/22923
loop3       squashfs          4.0                                                              0   100% /snap/lxd/23541
loop4       squashfs          4.0                                                              0   100% /snap/snapd/16292
loop5       squashfs          4.0                                                              0   100% /snap/snapd/16778
II f boot SSD's are active it seems that the following loops belong to sdb:

Code: Select all

loop0       squashfs          4.0                                                              0   100% /snap/core20/1587
loop2       squashfs          4.0                                                              0   100% /snap/lxd/22923
loop4       squashfs          4.0                                                              0   100% /snap/snapd/16292
and the others to sda:

Code: Select all

loop1       squashfs          4.0                                                              0   100% /snap/core20/1623
loop3       squashfs          4.0                                                              0   100% /snap/lxd/23541
loop5       squashfs          4.0                                                              0   100% /snap/snapd/16778

STEP 3: Backup and load partition table and randomize GUID

Code: Select all

$ sudo sgdisk --backup=table /dev/sdb
The operation has completed successfully.

$ sudo sgdisk --load-backup=table /dev/sda
Creating new GPT entries in memory.
The operation has completed successfully.

$ sudo sgdisk -G /dev/sda
The operation has completed successfully.
... and check if they have been added

Code: Select all

$ sudo ls -lah /dev/sd*
brw-rw---- 1 root disk 8,  0 Sep 17 14:54 /dev/sda
brw-rw---- 1 root disk 8,  1 Sep 17 14:54 /dev/sda1
brw-rw---- 1 root disk 8,  2 Sep 17 14:54 /dev/sda2
brw-rw---- 1 root disk 8, 16 Sep 17 14:50 /dev/sdb
brw-rw---- 1 root disk 8, 17 Sep 17 14:50 /dev/sdb1
brw-rw---- 1 root disk 8, 18 Sep 17 14:50 /dev/sdb2

STEP 4: Add new disk to RAID

Code: Select all

$ sudo mdadm --add /dev/md0 /dev/sda2
mdadm: added /dev/sda2

STEP 5: Watch resync process

Code: Select all

$ sudo cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sda2[2](F) sdb2[0]
      975526912 blocks super 1.2 [2/1] [U_]
      bitmap: 8/8 pages [32KB], 65536KB chunk

unused devices: <none>

STEP 6: Write bootloader

Well here I'm getting a bit unsure and busy googling what would be the correct procedure...

I think to just give it a go as there is not much to loose and I can always start all over again, but I got stuck at Step 5

So that's it for the time being, will be back tomorrow :-)
Last edited by Deekee on 2022-09-17 15:50, edited 3 times in total.

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#15 Post by p.H »

Deekee wrote: 2022-09-17 10:19 grub-efi/install_devices: /dev/disk/by-id/scsi-3500a0751e62401b8-part1, /dev/disk/by-id/scsi-0ATA_CT1000MX500SSD1_2214E62489DB-part1
As I suspected, the EFI partitions are recorded in debconf. The Debian bullseye package lacks this. Maybe in testing/sid ?
But according to earlier lsblk output only one EFI partition (sdb1) was mounted, so I wonder how does the package install or upgrade GRUB in sda1.
Can you show the contents of /etc/fstab ?
Too bad you removed the old SSD... It could have been useful to compare the UUIDs and PARTUUIDs.
Deekee wrote: 2022-09-17 10:19 I wonder how switching of the EFI boot partition is done when on SSD fails
The UEFI firmware will boot using any available EFI partition.
Deekee wrote: 2022-09-17 10:19 Wondering which files those pseudo devices /dev/loop0-5 make available as a block devices and for what the are used?
SquashFS filesystem images containing Snap packages. They neither belong to sda or sdb but md0 (RAID array containing the root filesystem).
Deekee wrote: 2022-09-17 10:19 STEP 4: Add new disk to RAID

Code: Select all

mdadm --add /dev/md0 /dev/sda1
mdadm --add /dev/md1 /dev/sda2
Err... no.
- /dev/sda1 is supposed to be an EFI partition, not a RAID member. It must be formatted as FAT.
- AFAICS there is no /dev/md1 RAID array, only /dev/md0.
Deekee wrote: 2022-09-17 10:19 STEP 6: Write bootloader
Well here I'm getting a bit unsure
From what I can see, that should be done by reconfiguring grub-efi-amd64

Code: Select all

dpkg-reconfigure grub-efi-amd64
You should then have the opportunity to include the new EFI partition.

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#16 Post by Deekee »

Her the output of /etc/fstab

Code: Select all

$ cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/md0p1 during curtin installation
/dev/disk/by-id/md-uuid-e3b73fb3:f7179803:669c7bb4:a27c053a-part1 / ext4 defaults 0 1
# /boot/efi was on /dev/sdb1 during curtin installation
/dev/disk/by-uuid/AF01-AC43 /boot/efi vfat defaults 0 1
/swap.img	none	swap	sw	0	0
Yeah, you'r right with step 4 my mistake

Now I'm stuck @ Step 5, as the RAID still shows

Code: Select all

[U_]
I probabyl have to format sda1 and sda2 first before the raid will start to reynch..

Will continue tomorrow

Have a nice evening and thanks so far for the help, learning a lot, ver much appreciated :-)

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#17 Post by p.H »

Deekee wrote: 2022-09-17 16:01 Her the output of /etc/fstab
It mounts only one EFI partition (on /boot/efi as usual). So I still don't know how GRUB is upgraded on the other one. Maybe the partition is mounted temporarily.
Deekee wrote: 2022-09-17 16:01 Now I'm stuck @ Step 5, as the RAID still shows [U_]
Can you show the full content of /proc/mdstat ?
Deekee wrote: 2022-09-17 16:01 I probabyl have to format sda1 and sda2 first before the raid will start to reynch..
Only sda1 (as FAT). As a RAID member, sda2 does not need to be formatted.

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#18 Post by Deekee »

After a restart suddenly sda became active

Code: Select all

$ cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sda2[0]
      975526912 blocks super 1.2 [2/1] [U_]
      bitmap: 8/8 pages [32KB], 65536KB chunk

unused devices: <none>
And sdb is even not listed in /dev

Code: Select all

$ ls -lah /dev/sd*
brw-rw---- 1 root disk 8, 0 Sep 18 05:37 /dev/sda
brw-rw---- 1 root disk 8, 1 Sep 18 05:37 /dev/sda1
brw-rw---- 1 root disk 8, 2 Sep 18 05:37 /dev/sda2
When rebooting there was a very long delay and think to have broken the system by now and will start over again

Code: Select all

[    6.755065] kernel: ata1: link is slow to respond, please be patient (ready=0)
[   11.435047] kernel: ata1: COMRESET failed (errno=-16)
[   16.791063] kernel: ata1: link is slow to respond, please be patient (ready=0)
[   21.471067] kernel: ata1: COMRESET failed (errno=-16)
[   26.827049] kernel: ata1: link is slow to respond, please be patient (ready=0)
[   56.527057] kernel: ata1: COMRESET failed (errno=-16)
[   56.530554] kernel: ata1: limiting SATA link speed to 3.0 Gbps
[   61.587085] kernel: ata1: COMRESET failed (errno=-16)
[   61.590626] kernel: ata1: reset failed, giving up
[   61.594597] kernel: scsi 1:0:0:0: Direct-Access     ATA      CT1000MX500SSD1  043  PQ: 0 ANSI: 5
[   61.598691] kernel: sd 1:0:0:0: Attached scsi generic sg0 type 0
[   61.598819] kernel: sd 1:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)
[   61.607371] kernel: sd 1:0:0:0: [sda] 4096-byte physical blocks
[   61.610852] kernel: sd 1:0:0:0: [sda] Write Protect is off
[   61.614309] kernel: sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00
[   61.614342] kernel: sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   61.647573] kernel:  sda: sda1 sda2
[   61.671351] kernel: sd 1:0:0:0: [sda] supports TCG Opal
[   61.673349] kernel: sd 1:0:0:0: [sda] Attached SCSI disk
[   61.685401] kernel: input: Apple, Inc. IR Receiver as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.8/2-1.8.2/2-1.8.2:1.0/0003:05AC:8242.0002/input/input7
[   61.747200] kernel: appleir 0003:05AC:8242.0002: input,hiddev0,hidraw4: USB HID v1.11 Device [Apple, Inc. IR Receiver] on usb-0000:00:1d.0-1.8.2/input0
[   61.764155] kernel: checking generic (90000000 7e9000) vs hw (a0000000 400000)
[   61.764159] kernel: checking generic (90000000 7e9000) vs hw (90000000 10000000)
[   61.764160] kernel: fb0: switching to i915 from EFI VGA
[   61.765690] kernel: Console: switching to colour dummy device 80x25
[   61.765734] kernel: i915 0000:00:02.0: vgaarb: deactivate vga console
[   61.797140] kernel: md/raid1:md0: active with 1 out of 2 mirrors
[   61.798174] kernel: md0: detected capacity change from 0 to 1951053824
[   61.799399] kernel:  md0: p1
[   61.836744] kernel: [drm] Initialized i915 1.6.0 20201103 for 0000:00:02.0 on minor 0
[   61.837807] kernel: ACPI: video: Video Device [IGPU] (multi-head: yes  rom: no  post: no)
[   61.837986] kernel: input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input8
[   62.054357] kernel: fbcon: i915drmfb (fb0) is primary device
[   62.155811] kernel: Console: switching to colour frame buffer device 240x67
[   62.182951] kernel: i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
[   62.559006] kernel: raid6: sse2x4   gen() 14939 MB/s
[   62.627007] kernel: raid6: sse2x4   xor()  5579 MB/s
[   62.695006] kernel: raid6: sse2x2   gen() 14764 MB/s
[   62.763006] kernel: raid6: sse2x2   xor()  9111 MB/s
[   62.831007] kernel: raid6: sse2x1   gen() 14138 MB/s
[   62.899006] kernel: raid6: sse2x1   xor()  8422 MB/s
[   62.899022] kernel: raid6: using algorithm sse2x4 gen() 14939 MB/s
[   62.899051] kernel: raid6: .... xor() 5579 MB/s, rmw enabled
[   62.899067] kernel: raid6: using ssse3x2 recovery algorithm
[   62.901275] kernel: xor: automatically using best checksumming function   avx       
[   62.902187] kernel: async_tx: api initialized (async)
[   62.969353] kernel: Btrfs loaded, crc32c=crc32c-intel, zoned=yes, fsverity=yes
[   63.034622] kernel: EXT4-fs (md0p1): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[   63.245237] systemd[1]: Inserted module 'autofs4'

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#19 Post by p.H »

Deekee wrote: 2022-09-18 06:16 After a restart suddenly sda became active
Or this is the former sdb, and the former sda did not respond. Device names sd* are not stable and depend on the discovery order (which is not deterministic), so they may be swapped at each boot. You can compare the UUIDs, PARTUUIDs, model and serial number.

Deekee
Posts: 91
Joined: 2022-07-02 17:50
Has thanked: 5 times
Been thanked: 3 times

Re: MacMini RAID1 ESP UEFI (SOLVED)

#20 Post by Deekee »

Ok, my system is back up and running nicely again :-)

Now I will try list all the data I could collect and which also just suggested to do even from the now reconnected second disk :-)

Code: Select all

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.6G  1.4M  1.6G   1% /run
/dev/md0p1      915G  6.8G  862G   1% /
tmpfs           7.8G     0  7.8G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/sdb1       1.1G  5.3M  1.1G   1% /boot/efi
tmpfs           1.6G  4.0K  1.6G   1% /run/user/1000

Code: Select all

$ lsblk
NAME        MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
loop0         7:0    0    62M  1 loop  /snap/core20/1587
loop1         7:1    0  79.9M  1 loop  /snap/lxd/22923
loop2         7:2    0    47M  1 loop  /snap/snapd/16292
sda           8:0    0 931.5G  0 disk  
├─sda1        8:1    0     1G  0 part  /otherboot
└─sda2        8:2    0 930.5G  0 part  
  └─md0       9:0    0 930.3G  0 raid1 
    └─md0p1 259:0    0 930.3G  0 part  /
sdb           8:16   0 931.5G  0 disk  
├─sdb1        8:17   0     1G  0 part  /boot/efi
└─sdb2        8:18   0 930.5G  0 part  
  └─md0       9:0    0 930.3G  0 raid1 
    └─md0p1 259:0    0 930.3G  0 part  /

Code: Select all

$ blkid /dev/sda1
/dev/sda1: UUID="2F57-ABFB" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="f7ca809b-fd13-4736-b599-c58846d28e4b"
$ blkid /dev/sda2
/dev/sda2: UUID="7873b6db-38fb-5528-2067-b03db0f6341b" UUID_SUB="9b0ef05c-1ef1-2418-ded5-f78e47d0360d" LABEL="ubuntu-server:0" TYPE="linux_raid_member" PARTUUID="16822710-8a1b-4ee9-b7ed-a9ac5c3d9604"
$ blkid /dev/sdb1
/dev/sdb1: UUID="2F34-5025" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="16d57ea3-e9a0-4928-aa37-cc6a4257eda4"
$ blkid /dev/sdb2
/dev/sdb2: UUID="7873b6db-38fb-5528-2067-b03db0f6341b" UUID_SUB="15d93a50-7d4d-d5f8-88d0-3208c81d6b7d" LABEL="ubuntu-server:0" TYPE="linux_raid_member" PARTUUID="eff9e8aa-6fae-4bdb-b757-e8514ae79f87"

Code: Select all

$ cat /etc/fstab
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
# / was on /dev/md0p1 during curtin installation
/dev/disk/by-id/md-uuid-7873b6db:38fb5528:2067b03d:b0f6341b-part1 / ext4 defaults 0 1
# /boot/efi was on /dev/sdb1 during curtin installation
/dev/disk/by-uuid/2F34-5025 /boot/efi vfat defaults 0 1
/swap.img	none	swap	sw	0	0

Code: Select all

$ sudo fdisk -l
Disk /dev/loop0: 61.96 MiB, 64970752 bytes, 126896 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop1: 79.95 MiB, 83832832 bytes, 163736 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop2: 46.96 MiB, 49242112 bytes, 96176 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: DC1CC795-962C-4EEF-9A7B-8D1DCB4BC148

Device       Start        End    Sectors   Size Type
/dev/sdb1     2048    2203647    2201600     1G EFI System
/dev/sdb2  2203648 1953521663 1951318016 930.5G Linux filesystem


Disk /dev/sda: 931.51 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: CT1000MX500SSD1 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 5EB5F715-4345-4ADF-8F1E-8FB3528A920B

Device       Start        End    Sectors   Size Type
/dev/sda1     2048    2203647    2201600     1G EFI System
/dev/sda2  2203648 1953521663 1951318016 930.5G Linux filesystem


Disk /dev/md0: 930.33 GiB, 998939557888 bytes, 1951053824 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: B6D70E03-E16D-470F-9F61-3B4FB7D7857E

Device     Start        End    Sectors   Size Type
/dev/md0p1  2048 1951051775 1951049728 930.3G Linux filesystem

Code: Select all

$ efibootmgr -v
BootCurrent: 0001
Timeout: 5 seconds
BootOrder: 0000,0001,0002
Boot0000* debian	HD(1,GPT,b67c0b2e-3921-45b2-83a6-2f8021c01ecc,0x800,0x1dc800)/File(\EFI\debian\shimx64.efi)
Boot0001* ubuntu	HD(1,GPT,16d57ea3-e9a0-4928-aa37-cc6a4257eda4,0x800,0x219800)/File(\EFI\ubuntu\shimx64.efi)
Boot0002* ubuntu	HD(1,GPT,f7ca809b-fd13-4736-b599-c58846d28e4b,0x800,0x219800)/File(\EFI\ubuntu\shimx64.efi)
Boot0080* 	PciRoot(0x0)/Pci(0x1f,0x2)/Sata(1,0,0)/HD(1,GPT,fbb8e9c7-869a-4bf6-8dfe-eb56718c1a6c,0x40,0x1fc0)/File(\EFI\BOOT\BOOTX64.efi)
BootFFFF* 	PciRoot(0x0)/Pci(0x1f,0x2)/Sata(1,0,0)/HD(1,GPT,fbb8e9c7-869a-4bf6-8dfe-eb56718c1a6c,0x40,0x1fc0)/File(\EFI\BOOT\BOOTX64.efi)
Contents of /dev/sdb1 (Current boot disk)

Code: Select all

$ ls -al /boot
total 122388
drwxr-xr-x  4 root root      4096 Sep 18 07:06 .
drwxr-xr-x 19 root root      4096 Sep 18 07:08 ..
-rw-r--r--  1 root root    261841 Aug 10 07:49 config-5.15.0-47-generic
drwxr-xr-x  3 root root      4096 Jan  1  1970 efi
drwxr-xr-x  5 root root      4096 Sep 18 07:06 grub
lrwxrwxrwx  1 root root        28 Sep 18 07:05 initrd.img -> initrd.img-5.15.0-47-generic
-rw-r--r--  1 root root 107265163 Sep 18 07:06 initrd.img-5.15.0-47-generic
lrwxrwxrwx  1 root root        28 Sep 18 07:05 initrd.img.old -> initrd.img-5.15.0-47-generic
-rw-------  1 root root   6245466 Aug 10 07:49 System.map-5.15.0-47-generic
lrwxrwxrwx  1 root root        25 Sep 18 07:05 vmlinuz -> vmlinuz-5.15.0-47-generic
-rw-------  1 root root  11530816 Aug 11 07:44 vmlinuz-5.15.0-47-generic
lrwxrwxrwx  1 root root        25 Sep 18 07:05 vmlinuz.old -> vmlinuz-5.15.0-47-generic 

Code: Select all

$ ls -al /boot/efi
total 12
drwxr-xr-x 3 root root 4096 Jan  1  1970 .
drwxr-xr-x 4 root root 4096 Sep 18 07:06 ..
drwxr-xr-x 4 root root 4096 Sep 18 07:06 EFI

Code: Select all

$ ls -al /boot/efi/EFI
total 16
drwxr-xr-x 4 root root 4096 Sep 18 07:06 .
drwxr-xr-x 3 root root 4096 Jan  1  1970 ..
drwxr-xr-x 2 root root 4096 Sep 18 07:06 BOOT
drwxr-xr-x 2 root root 4096 Sep 18 07:06 ubuntu

Code: Select all

$ ls -al /boot/efi/EFI/BOOT
total 1868
drwxr-xr-x 2 root root   4096 Sep 18 07:06 .
drwxr-xr-x 4 root root   4096 Sep 18 07:06 ..
-rwxr-xr-x 1 root root 955656 Sep 18 07:06 BOOTX64.EFI
-rwxr-xr-x 1 root root  85672 Sep 18 07:06 fbx64.efi
-rwxr-xr-x 1 root root 856232 Sep 18 07:06 mmx64.efi

Code: Select all

$ ls -al /boot/efi/EFI/ubuntu
total 3496
drwxr-xr-x 2 root root    4096 Sep 18 07:06 .
drwxr-xr-x 4 root root    4096 Sep 18 07:06 ..
-rwxr-xr-x 1 root root     108 Sep 18 07:06 BOOTX64.CSV
-rwxr-xr-x 1 root root     162 Sep 18 07:06 grub.cfg
-rwxr-xr-x 1 root root 1742728 Sep 18 07:06 grubx64.efi
-rwxr-xr-x 1 root root  856232 Sep 18 07:06 mmx64.efi
-rwxr-xr-x 1 root root  955656 Sep 18 07:06 shimx64.efi

Code: Select all

$ ls -al /boot/grub
total 2392
drwxr-xr-x 5 root root    4096 Sep 18 07:06 .
drwxr-xr-x 4 root root    4096 Sep 18 07:06 ..
drwxr-xr-x 2 root root    4096 Sep 18 07:06 fonts
-rw-r--r-- 1 root root    7626 Sep 18 07:06 grub.cfg
-rw-r--r-- 1 root root    1024 Sep 18 07:15 grubenv
drwxr-xr-x 2 root root    4096 Sep 18 07:06 locale
-rw-r--r-- 1 root root 2392289 Sep 18 07:06 unicode.pf2
drwxr-xr-x 2 root root   20480 Sep 18 07:06 x86_64-efi
Now I mounted the other boot partition.

Code: Select all

$ sudo mkdir otherboot
$ sudo mount /dev/sda1 /otherboot
Contents of /dev/sda1

Code: Select all

$ ls -al /otherboot
total 12
drwxr-xr-x  3 root root 4096 Jan  1  1970 .
drwxr-xr-x 20 root root 4096 Sep 18 09:20 ..
drwxr-xr-x  4 root root 4096 Sep 18 07:06 EFI

Code: Select all

$ ls -al /otherboot/EFI
total 16
drwxr-xr-x 4 root root 4096 Sep 18 07:06 .
drwxr-xr-x 3 root root 4096 Jan  1  1970 ..
drwxr-xr-x 2 root root 4096 Sep 18 07:06 BOOT
drwxr-xr-x 2 root root 4096 Sep 18 07:06 ubuntu

Code: Select all

$ ls -al /otherboot/EFI/BOOT
total 1868
drwxr-xr-x 2 root root   4096 Sep 18 07:06 .
drwxr-xr-x 4 root root   4096 Sep 18 07:06 ..
-rwxr-xr-x 1 root root 955656 Sep 18 07:06 BOOTX64.EFI
-rwxr-xr-x 1 root root  85672 Sep 18 07:06 fbx64.efi
-rwxr-xr-x 1 root root 856232 Sep 18 07:06 mmx64.efi

Code: Select all

$ ls -al /otherboot/EFI/ubuntu
total 3496
drwxr-xr-x 2 root root    4096 Sep 18 07:06 .
drwxr-xr-x 4 root root    4096 Sep 18 07:06 ..
-rwxr-xr-x 1 root root     108 Sep 18 07:06 BOOTX64.CSV
-rwxr-xr-x 1 root root     162 Sep 18 07:06 grub.cfg
-rwxr-xr-x 1 root root 1742728 Sep 18 07:06 grubx64.efi
-rwxr-xr-x 1 root root  856232 Sep 18 07:06 mmx64.efi
-rwxr-xr-x 1 root root  955656 Sep 18 07:06 shimx64.efi
I think, the only difference in the current boot partition is that the EFI directory lies within /boot/efi and contains all the files for booting, otherwise all files are the same, as diff returns nothing at all.

Code: Select all

$ diff -Nrq /boot/efi /otherboot
$

Code: Select all

$ diff -r /boot /otherboot
Only in /boot: config-5.15.0-47-generic
Only in /boot: efi
Only in /otherboot: EFI
Only in /boot: grub
Only in /boot: initrd.img
Only in /boot: initrd.img-5.15.0-47-generic
Only in /boot: initrd.img.old
Only in /boot: System.map-5.15.0-47-generic
Only in /boot: vmlinuz
Only in /boot: vmlinuz-5.15.0-47-generic
Only in /boot: vmlinuz.old
Contents of /boot/grub/grub.cfg

Code: Select all

$ sudo debconf-show grub-efi-amd64
  grub2/no_efi_extra_removable: false
  grub-efi/partition_description:
  grub2/kfreebsd_cmdline_default: quiet splash
* grub-efi/install_devices: /dev/disk/by-id/wwn-0x500a0751e62401b8-part1, /dev/disk/by-id/wwn-0x500a0751e62415f0-part1
  grub2/linux_cmdline:
  grub2/update_nvram: true
  grub-efi/install_devices_empty: false
  grub-efi/install_devices_disks_changed:
  grub-efi/install_devices_failed: false
  grub2/kfreebsd_cmdline:
  grub2/unsigned_kernels_title:
  grub2/unsigned_kernels:
  grub2/linux_cmdline_default:

Code: Select all

$ cat grub.cfg
#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
if [ -s $prefix/grubenv ]; then
  set have_grubenv=true
  load_env
fi
if [ "${initrdfail}" = 2 ]; then
   set initrdfail=
elif [ "${initrdfail}" = 1 ]; then
   set next_entry="${prev_entry}"
   set prev_entry=
   save_env prev_entry
   if [ "${next_entry}" ]; then
      set initrdfail=2
   fi
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="0"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}
function initrdfail {
    if [ -n "${have_grubenv}" ]; then if [ -n "${partuuid}" ]; then
      if [ -z "${initrdfail}" ]; then
        set initrdfail=1
        if [ -n "${boot_once}" ]; then
          set prev_entry="${default}"
          save_env prev_entry
        fi
      fi
      save_env initrdfail
    fi; fi
}
function recordfail {
  set recordfail=1
  # GRUB lacks write support for diskfilter, so recordfail support is disabled.
}
function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

if [ x$feature_default_font_path = xy ] ; then
   font=unicode
else
insmod part_gpt
insmod part_gpt
insmod part_gpt
insmod diskfilter
insmod mdraid1x
insmod ext2
set root='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'  f08fcff6-3957-414a-a9fc-4a50768b2cc7
else
  search --no-floppy --fs-uuid --set=root f08fcff6-3957-414a-a9fc-4a50768b2cc7
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=C
  insmod gettext
fi
terminal_output gfxterm
if [ "${recordfail}" = 1 ] ; then
  set timeout=30
else
  if [ x$feature_timeout_style = xy ] ; then
    set timeout_style=hidden
    set timeout=0
  # Fallback hidden-timeout code in case the timeout_style feature is
  # unavailable.
  elif sleep --interruptible 0 ; then
    set timeout=0
  fi
fi
if [ $grub_platform = efi ]; then
  set timeout=30
  if [ x$feature_timeout_style = xy ] ; then
    set timeout_style=menu
  fi
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/05_debian_theme ###
set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
### END /etc/grub.d/05_debian_theme ###

### BEGIN /etc/grub.d/10_linux ###
function gfxmode {
	set gfxpayload="${1}"
	if [ "${1}" = "keep" ]; then
		set vt_handoff=vt.handoff=7
	else
		set vt_handoff=
	fi
}
if [ "${recordfail}" != 1 ]; then
  if [ -e ${prefix}/gfxblacklist.txt ]; then
    if [ ${grub_platform} != pc ]; then
      set linux_gfx_mode=keep
    elif hwmatch ${prefix}/gfxblacklist.txt 3; then
      if [ ${match} = 0 ]; then
        set linux_gfx_mode=keep
      else
        set linux_gfx_mode=text
      fi
    else
      set linux_gfx_mode=text
    fi
  else
    set linux_gfx_mode=keep
  fi
else
  set linux_gfx_mode=text
fi
export linux_gfx_mode
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-f08fcff6-3957-414a-a9fc-4a50768b2cc7' {
	recordfail
	load_video
	gfxmode $linux_gfx_mode
	insmod gzio
	if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
	insmod part_gpt
	insmod part_gpt
	insmod part_gpt
	insmod diskfilter
	insmod mdraid1x
	insmod ext2
	set root='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'  f08fcff6-3957-414a-a9fc-4a50768b2cc7
	else
	  search --no-floppy --fs-uuid --set=root f08fcff6-3957-414a-a9fc-4a50768b2cc7
	fi
	linux	/boot/vmlinuz-5.15.0-47-generic root=UUID=f08fcff6-3957-414a-a9fc-4a50768b2cc7 ro  
	initrd	/boot/initrd.img-5.15.0-47-generic
}
submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-f08fcff6-3957-414a-a9fc-4a50768b2cc7' {
	menuentry 'Ubuntu, with Linux 5.15.0-47-generic' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.15.0-47-generic-advanced-f08fcff6-3957-414a-a9fc-4a50768b2cc7' {
		recordfail
		load_video
		gfxmode $linux_gfx_mode
		insmod gzio
		if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
		insmod part_gpt
		insmod part_gpt
		insmod part_gpt
		insmod diskfilter
		insmod mdraid1x
		insmod ext2
		set root='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'  f08fcff6-3957-414a-a9fc-4a50768b2cc7
		else
		  search --no-floppy --fs-uuid --set=root f08fcff6-3957-414a-a9fc-4a50768b2cc7
		fi
		echo	'Loading Linux 5.15.0-47-generic ...'
		linux	/boot/vmlinuz-5.15.0-47-generic root=UUID=f08fcff6-3957-414a-a9fc-4a50768b2cc7 ro  
		echo	'Loading initial ramdisk ...'
		initrd	/boot/initrd.img-5.15.0-47-generic
	}
	menuentry 'Ubuntu, with Linux 5.15.0-47-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.15.0-47-generic-recovery-f08fcff6-3957-414a-a9fc-4a50768b2cc7' {
		recordfail
		load_video
		insmod gzio
		if [ x$grub_platform = xxen ]; then insmod xzio; insmod lzopio; fi
		insmod part_gpt
		insmod part_gpt
		insmod part_gpt
		insmod diskfilter
		insmod mdraid1x
		insmod ext2
		set root='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'
		if [ x$feature_platform_search_hint = xy ]; then
		  search --no-floppy --fs-uuid --set=root --hint='mduuid/7873b6db38fb55282067b03db0f6341b,gpt1'  f08fcff6-3957-414a-a9fc-4a50768b2cc7
		else
		  search --no-floppy --fs-uuid --set=root f08fcff6-3957-414a-a9fc-4a50768b2cc7
		fi
		echo	'Loading Linux 5.15.0-47-generic ...'
		linux	/boot/vmlinuz-5.15.0-47-generic root=UUID=f08fcff6-3957-414a-a9fc-4a50768b2cc7 ro recovery nomodeset dis_ucode_ldr 
		echo	'Loading initial ramdisk ...'
		initrd	/boot/initrd.img-5.15.0-47-generic
	}
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/10_linux_zfs ###
### END /etc/grub.d/10_linux_zfs ###

### BEGIN /etc/grub.d/20_linux_xen ###

### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/35_fwupd ###
### END /etc/grub.d/35_fwupd ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg
fi
### END /etc/grub.d/41_custom ###

Code: Select all

$ cat /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=hidden
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_CMDLINE_LINUX=""

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"
Status of RAID1

Code: Select all

$ cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sda2[1] sdb2[0]
      975526912 blocks super 1.2 [2/2] [UU]
      bitmap: 0/8 pages [0KB], 65536KB chunk
And some details of RAID1

Code: Select all

$ cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10] 
md0 : active raid1 sda2[1] sdb2[0]
      975526912 blocks super 1.2 [2/2] [UU]
      bitmap: 0/8 pages [0KB], 65536KB chunk

unused devices: <none>
dirk@macmini:/etc/default$ sudo mdadm --query --detail /dev/md*
[sudo] password for dirk: 
/dev/md0:
           Version : 1.2
     Creation Time : Sun Sep 18 07:04:26 2022
        Raid Level : raid1
        Array Size : 975526912 (930.33 GiB 998.94 GB)
     Used Dev Size : 975526912 (930.33 GiB 998.94 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Sun Sep 18 14:10:20 2022
             State : active 
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : bitmap

              Name : ubuntu-server:0
              UUID : 7873b6db:38fb5528:2067b03d:b0f6341b
            Events : 8385

    Number   Major   Minor   RaidDevice State
       0       8       18        0      active sync   /dev/sdb2
       1       8        2        1      active sync   /dev/sda2
/dev/md0p1:
           Version : 1.2
     Creation Time : Sun Sep 18 07:04:26 2022
        Raid Level : raid1
        Array Size : 975524864 (930.33 GiB 998.94 GB)
     Used Dev Size : 975526912 (930.33 GiB 998.94 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Sun Sep 18 14:10:20 2022
             State : active 
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : bitmap

              Name : ubuntu-server:0
              UUID : 7873b6db:38fb5528:2067b03d:b0f6341b
            Events : 8386

    Number   Major   Minor   RaidDevice State
       0       8       18        0      active sync   /dev/sdb2
       1       8        2        1      active sync   /dev/sda2
Some more info about each partition

Code: Select all

$ sudo file -sL /dev/sd*
/dev/sda:  DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS (0x0,0,2), end-CHS (0x3ff,255,63), startsector 1, 1953525167 sectors, extended partition table (last)
/dev/sda1: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "mkfs.fat", sectors/cluster 8, Media descriptor 0xf8, sectors/track 63, heads 255, hidden sectors 2048, sectors 2201598 (volumes > 32 MB), FAT (32 bit), sectors/FAT 2152, reserved 0x1, serial number 0x2f57abfb, unlabeled
/dev/sda2: Linux Software RAID version 1.2 (1) UUID=7873b6db:38fb5528:2067b03d:b0f6341b name=ubuntu-server:0 level=1 disks=2
/dev/sdb:  DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS (0x0,0,2), end-CHS (0x3ff,255,63), startsector 1, 1953525167 sectors, extended partition table (last)
/dev/sdb1: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "mkfs.fat", sectors/cluster 8, Media descriptor 0xf8, sectors/track 63, heads 255, hidden sectors 2048, sectors 2201598 (volumes > 32 MB), FAT (32 bit), sectors/FAT 2152, reserved 0x1, serial number 0x2f345025, unlabeled
/dev/sdb2: Linux Software RAID version 1.2 (1) UUID=7873b6db:38fb5528:2067b03d:b0f6341b name=ubuntu-server:0 level=1 disks=2
Some more info about the RAID1

Code: Select all

$ sudo file -sL /dev/md*
/dev/md0:   DOS/MBR boot sector; partition 1 : ID=0xee, start-CHS (0x0,0,2), end-CHS (0x3ff,255,63), startsector 1, 1951053823 sectors, extended partition table (last)
/dev/md0p1: Linux rev 1.0 ext4 filesystem data, UUID=f08fcff6-3957-414a-a9fc-4a50768b2cc7 (needs journal recovery) (extents) (64bit) (large files) (huge files)

Now, I hope to have listed and documented all the information I need to do a successfull recovery on my 2nd try :-)

I will now disconnect one SSD again and try to recover...

CU and in the meantime have a nice and relaxing Sunday :-)

Post Reply