Raid5 does't start after blackout

[miky76]

Good Afternoon,
after a blackout, rebooting my linux Debian machine is not possible to make the raid5 group to start.
can you help me in what to do?
before do any action on the raid I want to be sure not to destroy the data on it

thank you in advance....

some info below about my raid5 with 3 disks

# mount /dev/md0 /raid
mount: special device /dev/md0 does not exist


# mdadm --assemble --scan
mdadm: /dev/md/0 assembled from 2 drives - not enough to start the array while not clean - consider --force.

# mdadm --examine --brief --scan
ARRAY /dev/md/0 metadata=1.2 UUID=582394e9:85ddedf1:1c3a63b5:cc9a6f0b name=artemide:0

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
unused devices: <none>

# blkid
/dev/sde: UUID="582394e9-85dd-edf1-1c3a-63b5cc9a6f0b" UUID_SUB="bc71213d-a98e-1b82-9d4b-aa5a16de33ef" LABEL="artemide:0" TYPE="linux_raid_member"
/dev/sda1: UUID="862B-0E38" TYPE="vfat" PARTUUID="d4b81290-5062-4ddd-a48b-90c1382987ee"
/dev/sda2: UUID="415c7d12-0d9e-4723-be4f-f54936ab4544" TYPE="ext4" PARTUUID="a55eda9b-85ce-412b-8fbd-9ae870dcb8ca"
/dev/sda3: UUID="067b788c-57fa-404b-afb8-351bda99bdd9" TYPE="swap" PARTUUID="c8bf43db-4565-430b-95b6-e2654004f319"
/dev/sdb: UUID="582394e9-85dd-edf1-1c3a-63b5cc9a6f0b" UUID_SUB="dcb52e3e-c520-8223-b3cd-f2c0f0d55ef0" LABEL="artemide:0" TYPE="linux_raid_member"
/dev/sdc1: UUID="af3414db-4591-48d1-92e1-396ec27e6f5f" TYPE="ext4" PARTUUID="9c5864df-206f-467e-b061-2019ee05a9e7"
/dev/sdd: UUID="TOaf2q-Wvfe-DWck-T50Y-7ZaB-g3RV-fnNYng" TYPE="LVM2_member"
/dev/mapper/vg--rbackup-mikbackup: UUID="00e22c83-e0e8-468b-aafa-27641bc3e7df" TYPE="ext3"
/dev/sdf: UUID="582394e9-85dd-edf1-1c3a-63b5cc9a6f0b" UUID_SUB="079d31f9-ec8d-6ce8-21e9-3cc1850f6c4d" LABEL="artemide:0" TYPE="linux_raid_member"
/dev/mapper/vg--rbackup-giubackup: UUID="f2ef8318-b29b-4b63-93f1-90dfbe94da3f" TYPE="ext3"

[p.H]

before do any action on the raid I want to be sure not to destroy the data on it

You have a backup don't you ? RAID does not exempt from doing backups.

Gather information with

Code: Select all

mdadm --examine /dev/sd?

[miky76]

I have an old backup...

Code: Select all

# mdadm -Q /dev/sdf
/dev/sdf: is not an md array
/dev/sdf: device 1 in 3 device inactive raid5 array.  Use mdadm --examine for more detail.

# mdadm -Q /dev/sde
/dev/sde: is not an md array
/dev/sde: device 3 in 3 device inactive raid5 array.  Use mdadm --examine for more detail.

# mdadm -Q /dev/sdb
/dev/sdb: is not an md array
/dev/sdb: device 0 in 3 device inactive raid5 array.  Use mdadm --examine for more detail.


I'm not sure the 3 disks show differents "Array State"

below the info about the disks

Code: Select all

# mdadm --examine /dev/sdb
/dev/sdb:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 582394e9:85ddedf1:1c3a63b5:cc9a6f0b
           Name : artemide:0  (local to host artemide)
  Creation Time : Tue Jan  7 23:16:02 2014
     Raid Level : raid5
   Raid Devices : 3

 Avail Dev Size : 5860271024 (2794.40 GiB 3000.46 GB)
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 5860270080 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=944 sectors
          State : active
    Device UUID : dcb52e3e:c5208223:b3cdf2c0:f0d55ef0

Internal Bitmap : 8 sectors from superblock
    Update Time : Thu Jan 31 00:17:01 2019
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 8d726d03 - correct
         Events : 9424009

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 0
   Array State : A.A ('A' == active, '.' == missing, 'R' == replacing)


Code: Select all

y# mdadm --examine /dev/sde
/dev/sde:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 582394e9:85ddedf1:1c3a63b5:cc9a6f0b
           Name : artemide:0  (local to host artemide)
  Creation Time : Tue Jan  7 23:16:02 2014
     Raid Level : raid5
   Raid Devices : 3

 Avail Dev Size : 5860271024 (2794.40 GiB 3000.46 GB)
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 5860270080 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=944 sectors
          State : active
    Device UUID : bc71213d:a98e1b82:9d4baa5a:16de33ef

Internal Bitmap : 8 sectors from superblock
    Update Time : Thu Jan 31 00:17:01 2019
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 838b1cda - correct
         Events : 9424009

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 2
   Array State : A.A ('A' == active, '.' == missing, 'R' == replacing)


Code: Select all

# mdadm --examine /dev/sdf
/dev/sdf:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 582394e9:85ddedf1:1c3a63b5:cc9a6f0b
           Name : artemide:0  (local to host artemide)
  Creation Time : Tue Jan  7 23:16:02 2014
     Raid Level : raid5
   Raid Devices : 3

 Avail Dev Size : 5860271024 (2794.40 GiB 3000.46 GB)
     Array Size : 5860270080 (5588.79 GiB 6000.92 GB)
  Used Dev Size : 5860270080 (2794.39 GiB 3000.46 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=944 sectors
          State : clean
    Device UUID : 079d31f9:ec8d6ce8:21e93cc1:850f6c4d

Internal Bitmap : 8 sectors from superblock
    Update Time : Mon Dec  3 02:23:34 2018
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 69e3e5f9 - correct
         Events : 474584

         Layout : left-symmetric
     Chunk Size : 512K

   Device Role : Active device 1
   Array State : AAA ('A' == active, '.' == missing, 'R' == replacing)


[p.H]

/dev/sdf has been out of sync since december.
Assemble the array in degraded mode with the two other members only, then re-add sdb to resync it.

[miky76]

/dev/sdf has been out of sync since december.

How can I monitor the state of the array? even with an email...

Assemble the array in degraded mode with the two other members only, then re-add sdb to resync it.

are these the right commands?

mdadm --force --assemble /dev/md0 /dev/sdb /dev/sde

mdadm --add /dev/md0 /dev/sdf
mdadm --assemble --run --force --update=resync /dev/md0 /dev/sdb /dev/sde /dev/sdf

[p.H]

mdadm has a monitor mode and associated service which can send e-mails when en event occurs.
You can also check the array status in /proc/mdstat at any time.

I would use only the first two commands, not the last one.

Also, you may prefer to backup recent data after assembling the array with two members and before adding the third one and starting rebuilding data on it. The rebuild is rather I/O intensive and long, and an active drive may fail during the operation.

[miky76]

mdadm has a monitor mode and associated service which can send e-mails when en event occurs.
You can also check the array status in /proc/mdstat at any time.

I would use only the first two commands, not the last one.

I did what you said, and seem it's worked
How can I be sure now that it's everything ok?

Code: Select all

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdf[1] sdb[0] sde[3]
      5860270080 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [U_U]
      [===>.................]  recovery = 15.9% (465938308/2930135040) finish=352.5min speed=116492K/sec
      bitmap: 18/22 pages [72KB], 65536KB chunk

unused devices: <none>


[p.H]

You can check the filesystem consistency.