Recovering data from overwritten Raid 5 array?

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
r_4_20
Posts: 3
Joined: 2017-07-25 14:23

Recovering data from overwritten Raid 5 array?

#1 Post by r_4_20 »

I have done something very stupid. I upgraded to Debian 9 and was having issues getting X to work. After tons of troubleshooting, I saw that one of my raid devices was showing as an invalid block size. I failed it from the array and removed it. I partitioned it and added it back into the raid array.

After a reboot, grub was unable to boot. I burned a Debian 9 live CD and booted into it. I wanted to run grub-install but it could not find a valid root (/). My raid array wasn't showing but all my hard drives were so I built the raid from the correct partitions. The Debian 9 rescue went on to write the changes to the partition table and started to install a new base system.

After that, I booted into the system and all my files are now gone. I've installed TestDisk and tried running it on each hard drive and on the /dev/md0 device but I cannot find my deleted files.

Because of how long TestDisk has taken, the resync of the raid 5 array has completed which I am now reading may have actually overwritten all my old data.

Is all hope lost? Are all my files gone? Any help is greatly appreciated!

User avatar
GarryRicketson
Posts: 5643
Joined: 2015-01-20 22:16
Location: Durango, Mexico

Re: Recovering data from overwritten Raid 5 array?

#2 Post by GarryRicketson »

Use the backup you made before starting to do this.
https://www.debian.org/releases/stable/ ... ng.en.html
4.1.1. Back up any data or configuration information

Before upgrading your system, it is strongly recommended that you make a full backup, or at least back up any data or configuration information you can't afford to lose. The upgrade tools and process are quite reliable, but a hardware failure in the middle of an upgrade could result in a severely damaged system.

The main things you'll want to back up are the contents of /etc, /var/lib/dpkg, /var/lib/apt/extended_states and the output of dpkg --get-selections "*" (the quotes are important). If you use aptitude to manage packages on your system, you will also want to back up /var/lib/aptitude/pkgstates.

r_4_20
Posts: 3
Joined: 2017-07-25 14:23

Re: Recovering data from overwritten Raid 5 array?

#3 Post by r_4_20 »

Unfortunately I did not make a backup. Like I said, I did something really stupid.

User avatar
dasein
Posts: 7680
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: Recovering data from overwritten Raid 5 array?

#4 Post by dasein »

Step 1. Swipe-and-right-click your thread title

Step 2. Select "Search <search engine name> for 'Recovering data...'"

Step 3. Pray
r_4_20 wrote:Unfortunately I did not make a backup. Like I said, I did something really stupid.
You did several somethings really stupid, starting with implementing RAID 5 simply because is sounded so l33t and kewl. The actual point of RAID is data protection, just like, y'know, backups.

Good luck.

r_4_20
Posts: 3
Joined: 2017-07-25 14:23

Re: Recovering data from overwritten Raid 5 array?

#5 Post by r_4_20 »

Hmm I've done a lot of searching on this problem already. That's why I decided finally to just post myself. I was hoping to get some new help. I'm going to try SystemRescueCD tonight. I've already tried a couple Linux Live CD's that are designed for data recovery. Fingers crossed.

ChristianMorris
Posts: 10
Joined: 2022-12-04 16:07
Been thanked: 2 times

Re: Recovering data from overwritten Raid 5 array?

#6 Post by ChristianMorris »

I'll try to explain what likely happened and what your chances are of recovering the data.

When you added a disk back to RAID 5 and the array started resynchronizing, it actually re-wrote the data to restore the integrity of the array. This is a standard process for RAID: the array rebuilds the data using the remaining disks, but it can overwrite everything that used to be on the rebuilt disk. That's why now, even after using TestDisk, you can't see your files - they've probably already been replaced by new data written during the resynchronization process.

Nevertheless, there's still something you can try to do. SystemRescueCD, which you're going to use, is a good choice. It includes tools like TestDisk and PhotoRec that can attempt to recover files at the block level. The way they work is that instead of trying to recover the file system, they look for the remaining chunks of data on the disk. It's important to realize that if the data has actually been overwritten, these tools may find nothing or only recover corrupted files.

For RAID arrays, there are also specialized programs such as R-Studio or ReclaiMe that know how to work with RAID configurations and can try to collect the remaining data from the array. Their advantage is that they can take into account the RAID structure and analyze data from multiple disks at once. If you try any of these, make sure you're working with copies of the disks and not the array itself so you don't make things worse.

For the future, even though RAID sounds like data protection, it's not a replacement for a backup. RAID protects against single disk failure, but it's not immune to human error or configuration issues like yours. And it's always important to disable automatic resynchronization if data is already corrupted or deleted to avoid overwriting it.

Now it all depends on how badly your data was overwritten. If you can't handle it on your own, you can contact companies that specialize in data recovery from RAID arrays. For example, such as SalvageData, but this is a matter of budget, because such services are usually not cheap.

CwF
Global Moderator
Global Moderator
Posts: 3197
Joined: 2018-06-20 15:16
Location: Colorado
Has thanked: 65 times
Been thanked: 280 times

Re: Recovering data from overwritten Raid 5 array?

#7 Post by CwF »

ChristianMorris wrote: 2024-12-03 01:51 they've probably already been replaced
...7 years ago or so...
You're not paying attention are you?
viewtopic.php?p=812146#p812146
Mottainai

Post Reply