Debian Server - Failed to boot due to xfs filesystem error.

If none of the more specific forums is the right place to ask

Debian Server - Failed to boot due to xfs filesystem error.

Postby lindsayrg1 » 2012-03-03 06:40

Hey, recently I decided to build a NAS server and install Openmediavault which is similar to Freenas but based on Debian. Anyway I'm afraid members of my family have been turning the server off at night via the power button (trying to save power) which has obviously caused filesystem issues. I have a 12tb raid 5 array with just over 8tb usable. Just before it stopped around 4tb of data was on it so I would like to try to get the server running and if that isnt possible attempt to recover some / all of the files.

When I boot the server I get the following message
Code: Select all
An automatic file system check (fsck) of the root filesystem failed. A manual fsck must be performed, then the system restarted. The fsck should be performed in maintenance mode with the root filesystem mounted in read-only mode...failed!A log is being saved in /var/log/fsck/checkfs if that location is writable. Please repair the file system manually. A maintenance shell will now be started. CONTROL-D will terminate this shell and resume system boot..Give root password for maintenance(or type Control-D to continue)


Log states
Code: Select all
Log of fsck -C -R -A -a
Sat Mar  3 15:43:31 2012

fsck from util-linux-ng 2.17.2
/sbin/fsck.xfs: UUID=6c763b53-3f3e-4f31-9d3c-3a0987821189 does not exist
fsck died with exit status 8

Sat Mar  3 15:43:31 2012:


Typing root password does nothing and entering Control-D just starts the server process / daemons etc allowing me to continue to access the web interface but the smb share is not working I assume due to the filesystem errors.

I honesly have no idea what I am doing so I have tried to searching for similar issues in this forum but most of the fsck errors don't seem to be for exit status 8. The other thread I found I did state the first steps to recover the filesystem are to un-mount all disks/partitions and then run a filesystem check from a live cd. (viewtopic.php?f=30&t=71688&p=399882&hilit=fsck+exit+code+8#p399882)

According to the web interface the fours hard drives are as follows
/dev/sda
/dev/sdb
/dev/sdc
/dev/sdd

They are listed as mounted and as an xfs filesystem but the status is listed as missing.
Do I just unmount all by typing in the following
Code: Select all
umount -a

and then run the following for each drive or is there a different /dev/xxx for an array?
Code: Select all
e2fsck -y -f -v /dev/hda1


Also as I said people recommend doing this from a live cd. When I boot from the openmediavault/debian installed on my usb it gives two boot options (default and recovery). Would not this usb booting from recovery be suitable or is a particular live cd recommended for filesystem recovery?
Finally I am sorry if it seems I jumped into the deep end but I didn't think I would have to learn much about debian and would just be able to rely on the web interface aspect of the server. Thank you if you managed to read this all the way through.
lindsayrg1
 
Posts: 5
Joined: 2012-03-03 05:53

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby pendrachken » 2012-03-03 18:42

Search around for how to manually restart your RAID array. Individual disks ( generally ) won't have recognizable file systems if they are part of an array unless the whole array is mounted.
fortune -o
Your love life will be... interesting.
:twisted: How did it know?

The U.S. uses the metric system too, we have tenths, hundredths and thousandths of inches :-P
pendrachken
 
Posts: 1047
Joined: 2007-03-04 21:10
Location: U.S.A. - WI.

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby brian.va » 2012-03-04 04:46

lindsayrg1 wrote:s there a different /dev/xxx for an array?.


generally they are referenced by /dev/mdX

try
Code: Select all
cat /proc/mdstat

and see what comes back, you should at least see what it is referenced as, and could try mounting it and see if it does anything
brian.va
 
Posts: 137
Joined: 2010-12-11 17:44

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby lindsayrg1 » 2012-03-04 06:42

I got the following.

Code: Select all
root@Openmediavault:~# cat /proc/mdstat
Personalities :
md127 : inactive sda[1](S) sdd[3](S) sdb[2](S)
      8790796680 blocks super 1.2
unused devices: <none>


So sda, sdb and sdd are inacative? What about sdc, it is showing up in the web interface?
Image

I tried mounting it but I'm probably doing it the wrong way here.
Code: Select all
root@Openmediavault:~# mount md127
mount: can't find md127 in /etc/fstab or /etc/mtab


Individual disks ( generally ) won't have recognizable file systems if they are part of an array unless the whole array is mounted.


Yes that is what I guessed and is why I asked. So essential I have to get all the disks active again and then run a raid xfs file system check somehow?
Thanks to both of you for your help so far!
lindsayrg1
 
Posts: 5
Joined: 2012-03-03 05:53

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby pendrachken » 2012-03-04 08:27

You most likely will have to assemble the array manually from the CLI. See the mdadm man page or http://linux.die.net/man/8/mdadm from a different computer. You may need to use the force flags, but only try that if assembling the array doesn't work without it. Once you have the array assembled you should be able to run the filesystem check on the array - they generally start with md0.
fortune -o
Your love life will be... interesting.
:twisted: How did it know?

The U.S. uses the metric system too, we have tenths, hundredths and thousandths of inches :-P
pendrachken
 
Posts: 1047
Joined: 2007-03-04 21:10
Location: U.S.A. - WI.

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby lindsayrg1 » 2012-03-05 00:20

ok I read the intro which talks about the several modes of operation of mdadm and the uses of assemble mode option.

So I tried the following
Code: Select all
mdadm --assemble --scan  (does nothing just goes to next line / no response)
mdadm --assemble md0 (md0 not identified in config file)


I try to get identity of the array with the following but no go.
Code: Select all
root@Openmediavault:~# /etc/mdadm/mdadm.conf
-bash: /etc/mdadm/mdadm.conf: Permission denied
root@Openmediavault:~# /etc/mdadm.conf
-bash: /etc/mdadm.conf: No such file or directory


The page says the following but I'm not sure what to type in to get identity? mdadm --assemble --config does not seem to do anything.
Code: Select all
If --scan is not given, then the config file will only be used to find the identity of md arrays.


Again I appreciate your help. I'll keep reading the article and hope that I understand it but it's all a little much as I have never used a CLI before.
lindsayrg1
 
Posts: 5
Joined: 2012-03-03 05:53

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby pendrachken » 2012-03-05 00:29

Use nano to edit mdadm.conf, nano is a CLI text editor.

Code: Select all
nano /etc/mdadm/mdadm.conf


you should see what the array is called in the conf file.

To exit nano, if you have changed things in the file you are editing press F3 and then enter, after that press F2. If you haven't changed anything just press F2.
fortune -o
Your love life will be... interesting.
:twisted: How did it know?

The U.S. uses the metric system too, we have tenths, hundredths and thousandths of inches :-P
pendrachken
 
Posts: 1047
Joined: 2007-03-04 21:10
Location: U.S.A. - WI.

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby lindsayrg1 » 2012-03-05 02:07

Wow thanks for the quick response. I have been puzzeling away and didn't even notice. I followed your recommendation and I believe the relevant part from the text is name=openmediavault:Raid5Array UUID=34eb0de1:60c528fa:b6c6d68d:c3616955

Code: Select all
 # mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
# Note, if no DEVICE line is present, then "DEVICE partitions" is assumed.
# To avoid the auto-assembly of RAID devices a pattern that CAN'T match is
# used if no RAID devices are configured.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# definitions of existing MD arrays
[b]ARRAY /dev/md/openmediavault:Raid5Array metadata=1.2 name=openmediavault:Raid5Array UUID=34eb0de1:60c528fa:b6c6d68d:c3616955[/b]


So I entered what is below and then nothing happened just got a new line?
Code: Select all
root@Openmediavault:~# mdadm --assemble openmediavault:Raid5Array
root@Openmediavault:~#
lindsayrg1
 
Posts: 5
Joined: 2012-03-03 05:53

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby lindsayrg1 » 2012-07-29 16:24

Hey its been quite awhile since my original post been busy with other stuff but I still have yet to get this fixed.

I have more time now so I am willing to investigate further into solutions if people can advice what I should do/try/read up on.

I basically got stuck last time and then other stuff happened so it just sat around. The data on it is not really important but I would like to try and get it back before i format the drives if possible.
If anyone can advice on what to do or who I can possible pay in australia to have a look and see if they can repair it would be appreciated.


Here is boot image of server attached to tv. Problem is UUID=xxxx does not exist.

Image
lindsayrg1
 
Posts: 5
Joined: 2012-03-03 05:53

Re: Debian Server - Failed to boot due to xfs filesystem err

Postby pendrachken » 2012-07-30 21:15

just let the --assemble --scan run for a long time, it may find the disks.


If that doesn't work...
Try manually re-assembleing the array:

Code: Select all
mdadm --assemble /dev/md0 /dev/sdX1 /dev/sdX1 /dev/sdX1


where /dev/sdX1 is the ALL of the drives ( and partitions ) from the array.
fortune -o
Your love life will be... interesting.
:twisted: How did it know?

The U.S. uses the metric system too, we have tenths, hundredths and thousandths of inches :-P
pendrachken
 
Posts: 1047
Joined: 2007-03-04 21:10
Location: U.S.A. - WI.


Return to General Questions

Who is online

Users browsing this forum: No registered users and 4 guests

fashionable