Bad superblock on ext4 filesystem

[dbbolton]

I added an ext4dev filesystem to my hard drive as /dev/sda3 and mounted at /share with the goal of putting all of my multimedia files on it and sharing it through NFS. I followed this guide:

http://ext4.wiki.kernel.org/index.php/E ... ing_Debian

Of course, this was from lenny (hence the -dev). The partition was fine, and I mounted it numerous times. I also performed a dist-upgrade to squeeze. I shutdown that computer with the usual 'shutdown -h now' and disconnected the AC power cord from it, because I wasn't planning on using that machine for awhile. As far as I could determine, everything was fine before I unplugged it.

Every time that computer is powered off, the clock resets to 1999 because the battery is bad. So I am used to getting the "last mount time is in the future" error.

Three weeks after halting, I powered the computer back on. It was unplugged until this point. I got an error message when it attempted to mount /dev/sda3

Code: Select all

e2fsck /dev/sda3
e2fsck 1.41.9 (22-Aug-2009)
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/sda3

The superblock could not be read or does not describe a correct ext2
filesystem.  If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
    e2fsck -b 8193 <device>

I then removed its line from /etc/fstab, rebooted. and tried to check it, but I just keep getting the same error-- even with the '-b 8193' option.

I am confused as to how this happened, since the machine was unplugged the whole time. I also don't exactly know what to do about it. It seems like the error message is generic. I have found a lot of forum posts containing it, but none seem to deal with my situation.

The machine is still using the 2.6.26 kernel from lenny. I am building a new one right now (2.6.30), hoping that the problem is caused by the ext4dev/ext4 module. Otherwise I don't know what to do.

[aspnair]

Why not try to boot using a recent live/rescue cd with ext4 support. Then try to set the clock and do fsck/mount your ext4 partition.

Are you sure that your hard disk does not have any problem.
smartctl -d ata -a /dev/sda
Install smartmontools and read smart information, see any errors are logged.

[dbbolton]

Why not try to boot using a recent live/rescue cd with ext4 support. Then try to set the clock and do fsck/mount your ext4 partition.

I already set the hardware clock and fixed the mount time error.

Here is the smartctl output:

Code: Select all

smartctl 5.39 2009-12-09 r2995 [x86_64-unknown-linux-gnu] (local build)
Copyright (C) 2002-9 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF INFORMATION SECTION ===
Model Family:     Maxtor DiamondMax Plus 9 family
Device Model:     Maxtor 6Y200M0
Serial Number:    Y63CFK4E
Firmware Version: YAR51HW0
User Capacity:    203,928,109,056 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   7
ATA Standard is:  ATA/ATAPI-7 T13 1532D revision 0
Local Time is:    Fri Jan 15 03:14:32 2010 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x80)	Offline data collection activity
					was never started.
					Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0)	The previous self-test routine completed
					without error or no self-test has ever 
					been run.
Total time to complete Offline 
data collection: 		 ( 363) seconds.
Offline data collection
capabilities: 			 (0x5b) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Suspend Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					No Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					No General Purpose Logging support.
Short self-test routine 
recommended polling time: 	 (   2) minutes.
Extended self-test routine
recommended polling time: 	 (  82) minutes.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  3 Spin_Up_Time            0x0027   179   176   063    Pre-fail  Always       -       17144
  4 Start_Stop_Count        0x0032   253   253   000    Old_age   Always       -       156
  5 Reallocated_Sector_Ct   0x0033   253   253   063    Pre-fail  Always       -       0
  6 Read_Channel_Margin     0x0001   253   253   100    Pre-fail  Offline      -       0
  7 Seek_Error_Rate         0x000a   253   252   000    Old_age   Always       -       0
  8 Seek_Time_Performance   0x0027   252   245   187    Pre-fail  Always       -       33553
  9 Power_On_Minutes        0x0032   238   238   000    Old_age   Always       -       935h+11m
 10 Spin_Retry_Count        0x002b   253   252   157    Pre-fail  Always       -       0
 11 Calibration_Retry_Count 0x002b   253   252   223    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   251   251   000    Old_age   Always       -       815
192 Power-Off_Retract_Count 0x0032   253   253   000    Old_age   Always       -       0
193 Load_Cycle_Count        0x0032   253   253   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0032   253   253   000    Old_age   Always       -       40
195 Hardware_ECC_Recovered  0x000a   253   252   000    Old_age   Always       -       2682
196 Reallocated_Event_Count 0x0008   253   253   000    Old_age   Offline      -       0
197 Current_Pending_Sector  0x0008   253   253   000    Old_age   Offline      -       0
198 Offline_Uncorrectable   0x0008   253   253   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0008   199   199   000    Old_age   Offline      -       0
200 Multi_Zone_Error_Rate   0x000a   253   252   000    Old_age   Always       -       0
201 Soft_Read_Error_Rate    0x000a   253   252   000    Old_age   Always       -       4
202 Data_Address_Mark_Errs  0x000a   253   252   000    Old_age   Always       -       0
203 Run_Out_Cancel          0x000b   253   252   180    Pre-fail  Always       -       1
204 Soft_ECC_Correction     0x000a   253   252   000    Old_age   Always       -       0
205 Thermal_Asperity_Rate   0x000a   253   252   000    Old_age   Always       -       0
207 Spin_High_Current       0x002a   253   252   000    Old_age   Always       -       0
208 Spin_Buzz               0x002a   253   252   000    Old_age   Always       -       0
209 Offline_Seek_Performnce 0x0024   203   203   000    Old_age   Offline      -       0
 99 Unknown_Attribute       0x0004   253   253   000    Old_age   Offline      -       0
100 Unknown_Attribute       0x0004   253   253   000    Old_age   Offline      -       0
101 Unknown_Attribute       0x0004   253   253   000    Old_age   Offline      -       0

SMART Error Log Version: 1
ATA Error Count: 2
	CR = Command Register [HEX]
	FR = Features Register [HEX]
	SC = Sector Count Register [HEX]
	SN = Sector Number Register [HEX]
	CL = Cylinder Low Register [HEX]
	CH = Cylinder High Register [HEX]
	DH = Device/Head Register [HEX]
	DC = Device Command Register [HEX]
	ER = Error register [HEX]
	ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 2 occurred at disk power-on lifetime: 74 hours (3 days + 2 hours)
  When the command that caused the error occurred, the device was in an unknown state.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  84 51 00 ef 91 08 e0  Error: ICRC, ABRT at LBA = 0x000891ef = 561647

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  25 03 22 ef 91 08 e0 00      17:43:25.952  READ DMA EXT
  25 03 28 b7 6e 0f e0 00      17:43:25.936  READ DMA EXT
  25 03 08 df 03 3a e0 00      17:43:25.920  READ DMA EXT
  25 03 20 87 71 31 e0 00      17:43:25.920  READ DMA EXT
  25 03 08 7f 71 31 e0 00      17:43:25.888  READ DMA EXT

Error 1 occurred at disk power-on lifetime: 38 hours (1 days + 14 hours)
  When the command that caused the error occurred, the device was in an unknown state.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  84 51 00 bf 25 78 e0  Error: ICRC, ABRT at LBA = 0x007825bf = 7873983

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC   Powered_Up_Time  Command/Feature_Name
  -- -- -- -- -- -- -- --  ----------------  --------------------
  25 03 40 bf 25 78 e0 00      03:35:13.728  READ DMA EXT
  25 03 02 57 34 02 e0 00      03:35:13.728  READ DMA EXT
  25 03 40 c7 53 7e e0 00      03:35:13.728  READ DMA EXT
  25 03 0a 4f 34 02 e0 00      03:35:13.712  READ DMA EXT
  25 03 40 ff 55 7e e0 00      03:35:13.712  READ DMA EXT

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]


SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[llivv]

.

[aspnair]

ATA Error Count: 2
.....
25 03 40 bf 25 78 e0 00 03:35:13.728 READ DMA EXT

Make sure that your hard disk is really good.
May be try another fsck, see whether the errors are going up.
Also look at the output of dmesg

[dbbolton]

I haven't used ext4 yet so I can't give you the solution.
I will suggest you take a look at the progs for repairing the superblock

The only program that I can think of that might do that is e2fsck, and I can't get it to check the partition.

Code: Select all

mke2fs -n /dev/sda3
mke2fs 1.41.4 (27-Jan-2009)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
9953280 inodes, 39797021 blocks
1989851 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
1215 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000, 7962624, 11239424, 20480000, 23887872

I tried using the -b option for each superblock backup mentioned there, and they all failed with the aforementioned error message.

ATA Error Count: 2
.....
25 03 40 bf 25 78 e0 00 03:35:13.728 READ DMA EXT

Make sure that your hard disk is really good.
May be try another fsck, see whether the errors are going up.
Also look at the output of dmesg

The other partitions are clean, and e2fsck won't check the bad one.

There is a lot of stuff in dmesg. What exactly am I looking for in it? Here are the last 50 lines from a live CD

Code: Select all

[    3.382450] Floppy drive(s): fd0 is 1.44M
[    3.399871] FDC 0 is a post-1991 82077
[    3.400068] hub 1-0:1.0: over-current change on port 5
[    3.434640] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[    3.434666] r8169 0000:00:0b.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[    3.434683] r8169 0000:00:0b.0: PCI: Disallowing DAC for device
[    3.434726] r8169 0000:00:0b.0: no PCI Express capability
[    3.435246] eth0: RTL8110s at 0xffffc20000152f00, 00:0c:76:b3:41:5b, XID 04000000 IRQ 16
[    3.457127] ohci1394 0000:00:0e.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[    3.509878] ohci1394: fw-host0: OHCI-1394 1.0 (PCI): IRQ=[19]  MMIO=[cfffe000-cfffe7ff]  Max Packet=[2048]  IR/IT contexts=[4/8]
[    3.516022] hub 1-0:1.0: over-current change on port 6
[    3.620033] hub 1-0:1.0: over-current change on port 7
[    3.724032] hub 1-0:1.0: over-current change on port 8
[    4.068009] usb 3-1: new low speed USB device using uhci_hcd and address 2
[    4.241322] usb 3-1: configuration #1 chosen from 1 choice
[    4.249197] usbcore: registered new interface driver hiddev
[    4.264583] input: Logitech USB Mouse as /devices/pci0000:00/0000:00:10.1/usb3/3-1/3-1:1.0/input/input5
[    4.288041] generic-usb 0003:046D:C00C.0001: input,hidraw0: USB HID v1.10 Mouse [Logitech USB Mouse] on usb-0000:00:10.1-1/input0
[    4.288060] usbcore: registered new interface driver usbhid
[    4.288063] usbhid: v2.6:USB HID core driver
[    4.469384] kjournald starting.  Commit interval 5 seconds
[    4.469396] EXT3-fs: mounted filesystem with ordered data mode.
[    4.484018] usb 3-2: new full speed USB device using uhci_hcd and address 3
[    4.503379] kjournald starting.  Commit interval 5 seconds
[    4.503389] EXT3-fs: mounted filesystem with ordered data mode.
[    4.670985] usb 3-2: configuration #1 chosen from 1 choice
[    4.796175] ieee1394: Host added: ID:BUS[0-00:1023]  GUID[0010dc00005c7cfd]
[    6.067058] ISO 9660 Extensions: Microsoft Joliet Level 3
[    6.091293] ISO 9660 Extensions: RRIP_1991A
[    6.335299] aufs 20080922
[    6.528117] squashfs: version 3.3 (2007/10/31) Phillip Lougher
[   52.759104] udev: starting version 141
[   52.997418] parport_pc 00:03: reported by Plug and Play ACPI
[   52.997530] parport0: PC-style at 0x378 (0x778), irq 7, dma 3 [PCSPP,TRISTATE,COMPAT,EPP,ECP,DMA]
[   53.419113] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[   53.562076] usblp0: USB Bidirectional printer dev 3 if 0 alt 0 proto 2 vid 0x03F0 pid 0x7904
[   53.562100] usbcore: registered new interface driver usblp
[   54.085373] input: PC Speaker as /devices/platform/pcspkr/input/input6
[   54.148775] ppdev: user-space parallel port driver
[   54.184957] ACPI: I/O resource vt596_smbus [0x400-0x407] conflicts with ACPI region SMOV [0x400-0x406]
[   54.184960] ACPI: Device needs an ACPI driver
[   54.767424] VIA 82xx Audio 0000:00:11.5: PCI INT C -> GSI 22 (level, low) -> IRQ 22
[   54.767583] VIA 82xx Audio 0000:00:11.5: setting latency timer to 64
[   58.148004] Adding 891596k swap on /dev/sda4.  Priority:-1 extents:1 across:891596k
[   67.159582] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   67.159585] Bluetooth: BNEP filters: protocol multicast
[   67.416960] Bridge firewalling registered
[   75.658620] lp0: using parport0 (interrupt-driven).
[   77.581884] r8169: eth0: link up
[   87.652013] eth0: no IPv6 routers present

An here is what happens when I try to mount it from the CD

Code: Select all

mount -t ext4 /dev/sda3 /mnt/devil/

mount: wrong fs type, bad option, bad superblock on /dev/sda3,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail  or so



dmesg | tail 

[   54.767424] VIA 82xx Audio 0000:00:11.5: PCI INT C -> GSI 22 (level, low) -> IRQ 22
[   54.767583] VIA 82xx Audio 0000:00:11.5: setting latency timer to 64
[   58.148004] Adding 891596k swap on /dev/sda4.  Priority:-1 extents:1 across:891596k
[   67.159582] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   67.159585] Bluetooth: BNEP filters: protocol multicast
[   67.416960] Bridge firewalling registered
[   75.658620] lp0: using parport0 (interrupt-driven).
[   77.581884] r8169: eth0: link up
[   87.652013] eth0: no IPv6 routers present
[ 1339.894240] VFS: Can't find ext4 filesystem on dev sda3.

[nadir]

http://forums.debian.net/viewtopic.php? ... =0#p265969
check especially the part about CONFIG_EXT4_FS=y
and from what i know 2.6.26 doesnt support ext4 (at least i can't mount my ext4 from elder live-cd's). if i am wrong: sorry, just an idea.

[smallchange]

Try fsck.ext4. It is part of e2fsprogs in sid.

[dbbolton]

The live CD that I was using is Ubuntu 9.04. I used it because I was pretty sure that it supported ext4.

[aspnair]

From your dmesg output, this is the only relevant one.

Code: Select all

[ 1339.894240] VFS: Can't find ext4 filesystem on dev sda3.

Try using some newer kernel.

[dbbolton]

This hard drive is a Maxtor. I have never bought any of their products myself, but I have used two Maxtor HDDs. The first one was on a Windows XP machine, and it went bad once I had to cut the power when Windows froze. I mean it really went to crap. I DBANed it twice, and still couldn't create a new filesystem on it.

The one mentioned in the OP appears to be going bad, and I have had other problems with it in the past. Although 2 samples isn't really enough to say "Maxtor sucks", 0 for 2 isn't a good record either.

I think I am going to bite the bullet and buy a new drive. In the meanwhile, I luckily have all my media files on my laptop, so I am just going to delete the bad ext4 partition.

[pcalvert]

I always format like so:

Code: Select all

mkfs.ext3 -c /dev/hdaX

The "-c" switch checks for bad blocks and excludes them. Formatting that way takes a long time, but I feel that the time spent is a worthwhile investment.

Ideally, though, I'd rather check for bad blocks before formatting to see if there are any and where they are located. If they are clustered together then I would prefer to partition around them. Of course, that's only if I believe that the drive is otherwise fine.

I'm not saying that the above is a solution to your problem, I'm only mentioning it in case it might be useful to someone.

Phil

P.S. BTW, based on fairly recent research I did, Maxtor has a rather bad reputation.

[dbbolton]

I found a 40 GB WD drive for under twenty USD total. I think I'll just use it for the OS, and put all my media on an external drive.

[bugsbunny]

As a last resort you can try recreating the fs using the option to only write the superblocks.

mkfs.ext4 -S <device>

Code: Select all

-S     Write  superblock and group descriptors only.  This is useful if
              all of the superblock and backup superblocks are corrupted,  and
              a  last-ditch  recovery  method is desired.  It causes mke2fs to
              reinitialize the superblock and  group  descriptors,  while  not
              touching  the  inode table and the block and inode bitmaps.  The
              e2fsck program should be run immediately after  this  option  is
              used,  and  there is no guarantee that any data will be salvage-
              able.  It is critical to specify the correct  filesystem  block-
              size when using this option, or there is no chance of recovery.

and 2.6.26 only supports ext4dev. I also think that there were some non-backwards compatible changes made to ext4 somewhere around .30 (or maybe that was btrfs). I'd have to research that one. In any case I would try the newest possible kernel first.

[dbbolton]

Thanks for the tip Bugs. But after trying it left me with the same error, so I went ahead and created a fresh ext3 partition.

[icegood]

Finaly found, have exactly same issue!
After creating ext4 via

Code: Select all

mke2fs -c -c -b 4096 -L backup2 -t ext4 /dev/sdc1

i found many bad sectors but troubles of such approach that superblocks info are created BEFORE read/write check could be done. So, i don't know wheather it just issue of bad kernel (2.6.31) ext4 support or bad ext4 organization at all. Anyway, partition created and even files could be read/written but still fsck.ext4 returns

Code: Select all

fsck.ext4: Superblock invalid, trying backup blocks...
fsck.ext4: Bad magic number in super-block while trying to open /dev/sdc

If this is bug of just kernel - please let me know because i just don't want to reformat partition to ex3 - "-c -c" gets a reeally long time.

[pcalvert]

Since ext3 works fine for me, I have no plans to switch to ext4 any time soon. I prefer to let others be the guinea pigs.

Some reading material to help you decide how to proceed:
http://www.linux-magazine.com/Online/Ne ... 3-and-Ext4
http://www.h-online.com/open/news/item/ ... 40671.html

Phil

[gradinaruvasile]

I used once ext4 (came default in Ubuntu) but the boot partition just kicked the bucket just like yours once i rebooted (normal restart). And no amount of fsck ing or cursing or anything brought it back in mountable shape. Something the like happened with a JFS partition.
Since then i use only ext3 for primary/system partitions - although i do have a JFS partition on my work desktop and one ext4 on my laptop and it works ok for data storage so far (1+ year).