orphans inodes after some time [solved]

Kernels & Hardware, configuring network, installing services

orphans inodes after some time [solved]

Postby Ilix » 2017-12-22 09:26

I don't know if I caught the right forum but it seems to me that "system config" is the best place to ask for support.
I tried a little 'around both here and googling but I was not able to find some help with my problem.

On an X3500 IBM server with debian jessie one of the 4 SAS disks in the hardware RAID 5 (made with the server controller) is damaged. From that moment sda1 (one of the resulting partitions on the raid sda disk) starts having problems with orphaned inodes.

After a while Debian detects 5 or 6 inodes orphans and goes into read only mode. The operating system remains on but many services are no longer able to write to the disk and stopped.

Restarting the server corrects sda1 and starts again. After a short it starts again with the orphaned inodes and so on.

If I boot the server with minimal lubuntu in rescue mode, the fsck.ext4 -y / dev / sda1 end successful. Everything seems fine, the system restart, debian starts again, everything runs smoothly (apart from ProFTP that does not start alone but I have to restart it) for half an hour and then again are always those 5/6 inodes orphans and the system sda1 is reassembled in read only mode. I I try to copy some files to sda1 the same but at the next restart orphans inodes are much more in quantity.

How do I get out of this infermal loop? I cannot understand if it is a HW problem (why the SAS controller don't detect problems?) or software.

TNX. Ilic
Last edited by Ilix on 2017-12-23 08:33, edited 1 time in total.
Posts: 4
Joined: 2017-10-25 18:11

Re: orphans inodes after some time

Postby Ilix » 2017-12-22 10:41

# tune2fs -l /dev/sda1
tune2fs 1.42.12 (29-Aug-2014)
Filesystem volume name: <none>
Last mounted on: /
Filesystem UUID: 215d2d4d-b400-476f-a634-7c1ae6644ea8
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags: signed_directory_hash
Default mount options: user_xattr acl
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 52690944
Block count: 210756864
Reserved block count: 10537843
Free blocks: 189421862
Free inodes: 52234282
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 973
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Flex block group size: 16
Filesystem created: Mon Sep 19 12:20:00 2016
Last mount time: Fri Dec 22 11:02:54 2017
Last write time: Fri Dec 22 11:02:51 2017
Mount count: 1
Maximum mount count: -1
Last checked: Fri Dec 22 10:57:14 2017
Check interval: 0 (<none>)
Lifetime writes: 1846 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 28
Desired extra isize: 28
Journal inode: 8
First orphan inode: 12320780
Default directory hash: half_md4
Directory Hash Seed: 6200ca7e-aa56-422a-854b-53d23139e269
Journal backup: inode blocks
Posts: 4
Joined: 2017-10-25 18:11

Re: orphans inodes after some time

Postby Ilix » 2017-12-23 08:32

Let's summarize how (I suppose) I solved:
- I tested all disks one by one (the SAS controller did not detect problems)
- I removed one disk at a time from the RAID and then reinserted it waiting after the previous one was "rebuilded" in the RAID

The goal was this second action, I think. My hypothesis (confirm please if I'm right or rebate if I'm wrong) is: the first disk of the array had kept track of the failure of the second disk (the one broken before replacing) and brought it behind deceiving Debian.

In essence, the errors were not really present but only mimed.

I will keep you updated in case of news and upheavals of this theory.
Posts: 4
Joined: 2017-10-25 18:11

Re: orphans inodes after some time [solved]

Postby llivv » 2017-12-25 17:51

replace the disk with the inode issues

find your hardware on IBMs site and readup on how it works
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.
User avatar
Posts: 5728
Joined: 2007-02-14 18:10
Location: cold storage

Return to System configuration

Who is online

Users browsing this forum: No registered users and 4 guests