diskspace is missing (SOLVED)

Kernels & Hardware, configuring network, installing services

diskspace is missing (SOLVED)

Postby Boronsbruder » 2018-08-06 06:18

Hello!
Here is my problem:
I'm running Linux 4.9.0-4-amd64 #1 SMP Debian 4.9.65-3+deb9u1 (2017-12-23) x86_64 GNU/Linux.

Yesterday my Server ran out of diskspace.

Now i'm trying to find out who is consuming the space.

I did df -h:
Code: Select all
Dateisystem              Größe Benutzt Verf. Verw% Eingehängt auf
udev                      7,9G       0  7,9G    0% /dev
tmpfs                     1,6G    9,2M  1,6G    1% /run
/dev/sda2                 222G    174G   39G   82% /
tmpfs                     7,9G       0  7,9G    0% /dev/shm
tmpfs                     5,0M    4,0K  5,0M    1% /run/lock
tmpfs                     7,9G       0  7,9G    0% /sys/fs/cgroup
/dev/sda1                 511M    3,6M  508M    1% /boot/efi
/dev/sdc1                 825G    394G  389G   51% /data2
/dev/sdb1                 8,2T    7,2T  528G   94% /data
//192.168.116.241/backup  5,5T    992G  4,2T   19% /mnt/NAS
tmpfs                     1,6G       0  1,6G    0% /run/user/0


and du -hx --exclude "/mnt" / | sort -h (posted only the last lines):
Code: Select all
1,9G    /usr/lib
1,9G    /var/log
2,6G    /root/tbsdriver
2,6G    /root/tbsdriver/tbs_git
2,6G    /usr/share
3,0G    /var
3,3G    /root
5,7G    /usr
14G     /


As you see there is a differénce of 160 GB for "/".
Where are they gone?

P.S.:
Sorry for my bad english - Germany speaking here :D
Last edited by Boronsbruder on 2018-08-06 18:16, edited 1 time in total.
PEBKAC -> Problem Exists Between Keyboard And Chair
User avatar
Boronsbruder
 
Posts: 4
Joined: 2018-08-06 05:54

Re: diskspace is missing

Postby dilberts_left_nut » 2018-08-06 06:35

Probably hiding “under” one of your mounts.
Unmount them and check again.
AdrianTM wrote:There's no hacker in my grandma...
User avatar
dilberts_left_nut
 
Posts: 4884
Joined: 2009-10-05 07:54
Location: enzed

Re: diskspace is missing

Postby Boronsbruder » 2018-08-06 07:45

I excluded my "/mnt" directory:

Code: Select all
du -hx --exclude "/mnt" / | sort -h


And the other mounted devices are listed apart when using df:

Code: Select all
/dev/sdc1                 825G    394G  389G   51% /data2
/dev/sdb1                 8,2T    7,2T  528G   94% /data
//192.168.116.241/backup  5,5T    992G  4,2T   19% /mnt/NAS


But i unmounted them and there is no change
PEBKAC -> Problem Exists Between Keyboard And Chair
User avatar
Boronsbruder
 
Posts: 4
Joined: 2018-08-06 05:54

Re: diskspace is missing

Postby dilberts_left_nut » 2018-08-06 08:10

... and are you still excluding /mnt?

That’s likely where it is if data has been copied there while the other volumes are unmounted.

The other common issue is deleted files where the process is still running.

Try ncdu for an easier ‘look’.

If still no joy, you’ll need to provide more info about your environment and usage.
AdrianTM wrote:There's no hacker in my grandma...
User avatar
dilberts_left_nut
 
Posts: 4884
Joined: 2009-10-05 07:54
Location: enzed

Re: diskspace is missing

Postby debiman » 2018-08-06 08:14

where are they hiding? maybe:
Code: Select all
du -hx /mnt | sort -h
User avatar
debiman
 
Posts: 2900
Joined: 2013-03-12 07:18

Re: diskspace is missing

Postby p.H » 2018-08-06 08:45

debiman : the command won't show anything useful if the files are hidden under a mount point.

BoronsBruder : typical scenario : you do backups on the NAS but don't check properly if the NAS is mounted, so once the backup went into the directory in the / filesystem.
Note : /mnt is not intended for permanent "production" mounts.

If you wand to check what's hidden under the mount points, bind mount / on some temporary mount point (what /mnt is meant for).

Code: Select all
mkdir /bareroot
mount --bind / /bareroot
du -hxd1 /bareroot
p.H
 
Posts: 595
Joined: 2017-09-17 07:12

Re: diskspace is missing

Postby Boronsbruder » 2018-08-06 09:09

Here we go!

Code: Select all
161G    /bareroot/mnt/NAS


Many thx to p.H!

And now the next question...
How can i do it the right way?
PEBKAC -> Problem Exists Between Keyboard And Chair
User avatar
Boronsbruder
 
Posts: 4
Joined: 2018-08-06 05:54

Re: diskspace is missing

Postby p.H » 2018-08-06 09:11

How to do what ?
p.H
 
Posts: 595
Joined: 2017-09-17 07:12

Re: diskspace is missing

Postby Boronsbruder » 2018-08-06 09:22

You told me:
Note : /mnt is not intended for permanent "production" mounts


How should i mount in the right way?
PEBKAC -> Problem Exists Between Keyboard And Chair
User avatar
Boronsbruder
 
Posts: 4
Joined: 2018-08-06 05:54

Re: diskspace is missing

Postby debiman » 2018-08-07 07:43

Boronsbruder wrote:You told me:
Note : /mnt is not intended for permanent "production" mounts


How should i mount in the right way?

use some dedicated (meaning you created it) directory.
there's a few options that should be avoided, one is /media.
fwiw, my NFS mounts are under /srv/nfs.
User avatar
debiman
 
Posts: 2900
Joined: 2013-03-12 07:18

Re: diskspace is missing (SOLVED)

Postby p.H » 2018-08-07 08:23

I am confident that the OP created /mnt/NAS.
Shouldn't /srv/nfs be used for NFS exports rather than mounts ?
Indeed /media is reserved for removable media, usually automatically mounted.
If the FHS does not define any standard location, the mount point may be created in any location that makes sense.

More importantly, you must check that the network share is actually mounted before doing a backup.
p.H
 
Posts: 595
Joined: 2017-09-17 07:12


Return to System configuration

Who is online

Users browsing this forum: No registered users and 8 guests

fashionable