updatedb not using sufficient CPU

If none of the more specific forums is the right place to ask

updatedb not using sufficient CPU

Postby Praxis » 2019-08-27 19:00

I installed mlocate on Debian stable and couldn't find a recent document so I did an "updatedb" but it didn't complete after 30 minutes on a 1TB file system, so I reniced it to -20 because it was only using 12% or so of a very anemic 1.6 GHz fanless Celeron quad core processor. Even then only about 16% of my CPU is being used. I could understand this if there were a bunch of other demands on my processor, but according to htop only about 25% of my cores' power are being used.

Is there some trick to getting updatedb to complete in my increasingly limited lifetime? Or is this a reportable bug (I didn't see anything in the mlocate bug page? https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=mlocate;dist=buster

EDIT: So an hour on I tried a locate command and got:
Code: Select all
locate: can not stat () `/var/lib/mlocate/mlocate.db': No such file or directory

But the process is veeeery slowly running, there is a gradually growing file in /var/lib/mlocate/mlocate.dbxxx
I suppose it will eventually finish if I leave my computer running all night, but I've never had to wait more than a few minutes for updatedb to finish, even on systems with low-powered processors and several large hard drives.

In an unrelated off-topic rant, I wish Debian would quit culling tiny little CLI utilities off of its default installations (mlocate, curl, rename come to mind). I could understand it if it were Ubuntu which isn't focused on more experienced users, but these things are minuscule and typical ISO images are over 2 TB (although I have no idea why they are so big since the typical install is pretty Spartan).
User avatar
Praxis
 
Posts: 76
Joined: 2006-12-19 00:00

Re: updatedb not using sufficient CPU

Postby Praxis » 2019-08-28 17:37

It doesn't look like my problem has to do with mlocate, specifically. After letting updatedb run all night and getting a 5GB+ file with no end in site I purged the mlocate package again and reinstalled. Then ran updatedb and it completed quickly (a few seconds, producing a 16.8 MB mlocate.db). Then I edited /etc/updatedb.conf as is my wont, and removed 'cifs' from PRUNEFS and it looked like I was going back into the eternal growing updatedb process (my always-on samba server has about 600 GB of data, this has never been a problem for mlocate before). So I purged mlocate and installed locate. locate ran updatedb quickly, but then I went in to adjust the program to remove cifs from /etc/alternatives/updatedb (which seems to be a script rather than a binary, as it seems to be for mlocate) and it looked like it was going to take forever again. So I purged locate and went back to mlocate. With just the local 120GB being indexed it took updatedb about 3 seconds to generate a 16.8 MB mlocate.db. After removing 'cifs' from /etc/updatedb.conf it took 2.5 minutes to hit 16.8 MB with no end in sight. Restarting samba on my server and removing a line I'd recently added to my server's smb.conf did not help. I played around with my /etc/fstab settings for the auto-mounting share, but didn't find anything useful.

So I'm guessing that because the problem impacts both locate and mlocate the issue must be with the configuration of samba. So I guess I'll have to add 'cifs' back to PRUNEFS and hope this symptom doesn't occur on other computers. If it does, I gather from the mlocate man page that you can make 'locate' search multiple databases, so I'll have to figure out how to get updatedb to include a link to the /var/lib/mlocate/mlocate.db file on my server. And maybe file a bug report.
User avatar
Praxis
 
Posts: 76
Joined: 2006-12-19 00:00

Re: updatedb not using sufficient CPU

Postby Praxis » 2019-09-07 21:27

After the latest round of updates I removed CIFS from PRUNEFS and ran updatedb and it completed in a reasonable period of time, problem solved. I'm really too stupid to run Linux, I never have any idea what is going on.
User avatar
Praxis
 
Posts: 76
Joined: 2006-12-19 00:00


Return to General Questions

Who is online

Users browsing this forum: 4D696B65, cooler01 and 9 guests

fashionable