will tracker-extract ever settle down?

[HankB]

New install (earlier this week)

Code: Select all

barta@yggdrasil:~$ inxi -S
System:    Host: yggdrasil Kernel: 4.8.0-2-amd64 x86_64 (64 bit) Desktop: Gnome 3.22.2
           Distro: Debian GNU/Linux 9 (stretch)
hbarta@yggdrasil:~$ 

Ever since one or the other of the indexing back end programs is running at 100% CPU (on a single core.) Hardware is a Lenovo Y-50 (i7-4710HQ) with 16MB RAM and an SSD. I also run with an NFS share. Disk usage is:

Code: Select all

hbarta@yggdrasil:~$ df -H
Filesystem           Size  Used Avail Use% Mounted on
udev                 8.4G     0  8.4G   0% /dev
tmpfs                1.7G   31M  1.7G   2% /run
/dev/sda5             32G  6.5G   24G  22% /
tmpfs                8.4G  511M  7.9G   7% /dev/shm
tmpfs                5.3M  4.1k  5.3M   1% /run/lock
tmpfs                8.4G     0  8.4G   0% /sys/fs/cgroup
oak:/export/share    3.0T  2.0T  884G  69% /mnt/share
/dev/sda9            529G  308G  194G  62% /mnt/home
tmpfs                1.7G   21k  1.7G   1% /run/user/118
tmpfs                1.7G   58k  1.7G   1% /run/user/1000
oak:/export/redwood  3.0T  2.0T  884G  69% /mnt/oak
hbarta@yggdrasil:~$ 

I have an unusual $HOME directory setup. I often try different distros and mounting my home directory has resulted in problems due to different settings. I get around this by keeping my home directory in the root directory and symlinking to directories I wish to share between distros (such as ~/Documents, ~/Pictures ...) which reside in /mnt/home/hbarta.

Code: Select all

hbarta@yggdrasil:~$ ls -l Documents
lrwxrwxrwx 1 hbarta hbarta 26 Jan 19 19:32 Documents -> /mnt/home/hbarta/Documents
hbarta@yggdrasil:~$ 

Maybe that's the problem. If indexing follows symlinks, there are two paths to a lot of my files. I've just added /mnt to the exclude list and perhaps that will help. (Now both tracker-miner-fs and tracker-extract are each consuming a CPU and the processor is at 75.0°C with the fan sounding like it's going full blast.)

I searched and see lots of posts about the tracker using a lot of CPU and the usual solution is to uninstall it. I'd like to keep it if possible to get the full gnome shell experience.

I find it difficult to believe that indexing is still doing something useful. Is there any hope that it will eventually settle down?

Thanks!

Edit:I changed the settings to not run when on battery power but tracker-extract seems not to respect that setting. I killed it and it started right back up.

[dcihon]

I don't know if it will help or hurt you but I am running dual boot with 2 distros. One being Solydkee64 and the other Ubuntu Testing.
What I have is a separate partition that I put my /home folder on and my 2 distros point to that folder for their /home.
It does cause some issues but works good for most settings between distros. This way when I replace the OS I just make the /home folder for each one and don't format it.

[HankB]

I used to use a separate /home partition but had continuing problems with settings from one distro interfering with settings from another. I went with the symlink solution and some dirs like ~/.config, ~/.gnome and ~/.gconf are not shared between distros.

thanks

[dcihon]

Yeah that is probably a better solution than what I am doing if you are going to use a lot of different distros.

[HankB]

Back to the original issue (CPU load.)
I found a useful page at https://wiki.gnome.org/Projects/Tracker ... /Debugging. Debian 9 is on Tracker version 1.10.3 and the most recent seems to be 1.14. Gnome is at 3.22.2 and Google autocomplete seems to suggest that 3.4.2 is most current. (A little googling did not find me the most current Gnome version.) It is not clear to me whether the Tracker version is tied to the Gnome version or if I could try a newer version of Tracker. I would probably just abandon it instead.

I could not find any configuration files or figure out what should go in them to provide some sort of logging or reporting. I tried adding some environment variables to ~/.bashrc:

Code: Select all

export TRACKER_VERBOSITY=3
export TRACKER_USE_LOG_FILES=1

I rebooted the PC to make sure that these were all properly exported after logging out and back in showed no difference. Still no difference. 'tracker-extract' still ran 100% on one CPU thread.

I decided to take the 'nuclear' option

Code: Select all

barta@yggdrasil:~$ tracker reset --hard
CAUTION: This process may irreversibly delete data.
Although most content indexed by Tracker can be safely reindexed, it can't be assured that this is the case for all data. Be aware that you may be incurring in a data loss situation, proceed at your own risk.

Are you sure you want to proceed? [y|N]: y
Found 5 PIDs…
  Killed process 1443 - 'tracker-miner-apps'
  Killed process 1446 - 'tracker-extract'
  Killed process 1448 - 'tracker-miner-fs'
  Killed process 1454 - 'tracker-store'
  Killed process 1455 - 'tracker-miner-user-guides'
Setting database locations
Checking database directories exist
Checking database version
Checking database files exist
Removing all database/storage files
  Removing first index file:'/home/hbarta/.cache/tracker/first-index.txt'
  Removing last crawl file:'/home/hbarta/.cache/tracker/last-crawl.txt'
  Removing database:'/home/hbarta/.cache/tracker/meta.db'
  Removing journal:'/home/hbarta/.local/share/tracker/data/tracker-store.journal'
  Removing db-version file:'/home/hbarta/.cache/tracker/db-version.txt'
  Removing db-locale file:'/home/hbarta/.cache/tracker/db-locale.txt'
hbarta@yggdrasil:~$ 

At present Tracker is quiet. Too quiet. It seems to be no longer indexing files. Logging out and back in solved that.

Code: Select all

top - 12:29:47 up 55 min,  2 users,  load average: 2.58, 1.87, 1.12
Tasks: 272 total,   2 running, 269 sleeping,   0 stopped,   1 zombie
%Cpu(s):  2.0 us,  0.5 sy, 25.1 ni, 72.2 id,  0.2 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 16351016 total, 10931968 free,  2769692 used,  2649356 buff/cache
KiB Swap: 16777212 total, 16777212 free,        0 used. 12823016 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                       
 6223 hbarta    39  19 1140448  62364  13156 R 103.7  0.4   3:45.36 tracker-miner-f                               
 6219 hbarta    39  19 2848952 164124  43128 S 100.3  1.0   3:31.84 tracker-extract                               
 6241 hbarta    20   0  493948  91096   9092 S   8.6  0.6   0:50.80 tracker-store                                 
 6916 hbarta    20   0  878968 113296  67564 S   2.3  0.7   0:07.07 chrome                                        
...

I did find this in /tmp:

Code: Select all

lrwxrwxrwx 1 hbarta hbarta 63 Jan 22 12:27 2-dbbde73aeedfd9033c8e7d2021f0f17a -> /home/hbarta/Documents/CRBMWOA/roster/2009-07NewsLetterList.xls
hbarta@yggdrasil:/tmp/tracker-extract-files.1000$ 

Is it possible that this single file is giving causing tracker-extract to go into an endless loop? That may be it. I added that file to the Tracker ignore list, killed tracker-extract (and it immediately restarts.) Now the link in /tmp/tracker-extract-files.1000 is continuously changing.

Edit: After three hours I checked and Tracker had settled down. It seems that that single file was the issue. Problem solved!

[Francewhoa]

Hello HankB and all contributors I'm facing the same challenge here. One of my small devices has only 2 CPU. tracker-extract use 50% CPU for hours. The device is overheating. On other devices with up to 64 CPU this is not a challenge. But on devices with only 1 or 2 CPU it is.

Steps which resolved this challenge in my case:

1. Using Debian 8 or 9, using GNOME, search for "Indexing Preferences"
2. Open it. Then configure it appropriately. So that tracker-extract uses less CPU. While at the same time you can keep all or some of GNOME indexing features. While configuring it, review all "Indexing Preferences" tabs: "Indexing", "Location", "Ignored Content", "Control", "System".
   For those not familiar with "Indexing Preferences", find this screenshot at https://i.postimg.cc/59vvJv9H/GNOME-Ind ... cewhoa.jpg
3. Optionally, if you need tracker-extract to use zero (0%) CPU, and you do not need any of its features, then in "Indexing Preferences" simply deactivate all features.
4. For some configuration changes to be effective, you need to either restart your GNOME session or restart your device