no progress updates from gnome tracker

Question

I very much want to have desktop search working on a new Ubuntu Gnome 17.04 installation. I realize that initial indexing might take a while, but for more than 12 hours, tracker status has returned:

Currently indexed: 93634 files, 6371 folders
Remaining space on database partition: 226.6 GB (45.10%)
Data is still being indexed: Estimated less than one second left

There should be more than 94000 files indexed, after excluded files have been removed. I can't tell if the process has crashed, or is still working on files.

tracker daemon has returned the same result all this time:

Store:
12 May 2017, 15:45:13:  ✓     Store                 - Idle 

Miners:
12 May 2017, 15:45:13:  ✓     Userguides            - Idle 
12 May 2017, 15:45:13:  ✓     File System           - Idle 
12 May 2017, 15:45:13:    0%  Extractor             - Extracting metadata 
12 May 2017, 15:45:13:  ✓     Applications          - Idle 

and using the -f and -w options return no updates. tracker-extract is using one of my cores at 100% and has been all of this time.

How can I tell if tracker is having problems or just taking its time indexing roughly 200 GB of files?

Solution

It seems that tracker-extract was having problems with a couple of Excel XLS files from the same auto-generated source. I suspect they were hitting bugs in tracker's extract code. Tracker has now successfully indexed, and is using negligible resources.

This Debian User Forums post was the key: will tracker-extract ever settle down?. Diagnosing and fixing the problem required me to look in /tmp/tracker-extract-files.1000. If a symlink to the same file persists for any amount of time and tracker-extract is hitting 100% CPU usage, you have a problem file. For me, a problem file symlink looked like:

$ ls -l tracker-extract-files.1000/
total 0
lrwxrwxrwx 1 scruss scruss 55 May 12 16:25 1-9eaf433878d0c8e604486b798d035882 -> /home/scruss/Documents/toronto_hydro/SmartMeterData.xls

To fix this particular file:

• stop tracker with tracker daemon --terminate

• Resave the offending file in a different format, delete it, or set it as excluded in the tracker config GUI. It's important that the file with problems is gone from anywhere that tracker will try to index, or the problem will persist.

• Delete the broken symlink in /tmp/tracker-extract-files.1000

• restart tracker with tracker daemon --start

If you monitor tracker with tracker daemon --follow, you should see Extractor progress lines increase from 0, 1, 2 ... 100%. If it hangs at anything less than 100%, check /tmp/tracker-extract-files.1000 again.

For me, tracker-extract used to throw the error tracker-extract crashed with signal 31 in __libc_message() when it finished. It didn't seem to affect tracker running or indexing new content, though.

Comments

[scruss]

Running tracker daemon -f as soon as I could after a reboot, I did see updates about File System - Crawling recursively directory ... that have now fallen idle. Extractor is still extracting metadata, which may be the problem

[Shaheer Ahmed]

That was most helpful, thank you ! I could pinpoint the culprit. A "playsid" file from a huge C64 collection. Cheers !

[scruss]

@mahen I just had to refer back to this post to fix a block tracker miner, And sure enough, the offending file was "Lunatico_Side_2.sid" from CSDB …

[DanMan]

Thanks for this. I had the tracker choke on an .exr file created by Meshroom. That caused my Ryzen 3700 to not go into idle, so it remained at 60°C. Now it goes as low as 40°C again after adding exceptions via dconf editor under org/freedesktop/tracker/miner/files.

[scruss]

In yet more fun with tracker news, on one of my systems it had forgotten how to find things despite large databases in ~/.cache/tracker. tracker daemon --follow showed ✗ Store - Unavailable. A brute-force solution appears to be tracker daemon --terminate, followed by tracker reset --hard (it's interactive, so you can't chain these commands), followed by tracker daemon --start. This will force the reindexing of your whole system, so is not ideal