#mdworker

Fix Spotlight continually re-indexing your hard drive

Apple’s Spotlight technology is its built-in searching routine for finding almost anything you will use in your account, and to do this it will create and then maintain a metadata index for each locally attached drive on your Mac; however, there may be times when you open the Spotlight menu and always see the progress bar indicating the system is indexing your hard drive. In addition, if you open…

I recently had a Mac misbehaving; weekly kernel panics, kernel_task consuming excessive CPU and leaking memory, and spotlight appearing to delay boot up. To be fair, the poorly Mac had never seen an OS reinstall after three years, taking it from 10.4 to 10.6.6. I could've reinstalled the OS, but curiosity got the better of the me - I wanted to learn exactly what was causing these issues.

Most Mac users know about Onyx, a tool for tweaking/maintenance for OS X. I gave that a shot and while the boot time appeared to decrease slightly, it didn't really make much difference. In fact a day later, I observed another kernel panic.

I knew things weren't right with kernel_task, so I directed my energies into understanding what the process does, and how to debug it. Tyler Ballance explains what kernel_task is quite well. Then I stumbled across gold - Consultants Canary by KHI, a script that rattles through your system, finding kernel extensions, app plugins, frameworks and start-up items. After a couple of minutes of pouring through the output, I quickly identified several unwanted extensions and tasks -

udmarket

com.caiaq.driver.NIUSBHardware Driver

org.defyne.driver.MMInputFamily

org.defyne.driver.ConexantCX2388X 

org.defyne.driver.PhilipsSAA7146 

org.defyne.driver.UDMUtils 

org.defyne.driver.PrincipalMMInputs

com.taoeffect.ispy.kext

com.bombich.ccc

Pando Web Installer / PandoWebInst.plugin

I use App Zapper to uninstall unwanted apps, but obviously it wasn't removing absolutely everything. Once I'd removed all these extensions, I gave the system a reboot - it didn't seem any faster, and still got stuck loading the menu bar with just spotlight showing. [After three weeks of use, I've not seen a single kernel panic - so it probably was a kernel extension that caused them]

Next I turned my attention to Spotlight. After reading a ton of information about how Spotlight (aka mds and mdworker) works, and trying to find others with systems exhibiting the same behaviour, I eventually worked out the Cisco VPN client I had installed was the culprit. To uninstall it properly (there's no uninstaller app), I had to:

sudo /usr/local/bin/vpn_uninstall

Sure enough, after I'd removed it and rebooted, no more pauses as the menu bar and dock loaded.

Over the next 48 hours I kept Activity Monitor open, keeping an eye on CPU usage and memory consumption. Kernel_task was indeed behaving itself, but I noticed mdsworker chewing up CPU and RAM. After reading a couple of articles (here and here), I used fs_usage to identify a couple of places on the hard disk where Spotlight seemed to be spending a long time indexing - and once excluded in Preferences > Privacy, cleared that right up.