Ancient thread alert! But to answer your question, Clean Audio Folder searches the whole system for projects that might reference the audio in the folder you're cleaning in case a project was moved without its audio or vice versa. Clean Audio Folder is just trying to save users from themselves.
I recommend you try using the Cakewalk Audio Finder (CWAF) utility instead. It's no marvel of modern software development, either, but it's far more flexible and powerful - and provides more info about how projects and audio are related - than CAF. The main difference is that it searches only one drive, and you can exclude folders from the search. Depending on how your file system is structured, it can be pretty easy to restrict it to just your per-project and Global audio folders.