There's a few ways to do this.
The CWAF tool is specifically designed to scan your audio folder and your projects and give you a list of missing, referenced, and orphaned files. Theoretically the orphans can be deleted which would clear up space. The problem? It's not uncommon to have it not work right. Of course that basically makes it a non-starter. So, lots of people use method 2:
Do a "Save As" with the "copy audio" option ticked. By basically copying the project and moving over just the audio referenced you can delete the old project and audio folder which will delete unused audio. The problem? Well doing this for EVERY project if you have them organized in a specific way or you've never done it before is a long and time consuming process. But, in the end you'll be caught up and consolidated.
IIRC, there's also some kind of "clean project audio" command that does method 1 on a specific project. It's been a while so I don't remember much about it though, others may chime in on the topic though.