Yes, I totally agree about having "Per Project Folders" activated.
Once you've done this, open up each project in turn, then do a "Save As" to a different location. Make sure you've got "Copy Audio With Project" checked.
Once you've done this for all your projects, you can safely delete the contents of the global audio folder.
But don't delete the folder itself - this needs to be present for those times when you start a new project by the "Ctrl + n" shortcut