Grounding is not mysterious, nor is it voodoo. It is, however, difficult to manage because of the myriad ways manufacturers apply basic grounding rules, and that's complicated by the mix of balanced and single-ended interfaces.
 
The basics:
 
A balanced output provides an equal impedance from each leg to reference ground. There is no requirement that both sides of the line are driven, although the most common topology applies a signal of equal amplitude, and opposite polarity, to the two legs. Signal symmetry has nothing to do with balancing a line, other than you can't have signal symmetry on a single-ended output.
 
A balanced input takes the difference between the two signal lines, which is probably why this is confusing. But it is the impedance balance between the two legs that provides the noise immunity. If each leg is impedance balanced, and noise is impressed on both legs it will cancel out. But in order to accomplish this both input pins must also have equal impedance to the reference ground. The cool thing is that the reference does not have to be remote!
 
Another thing people trip over - the shield on a balanced line is not required to transmit, and recover the signal. In fact, it is a really bad idea to connect the shield anywhere other than the chassis itself. If the shield enters the device you have problems - most commonly referred to as the "Pin 1" problem.
 
And now you probably see the problem... a single ended interface usually uses the shield as one of the signal conductors, which means that it necessarily has to enter the chassis. Well, ok, there are ways around even that, but they are not terribly effective.
 
The really bad news? It takes only one poorly implemented interconnection in an entire system to drag it into noise heaven. And USB is very often that culprit.
 
The good news is that most audio manufacturers seem to have figured out how to properly handle USB. My new Presonus 1818VSL exhibits no pin 1 problems.
 
There is a lot more to audio interfacing, but my fingers are getting tired, and I sense many eyes are glazing over.