I really need to write this up<G>...
First - ground loops, and ground potential differences are NOT a problem, they exist everywhere, and you can't make them go away.
Second - single-ended interfaces, especially inputs, make the problem of noise pickup worse, but once again, they are not the problem, well, I think the world would be a better place without them, but that's cause I'm lazy<G>!
Third - you have to differentiate between different mechanisms for noise couping. And there are a bunch - it can be capacitively or indutively coupled, it can be coupled to the shield or to a signal conductor, etc. And the #1 problem is poorly designed equipment where the noise is allowed inside the case because the manufacturer violated the Pin-1 rule!
Fourth - Ground is not common, and common does not have to be ground. Yeah, makes my head spin too!
Fifth, probably should be first - Safety Ground is there for safety, not noise reduction. NEVER defeat it, not even temporarilly. It's marginal as a trouble-shooting tool, and it's dangerous. If you must break galvinic connections break them on the signal paths. And be wary of any device that claims to do so, the ONLY device that works is a transformer. If you are using it for troubleshooting you can get away with a cheap one, usually, If you are using it as a solution you'll need to spend the money on a good one, or suffer the audible penalty.
Trouble-shooting noise problems is, or can be, complex. So it's generally better to build for noise immunity.
The first step to a low noise system is a clean common reference, and power distribution. Note I say common, not ground. Ground is the final reference point, for safety reasons. Make every path to your reference as short and low impedance as possible.
You'll also need to think through the grounding scheme. There are two camps - single point grounding and mesh grounding. They both work, and they can even be combined, but you have to think it through. Single Point grounding is easier and less expensive, and has become the most common practice, at least in the US.
Power should be fed from a single breaker, or at least one leg. And the leg shouldn't have a lot of noise makers on it.
Balanced output to Balanced input is far and away the best interconnection... assuming the designers didn't do anything dumb - and you can't assume that I'm afraid.
Single-ended output to Balanced input is the second best solution. Wire it with a twisted pair, and connect the shield at the send only for starters. You may need to terminate the shield at the recieve end too in high RF environments.
Note that it is the twist of the signal pair that provides protection from magnetic fields. Shields have no effect at low frequencies.
While it's a bit more work, terminate the shields to the chassis, not the connector. You can, of course, do this only for devices that have a Pin-1 problem, but it's just as easy to start here.
If you are in a high noise environement you might want to modify single-ended sources to impedance balanced sources by adding an impedance between the low conductor and signal common.
Speaking of which - a balanced connection is balanced because the impedance to ground from either conductor is equal. It has nothing to do with the signal, and there is no requirment that both conductors carry a signal, only that each one has the same impedance to ground at the input. This allows the input stage to maximize Common Mode Rejection.
Single-ended source to Single-ended input, or Balanced source to single-ended input should be a last resort!
The worst is Balanced source because now you need to figure out if you need to ground or float the unused pin.
Avoid them and save me some typing<G>!
Other thoughts... impedance matching is not required, and in fact can make matters worse. Modern equipment (except guitars and guitar amplifiers) is designed for voltate transfer, not power transfer, and contrary to popular belief, characteristic impedance does not play a role in typical studio designs.
This is a huge topic - and an interesting one. I only skimmed it, so feel free to ask questions...