The higher the bit depth, the greater the signal to noise ratio (S/N). For a given bit depth, increasing bit rate increases the highest frequency that can be reproduced.

I think the OP was referring more to mp3 formats, which isn't an apples-to-apples comparison with PCM. I started a thread a few years ago now that got a lot of nice input and was started from an article I had found saying that mp3s are "effectively 20-bits for bit depth." That thread got some nice discussion and insight into mp3s.
 
This is a "necro thread," yes, but the mp3 format is still relevant due to its high level of usage.

John T
You're thinking of sampling frequency (for "speed of the tape"), which does not directly relate to bit rate.

Yes, you're right!  my bad.