Incidentally, both analog tape and vinyl in an abstract way can be thought of as sampling systems due to the mechanics of the stylus and tape head contact. The stylus actually bounces up and down in the groove of the vinyl and the tape does not remain perfectly, under constant tension continuously, in contact with the tape head.  These inconsistencies could be viewed as sampling of the medium. There could also be a plethora of arguments against those statements. :-)

...double post

rabeach
Perfect band limiting is not possible. Therefore we are left with optimal reconstruction not perfect reconstruction.

Which in the real world means:
 
1. We start rolling off a little before one half the sampling rate.
2. We don't have perfect (infinite) attenuation of frequencies greater than one half the sampling rate. 
3. We can have some imperfections in the frequencies we want to keep.
 
 
If you look at the spec sheets of modern ADC/DAC chips, you will find that the designers typically start rolling off at ~20kHz for 44.1kHz (#1), try to attenuate so that any aliasing will be below the existing noise floor of the chip  (#2), and reduce #3 as much as possible. Typically they also use the exact same filter at 96/88.2kHz as 48/44.1kHz, but the cutoff frequency is an octave higher.
 
Some chips allow for different compromises to be made, such as a "low latency" mode where it might start rolling off at 17kHz and less attenuation.

drewfx1

craigb

drewfx1

jamesg1213
The more samples taken per second, the more accurate the digital representation of the sound. For example, the sample rate for CD-quality audio is 44,100 samples.

Be careful  here -  it's only "more accurate" in that it contains higher frequencies.
 
What the sampling theorem proves is that if you filter out everything greater than one half the sample rate, then everything that happens between the samples is stored in the samples. It can't be any "more accurate" once you already have all of the data.
 
In theory it works perfectly, but in practice we end up with some filter artifacts and also have to start filtering things out a little below 1/2 the sampling rate. A higher sampling rate allows one to move artifacts to a higher frequency as well as capture higher frequencies, but otherwise it's not any "more accurate".

Wait a minute...  is your rebuttal about sampling rate or resolution (e.g., usually 16-bit for CD's)?  'Cause it sounds like you're talking about resolution.  Sampling rate shouldn't have much to do with the frequency, only in the accuracy of tracking the changes in the music, no?

Sampling rate really only has to do with frequency, not resolution.
 
From a practical standpoint in the real world all anyone really needs to know about audio sampling can be summarized as follows:
 
Higher sampling rate = higher frequencies
Higher bit depth = less noise
 
So once you can reproduce a signal to the high frequency limits of your hearing with the noise too quiet for you to hear at a given listening level, you're done. Simple as that.

Duh.  Of course it is!  I really shouldn't try to think right after I wake up and before having coffee... 

Optimal sample rate depends on whether or not you're diabetic, and who's watching.