it is 96 THOUSAND time per second.... not 96.
That is the AUDIO sampling rate. It takes the audio coming in and samples it at that rate to reproduce it in digital form for recording.
The higher the sampling rate, the more accurate the picture it can paint. However, the normal rate for CD of 16 bits and 44.1k sample rate is more than sufficient to reproduce a highly accurate and detailed musical note. That's why it's a standard. Commercial CD players are all set up to read that particular standard. Other non-standard rates may or may not be read on a given machine. Although a computer will generally read them.
Going above 44.1khz is OK, but 99.9% of listeners can not differentiate between a 44.1khz sampled recording and the 96khz ones in a side by side listening test with any degree of accuracy beyond the random correct guess.
Midi is already digital. However, coming out of the DAW, when it is converted back to audio, yes, the sampling rate selected is used to convert the digital back into audio..... that is why you select a bit depth and sampling rate to export to audio.