VS-100 Sampling Rate

I have a VS-100 who's sampling rate is 24bit/96kHz.   Thinking of adding an A/D D/A converter (RME ADI-2) to my chain.  The RME ADI-2 has a sampling rate of 24bit/196kHz.
 
Would I be losing something sonically because the VS-100 can only max at 96kHz?

It's not quite true to say that "20Hz-20kHz = 44.1"
The Nyquist Theorem (google it!) states that the sampling rate must be twice the maximum frequency being sampled, so (theoretically) only a 40kHz sampling rate is required to sample an audio signal with a maximum frequency of 20kHz. 
However any frequencies above 20kHz in the input signal are not lost - they are mangled and come out below 20kHz in the output - e.g. a signal of 22kHz being sampled at 40kHz would be "reflected" about the 20kHz bandwidth limit, and come out at 18kHz in the output.  So the input signal is filtered with the steepest possible filter above 20kHz.  In practice, the 44.1kHz sampling rate was chosen so that the maximum frequency of 22.05kHz gives a bit of room for the filter to work between 20kHz and 22.05kHz.  If you think about, though, 20kHz - 22kHz is only 1/10th of an octave, so a 24dB/oct filter would only reduce signals at 22kHz by 2.4dB, which would still leave the artifacts clearly audible in the output, so MUCh steeper filters are used, which inevitably introduce distortions in the audible frequency range.

So it's a good idea to have a sampling rate higher than 44.1kHz.  Not just because it allows the frequency range to be extended, but also because it allows any filtering artificats to be moved further out of the audible spectrum.

Whether you can hear the difference between 96kHz and 192kHz is another matter altogether, though.  I suspect that the quality of A/D converters and filters are FAR, FAR more important than the sample rate at those sort of numbers.

Sorry if I caused offence.  It wasn't obvious to me from your response that you understood more than you were letting on.  And you really shouldn't be insulted by advice to look things up on Google, there's far more information there than you, I or any other mortal will ever know!

Thanks everyone for your help!