Recording at 96 kHz will solve
most issues that upsampling is intended to solve. However as the video shows, there's a significant audible difference between recording at 88.1 kHz and 176.2 kHz with a
very specific type of waveform that's generated
in the box. Then again, how often will you use a sound source that is affected in this way (answer: not often!), and how many people would even know that it was supposed to sound different (I doubt any).
Just to bring things back on course, all that matters is the emotional impact of the music on the listener. A great song is a great song at 44.1 or 96 or 384 kHz, and while a particular virtual synth might sound slightly different at these different sample rates, I don't think it would interfere with anyone's listening pleasure.
That said, though...you pretty much can't go wrong with higher sample rates, if your system is up to the task. If not, well, there's always upsampling.