...after having soloed it using Shift + click on its number?

'Ctrl + click' seems to work for me.  Or alternatively, you can select the number that is now off and click on the little power button icon to switch back on.
 

Ctrl + click turns the oscillator on and off. Soloing is done via shift + click. But un-solo doesn't work. You have Ctrl + click all the other (now muted by the solo process) oscillators' numbers to power them back on, or - even more tedious - as you say, click their little power button.