If it is a line level switcher then the popping problem is easily solved by using make-before-break switches.
The bigger challenge is avoiding loading problems if you decide to drive all four loudspeaker pairs at once!
I built a similar box several years ago, but I used relays (make before break) and momentary switches (and a handful of diodes<G>) so that only one loudspeaker pair was active at any time.
I also had four sources on a rotary switch, which worked, except that I only needed two inputs, the stereo converter and the PC audio output, for those times when I just wanted to surf the net and didn't want to re-route the audio<G>!
Total cost was under $60.