I don't think so. Ethernet switches expect to see ethernet-formatted packets, with ethernet headers, etc. Cat5 is just a cable and connector spec that is robust for a certain level of digital information capacity over a certain distance and a certain physical strength (bending & tension) for dragging around. It is used in a variety of digital applications, not just ethernet.
In the converters you mention, I believe the digital signal carried over cat5 is most likely formatted around the toslink or ADAT specs, not ethernet. Here's a quote from one Toslink / ADAT product: "Without any kind of compression or other potentially sound-degrading conversions, the input signal is 100% bitwise transmitted in real time to the output." I.e. there is no mention of converting to some kind of ethernet packet format.
On the other hand, if you always want to send digital signals from one source to one destination, all you need is a patch bay you could make yourself with a panel of connected RJ45 jacks.