Roundtrip latency of 16.4 ms is okay. You probably won't hear any delay. I'm pretty sure I couldn't hear it.
You may be able to use WDM or WASAPI Driver Mode and get acceptable performance, if they're available. You'll be able to use 2 devices in WDM... not sure about WASAPI.