Unless you use audio over DisplayLink, there is a fix. With the DisplayLink adapters connected, disable the audio drivers for them.
Control Panel -> Device Manager -> Sound, video and game controllers -> Right-click on the associated display adapter device and select "Disable"