I use ASIO4ALL for several devices. It wins every time compare to all not native ASIO approaches with the same hardware.
But in case there is a working native ASIO driver, it should be used instead (as was mentioned several times already).
For ASIO4ALL: in case there are more then one audio device exists in the system, ASIO4ALL will by default connect to all of them and neither are going to work. You can disable not required devices in the ASIO4ALL settings (I do not mean SONAR settings), and do not forget to restart the driver/system after that change. Start with really big buffer, like 1024. From my experience ASIO4ALL can not go really low with USB devices.
For hardware: check that you do not have any USB hub / long cables between your interface and the computer. Start testing with all other USB devices (and hubs) disconnected. Try several ports before you come to any conclusion (depending from the system, different ports can even have different hardware on computer side and in all cases interrupts distribution makes some ports better then other).
With Latency Monitor check that you do not have something "heavy".
I could always get ASIO4ALL working ok, even on 10m long USB extender with hub and crappy interface. But I will not claim I could enjoy playing software synth throw such system