Make sure your MB/USB drivers are up to date and try different USB ports. The specs call for a "storage class" USB port.
Mine is working great plugged into a USB3 port, but I did have to hunt down drivers for the MB ports.
I can run all day at 64 samples, but my typical projects are modest. 64 samples @48K gives me a measured 8.3ms RTL. There other general audio considerations that Anderton has posted about well, such as disabling unused integrated HD audio ports.
My system is an i5 quad, 16G memory running Win8.1x64. I went straight from XP to 8, so I can't speak (type?) to anything specific for in-between versions.