Little late to the game but, I have the us1641 which is more or less the same interface.
What I found was when I switched to Windows 7 with the 64 bit drivers everything got way better. Under XP 32 bit the interface had issues, terrible latency and drop outs. MIDI would stop working too.
I've been using it full time now for a few years with W7 64 bit and all is good. It still is not a great performer with Round Trip Latency, mine is around 9ms, but if you look, many interfaces are between 5 and 10 ms so I figure it is par for the course with any budget interfaces.
There is no other issues with drop outs or tracks not syncing up etc.
For the money it's a good piece of gear.
So I don't know why your only using 32 bit but I would change to 64 bit pronto.