What are you using for the Firewire controller?
If it's not a good TI chipset Firewire controller, that's where you want to start.
Video is most likely not the culprit.
Also, check the system's DPC Latency using Latency Mon.
High DPC Latency will cause audible gliches.
Essentially, High DPC Latency equates to hiccups in data flow.
If your audio interface's buffer isn't filled in time, you'll hear audible glitches or (even worse) the transport will drop-out.