Anderton
Could be memory, could be CPU, could be the driver...could be all three. Try increasing latency to see if that solves the problem.
+1 to this - increasing latency usually solves this.
If you're finding that an increased latency is sluggish for recording though, consider freezing the tracks you're not planning to edit further. This will reduce the CPU usage, although it might increase disk I/O.