There are so many variables with video and audio hardware and software that its really hard to make a recommendation.
One sure fire way that helps is to use a low resolution proxy (copy) of the video, using a low overhead codec. This greatly reduces the CPU load, leaving more for audio and plugins. Once you're done scoring or mixing on that low resolution copy, you can then mate the resulting audio to the original resolution video.
I should note that doing a proxy annoys some folks, but this is the way things have always been done in the video community when starved of resources. Back, say 10 years ago, people used to make DV proxies of HDV mpeg-2 video for editing, and then replacing the DV proxies with HDV for final rendering. With today's PC's, mpeg-2 is now a great choice for editing mpeg-4 video, which can tax many computers.