Latency Issues

Every few beats the tempo slows down then speeds up. This happens on my own midi files and midi files I’ve down-loaded. It happens during record and play back. I can not only hear it audibly but can actually see the time meter pause a few milliseconds as well.  It’s as if the program stops or pauses, then continues. It would drive a drummer mad! 
  
I’ve seen other threads of people having similar problems. I’m learning that there are “different kinds” of latency issues and different tips have helped different people but haven’t found a fix for my particular latency issue yet. 
  
I’ve tried some of the tips I’ve ran across like making sure the ASIO driver is clicked, increasing the buffer size, changing the ticks/beat in the time code, un-installing and re-installing the UA-25 EX drivers, deleting the AUD.ini file, cutting and pasting tracks into a new project, and other less critical tips as well, but still have the problem. I have the “advanced driver” switched to the “on” position on the UA-25 EX card as recommended. I believe I have plenty of RAM because it worked just fine until a few days ago. 

I've read where people can “change” the latency. Where is this done? 
 
Incidently, I don’t have this latency issue when I record audio; just MIDI. 
 
Also, when I dis-connect the UA-25 EX card and play midi files with Windows Media Player through the internal (stock sigma-tel) sound card I don’t experience the latency. 
  
Thank all of you who have offered advice. 

Any more help or tips would greatly be appreciated. I’m dead in the water until I resolve this issue.