20ms (not samples) is roughly a 1024 sample AUDIO buffer. Higher buffer (2048) may help but you are then putting stress on your drives since you are pulling samples (EW).
First, check your "prepare using ms buffer" for MIDI (defaults to 250) and jack that up to 1000 or 1250. See if that helps, and this is a MIDI parameter. Also check your disk I/O buffers (both read and write) and set those to 512 (default is 256 IIRC). My system freaked on a large project once and that 512 cured it.
Last resort (ONLY)... If that still doesn't cut it, save the project with a new name and try bouncing all the tracks to new tracks, then delete the originals (BE SURE TO SAVE THE PROJECT WITH A NEW NAME FIRST). This will turn an export into a wav file mix only (as simple as it gets... very little stress to your system). A 1024 sample audio buffer should suffice.