MIDI latency after play/record

Using a MIDI keyboard for sketching in drums and keys, I have notes the latency increases after I either rehearse to a playback, or record.
 
Before playback/record, there is no apparent latency from pressing a key and all of seeing a meter jump on Sonar and hearing the synth play back the sound.
 
However, the moment I press play/record, a disgustingly long latency (perhaps an 1/8th of second) is induced and it won't go away until I close the project.
 
I have mucked around with all the MIDI controls, and even changed the drivers (ASIO is the normal mode for me).
 
Any suggestions as to cause and fix?