latency issue - SOLVED

Hi all
 
With a couple of sessions in X2 Producer, I'm getting a strange latency issue.  I'm using a Focusrite Scarlett 2i4.
 
So I start with recording a few audio tracks - guitar, vocal, etc, everything goes smooth.  I usually record with input echo on, so to add some reverb/compression for the performer.  I always set latency at 2ms for this.
 
Then I go to add some piano or other soft synth track.  Latency is fine until I hit record.  It gets bad enough that it's impossible to play.  If I then open the driver control, the latency is still set to 2ms.  I have to change it, apply the change, then change it back to 2ms to get it back to normal.  But as soon as I hit record again, the long latency comes back.  Then I notice it also happens with audio tracks with input echo on.
 
I had to work around by exporting the mix into another fresh session, recording my MIDI on a sft synth track, then going back to the original file, insert the same soft synth, and copy the MIDI clip(s) into place.  Annoying and tedious to say the least.
 
This only happens with some files and as far as I can tell, there's no pattern with which soft synth I'm using.  Generally I use True Pianos and or Kontakt, both of which have been perfectly reliable in the past.
 
Any ideas how to avoid this strange problem?
 
Thanks guys!
 
 

Please respond with ALL of the following information:
 
1.  From audio interface: Sample Rate, ASIO Buffer Size
 
2.  From Sonar: Sample Rate, Driver Mode, Total Roundtrip Latency
 
3.  For the project with the latency issues, please list EXACTLY what plugins are loaded.
 
4.  What happens if you hit 'E' on your computer keyboard, then try to record?  ('E' is a shortcut key that is bound to Sonar's 'Bypass All Effects' functionality, hit 'E' again when done testing record out to turn effects back on).
 
5.  Do you have a Wi-Fi adapter present on this computer?  If so, disable it temporarily, just prior to launching Sonar, and you can enable it again after you are done with your Sonar session.
 
If the Bypass All Effects makes the latency go away, then one or more of your plugins loaded in the project are causing the latency problems.  Plugins that do this are those that are meant for mixing/mastering and NOT meant for tracking (recording).  These effects will either use look-ahead processing, like Boost 11, or will be very CPU hungry.  What most folks do is to use less latency-prone effects while recording, then they deliberately raise the ASIO Buffer size a LOT, then they put in the latency-prone effects for mixing/mastering.
 
SOOOO, please review and respond to ALL the above - thanks, 
 
For whatever the worth, look at the bottom of this post at my settings - these are good for recording, and unless your plugins are a problem, you should be in good shape running with those.
 
Also, be aware that keeping your latency TOO low can also cause latency problems.  You want to shoot for a Total Roundtrip Latency value of around 10 milliseconds or just a bit less, as a reasonable balance that will give you good sound quality without dropouts or making your system work too hard.
 
Bob Bone
 

Oh, thanks Bob for the additional insight.  I guess the LP64 is really meant for mastering, etc.  I just use it as my go-to for some crazy "telephone effect" EQ'ing because it's easy to draw a curve on there quickly to get the effect.
 
But just turning off the LP64 while recording has solved the problem too.
 
Thanks!