Just to be clear, are you using Z3TA+ 2 as a stand alone application driven by your controller, or as a plugin/client synth in a host program like Sonar, when you notice this? If in a host, the latency should be pretty much controlled by the host settings (buffers) and the audio interface driver latency settings. If the synth is using more resources than other synths, it should not cause an increase in latency under Sonar, it would cause dropouts if the buffer sizes were too small. If you set those buffers higher to prevent dropouts, that would cause latency.
It is possible that you are noticing a delay from keypress to sound that is not due to latency per se. Some patches have envelopes that do not ramp up until the key has been pressed for some time. The envleope may not produce audible sound at your volume settings until it reaches a threshold. Look at the envelopes for the patches you are using that seem to cause this slow response.