Normally, unexpected latency in a particular project would be due to some pluging using a lookahead buffer that induces automatic plugin delay compensation, but not if the latency is more than a small fraction of a second. Latency that high would make me suspect a driver probelm... but not if it's only one project.
Are you sure the latency is really more than a second? If you hit the PDC (Override) or FX Bypass buttons in the Mix module, does it help?
Also, it's a little confusing that you refer to the Alesis kit as a 'virtual instrument'. The Alesis kit is a MIDI controller with combined hardware sound module and MIDI interface that I gather you're using only send MIDI to EZdrummer which is the virtual instrument. Are you connecting the Alesis MIDI output by the USB port or by the DIN connector to another interface?