Cakewalk UA 25 EX & Cubase 7 Latency issues

Hi! I recently bought a new PC and Cubase 7 full version, but decided to keep my UA-25 EX since it has delivered on my earlier system.
However, I got this weird problem that hopefully someone can help med out with.
Whenever I start a new project in Cubase, it works out great. I usually make pop/rock containing 5-10 tracks with midi-drums and couple of guitar-tracks and bass.
Everything works fine and I save and exit Cubase after recording a couple of tracks, and sometimes get the whole song-structure finished.
The problem is when I open Cubase later on, perhaps the next day or the next week, the input-latency is horrible on the UA 25-Ex input (Hi-z).
Whatever I do, the latency is like 1 second (a lot) when I strum the guitar and it seems like there's an driver issue of some sort. This is of course unbearable, try making a kick-ass tune at one take without the option to correct anything later on ;)
Playback works nice, the midi works nice, but the Guitar-input is having a lot of latency.
I can start from scratch on a new project in Cubase, and the latency is gone. Only when I reload a previous project, the latency-issue appears.
I do realize that this probably is a Steinberg/Cubase issue, but I'm posting it here to see if others have the same issue.
System Spec
Windows 7 Home Premium 64-bit
Memory: 16384MB RAM
Card name: NVIDIA GeForce GTX 660 Ti
Processor: Intel(R) Core(TM) i7-3820 CPU @ 3.60GHz (8 CPUs), ~3.6GHz
Midi: Axiom 49 Cakewalk UA-25 EX
Cubase 7 Full version
Thanks!