Am I missing something? The OP says that audio "keeps moving out of synch." His description seems to indicate that it only moves out of synch one time, at the end of the recording.
The symptom is different if what is happening is that:
1.the entire audio track shifts ahead or backward along the timeline (the direction of the shift is important), without changing the relative note to note relationship,
than if
2.the track timing drifts continually from a steady beat.
1. Suggests delay compensation error, which may be quite long if a plugin is reporting a long delay.
2. Suggests a clocking issue.