If I may ... (LOL),
I don't see this kind of behavior when I use the Step Sequencer, but I do see it when I create my own loops or use the Loop Construction View tool.
There is something that is not quite right when you create your own loops from a Midi clip, or Audio clip. It's been that way as far back as I can remember.
Try converting your clips in to Step Sequencer clips and see if that helps.
I completely gave up on making my own Groove Clips because no matter how far in I zoom, and how exact I make them to a measure ... they almost always end up drifting. It's especially evident if you use the Metronome.
Please try the Step Sequencer tip and let us know if that helps.
Good luck!
Bub.