Frankly, I think the whole clip/loop boundaries thing is a red herring. I've been playing around with this for the last hour or so. I ginned up 4 lanes of MIDI driving Addictive Drums for 8 bars with a two-bar repeating pattern. I recorded all parts live, and quantized 50% just to tighten things up a little so the clips still started and ended randomly.
I then bounced each lane's output to a separate audio track, and started playing around. Addictive Drums doesn't null like Session Drummer due to sample randomization so I didn't try to do that; I just let the parts play together and relied on hearing any sync errors that were more than just phase variations.
First I tried every randomly sized loop imagineable with snap disabled, changing lengths, dragging them back an forth endlessly and clicking randomly inside the loop boundaries to change the Now time, then stopping and letting loops play for dozens of iterations without moving them. The only problem I managed to generate was having the MIDI notes drop out a few times (maybe once in 20-30 moves of the loop), but this would clear up as soon as I moved the loop range again. Nothing ever went out of sync.
Then I enabled Audiosnap with Autostretch on the audio, and started alternately changing tempos and randomly sizing and dragging loops. No problem.
Then I disabled Audiosnap, groove-clipped all the MIDI and Audio and dragged it out to measure 217 or something like that. I repeated the random loop sizing and dragging exercise, and could not generate any real problems. One time when resizing a loop, I managed to get MIDI and audio to go out of sync, but it was a fixed offset - not a drift - and cleared right up with the next loop move.
I don't understand why some environments seem to have this problem consistently, because I find it extremely difficult to replicate any thing resembling a MIDI-Audio sync drift, no matter how I abuse loop start and end points. The only thing that really matters is to make sure that groove clips repeat at whole measure/beat
intervals, even if not
on the beat, and that's easily accomplished by slip-editing their ends to the nearest measure/beat before groove-clipping them.
I dunno. It's a mystery.

Makes me never want to upgrade my machine or OS again.