Multi-tracking without a metronome

Using X1 Producer, and I'm working with a singer-songwriter who plays piano and sings.  Her songs are fairly dynamic, with tempo changes, pauses, etc, so it doesn't make sense to record her to a click.

So she lays down a MIDI piano track freely.  Then I record her singing and small guitar parts here and there.  Her timing is pretty good, only problems occur after a "hold" when the piano comes back in and she needs to sing at the same time.

Anyway, I'm also adding things like bass and drums (all MIDI on my keyboard).  So I figured on re-aligning the ruler to match her piano part by using the "Shift-N" function - insert meter, so I can quantize the drums and bass parts.  I bounce the MIDI piano to audio, set my snap to transients, and set the downbeat for each measure.  Tempo map updates as I go along, everything seems fine.

Now the problem - vocal and guitar parts that are already recorded begin to slip out of sync.  It get worse the further along in the song it is.  Why does this happen?  I figure if I'm changing the ruler, and I have a vox clip at measure 58, it should remain at measure 58, even if the location of measure 58 changes.  Am I wrong?

Would it help it I bounced each audio track as a single clip starting at measure 1 before I change the ruler?  Or should I be doing the ruler right after the piano part is recorded?

Thanks!

I use this method all the time to get a genuine live feel. I usually track midi on keys or V-Drums and use the midi tracks to set the tempo map.
 
It's time consuming but I've got the best results by going through the midi tracks and setting the 'now' time using Shift N, as you say, at the start of each bar. The audio is completely unaffected.
 
The only issues I've had is at the very start of the process. I've found that it helps having a rough guess at the starting tempo and setting this as the default, and then sliding all the tracks over so that the first beat is at the start of bar 2 on Sonar, i.e. there is a gap of one bar. This seems to help.
 
I hope this helps. Let me know if I can help further.
Best wishes,
Toby