Removing tempo map after recording

I had a session yesterday where the plan was to use a tempo map w/ the metronome, but the drummer wasn't playing well with it.  So we scratched that idea and had the rhythm section bang it out with him with some scratch guitar and whatnot.  Mucho better.

We have clips split here and there because I had a few edits to do and they wanted to punch in at times.  No big deal.

Now I have a session that has tempo changes at the wrong spots in the song.  Nobody will ever know once all is said and done, but I'm curious if there is a way to remove all tempo changes while keeping the clips all in the same place relative to one another.  I tried to lock the clips in place, but that didn't work (as I suspected it wouldn't).

If I remove all temp changes the clips obviously lose their spot relative to one another due to what i suspect is overlap into the next tempo change.

I assume the answer will be "no, you're stuck sorry man".  But hopefully I'm wrong.