and here comes the truth by an expert below 

Latency has nothing to do with the sound of an individual track. It's impossible to "hear" latency in a track. If track alignment were off, this would be easy to check by rendering (which, by the way, is basically the same thing as freezing), but it's not the case. This has nothing to do with ears or opinion, it's easily verifiable. This is psychoacoustics at work.

Thank you to everyone who added insight to this topic. I just wanted to make sure I wasn't upsampling any plugin that would cause aliasing/phase or other negative side effects. Sounds like other than causing the rendering to take longer, it's a fairly safe approach to upsample / render any plugin I choose.
 
Thanks for easing the worry too much side of me. Appreciate it.