When you insert things that make the computer "think a lot" (FX, certain VSTis, etc.), the audio engine loses ability to keep up. These are easily noticeable and situations where increasing buffer, bouncing, or freezing tracks become helpful.
In general, when recording you want low buffers (latency) where the global bypass can be handy in a project already loaded. When mixing, latency is not an issue, so bumping up buffers is useful to keep the CPU happy.
Getting comfortable with shifting buffers and understanding plugins that load CPU is important.