A larger bit depth creates a larger file and larger numbers for your computer to calculate with, so it increases the load on your system somewhat.
Larger bit depth has to do with the accuracy and dynamic range with which a waveform is drawn. Generally I don't believe it's useful to set the recording file bit depth higher than your soundcard supports. There will just be extra zeros tacked on the way I understand it. However if you're using a lot of FX you're doing many calculations on the source material/waveforms. So extra accuracy could theoretically be beneficial. As an example; 5.253 gets rounded off to 5.25. But twice 5.253 is 10.506, which becomes 10.51, not 10.50 (twice 5.25). Conversely, dividing 5.253 by two becomes 2.6265, another decimal, which may be lost without sufficient bit depth.
In practice, I haven't been able to hear a difference between 32 bit and 24 bit, but I do hear the difference between 24 and 16. I always stay at 24.