Too high often means higher latency, i.e. a music keyboard feels unresponsive as there is a delay.
Too low and you risk CPU spikes.
How high or high low it should be is down the the driver efficiency and the hardware, and how much CPU the rest of your software takes (i.e. melodyne).

This appears to be more like drivers to me anyway...
I must be getting muddled, I thought I wrote this somewhere already, I lose track of the thread that are everywhere...
 
If you have lag when pressing buttons such as arm/record (classic hardware/drivers issue) please under preferences -> Project -> Record - Disable "Allow arm changes during playback/record".
 
Also please try disabling "always stream audio through FX" if that doesn't help in preferences.
 
Thanks.

The only time I get pops and crackles is when I time stretch audio clips. When that happens the track meter peaks and basically it corrupts the audio clips. Even bringing them back to 100% time doesn't help. This happens even at very high latency settings.

Small buffers fill faster but don't move as much data. You get drop outs when there is more data than can be moved in time

Large buffers take longer to fill but move more data at any one time. Dropouts occur when the filling of the buffer takes too long.

So there's a minimum where the buffers are just big enough that they can fill in response to how much data is being fed to them.

There is also a maximum where the buffers large size doesn't slow the data down waiting on buffers being filled.

Alex: if your read the 2 posts I did you would remember that im not here much anymore, I dont have time ..as for reading the threads , the longer they get , the more I know I should help out. Keep up the good support . Il drop in to see how you all are doing !