You usually wouldn't change that figure. It allocates the number of buffers not the size of the buffer. Depending on the driver for your device, latency is usually expressed in ms (milliseconds) or samples.
During mixing you usually increase your latency. During recording if you are monitoring while processing live fx and or VSTi's, you would reduce your latency. If you are able to monitor direct from your sound card and are not processing live fx or synths etc, there is no need to change your latency. Expressed in samples this is usually from 32, 48, 64, 96, 128, 256, 512, 1024 etc
At a given sample rate eg 44.1 kHz, 48 khz, 88.2, 96, 192, 384 kHz, a buffer of 48 samples would be a certain number of ms get the idea. Eg a buffer of 1024 samples at a sample rate of 44,100 Hz is equal to 1024/44,100 *1000 = a latency of 23.22ms