I believe that mathematically you take the sample buffer size and divide it by the sample rate to give you the latency, so that:
512 sample buffer size at 44.1 khz sample rate gives you 11.6 (rounded) ms latency
512 sample buffer size at 96 khz sample rate gives you 5.3 ms latency
It of course comes at a price, that being curb-stomping your CPU at higher sampling rates.
I have a crazy fast CPU and 43 GB of memory, with what I believe is a reasonable audio interface (specs are below), and I still record at 44.1 because I don't want things pushed to the edge where it is a constant worry.
I suppose I will spend some more time tweaking things - but if there is not that much gain to it, I will probably not do too much. For example, I found that I could trim off about 2 ms in total round trip latency by turning off all kinds of services and disabling my antivirus software and all of that, but that just didn't float my boat for having to lose internet capability during audio processing, so I undid it.
Remember too, that it is the Total Round Trip Latency that is ultimately sets the quality of sound output - that initial calculation is only a piece of that.
Bob Bone