The beauty of "lowest possible latency" is in the eye of the beholder.
If you're monitoring via hardware... and that's the extent of your needs (low latency monitoring), then you're set.
However, if you're trying to do something like trigger drum samples from an electronic kit (and have them feel as immediate/responsive as possible), having lowest possible latency is a much bigger deal.
Playing live:
When you get fairly distant from a monitor speaker or the mains (say going well out into the audience without wearing in-ear-monitors), the latency can be huge/distracting.
Can you compensate? Yes
Is it comfortable? No
All audio interface drivers are NOT created equal.
ie: Presonus Audio Box and RME Fireface UFX can both be set to a 64-sample ASIO buffer size.
Reported latency will indeed be similar.
Guess which one allows running heavier loads glitch-free?
That's part of what you're paying for when buying an RME audio interface.
I know Aaron (spear-headed pooling the round-trip latency information in the link above).
He's a good guy... and certainly means well (to help other folks).