Based on what others report, it seems that ~27 ms may be about as low as you can get with the Focusrite at 44.1 KHz. If your CPU can handle it, moving to a higher sample rate will also lower latency. I'd consider that a last resort, however, since you'll also be lowering the threshold at which the CPU runs out of steam on a large project.
Despite that limitation, I am a happy Focusrite user. That's because I just don't care about latency most of the time - the interface is capable of zero-latency monitoring when recording audio from outside sources such as vocals and guitars. Consequently, my buffers stay at 1024 all the time. The only exception is when I'm auditioning a new sample library and want to play it in real time with minimal latency, at which time I go down to 64 samples and run the sampler standalone.

All else is in doubt, so this is the truth I cling to.
My Stuff