I realize that ultimately I have to rely on my own ears and wallet to make this decision for myself, but I'm really curious what latencies different folks experience and find acceptable in different situations.  For reference, the most I ever record at one time is a single stereo pair.  I have several devices with USB audio capability as well as an older PCI audio interface.  The worst of the USB devices can NOT operate reliably at a measured round-trip latency of less than 19.5ms. 
 
What are you guys getting?
 
 
 

Well, here are my thoughts on the subject, check out #3.

 My buffers are usually set to 512 so I get around 20ms round trip.  I don't do a lot of recording so that works for me. If I have to record I can set it lower and revert back to 512.
 From what I've read RME devices are great low latency devices. You can't touch one for under $500.  If I could do it over again I would have bought a Hammerfall PCI instead of the Yamaha SW1000XG and they were about the same price.  Any PC hardware today will not run the Yamaha.
 

Latency is a bummer with high counts of VSTs and VSTi's... fortunately, UAD has a somewhat solution to this in that the APOLLO interface has a monitor mode that is virtually  ZERO latency even with UAD plugins installed in the audio path.... you can specify if you want these PLUGINS to be part of the recorded signal or used only for monitoring.
 
Either way, this is quite an elegant way to get audio recorded into your DAW without any latency. I sometimes send the signal without plugs to the DAW and use the plugs for performance reasons (reverb, eq, effects, virtual amps, etc.) and as such get a good feel performance and a unprocessed Direct input into the DAW as audio (wherein, you can mix and insert plugins during mix down).
 
I am sure that there exists other solutions but this is the one that works best for me.  Albeit, the APOLLO isn't cheap but it is worth it to me.
 
Jim

At 24bit/48KHz, 12 input channels open, of which one stereo pair is recording and another 8 are being monitoired via track echo with delays on several and a low cpu load reverb buss I get a (measured) round trip latency of 4.8 milliseconds.
 
So long as I don't use any plugins that add latency of their own of course. I should probably add that I rarely use VSTi synths other than for drums so the cpu load is usually pretty low.
 
RME really are worth the money of you need stable, ultra-low latency operation. To be honest my previous UA-101 gave just under 10 ms round trip (the minimum possible at 24/48) without a hitch, but any higher latency than that and I start to have problems.
 
Oh, and add another 3 or 4 ms for the time it takes for the sound to get from the nearfields to my ears.

My interface driver's buffers stay at 2048 samples all the time. That's 46+ ms latency, and additional plugin latency can double that. I'd go even higher if my interface allowed it.
 
Such high latency is never a problem because a) my interface supports zero-latency monitoring, b) I don't use ITB effects while recording, and c) I don't play software synthesizers and samplers in real time.
 
I monitor and play outboard hardware synths, set to any patch that's roughly close to what I intend to use later via Kontakt / SampleTank / Omnisphere / Zebra / etc. and record just the MIDI output.
 
Latency is therefore a non-issue.

On FW with the TC Konnekt the newest drivers are down to 4 ms.  W/ USB on the Tascam UH-7000 it is like 12 ms. round trip.   I couldn't work at 20 ms - since I do a lot of soft synths.  Far too spongy.
 
@

bitflipper
My interface driver's buffers stay at 2048 samples all the time. That's 46+ ms latency, and additional plugin latency can double that. I'd go even higher if my interface allowed it.
 
Such high latency is never a problem because a) my interface supports zero-latency monitoring, b) I don't use ITB effects while recording, and c) I don't play software synthesizers and samplers in real time.
 
I monitor and play outboard hardware synths, set to any patch that's roughly close to what I intend to use later via Kontakt / SampleTank / Omnisphere / Zebra / etc. and record just the MIDI output.
 
Latency is therefore a non-issue.

Ditto. I monitor from the interface through a mixer which also has my Yamaha keyboard plugged into  it. I record MIDI using the keyboard's sounds and after recording I will add some VST instruments and effects.
I have the Saffire's latency set at it's max, 40 ms.

I do everything in the box so a project usually has 1 instance of Kontakt, 1 or 2 Superior Drummers ( one for the kit, another for latin percussion!), 2 or 3 TH2s, 1 GK Amp Pro (or Mark Studio 2/ just bought!), 4 or 5 synth vsts, and all the prochannel toys as needed on the channels and busses.
Since Craig's eye-opening tweak for lower latency I have been able to hover around 128 and 256 samples with no problems. Even while tracking! I do set the latency back up to 512 for playback when the mixing and mastering is ready. If I trim things down some, I have actually gotten down to 64 samples!

Sidroe
Since Craig's eye-opening tweak for lower latency I have been able to hover around 128 and 256 samples with no problems. Even while tracking! I do set the latency back up to 512 for playback when the mixing and mastering is ready. If I trim things down some, I have actually gotten down to 64 samples!

Here's the link for the "eye-opening tweak."