Well, for me, coming in at about $1000 cheaper than the RME, is the Mackie Onyx Blackbird. Crystal clear sound with virtually non-existent latency, in a firewire-400 platform. Lots of great options for routing and daisy-chaining.
I saw someone already brought up wi-fi. That is definitely a latency monster.
Of course it should go without saying, but it's always crucial to trim the digital "fat" from your PC's background and redundancy functions. I always disconnect the internet entirely and disable virus software as well. I've eliminated as many useless programs and applications as possible. In all, I find that I never run into the least noticeable bit of latency until I start piling on tracks with soft synths and adding a lot of effects plug-ins.
Another huge help is to freeze tracks that are complete, or at least not the focus of the day's work.
I still have an old interface from 2004 that has only 2 audio inputs. One of the first models on the market to include 24bit/96k conversion. It works flawlessly with my PC set up. Like it was just released on the market.
Moral of the story ... dig around in the software and of course maintain drivers, and you'll solve most of your latency problems before you even consider an interface.