I've tested a bazillion interfaces going back decades. Operating systems for both Windows and Mac have settled down over the years, and it's not as difficult to interface with those systems as it once was. Most drivers are at least decent and work fine, but much depends on your specific configuration. I'm not seeing particularly big variations in latency depending on different drivers. If my system works with one driver/interface at 96 samples, it pretty much works with any driver/interface at 96 samples.
Specs also have more similarities than differences. Even an inexpensive interface like the Mackie Blackjack has very low noise and distortion. A few interfaces will be
outstanding in some respects, but the overall level of quality is quite good.
The main differences among interfaces are the tradeoffs made in the feature set. For example TASCAM just came out with a new interface, the UH-7000. They put all the design work into the fidelity and not the number of inputs. So it's basically a 2 x 2, but the mic pres are excellent (e.g., isolated power supplies for each channel, great converters on the component level) and they market it as a dual mic pre as well as an interface.
Or consider PreSonus. If you have a StudioLive mixer, their software is designed for seamless integration so it would make sense to "keep it in the family." Focusrite really has the FireWire thing down. MOTU makes some industrial-strength interfaces with enough inputs to record an orchestra.
What I suggest is you "design" your ideal interface. How many inputs do you need? Will you need to add lots of mics sometimes (e.g., to record a drum set), and therefore an ADAT interface would be helpful? How many hi-Z inputs will you need to use at one time? Firewire or USB? Do you need onboard DSP so you can add processing when doing zero-latency monitoring (e.g., reverb in the headphones for the singer?) Does it handle the sample rates you need to use? Do you need 5-pin MIDI connectors or does all your MIDI gear communicate over USB?And so on...
Compare that list of features to what's out there and look for the closest match. Then, search for "'name of interface' problems issues" and see if it acts up with some software or hardware more than others. The only other recommendation I'd make is that if you get something with more inputs than you think you need, you can leave everything patched in and ready to go...just select whichever input has the source you want to record.