The audio interface's contribution to overall latency is small, and there is perhaps a 1ms difference between a cheap one and a top-of-the-line model. A better criterion for choosing a soundcard manufacturer is the company's reputation for solid drivers and rapid bug fixes. RME ranks very high in that department.
The good news is that all modern audio interfaces are really quite good! Even the little $5 one built into your motherboard is an amazing piece of technology (although you probably don't want to use it for serious audio production). First and foremost, choose an interface that has the
features you want. Number of inputs, front vs. rear-mounted connectors, line vs. mic inputs, phantom power, number of outputs, support for MIDI, S/PDIF and maybe ADAT, zero-latency monitoring, number and placement of headphone outputs, maybe supported sample rates, maybe built-in DSP.
Start with how many inputs you need, how many of them should be line versus mic/line combos, and how many need phantom power. Figure out how many channels you expect to record simultaneously (e.g. will you ever record an acoustic drum kit?) and how many things you might want to keep plugged in all the time (e.g. multiple synthesizers). It's a hassle getting behind the interface to plug and unplug devices, especially if you mount it in a rack. I prefer to keep most things plugged in all the time, which requires many inputs even if they're not all used at once.
Then think about how many outputs you need. Even though you only use headphones now, eventually you're probably going to want some quality speakers, maybe even two pairs, or someday need to route to a PA system for rehearsals. You may also want to set up separate headphone mixes, which will require separate outputs. I wouldn't buy an interface with fewer than 4 outputs. Mine has 10, although I've never used more than 6.
Since you're using a MIDI controller, you'll want an interface that accepts either 5-pin DIN or USB for MIDI, preferably both. I have 6 MIDI controllers and prefer DIN connections, so I've added a MIDI matrix so I can have them all plugged in all the time and can route them any way I like. Probably not something you need today, but you never know.
How the interface connects to your computer is a consideration, too. Firewire was the standard for a long time because of its better efficiency relative to USB. However, many computers do not have a Firewire interface but all of them have USB. If you go with USB, check how many ports your computer has. You can use them all up pretty easily, and you don't want to use hubs if you can avoid it. Buying an add-on Firewire card might make sense. Note that many of the newer interfaces support both Firewire and USB, a nice feature.
You'll have to decide between PCIe or external (Firewire or USB). PCIe cards offer the lowest latency, but introduce their own set of potential problems. PCIe cards don't have room for all the I/O connections you'll need, so they typically have a breakout box or cable harness. The best solution, IMO, is a rack-mountable breakout box with a single cable between card and box. Multi-headed cables add to clutter behind the computer and can be a hassle to change connections. PCIe cards, of course, also require a free slot that's not immediately adjacent to noisy devices. They are obviously not an option for laptop users, but may also not be possible with some compact form factors with limited slots. PCIe cards are also more prone to heat-related problems, add to the draw on your power supply, are subject to IRQ conflicts and are more difficult to install and swap out. Most lack the front-panel controls that external interfaces have for setting sample rate, bit depth, phantom power and channel enables.
Make sure your interface supports S/PDIF, even if you have no need for it right now. It adds little to the cost of the device, but if the day comes when you realize you need it, you don't want to have to buy a new interface to get it. There are two types of S/PDIF connections: RF and optical (also called TOSLink). The former uses RCA connectors and is the more common of the two for prosumer gear; the latter allows longer cable runs but may not be compatible with all your devices, requiring a converter. I'd recommend going with RCA connectors.
Some interfaces have features you probably never thought you'd need. Zero-latency monitoring is one, and
most interfaces offer it. Make sure yours does, too. Another feature you might not have thought of is an effects loop. This allows you to insert things like reverbs and compressors into the signal chain. Another feature you didn't know you needed: onboard DSP. This provides effects such as reverb, echo, phaser, and compression. If this is a factor in your decision, make sure the device gives you the option of monitoring effects without recording them.
Then there's sample rate and bit depth. All of the better units support 24-bit recording and the most common sample rates (44.1KHz, 48KHz and 96KHz). However, some have restrictions when using 96KHz. Many interfaces proudly advertise 192KHz capability, but this is a non-consideration. Nobody who understands what they're doing records at 192KHz. Don't assume that a 192KHz-capable device is higher quality than one limited to 96KHz.
Oh, cheez whiz, I'm getting overly verbose. Who do I think I am, Danny Danzi? I'll stop here and just reiterate what's been said above: choose your card based on features first.