I've done more research, and it seems like USB is the way to go.  So now I'm trying to pick a brand.
 
1) RME gets the most recommendations, but those things are very expensive!  Since I actually do very little audio recording, RME seems like total overkill.
2) MOTU is also recommended, but, again, it seems like overkill.
 
3) Focusrite has been recommended several times, and seems to have options at a decent price.
 
4) M-Audio is what I am using now.  My Delta-44 is about 15 years old, so I'm sure technology has advanced since then.  Although my Delta-44 picks up the computer noises, that doesn't mean an external M-Audio card will have the same problem.  Still, it's natural to have an instinctive aversion to whatever brand is currently giving problems.
 
Any recommendations?
 
I'd love to find something for less than $200 USD.  Once we go above that, I have to start paying import duties and GST, which are 20%.  Otherwise, I'll have to buy something here in NZ, and prices here tend to be a lot more than 20% higher than those in the US!

IMO focusrite is best bang for the buck, under $200, but may not get the latency under 10ms for that money,maybe one of their higher end models... get RME if u want to spend more money for lowest latency.

Tripecac
I've done more research, and it seems like USB is the way to go.  So now I'm trying to pick a brand.
 
1) RME gets the most recommendations, but those things are very expensive!  Since I actually do very little audio recording, RME seems like total overkill.
2) MOTU is also recommended, but, again, it seems like overkill.
 
3) Focusrite has been recommended several times, and seems to have options at a decent price.
 
4) M-Audio is what I am using now.  My Delta-44 is about 15 years old, so I'm sure technology has advanced since then.  Although my Delta-44 picks up the computer noises, that doesn't mean an external M-Audio card will have the same problem.  Still, it's natural to have an instinctive aversion to whatever brand is currently giving problems.
 
Any recommendations?
 
I'd love to find something for less than $200 USD.  Once we go above that, I have to start paying import duties and GST, which are 20%.  Otherwise, I'll have to buy something here in NZ, and prices here tend to be a lot more than 20% higher than those in the US!

 
You can pick up RME PCI cards dirt cheap and they still develop drivers for them!  (9652 and 9632)

I ended up ordering a Focusrite Scarlett 2i4 (USB).  I did this mostly because I don't have to fiddle with internal cards, and it seems like a USB device is less likely to pickup the internal computer sounds than a PCI card.  We'll see how it goes...