I'm about to buy Scarlett Solo.
Not 2i2 because I don't need even one microphone input, let alone two, once I have a YETI usb microphone that is enought for me for now.

But before I spend my precious $100, I would like to get opinions as to whether this interface solves my current problem: MIDI latency while recording with Synths.
My most common case: I have my Axiom 61 MIDI controller connected via USB and want to record an instrument via Kontakt. 
Even using ASIO4ALL, the lowest acceptable latency is 512 samples. And I paid a high price for use ASIO4ALL, with all its bugs ...
So my main question is: will I have significantly reduced latency for MIDI recording with Scarlett Solo?
Or is there still cheaper solution to give the same result?

Hi. It would also depend on the specs on your pc, but I believe the answer is yes. And use the proper ASIO drivers for the AI you buy. (Ps. ASIO4ALL is not a real ASIO driver...)
Best of luck.

The answer is yes.

A USB mic is going to be really hard to use with other gear. They have notoriously lousy drivers even though some big name mic companies are getting in the game.

Get rid of Asio4all driver. It become a modern day horror on a PC and even though some get it working aoK many have problems and for just as many just having it installed can create problems for your audio device drivers.

And IIRC, you cannot use two ASIO drivers at the same time. Meaning that you would have to decide which component to use at any time (cannot use audio interface and usb mic simultaneously under ASIO). 

The Scarlett mic pre-amps are reportedly pretty decent. A USB mic will have to include the transducer, the pre-amp and the USB convertor all in one unit. So it is going to be difficult to find the quality in a USB mic that matches a good standard mic using the Scarlett circuits for the same price. For $130.00 you can probably do better. Plus the USB mic can't share ASIO drivers, and it will represent a second audio interface in the same computer which is notorious for causing problems.

Given the specs on your PC, all things being equal (e.g., not having elements installed that degrade performance), I think you would have no problem getting 256 samples on complex projects and probably could go down to 128 or even 96 samples much of the time.

Thanks for the tips. 
I've ordered an Scarlett Solo and I will be getting this in a few days.
New audio interface, new life!

Buy this:
http://www.sweetwater.com/store/detail/US2X2
 
You will find ways to use it.

You asked for tips but just went ahead and bought what you were going to buy anyway. 
The Solo will be "OK" but for a few more dollars you could have bought something with a lot more flexability. Like the above mentioned Tascam. For the money it blows away the Solo (and the 2i2 and the 2i4). Main reason is it can use external power. USB powered interfaces can be a PITA on many systems. Not only that, but having a matching set of inputs is super important for a lot of situations, like a stereo board mix. And having MIDI ports is a must have for many. 
But the Solo will give you better drivers and stability. USB mikes are the bigest waste of money I've ever seen because they are useless for interfacing with any pro audio gear. 
For the same money you can buy a proper XLR mike which can be used with all proper audio equipment.