The guitar, the quality of input you are plugging into and perhaps the audio to digital converters themselves.
IME though none of these sims sound as good as they are presented right "out of the box". They need some tweaking.
The other thing is that many of the models are designed to respond to the input level like a real amp. So if you had a tube amp and you tossed a simple "gain" pedal in front of it or used a guitar with really high output pickups you are gonna work the tubes harder and get more bite/drive/whatever.
A perfect example is in GR5. One of the most useful "Components" in the suite is the "Gain" module. If you put that in front of the various amp models and turn it up you can see how the amps respond to the extra "virtual" input level.
The gain module doesn't actually add any real distortion or anything (you can play through it on it's own and it mostly just boosts the volume). It's just making those phoney baloney "tubes" light up and do their thing.
Also, of course, different pickups are going to give different tones. I never expect a good "blues" tone until I engage the single coils on my Pacifica (it has been modded with a "Hot Strat" config so it has a humbucker at bridge position that can turned into single coil with the push pots... and it has single coils in middle and neck positions).
Try adding some quick reverb to the sound too for blues or choose a "large" room for the cab IR stuff. That helps a lot for that style (and... well many styles to certain degrees).
Cheers.
Edit: and for the reverb thing set your reverb (if you can) to focus on the higher frequency spectrum instead of EVERYTHING. Reverb on the low end tends to muddy things up so you want the bottom quick and tight or completely dry and the room "echo" dancing around the higher frequencies.
This is just my own experience. I am, as always, just a student... not a hardcore pro.