Mastering (properly) is a VERY technical / complex process. Plugins and stuff make it easy to do with presets and stuff like that, but there are a lot of things that could factor into that. As an example, you could have some low frequencies in one song, that are triggering the limiter (but aren't very audible, because they're either outside of the range of human hearing, or just not very noticeable) whereas, if you have higher frequencies (midrange frequencies, etc...) that are causing the limiter to trigger, then they can cause pain to your ears, because they are RIGHT THERE, hitting your ear sort of at "bullseye" or whatever. (if that makes any sense).
So even though in the end result, you might have two waveforms for the mastered tracks that look similar, if you look at them through some sort of spectral analyzer or something, you'll PROBABLY see that the "louder" tracks have more action happening in the midrange frequencies, where the quieter tracks might have more mud & stuff, happening down in the sub-bass frequencies where you can't hear them.
There are other things that could factor into that, but that's one thing I would suggest checking (if you haven't already).