Great question, got me thinking...
I'd always assumed it was high frequencies that did the damage, based on the ideas that a) those are what are most likely to be physically painful at a loud concert, and b) that it's the high end you lose first as hearing degrades from long-term damage.
But sharke's comment got me thinking. Low frequencies represent far more energy than highs, perhaps 10,000 times more sound pressure at the lowest end of the spectrum. When the OP upgraded his speakers, what was he now hearing that he couldn't hear before? The sub-100 Hz range.
I recently joined a band and have begun using musicians' earplugs, something I'd never done before. They definitely help, but since they're off-the-shelf rather than custom-molded they do little to attenuate low frequencies. Now I'm wondering if they're protecting me as well as I'd hoped they would.
I do agree with the assessment that minimal damage is probably being done, as the OP is monitoring at sensible volume. But the fact remains that, statistically, audio professionals lose their hearing faster than the general population. I'm sure George Martin was not in the habit of listening to loud music, but even he had to retire due to hearing loss.