I've never been happy with this idiom, it doesn't make sense to me at face value. I've always just shrugged my shoulders when I've heard it.
But I was bothered by someone saying it today and decided to look it up. It turns out that it originates from the
old meaning of
prove - "to test," so that the exception in question
tests the rule.
It can also mean an exception the existence of which proves that a general rule exists. Wikipedia gives by way of an example "a sign that says “parking prohibited on Sundays” (the exception) “proves” that parking
is allowed on the other six days of the week (the rule)"
So glad I've gotten that cleared up!