I had an issue, not with Sonar in particular but with RME Totalmix.
All outs were also routed to analog 1+2 out - so running headphones out on one and then on main - became 6dB extra which distorted.
So I removed that extra routing to 1+2 in Matrix view - and that fixed it.
Don't know if applicable in your case - do you run a headphone mix too or something?
A bit puzzled over description - you say actual signal is below -12dB, but you had to set a limiter on -3dB:
a) why do you need that if so low on everything?
b) does limiter say any limiting done?
Either way, I would look at Totalmix meters what is moving with a playthrough.
No luck, look through dc offset and remove that.
Still no luck - did you have offset mode for fader on at some time - meaning there might be an internal offset done to the automation your don't think you have, because it's a straight line. Check the level on it!
Meters set to peak, and not rms?
Now, I'm out of ideas....