Hope it works for you, Konrad.
FBB pointed out the other day that he found these settings to be a balancing act, between the audio buffers and the I/O buffers.
Each system varies, and. You need to find your system's sweet spot. So it's a case of increasing one and lowering the other, and vice versa, until you find the right balance.
I just read that back and it's not very clear, but I have to shoot out now to watch my son's Nativity Play.