ISSUE RESOLVED! Thanks to everyone for all the suggestions.
I changed buffer size to 128 samples and the distortion was much worse. I changed to 512 samples and it disappeared. The cost is a tad more latency than I want. But I hope some fine tuning can reduce that or learn to live with it
Many thanks again for all the suggestions.