It doesn't bother me to hear about one leaving something to move on. I don't think physical limitations is the only valid reason either.
The downer to me is when one may be forced to move on to something else when they don't want to.
In a world that has so much to offer and one is hung up on something they're not pleased with and can't or won't move on for whatever reason is a very bad situation the way I see it.
In your case, good for you. Nothing wrong with quitting something and moving on to find other pleasures. Hope you find them.