I suggest you use the "N" key to turn of snap when you are about to perform an edit that does not require snap. Then just hit "N" again when you are done to enable the smart grid. This is how I work with it.
Another suggestion is to try lowering the snap intensity in the preferences. This will achieve what you are looking for as well. The only draw back is you need to go in and out of the preferences to toggle that behavior on and off.
Using the "N" key is the best option I think.