This is a really strange one.  Let's say bars 3 to 40 are selected and I want to alter or affect one note somewhere within that selection.  Right now, to do this, I have to go to piano roll view even if I am busy somewhere else and scroll up or down to where I think no notes are in the track and click on a note on the piano roll keyboard and this removes the selection.  What I'm looking for is something easy like hitting the escape key or something similar.
 
I understand that if I want to select anything else that the new selection removes the old one but in most cases I simply want nothing selected.  What am I missing?

CNTRL+SHIFT+A = Unselect all tracks and their selections.
This shortcut will become your best friend.

Assuming I understand what you want correctly, and you're in Track View, just click on track's blank space in the Clips pane.

the 5 on my numbers pad deselects all

Thank you guys.  Three better ways than what I had.