Does the average listener really notice if the drums are panned the opposite way they would be if they were listening to a band on stage? I would say for most people, that somewhere in the region of 95-100% of music they listen to is recorded, not live. And I'm not sure if the mixing philosophy behind most of the recorded music they listen to was "make it sound as much like a live performance as possible." I would just focus on making everything sound good together instead of worrying about whether or not it sounds "real."

I position the drums from the audience perspective.  Hi Hats on the right side and ride on the left with different sounding crash cymbals on both sides of the kit.
 
I don't know anything about mixing fake drums but when recording live drums I use two mics over the kit.
 
There are several ways to do this but ... the point I want to make is that if you record live such that the overhead mics are picking up the whole kit with the ultimate intention of panning those mics hard left and right ... then, when you bring in the close mics on the individual drums, their positioning (i.e., panning) has to be consistent with their positioning in the stereo field created by the overheard mics.  

It certainly cannot matter from a perspective of how real the result sounds.  For an acoustic recording of a drum kit in a medium size or larger sound stage, the live sound source will sound like a mono source coming from whatever pan position the drum kit occupies on the stage.  Widely panned kit elements place the listener either right in front of the drums or sitting in the drummer's lap.  It's all just illusion anyway, so how could it matter where the kit elements are placed pan-wise?

well, as far as the overheads go, they capture not only the kit, but the sound of the room.... and the relative placement of the kit in the room... and all those reflections and whatnot are what the really smart part of your brain tells you what a 'direction' is....
 
so if a tom is banging off the side wall of the studio or stage, it is going to sound a certain way to it's own mic, and that also translates to the overheads, and if you mess with that relationship, you are in sonic trouble.
 

i'm talking real drums here, not fake drums.
 
fake drums (like superior ) all bets are off.
LOL
 

dmbaer
For an acoustic recording of a drum kit in a medium size or larger sound stage, the live sound source will sound like a mono source coming from whatever pan position the drum kit occupies on the stage.

A kit has several resonating heads in different locations that can be many feet apart from each other on larger kits and the resonating heads are oriented in different directions (i.e. snare/horizontal and bass drum/vertical).  I can certainly make an argument that the sound of the drums in a live setting are not localized as a single point source. 

I can certainly make an argument that the sound of the drums in a live setting are not localized as a single point source.

 
Though from an audience member's point of view those separate drums are only half a degree off from each other occupying maybe 2% of their stereo localization.  
 
One of the reasons why audience perspective is not always the best recording perspective.  A live concert usually means washed out mono with 80% room sound.  It can sound good when you are there, but you certainly should not record it from where you sit.  

If you see a concert at a large venue what you are hearing is the mic'ed (or triggered) drum kit through the PA.
 
If you are listening to an acoustic set of music with a un-mic'ed drum kit in a small venue and you are hearing washed out mono with 80% room sound then the acoustics of the venue suck.

If you are listening to an acoustic set of music with a un-mic'ed drum kit in a small venue and you are hearing washed out mono with 80% room sound then the acoustics of the venue suck.

 
Most bands play in rooms with poor acoustic, granted, but the ratio of direct to reflective sound actually favors the reflective sound more in good acoustics.  If you are standing 20 feet away from a kit, say in an acoustic jazz concert, there is not going to be any left or right information, just a single point and whatever is reflected off the room surfaces.  
 
If you ever go to an orchestra concert, you will also notice there is not much clear left or right information even though the orchestra angle is huge by comparison, it is mostly just a wall of reflective sound.  Unfortunately that wall does not record well.  
 
And in rock concerts, the audio is almost always mixed mono for the PA anyway.  

Bass is the easiest for me to make panning decisions on.