Hey Beeps!
In a nutshell, you should eq each guitar part as its own entity. If one part is to be the bass leader, you have to treat it that way while always keeping in mind that as soon as you feel that guitar whoomf or rumble, you have too much low end in it. In the same breath, less EQ is better. The sound selection tells all. I barely eq anything these days other than little things here and there.
The same with mid guitars and high end guitars. The mid guitars can sound congested and muddy which will fight with certain keyboard lines, vocals and even snare drums. Add a lead guitar in and you'll wonder why you can't hear it.
The high end guitars can sound so razor sharp that they sound like little bees buzzing about. That said, that may be just what you want if they are in orchestration with other teams of guitars.
Guitar layer orchestration is challenging. I stay away from it unless I really need that type of production. Can I do it? Definitely....there just isn't much of a need in the stuff I write as well as the stuff my clients record with me.
The one thing you have to keep telling yourself with layers is....just about nothing will sound good on its own. As soon as you tweak for that "good all alone sound", you lose. The idea is to make the entire stack of guitars sound like a wall of "dear Lord, that slays!".
Another thing to keep in mind that gets said on here constantly is the arrangement. I can't stress how important that is. As you probably know with your theoretical background, you can't just glue a bunch of great ideas together. What happens there is....certain parts technically walk on top of other certain parts and you may NEVER be able to eq that stuff out of each others way.
I just completed a Deep Purple tune (Perfect Strangers) just for my head while testing some stuff out. It's scaled down and simple, but man, every time I hear it, I couldn't be happier with my sound selection as well as my performances. 2 rhythm guitars, one lead guitar, 4 keyboard instruments, one lead vocal, one back up vocal, bass and drums. I'm stoked about it as it is my own take and it just really moves me.
Here's how I record my stuff these days both for vocal stuff and my instrumental stuff:
There are times when I'll layer 6 backing vocal harmonies on something for effect purposes (high mid and low harms 3 left, 3 right)....but most times you'll hear two-part or three-part vox with an occasional 4 or 5 part. I have never and will never double my lead vocals. One and done.
On guitar, just about always 2 independent rhythms or one with a HAAS effect and then maybe one or two more to make a chorus part hit a little harder.
Lead guitars....always one and at times 2-3 for harmonies. I never double my leads for thickness. 1 because I never play the same thing twice, and 2 I've never felt the need.
Bass, always one track...or sometimes a DI and an amp or totally different signal. But just about always, one solid bass track.
Drums...I'm a hybrid whore. I love mixing and matching kits. But even there, it's usually kick and snare that are the choices for the hybrids. I have a great assortment of toms and cymbals both real and in sample format.
Keys, I play what's needed. Piano, strings, pipes, Hammond....special effects.
My REAL feelings on the above....I don't want to hide my raw/human capabilities behind massive orchestration production any longer. I did that in the 80's. Though I'm still a product of the 80's and proud, I don't miss the production pains I used to go through back then. I mean seriously.....do I really need to layer 10 guitar tracks? If you do, God bless you brother. LOL!
The point is, don't stress out with stuff that you might not be ready for right now. If you never stress over this stuff, then hey, you are a better man than I with more control. I can't tell you how many times I wanted to hang it up mixing to the point of frustration!
The more you have to work with, the thinner the mix (you have to make room for all that stuff) and the more headaches you can have. I just don't want to see the hard stuff deter you from being really good at the more simplistic stuff....and trust me, it CAN happen!
Record some cool stuff and post it up. There are guys that sit in their rooms messing around while reading....testing, and they might do 1 song in a year. Some never show anything because all they do is read, experiment and bite off way more than they can chew. This field is way easier than people lead you to believe. To really see that part, we have to get the basics down. THEN you can grow and move into other areas.
But don't let me discourage you. Do what you feel is best man. Like I've said before....if you can do a basic mix that sounds great everywhere that others give you props on as well and it gives you "wood".....do that 30 more times to see if you really know your stuff. THEN tackle the the bigger beasts. :) As always, good luck brother...always pulling for you and here for you. :)
-Danny