DIsk I/O Issues...some questions....and no mocking from the peanut gallery!

I'm currently working on a project with a fairly impressive (some might say I'm using the wrong word here, that "ludicrous" might be more appropriate) track count.  I essentially use Sonar X3 as an unlimited multitrack recorder (no soft synths at all), but there are a few plugins on some tracks.  I am currently at 97 tracks, and it's likely that this project will exceed 150 tracks by the time we're finished with it (we are trying to achieve a pretty dense sound).  I am a firm believer in reamping, so every guitar track on this project (and there are LOTS of guitar tracks) has a corresponding direct track so that if the band decides they want a different sound later on, we can simply reamp without having to re-record.  Probably half the tracks are scratch tracks, and all those tracks have been archived, along with all the direct tracks.  Yesterday, Sonar started halting on one particularly dense section of the song, and I noticed that the Disk I/O icon had gone red.  I monitored the disk I/O, and it was hovering between 60% and 88% most of the song, but upon attempting to play that section, went to 100%, and Sonar would just give up.
 
I've been poking around here on the forums this morning, and found a couple of posts about I/O buffers, which were set to 256 for both recording and playback.  I doubled the recording buffers (512) and quadrupled the playback buffers (1024) and disk I/O dropped pretty dramatically.  I then doubled each of those values (1024 recording, 2048 playback) and the disk I/O dropped even more (now around 25-30%).
 
Prior to reading on the forums, I was prepared to buy a faster hard disk (the disk I'm currently recording on is a 1TB 7200rpm SATA drive, and yes, I have the scratch disk on C: and the track disk set to D:) to alleviate the problem.  Now, with the improved I/O numbers, I'm thinking that isn't necessary.  But I read that increasing the buffers can lead to other issues.  What might some of those issues be? Did I raise the buffers correctly?  Should the recording buffer be higher than the playback buffer, since I'm trying to play all zillion tracks while recording two new ones?
 
Thanks to anyone who can lend insight to this issue and these questions....and thanks for reading....
 
Paul
 
PS...this is my first project with this band, and the leader of the band is a well-know artist in my area.  He used to run one of the more popular and successful studios in the area, and I'm trying to convince him he should continue to work with me (this is a hobby, we're both a million years old, neither of us is gonna make any money off this project), and once we're finished with the project, I'd like to maybe find some other clients to work with, so this project's success is very important to me.  I'm working with people who have 30-40 years of professional musicianship under their belts.  On a side note, they are all blown away with what we've recorded so far...several people have commented that they are surprised that something of this quality came from a "home studio".  So far, Sonar has impressed a LOT of people, most of whom are Mac users using ProTools.
 
Specs:  Sonar X3e Producer on home-built dual Xeon quadcore server motherboard, 8GB RAM, Mackie 1640i audio interface.

Also you can freeze tracks you are not going to adjust (except for say volume / panning) - you can quickly unfreeze if you need to work on the track again
Nigel
 

Whilst this may sound a little silly some recent research I did revealed that XEON CPU's and Motherboards are not really suited to music production.  Most music software is not optimised to take advantage of the XEON functions and they may in effect be missing some of the advantages of the i7, i5 and even i3 processors.

I would second Bristol_Jonesey's and Baja Blue's comments to reduce CPU load.

Spinning hard disks are good (fast) with small numbers of large files but struggle with large numbers of small files.
 
While the disk spins fast and sequential data can be read fast, the speed of the head moving is very slow (by comparison) so every time the read head moves to a new area of the disk to read a different file (different guitar track) time is wasted.
 
Increasing the I/o buffer size makes Sonar load more from each file on each read, so the disk spends more time reading data and less time moving the read head around.
 
The issue is avoided by using SSD not HDD.

Don't discount the power of degrag.  With that much audio, I wouldn't be surprised if a defrag improved things for a bit.

Don't defrag SSD!!

I have run with zero issues at 512 for years and years.  I DO switch ASIO Buffer Size from either 64 or 128 up to 1024 when mixing, and back down to 128 or 64 for tracking/recording.
 
Bob Bone
 

I would agree with some of the replies here.  However the OP mentioned he does not have any synths in his project.  I keep my playback buffer at 1024 and have had pretty stable high track count playbacks.  When Im tracking its not unusual for me to bypass my fx vcia the icon.
 
In my experience with higher track counts - most of it will be bussed down in some fashion... and its a good idea to use alot of foresight when compiling more tracks on top of that.  What I mean is high track counts usually have some doubling, or quad tracking etc...  Ill always use a buss processor instead of each track, and then mix down the buss and archive so the pc only has to play back 1 stereo mix instead of 12 individual tracks, especially if I know during the current tracking session that Ill be adding another high track count ensemble part... such as - "stomps" - or "Claps" - or " gang vocals" which can reach 8 individual tracks or so -   Then... for the final mixdown ill open back up the archives.  Tracking and Mixing are two different stages of the recording process, so its ok to treat them as such and get to know your hardware better, in my opinion all this is.  

I would totally use that. I end up never flattening comps because while I'm doing edits with the performer I'm on the clock and it's always possible I'll need to recomp a bit here and there. If I could flatten and preserve/archive my takes (similar to freezing/unfreezing a softsynth) I would definitely use that option. Or maybe I'm just misunderstanding how this works right now. Fortunately my system doesn't seem to mind too much most of the time.

Sanderxpander
I would totally use that. I end up never flattening comps because while I'm doing edits with the performer I'm on the clock and it's always possible I'll need to recomp a bit here and there. If I could flatten and preserve/archive my takes (similar to freezing/unfreezing a softsynth) I would definitely use that option. Or maybe I'm just misunderstanding how this works right now. Fortunately my system doesn't seem to mind too much most of the time.

BobF
The originals stay there in a flattened comp.  A new lane is created called Comp.  You can delete that lane and redo the comp and flatten again.

No need to delete it. I create multiple comps in the same track quite often. I record so many takes when I'm in the "zone" that I usually have a large enough pool to comp together doubled guitar tracks.
 
You just need do some fiddle faddling with the Solo/Mute buttons on the lanes after the Flatten process (which is actually a little anoying but doable).
 
And it doesn't even have to be for doubles. It can just be to have different performances/solos/etc that I can check out later after my brain/ears have settled from tracking.
 
So I'll create Comp 1 and call it C1 in the Notes section of the lane. Then I'll go through and make notes of which sections I used for C1 (sometimes using timecodes or just names I give sections... sometimes I'll use Markers until it's all done and name the markers and use those marker names in my lane notes).
 
Then I'll unsolo the C1 lane (and mute it) and create another comp which I call C2. If I'm creating a double I avoid using the sections I used before for C1 (because otherwise it will dump to mono in the mix and just be louder instead of a double... which is actually a cool effect sometimes in the middle of a solo but not a true double) but if I'm just making an alternate take it doesn't matter what sections get used.
 
You can edit and flattened as many as you want to like this.
 
I've actually gone so far as to create a bunch of comps of all the best material (to get rid of mistakes and stuff I didn't like) then unlocked the comped clips and created a NEW comp from those clips.
 
Kind of like "sifting" the takes to get at the best of the best.
 
It really depends on what I'm doing and how the recording is working out. I roll with a bunch of different weirdo workflows I've concocted based on what I think is gonna work best/save me time.
 
Meh. Kind of "not as designed" usage but I'm a pigheaded bastard so I force things to work how I want.

I haven't tried this in a while but I thought I lost all my original comping choices when flattening? Maybe I just did it wrong. Never mind, I need to play with this more myself before asking silly questions in a thread about someone else's problem!

Carry on!

My vote is for Sander's original suggestion of doing a save as version that's stripped of all of the muted/archived tracks. Having said that, I would definitely defrag first. See if that fixes it. It definitely can't hurt and should be a regular part of your system maintenance if you are using HDD drives. Hint, name your save as something that REALLY tells you that it is not the original. I have done save as "song 1" "song 2" etc. that has left me scratching my head if I leave it go for any length of time