Kontakt Load Times

I apologize if this kind of topic is verboten.  Everywhere you see people talking about issues loading sample libraries in their DAWs, but folks don't tend to actually say how much and how long - they just say "long".  I would like to understand if my situation is normal.
 
I am on Sonar Platinum, my DAW is an Intel i7-4820K quad core, Asus X79 mobo SATA 6GB/s with 32 GB RAM and separate Samsung EVO SSDs for sample libraries and OS.
 
I just timed loading one of my projects that has a big template spanning six instances of Kontakt 5.4, having a total loaded RAM consumption of 8.1 GB.  The project "comes up" (the UI is completely up and I can see my track view and multidock fully populated) in maybe 20 seconds.  However, I can see each actual instrument fully loading samples one instrument at a time (I assume this is the so-called background loading) until they are all done.  Loading progresses through each Kontakt instance in the order they appear in the synth rack.  It takes about 2 minutes and 30-45 seconds to completely load all Kontakt instruments in all instances (to where I could hit play and hear no drop out at all because of unloaded samples).  By the way, as far as I know each library has been batch-resaved.
 
So, does 2.75 minutes seem a reasonable load time for 8 GB of samples hosted on SSDs?
Thanks for any comments - moderator please feel free to chuck me in the offender's bin if I committed an indiscretion with this post.
 
Lum

Per comments from Noel, you give up multiple thread processing when you lump a bunch into a smaller number of Kontakt instances, and you might be better off on your load times to split the template into a bunch more instances of it.
 
I myself don't build massive templates, but orchestral folks have different needs, and for them (and others), big templates may be desired.  The more you load, the longer it takes, so, balance convenience with consequence is about the best advice I can give you.
 
Also, even if you split your OS from your sample libraries, on different drives, if you are loading the world into Kontakt, it WILL end up fighting with itself doing disk access at some point.
 
You may want to consider further splitting up sample libraries among additional drives, to split things up even more, and I suppose the order of loading into a given Kontakt instance would potentially be something to consider, in terms of which libraries from which instance and which drive are optimized by which ones appear in which order in the various instances.
 
Bob Bone
 

That 500 MB/s figure is theoretical. You mentioned that it was an external drive; if it's USB 2.0, the fastest it can go is 60 MB/s but that's virtually never attained - in the real world, 40 MB/s is more like it. Also, you mentioned separate drives but whether they're 2.0 or 3.0, they're probably being driven by the same controller chip. The USB controller has to wait for the drive to say it's ready before it sends any data, which means wait times especially if multiple devices are sharing the same controller.
 
Furthermore, SSDs work most efficiently when streaming single files. Accessing lots of little files - exactly what you'd find in sample libraries with tons of articulations and hundreds of samples per instrument - will slow things down dramatically. Even without taking the interface bottleneck into account, that 500 MB/s could drop easily to 100 MB/s or less.
 
USB 3.0 drives have a maximum theoretical speed of 640 MB/s, but most top out at around 100 MB/s. Also, you have to read specs VERY carefully - 100 mbps is NOT the same as 100 MBps; 100 MBps is 800 mbps. Not that drive manufacturers want you to read the specs carefully...
 
There's also confusion about what a spec says is the maximum speed vs. whether real-world devices can attain that speed, which is rare and happens only as the technology improves over time. Think of it this way. A planning commission specs a highway to have four lanes, even though the current traffic could be handled easily by two lanes, because the existence of the highway will lead to increased traffic and maybe someday, four lanes of traffic will go down that highway. They're spec'ing for the future.
 
Or in other terms, the engine is my car is spec'ed  to go up to 140 MPH. I've never taken it over 95 

Thanks again Craig.  Not sure where the "external" came from.  These are internal SSDs on 6GB/s SATA.

Frankly, I am not super-stressed about the load times.  I usually get up to have a wee or go get a pop when I load.  But this large template issue nags at me.  Most of us "feel" that putting super-fast SSDs into a new system with the fastest SATA pipe available is going to make our load time delay go away.  Either that's just not the case or I have a Clark Griswold switch problem still lurking (maybe I can get Beverly D'Angelo to come over and flip some switches).
 

So, all this sort of caused me to dig and think a little harder.  I have Kontakt set to override the instrument preload buffer size, choosing 18 kb vs the default 60 kb (this relates to the question Craig was asking about in terms of streaming).  The reason is, with SSDs the DAW can load less of the sample beginning into memory since it will be able to stream off disk for the remainder so much faster (than a mechanical drive).  I have seen people say, "go for it wild man, set that preload down to 6 kb", which is the lowest it can go, but I found with these large templates that 6 kb is impractical and I get dropouts, even with the SSDs ( and I do have two of them for samples, there are actually 3 SSDs in the box... details, details).  18 kb never drops out (yet anyways).
 
But here's the deal and the reveal.  When I say I am loading 8GB into RAM, that's terribly misleading.  That's what Sonar reports for memory consumption.  That's ONLY the RAM being used to cache the PRELOAD portion of the samples.  In fact, the actual sum of the library sizes I am loading in this template - face value as it were - is up in the 40 GB range.  The DAW needs only to load a fraction into RAM because of the SSD streaming, but the loaded instruments have an enormous amount of sample references.  So the upshot is, I believe... that's simply a barnload of samples.  The 8GB Sonar reports is only the RAM used to load that 18 kb preload part (plus whatever other VST RAM I am using for things like Superior Drummer, etc.)
 
So I think this is merely an issue of just a zillion sample files being preloaded.  I think this is where I count my blessings and remember when I used to live in that world Craig was thinking about... the multi-box world connected by midi-over-ethernet with 3 gigastudio licenses running.   Ow... that hurts to remember.

I would still love to hear if there are any other orchestra freaks out there who may comment on big templates, but thanks very much Craig and Bob for taking the time to help me think.
 
Cheers mates.

With some Kontakt instruments, load times can be sped up considerably by saving the instrument/multi you are loading once, after it is fully loaded. Make sure you don't make any changes unless you want those changes to become part of the instrument/multi.
 
Regards,
Dan
 

You can also try Purging memory in Kontakt instruments and saving that. It may make it load faster.