Glyn Barnes
Max Output Level: -0.3 dBFS
- Total Posts : 7564
- Joined: 2009/06/10 05:12:31
- Location: A Stone's Throw from the Line
- Status: offline
Compressing Kontakt Samples
It looks like I could save a lot of disk space by compressing samples in third party Kontakt libraries. As an experiment I reduced the size of my Lyrical Distortion Charango from 551 to 332 Mb by saving as a monolith and compressing the samples. Everything seems to work OK. What, if any, is side of doing this? Edit - thinking about it saving as a monolith will only save space if one NKI is accessing the samples. In other cases it looks better just to compress the samples.
post edited by Glyn Barnes - 2011/12/30 14:38:33
|
bitflipper
01100010 01101001 01110100 01100110 01101100 01101
- Total Posts : 26036
- Joined: 2006/09/17 11:23:23
- Location: Everett, WA USA
- Status: offline
Re:Compressing Kontakt Samples
2011/12/30 15:52:51
(permalink)
The only downside is that any old projects that used the uncompressed libraries in an earlier version of Kontakt won't load anymore. If you want to bring them up and work on them, you'll have to replace Kontakt in that project with the later version. I have a lot of Kontakt 3 projects, so I don't compress any of the old libraries that I've ever used with K3. Anything I get new, though, gets compressed.
 All else is in doubt, so this is the truth I cling to. My Stuff
|
wst3
Max Output Level: -55.5 dBFS
- Total Posts : 1979
- Joined: 2003/11/04 10:28:11
- Location: Pottstown, PA 19464
- Status: offline
Re:Compressing Kontakt Samples
2011/12/30 19:54:16
(permalink)
Agreed - no downside, and in fact I believe that not only do you save disk space, but the instruments complete their initial load more quickly as well. I don't know about the monolith bit, but compressing the individual wav or aif files to ncw seems to work quite well.
-- Bill Audio Enterprise KB3KJF
|
Glyn Barnes
Max Output Level: -0.3 dBFS
- Total Posts : 7564
- Joined: 2009/06/10 05:12:31
- Location: A Stone's Throw from the Line
- Status: offline
Re:Compressing Kontakt Samples
2011/12/31 06:40:54
(permalink)
Thanks guys.
I am very now cautiously working through my collection.
|
Kalle Rantaaho
Max Output Level: -5 dBFS
- Total Posts : 7005
- Joined: 2006/01/09 13:07:59
- Location: Finland
- Status: offline
Re:Compressing Kontakt Samples
2011/12/31 09:01:50
(permalink)
How about disc streaming if everything on the disc is compressed? Doesn't it slow down? Or does Kontakt automatically decompress files that are used in the project under construction?
SONAR PE 8.5.3, Asus P5B, 2,4 Ghz Dual Core, 4 Gb RAM, GF 7300, EMU 1820, Bluetube Pre - Kontakt4, Ozone, Addictive Drums, PSP Mixpack2, Melda Creative Pack, Melodyne Plugin etc. The benefit of being a middle aged amateur is the low number of years of frustration ahead of you.
|
Starise
Max Output Level: -0.3 dBFS
- Total Posts : 7563
- Joined: 2007/04/07 17:23:02
- Status: offline
Re:Compressing Kontakt Samples
2011/12/31 12:10:25
(permalink)
I will probably get a bigger HDD in the near future. If I tried that I would likely accidentally hit a key and erase all samples or some other nonsense.
Intel 5820K O.C. 4.4ghz, ASRock Extreme 4 LGA 2011-v3, 16 gig DDR4, , 3 x Samsung SATA III 500gb SSD, 2X 1 Samsung 1tb 7200rpm outboard, Win 10 64bit, Laptop HP Omen i7 16gb 2/sdd with Focusrite interface. CbB, Studio One 4 Pro, Mixcraft 8, Ableton Live 10 www.soundcloud.com/starise Twitter @Rodein
|
inaheartbeat
Max Output Level: -77 dBFS
- Total Posts : 679
- Joined: 2011/08/11 12:19:15
- Location: Nashua, NH USA
- Status: offline
Re:Compressing Kontakt Samples
2011/12/31 13:01:40
(permalink)
One additional down side to be aware of is that if you compress your sample libraries you can't get at the samples as individual .wav files if you want to use them for something else. That is a pretty minor inconvenience most likely and probably a non-issue for most people.
Because inquiring minds want to know...;-)
Ken
PC Audio Labs mobile i7 MC, 3.46 Ghz i7 990X, 12 Gb RAM, 3 750 Gb 7200 RPM drives, 3 USB2, 2 USB 3 ports, firewire, Windows 7 64 bit Pro, Sonar X3e Producer 64 bit,
|
bitflipper
01100010 01101001 01110100 01100110 01101100 01101
- Total Posts : 26036
- Joined: 2006/09/17 11:23:23
- Location: Everett, WA USA
- Status: offline
Re:Compressing Kontakt Samples
2011/12/31 13:16:19
(permalink)
How about disc streaming if everything on the disc is compressed? Doesn't it slow down? Or does Kontakt automatically decompress files that are used in the project under construction? Yeh, that was my concern, too. But the decompression overhead is apparently offset by the time savings of loading smaller files. There does not seem to be any noticeable penalty during playback. But even if there was a slight performance hit I'd probably still do it because the load times are drastically improved.
All else is in doubt, so this is the truth I cling to. My Stuff
|
Fog
Max Output Level: 0 dBFS
- Total Posts : 12302
- Joined: 2008/02/27 21:53:35
- Location: UK
- Status: offline
Re:Compressing Kontakt Samples
2011/12/31 18:18:42
(permalink)
as well as the obvious size reductions, perhaps there are loop / sustain points etc that are not being kept in the conversion ?
also if the programming for say multi samples is in 1 format, then maybe again there is the issue of finding the file it relates to also.
I remember doing something simular with DP's lib.. but the format I converted to wasn't supported in aspects.
|
digitalboy
Max Output Level: -87 dBFS
- Total Posts : 173
- Joined: 2007/08/20 12:38:42
- Status: offline
Re:Compressing Kontakt Samples
2012/01/01 02:05:23
(permalink)
Compression to .ncw is the way to travel... When they first released it,there were some problems with the load times,but the mechanics at NI got it sorted out... I batch processed my entire library a while back and it's all good  Less space on the hard drives and faster loading times...Vrooooooooooooooooooooooom
Sorry - I don't use Autotune :)
|
Freddie H
Max Output Level: -39 dBFS
- Total Posts : 3617
- Joined: 2007/09/21 06:07:40
- Status: offline
Re:Compressing Kontakt Samples
2012/01/01 07:38:14
(permalink)
digitalboy
Compression to .ncw is the way to travel...
.......
Less space on the hard drives and faster loading times...Vrooooooooooooooooooooooom
How come it makes faster loading times working with compressed library? Technical It should be the opposite??  0+ Compression will always slow down your computer because it induces a certain latency in accessing files, because the processor always have to decompress before being able to do anything with the data. • You can work with NTFS-compressed files without decompressing them, because they are decompressed and recompressed without user intervention.
• You may notice a decrease in performance when you work with NTFS-compressed files. When you open a compressed file, Windows automatically decompresses it for you, and when you close the file, Windows compresses it again. This process may decrease your computer performance.
• NTFS-compressed files and folders only remain compressed while they are stored on an NTFS Volume.
-Highly developed spirits often encounter resistance from mediocre minds. -It really matters!
|
keith
Max Output Level: -36.5 dBFS
- Total Posts : 3882
- Joined: 2003/12/10 09:49:35
- Status: offline
Re:Compressing Kontakt Samples
2012/01/01 08:51:47
(permalink)
Freddie H
digitalboy
Compression to .ncw is the way to travel...
.......
Less space on the hard drives and faster loading times...Vrooooooooooooooooooooooom
How come it makes faster loading times working with compressed library?
Technical It should be the opposite??
0+
Compression will always slow down your computer because it induces a certain latency in accessing files, because the processor always have to decompress before being able to do anything with the data.
• You can work with NTFS-compressed files without decompressing them, because they are decompressed and recompressed without user intervention.
• You may notice a decrease in performance when you work with NTFS-compressed files. When you open a compressed file, Windows automatically decompresses it for you, and when you close the file, Windows compresses it again. This process may decrease your computer performance.
• NTFS-compressed files and folders only remain compressed while they are stored on an NTFS Volume.
Specualting here, but based on my basic understanding of the mechanisms... Disk streaming generally requires a small buffer of each sample to be loaded into memory, the rest is read from disk as needed. This buffering avoids the initial latency of reading the first bit of each sample from disk. The latest kontakt engines don't require quite the amount of buffering that the old v1, 1.5 and 2.x versions did, IIRC. The streaming is handled in chunks... once a chunk of sample data is in memory, it gets streamed. The engine could either decompress as it reads from disk => memory, or leave it compressed in memory and decompress going memory => audio engine.... not sure what kontakt does, and whether it makes a difference performance-wise, since you need to take a hit on the decompression either way. Now... regardless of where the decompression happens, I still need to read chunks of samples. The reason disk performance for compressed samples would be better is simply because you're able to read more sample data per unit disk space in one chunk. E.g., let's say, hypothetically, I'm dealing with 50% compression, and I'm reading 128K chunks (which is probably much larger than the chunks kontakt reads)... If one chunk of uncompressed sample data is 128K, then I could read two chunks of compressed data (2 x 64K) in that same operation. The decompression is going to hit your CPU. With modern CPUs with their overall massive throughput and specialty DSP instruction sets you're probably not seeing more than a couple percent increase in CPU usage over uncompressed samples... That's my guess anyway...
|
bitflipper
01100010 01101001 01110100 01100110 01101100 01101
- Total Posts : 26036
- Joined: 2006/09/17 11:23:23
- Location: Everett, WA USA
- Status: offline
Re:Compressing Kontakt Samples
2012/01/01 13:10:34
(permalink)
That's a pretty good explanation. The tradeoff is between CPU and disk efficiency. With sample libraries, disk I/O is more likely to be the bottleneck than CPU. Plus most of the decompression overhead happens when you first load the library and is therefore not an issue during playback, when CPU cycles are most precious.
All else is in doubt, so this is the truth I cling to. My Stuff
|
inaheartbeat
Max Output Level: -77 dBFS
- Total Posts : 679
- Joined: 2011/08/11 12:19:15
- Location: Nashua, NH USA
- Status: offline
Re:Compressing Kontakt Samples
2012/01/01 13:50:20
(permalink)
bitflipper
That's a pretty good explanation. The tradeoff is between CPU and disk efficiency. With sample libraries, disk I/O is more likely to be the bottleneck than CPU. Plus most of the decompression overhead happens when you first load the library and is therefore not an issue during playback, when CPU cycles are most precious.
This is it exactly. It gets REALLY valuable also if you can avoid disk fragmentation by having smaller files that are allocated in contiguous blocks cause once you have to seek your disk head you are just dead for performance. So, besides compressing your libraries definitely make sure that you have defragmented your disk drives. Ken
PC Audio Labs mobile i7 MC, 3.46 Ghz i7 990X, 12 Gb RAM, 3 750 Gb 7200 RPM drives, 3 USB2, 2 USB 3 ports, firewire, Windows 7 64 bit Pro, Sonar X3e Producer 64 bit,
|
digitalboy
Max Output Level: -87 dBFS
- Total Posts : 173
- Joined: 2007/08/20 12:38:42
- Status: offline
Re:Compressing Kontakt Samples
2012/01/01 18:05:07
(permalink)
Freddie H
digitalboy
Compression to .ncw is the way to travel...
.......
Less space on the hard drives and faster loading times...Vrooooooooooooooooooooooom
How come it makes faster loading times working with compressed library?
Technical It should be the opposite??
I'm not sure what the deal is because I'm not a mechanic...
I'm just an end user and I see the results in front of me....
But it could be something to do with how they are streaming the .ncw samples...
There is a significantly faster load time for the libraries now...
Sorry - I don't use Autotune :)
|
keith
Max Output Level: -36.5 dBFS
- Total Posts : 3882
- Joined: 2003/12/10 09:49:35
- Status: offline
Re:Compressing Kontakt Samples
2012/01/01 21:34:30
(permalink)
As a sidenote... gigasampler had the option of compressing samples, and that was late 90's/early 2000's... I believe the compression was not just for distribution, but the GIG content stayed compressed until streamed, IIRC. Not that it matters much now either way. The gigapiano that shipped w/ GS was marketed asa 1GB piano, and fit on a single 640MB CD.
|
Freddie H
Max Output Level: -39 dBFS
- Total Posts : 3617
- Joined: 2007/09/21 06:07:40
- Status: offline
Re:Compressing Kontakt Samples
2012/01/02 01:00:56
(permalink)
keith
Freddie H
digitalboy
Compression to .ncw is the way to travel...
.......
Less space on the hard drives and faster loading times...Vrooooooooooooooooooooooom
How come it makes faster loading times working with compressed library?
Technical It should be the opposite??
0+
Compression will always slow down your computer because it induces a certain latency in accessing files, because the processor always have to decompress before being able to do anything with the data.
• You can work with NTFS-compressed files without decompressing them, because they are decompressed and recompressed without user intervention.
• You may notice a decrease in performance when you work with NTFS-compressed files. When you open a compressed file, Windows automatically decompresses it for you, and when you close the file, Windows compresses it again. This process may decrease your computer performance.
• NTFS-compressed files and folders only remain compressed while they are stored on an NTFS Volume.
Specualting here, but based on my basic understanding of the mechanisms... Disk streaming generally requires a small buffer of each sample to be loaded into memory, the rest is read from disk as needed. This buffering avoids the initial latency of reading the first bit of each sample from disk. The latest kontakt engines don't require quite the amount of buffering that the old v1, 1.5 and 2.x versions did, IIRC. The streaming is handled in chunks... once a chunk of sample data is in memory, it gets streamed. The engine could either decompress as it reads from disk => memory, or leave it compressed in memory and decompress going memory => audio engine.... not sure what kontakt does, and whether it makes a difference performance-wise, since you need to take a hit on the decompression either way. Now... regardless of where the decompression happens, I still need to read chunks of samples. The reason disk performance for compressed samples would be better is simply because you're able to read more sample data per unit disk space in one chunk. E.g., let's say, hypothetically, I'm dealing with 50% compression, and I'm reading 128K chunks (which is probably much larger than the chunks kontakt reads)... If one chunk of uncompressed sample data is 128K, then I could read two chunks of compressed data (2 x 64K) in that same operation. The decompression is going to hit your CPU. With modern CPUs with their overall massive throughput and specialty DSP instruction sets you're probably not seeing more than a couple percent increase in CPU usage over uncompressed samples... That's my guess anyway...
Thank you for that excellent answer Keith. I think you nail it.
-Highly developed spirits often encounter resistance from mediocre minds. -It really matters!
|
keith
Max Output Level: -36.5 dBFS
- Total Posts : 3882
- Joined: 2003/12/10 09:49:35
- Status: offline
Re:Compressing Kontakt Samples
2012/01/02 06:22:58
(permalink)
COMPRESSION, IT REALLY MATTERS!!!
|
Nadine
Max Output Level: -90 dBFS
- Total Posts : 45
- Joined: 2010/12/05 19:56:42
- Location: England
- Status: offline
Re:Compressing Kontakt Samples
2012/01/02 12:07:46
(permalink)
Total novice question... how do you compress to ncw?
|
bitflipper
01100010 01101001 01110100 01100110 01101100 01101
- Total Posts : 26036
- Joined: 2006/09/17 11:23:23
- Location: Everett, WA USA
- Status: offline
Re:Compressing Kontakt Samples
2012/01/02 12:30:50
(permalink)
how do you compress to ncw? The functionality is built in to Kontakt. Click on the "Files" button at the top of the Kontakt screen and select "batch re-save" or "collect samples / batch compress". Use the former if you'll be overwriting the old files, or the latter if you'll be preserving the old files and putting the compressed versions elsewhere.
All else is in doubt, so this is the truth I cling to. My Stuff
|
Nadine
Max Output Level: -90 dBFS
- Total Posts : 45
- Joined: 2010/12/05 19:56:42
- Location: England
- Status: offline
Re:Compressing Kontakt Samples
2012/01/02 17:19:30
(permalink)
bitflipper
how do you compress to ncw?
The functionality is built in to Kontakt. Click on the "Files" button at the top of the Kontakt screen and select "batch re-save" or "collect samples / batch compress". Use the former if you'll be overwriting the old files, or the latter if you'll be preserving the old files and putting the compressed versions elsewhere.
Thanks Bit X
|