Helpful ReplyIssue with bundle corruption

Author
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
2014/11/09 15:52:11 (permalink)

Issue with bundle corruption

Hi everyone,
 
Haven't been on the forum (posting, responding) for a few years, old username inaccessible, starting from scratch.
 
I have an old bundle that was created with Sonar 8.x (not 8.5 -- a patched version of 8.0) that is having problems with corruption.  When I try and open this bundle with a newer version of Sonar, I have the same problem as when I try to open the bundle under Sonar 8.x and XP (32-bit).  I am pretty sure the issue is not related to RIFF format and Windows 7 (64), as I've seen mentioned here in the past.  I am able to open all of my older bundles perfectly fine.  Just this one bundle is a problem.  Have tried in a friend's Sonar and it's the same thing.
 
Here's what happens: I try and open the bundle and Sonar crashes.  When I open the bundle in a basic audio editor, or rename the cwb to .wav and open in Sonar, some of the samples look and play fine, and some are demolished (corrupted, no wave forms drawn, silence, garbled audio, etc).
 
As a result, I think I have reached the point where I need to try and extract as much usable audio as possible.  Ideally this audio would be extracted in its original form (i.e. length) rather than me copying and pasting from an audio editor back into Sonar and then having to try and line everything up.  Several of these tracks are for drums, so for the obvious reasons I don't want to brute force copy/paste if there is a better way.
 
Having written that, I have a passing knowledge of the RIFF format and I suspect I may have to resort to a binary editor to do what I want to do (i.e. to extract the audio from a bundle) -- but after I dig into my bundle (e.g. using hexdump under Linux), and reach the data section of the bundle, things are not as I had expected.  Earlier reading of the Cakewalk forum, archives, and various RIFF data sheets via Google suggest that what I should be seeing is a LIST chunk of some sort.  However all I'm seeing is a 'data' marker followed by length of the data chunk, followed by the (I presume to be) data chunk payload.  I have looked at bundles I've created over the years and I can't find a single bundle that has an explicit LIST chunk.  What I see is:
 
RIFF
file length offset
WAVE
fmt data
PAD
pad length
(maybe another PAD and pad length)
data
data payload
cwp1 (I presume Cakewalk project)
 
But, again, no information that I can see that describes the list of audio chunks in my bundle.
 
The only commonality I can find between bundles is that data chunks seem to be padded with (I'm guessing) a 16-byte boundary?  Is my understanding of this correct or am I missing where LIST may appear in the bundle file?  The difference between my corrupted bundle and perfectly-find bundles is that in certain spots in the data section, I'm seeing what appears to be less than a 16-byte boundary, sometimes an odd count of 0s, and an odd count of what I presume is audio mixed in between.
 
Could anyone shed some light on this?  Am I on the right track by trying to use some kind of binary editor to grab the "tracks" (i.e. audio chunks) from the data chunk payload?
 
IMO this is one of those corner cases that I don't believe is an issue with Sonar but rather data corruption (i.e. bit rot) that happened over the years when I migrated to new systems and backups.  Speaking of backups, the backups that have been retained all show the same problem for this one bundle.  Definitely looking for some direction if you have time to entertain this.
#1
kakku
Max Output Level: -59 dBFS
  • Total Posts : 1646
  • Joined: 2014/08/31 21:37:39
  • Location: Finland
  • Status: offline
Re: Issue with bundle corruption 2014/11/09 18:48:58 (permalink)
Did you try opening the bundle in safe mode by holding shift while opening?
kakku
#2
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/09 19:13:50 (permalink)
Yes, doesn't crash Sonar in safe mode, but still some issues with the audio files.
 
#3
Anderton
Max Output Level: 0 dBFS
  • Total Posts : 14070
  • Joined: 2003/11/06 14:02:03
  • Status: offline
Re: Issue with bundle corruption 2014/11/09 21:29:25 (permalink)
We've been through the bundle file corruption controversy before, here's the short form:
 
  • I've asked if anyone saved a bundle file, opened it after saving, and there was corruption. So far no one has said that has ever happened, which would indicate the file format is not a problem. All examples of corruption occur with older files, typically many years old.
  • Bundle files are like zip files because they are just a single file. If something goes wrong, it can screw up the whole file. With per-project folders, there could be media damage that destroys portions of an audio file but the rest could be recovered.
  • You cannot trust storage media long-term. Flash RAM loses cells, CD-Rs (which are not as robust as CD-ROMs) deteriorate especially if exposed to heat or humidity, hard drives lose sectors, etc. A single backup to a bundle is risky. A bundle file is what I would use to transfer a project to another user, or if used for backup, as tertiary backup or temporary backup only. 
 
So...I don't know of a solution, but I seem to recall some people having success with recovering substantial portions of corrupted bundle files.

The first 3 books in "The Musician's Guide to Home Recording" series are available from Hal Leonard and http://www.reverb.com. Listen to my music on http://www.YouTube.com/thecraiganderton, and visit http://www.craiganderton.com. Thanks!
#4
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/09 22:07:21 (permalink)
Thanks for your response.  Yes, what I'm looking for is a way to recover as much audio out of the bundle as I can.  I suspect the answer lies in the realm of either deleting corrupted portions of the data section (and rewriting the length ## of the data section) or understanding how audio is stored in the data section -- then constructing a new and separate wav from each chunk.
 
FYI, I pretty much stopped using bundles in 2004 (or maybe 2003?) once per-project folders were the norm.  It's always the old stuff that gets you.  :-(
 
#5
mudgel
Max Output Level: 0 dBFS
  • Total Posts : 12010
  • Joined: 2004/08/13 00:56:05
  • Location: Linton Victoria (Near Ballarat)
  • Status: offline
Re: Issue with bundle corruption 2014/11/09 22:37:59 (permalink)
I believe Cakewalk have some tools they can use to extract audio from corrupt bundles. Best contact tech support

Mike V. (MUDGEL)

STUDIO: Win 10 Pro x64, SPlat & CbB x64,
PC: ASUS Z370-A, INTEL i7 8700k, 32GIG DDR4 2400, OC 4.7Ghz.
Storage: 7 TB SATA III, 750GiG SSD & Samsung 500 Gig 960 EVO NVMe M.2.
Monitors: Adam A7X, JBL 10” Sub.
Audio I/O & DSP Server: DIGIGRID IOS & IOX.
Screen: Raven MTi + 43" HD 4K TV Monitor.
Keyboard Controller: Native Instruments Komplete Kontrol S88.
#6
johnnyV
Max Output Level: -48.5 dBFS
  • Total Posts : 2677
  • Joined: 2010/02/22 11:46:33
  • Location: Here, in my chair
  • Status: offline
Re: Issue with bundle corruption 2014/11/09 22:58:06 (permalink)
Oh oh, here we go agin! Anyhow, my take is like Craig is saying that Buns when they go south are way worse than a CWP because you loose it all. Cakewalk should pitch in to help out with this issue because they are the ones who actually recommended Bun files as a back up when this was obviously bad advice. At least with a CWP file you still have the audio folder. 
I now recommend we also back up as a MID file as well. Mid files will always open, some of mine are 30 years old. Takes 20 seconds to back up any project as a Mid type 1 file. After all a lot of use put a lot of work into our drum tracks ect. Audio we can always re perform and do a better job right? 

Sonar X3e Studio - Waiting for Professional
 Scarlett 6i6
Yamaha Gear= 01v - NSM 10 - DTX 400 - MG82cx
Roland Gear= A 49- GR 50 - TR 505 - Boss pedals
Tascam Gear=  DR 40 - US1641 -
Mackie Gear= Mix 8 - SRM 350's 
i5 Z97 3.2GHZ quad 16 Gig RAM W 8.1  home build
Taylor mini GS - G& L Tribute Tele - 72 Fender Princeton - TC BH 250 - Mooer and Outlaw Pedals  Korg 05/RW
 
#7
Anderton
Max Output Level: 0 dBFS
  • Total Posts : 14070
  • Joined: 2003/11/06 14:02:03
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 00:40:36 (permalink)
johnnyV
Oh oh, here we go agin! Anyhow, my take is like Craig is saying that Buns when they go south are way worse than a CWP because you loose it all. Cakewalk should pitch in to help out with this issue because they are the ones who actually recommended Bun files as a back up when this was obviously bad advice.

 
To be fair, I think they pre-dated per-project folders and budget hard drives. Also, regardless of what backup anyone uses, "digital data is not real unless it exists in at least two places." I still have some bundle files remaining, but have duplicates of them just in case. One of these days I'll open and re-save as per-project, but I've already done that with the most important ones. 
 
I now recommend we also back up as a MID file as well. Mid files will always open, some of mine are 30 years old. Takes 20 seconds to back up any project as a Mid type 1 file.



Yes yes yes. And don't forget that if you're using any external MIDI gear, Sonar has great sys ex storage capabilities. When I was using Sonar with a digital mixer, it was great...I just played back the sys ex at the start of the file, and the mixer was magically set up and ready to go.
 
One more very important thing: You need to "refresh" backups. I archive all the articles I've written over the years. Material I wrote on 8" floppy drives back in the 70s was re-saved a few years later to 5.25" disks, along with more recent material. Then to 3.5 floppies, then zip drives. Finally CD-Rs become reasonable and then they got saved again along with other work I'd done up to that point. When DVD-Rs became affordable the data was transferred again, but I'm going to transfer again to Blu-Ray in a month or two. However, I'm keeping the previous CD-Rs and DVD-Rs so now my digital exists will exist in at least four places - the three optical media. a hard drive, and in the case of my Mac, the Time Machine backup hard drive.
 
Digital storage media is quite robust, but it's by no means purfekt. There have been several times I haven't been able to open files, and it was great to just reach for a twin and open from that instead.

The first 3 books in "The Musician's Guide to Home Recording" series are available from Hal Leonard and http://www.reverb.com. Listen to my music on http://www.YouTube.com/thecraiganderton, and visit http://www.craiganderton.com. Thanks!
#8
Paul P
Max Output Level: -48.5 dBFS
  • Total Posts : 2685
  • Joined: 2012/12/08 17:15:47
  • Location: Montreal
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 08:00:05 (permalink)
Anderton
Also, regardless of what backup anyone uses, "digital data is not real unless it exists in at least two places."



I'd add "which aren't under the same roof".
 

Sonar Platinum [2017.10], Win7U x64 sp1, Xeon E5-1620 3.6 GHz, Asus P9X79WS, 16 GB ECC, 128gb SSD, HD7950, Mackie Blackjack
#9
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 09:05:31 (permalink)
BTW, I in no way intended to start a holy war about bundles (good, bad, indifferent, etc).  While my implementation and infrastructure for creating backups, testing backups and restore, maintaining multiple copies, and generally being a competent IT guy has improved over time, this is obviously a project where I didn't follow my own rules way back when.  I can't fault Sonar for that.
 
At the same time I am also somebody who tries to dig in and solve a problem if I can.  In this case I am trying both to learn something and to show a little bit of ingenuity to dig myself out of a hole if I can.  :-)
 
BTW, this is the hardest I've ever worked to copy 8 drum tracks from one place to another -- just so I can put them in a new project and make samples for another project where the drum sound would be "perfect".
 
#10
Anderton
Max Output Level: 0 dBFS
  • Total Posts : 14070
  • Joined: 2003/11/06 14:02:03
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 09:57:11 (permalink)
nvp1971
BTW, I in no way intended to start a holy war about bundles (good, bad, indifferent, etc).  While my implementation and infrastructure for creating backups, testing backups and restore, maintaining multiple copies, and generally being a competent IT guy has improved over time, this is obviously a project where I didn't follow my own rules way back when.  I can't fault Sonar for that.



Thanks for the clarification. There have been several threads about how the bundle format is "flawed" from people who don't understand the limitations of digital storage media. Although bundles are not my first choice for backup, they do work. As long as people think the problem doesn't relate to storage media, then they'll continue to experience problems with backups, whether Sonar or some other program. So I take any opportunity I can to remind people that backup is an ongoing process, not a set-and-forget one.

The first 3 books in "The Musician's Guide to Home Recording" series are available from Hal Leonard and http://www.reverb.com. Listen to my music on http://www.YouTube.com/thecraiganderton, and visit http://www.craiganderton.com. Thanks!
#11
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 10:31:36 (permalink)
I don't think that bundles are flawed at all (in as much as we can rely on any binary-packed format -- which we do A LOT), but if I have one comment it would be that there is a void of information about the construction (i.e. the spec) of bundles or a desire to have a diagnostic or repair tool to help recover from a situation where things go wrong.  It's not like the folks who wrote Sonar are hiding these details; I just figure that most times nobody cares so long as nothing goes wrong.
 
Years ago when I was more active on the Cakewalk forums, maybe 2006 or 2007, I could swear there was a detailed thread about bundles and the same kind of issue I'm experiencing now, where Noel himself gave a basic breakdown of the composition of a bundle, how it's basically a gigantic RIFF WAV, how padding is used in the data sections, and so forth.  I keep trying to Google for Noel's detailed response but I'll be darned if it I can find it when I would really probably find my answer just by looking at it.  :-(
 
#12
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 10:44:05 (permalink) ☄ Helpfulby mettelus 2014/11/10 12:03:37
Most detail I recall about bundle layout http://forum.cakewalk.com/FindPost/1417418
#13
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 11:03:03 (permalink)
That's exactly the thread I was looking for and somehow could not find on my own.  Thanks, scook!
 
Two interesting takeaways from this thread:
  1. Now I understand why certain myths have been mongered about bundles.
  2. I will have to dig in deeper with my understanding of how the data section is comprised.  I understand what Noel was saying about the data section being broken down into audio chunks of various bit depths, seemingly ordered from high to low bit depths, but there is still the issue of determining size of each clip (in bytes) or maybe reading each clip to a boundary (maybe the 16-byte zero-padding?).
Very interesting information and thanks for your response!
 
Hopefully my next response with be with findings.  :-)
 
 
#14
FCCfirstclass
Max Output Level: -71 dBFS
  • Total Posts : 969
  • Joined: 2003/11/15 15:02:42
  • Location: Las Vegas, Nevada
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 11:09:21 (permalink)
Yes to multiple backups and media, and as Paul says to another physical place away from your work area.

Win 10 Pro x64, 32Gb DDR3 ram, Sonar Platinum, Cubase 9.5, Mackie MCU Pro, Cakewalk VS 100, Roland Octa-Capture,  A 800 Pro, Carver M-1.5t amp & C4000 pre amp, various mics, drums and brass instruments.
 
And away we go!
#15
Seth Kellogg [Cakewalk]
Administrator
  • Total Posts : 814
  • Joined: 2009/02/06 15:25:40
  • Location: Boston, MA
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 11:23:12 (permalink) ☄ Helpfulby mettelus 2014/11/10 12:04:05
mudgel
I believe Cakewalk have some tools they can use to extract audio from corrupt bundles. Best contact tech support


This gets mentioned quite a bit, but it's not so. In the past we could attempt to recover CWP's, but this functionality was integrated into SONAR a few versions ago.

If we did have a tool that could extract audio from corrupt bundles, we would definitely share in publicly as there's no reason to keep that internal.  

Best Regards,
Seth
#16
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 11:42:00 (permalink)
nvp1971
That's exactly the thread I was looking for and somehow could not find on my own.  Thanks, scook!
 

Here is the magic Google search string I used
site:forum.cakewalk.com noel bundle riff wave
The thread  with message 1417418 was the first returned
#17
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 11:57:36 (permalink)
scook
nvp1971
That's exactly the thread I was looking for and somehow could not find on my own.  Thanks, scook!
 

Here is the magic Google search string I used
site:forum.cakewalk.com noel bundle riff wave
The thread  with message 1417418 was the first returned




It's absolutely crazy to me that I could not find this on my own but thanks again for pointing me in the right direction!
#18
Grem
Max Output Level: -19.5 dBFS
  • Total Posts : 5562
  • Joined: 2005/06/28 09:26:32
  • Location: Baton Rouge Area
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 14:39:48 (permalink)
nvp1971
scook
nvp1971
That's exactly the thread I was looking for and somehow could not find on my own.  Thanks, scook!
 

Here is the magic Google search string I used
site:forum.cakewalk.com noel bundle riff wave
The thread  with message 1417418 was the first returned




It's absolutely crazy to me that I could not find this on my own but thanks again for pointing me in the right direction!





No it's not crazy!! It was just a very good description scook came up with. I run into this often. I can't find what I am looking for. Iask around and someone finds it. When I ask how they did it when I couldn't, they give me the search "string" they used. And I keep saying to myself, "I have to learn to do that!"

Grem

Michael
 
Music PC
i7 2600K; 64gb Ram; 3 256gb SSD, System, Samples, Audio; 1TB & 2TB Project Storage; 2TB system BkUp; RME FireFace 400; Win 10 Pro 64; CWbBL 64, 
Home PC
AMD FX 6300; 8gb Ram; 256 SSD sys; 2TB audio/samples; Realtek WASAPI; Win 10 Home 64; CWbBL 64 
Surface Pro 3
Win 10  i7 8gb RAM; CWbBL 64
#19
bapu
Max Output Level: 0 dBFS
  • Total Posts : 86000
  • Joined: 2006/11/25 21:23:28
  • Location: Thousand Oaks, CA
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 15:07:07 (permalink)
Craig brings up a great point adn I'm going to implment it.
 
Refresh the backups.
 
My project backup strategy is a delta based system. Copy all from DAW to backup deleting from backup all no longer in DAW.
 
The other aha! moment was that I should probably open every project once after any major upgrade from CW (unless a patch was to address a specific project related opening problem).
 
Since I pretty much have upgraded (thus far) for each new version that process insures that at least once a year I will be verifying that all my past projects still "work" .
 
#20
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 17:24:53 (permalink)
scook
nvp1971
That's exactly the thread I was looking for and somehow could not find on my own.  Thanks, scook!
 

Here is the magic Google search string I used
site:forum.cakewalk.com noel bundle riff wave
The thread  with message 1417418 was the first returned




Also, you referenced forum thread 653751 in that thread but that item seems to be gone now.  Do you recall the details of this?
#21
scook
Forum Host
  • Total Posts : 24146
  • Joined: 2005/07/27 13:43:57
  • Location: TX
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 17:30:49 (permalink)
The message is still there, use the new FindPost function http://forum.cakewalk.com/FindPost/653751.
#22
nvp1971
Max Output Level: -90 dBFS
  • Total Posts : 9
  • Joined: 2014/11/09 15:21:17
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 17:36:08 (permalink)
scook
The message is still there, use the new FindPost function .




Interesting!  So looks like I would have to extract each audio chunk (guess it's 16-byte boundaries between those), then dig into the cwp to see what length of audio I would need to extract from which chunk.
 
Maybe not worth the time but now I'm intrigued.
 
#23
slartabartfast
Max Output Level: -22.5 dBFS
  • Total Posts : 5289
  • Joined: 2005/10/30 01:38:34
  • Status: offline
Re: Issue with bundle corruption 2014/11/10 19:25:35 (permalink)
Your best shot without help from Cakewalk tech support would be to try to open the bundle with the version of Sonar that you used to create it. Then save it from that version as a per project folder and open that in a more recent version of Sonar.
#24
SONARtist
Max Output Level: -79 dBFS
  • Total Posts : 599
  • Joined: 2005/10/03 17:10:31
  • Location: Switzerland
  • Status: offline
Re: Issue with bundle corruption 2014/11/11 11:50:03 (permalink)
bapu
 
The other aha! moment was that I should probably open every project once after any major upgrade from CW (unless a patch was to address a specific project related opening problem).
 
Since I pretty much have upgraded (thus far) for each new version that process insures that at least once a year I will be verifying that all my past projects still "work" .




 
Yep, I do that too, and that's why I precede each filename with the SONAR version, e.g. "X2_", X3_"
Helps tons when refreshing old projects in new DAW s/w.
#25
Johnbee58
Max Output Level: -72 dBFS
  • Total Posts : 946
  • Joined: 2014/06/21 10:45:53
  • Location: Reading PA
  • Status: offline
Re: Issue with bundle corruption 2014/12/07 12:10:32 (permalink)
You know, I got SOOOOOOOOO tired of trying to figure out the best way to back up projects that all I do now is back up the  Cakewalk Projects folder on the C: drive and back up the Audio folder on the desktop.  Then if I ever have an HDD crisis I will just put the backed up Cakewalk Project folder into the C: drive and the Audio folder back on the desktop.  Then everything should link from there, right?
 
John B.

Lenovo Core i5 4460 Desktop PC (Windows 8 64 bit), Focusrite Scarlett 6i6, Nektar LX61 Keyboard MIDI Controller, Avantone Pro CV-12 tube condenser microphone, JBL LSR308 8" active monitor speakers.  Cakewalk by Bandlab, Reason 7,
NI Session Strings Pro, NI Strummed Acoustic Guitar, Miroslav Philharmonic, Auturia DX7 V, Garritan JABB 3, EZ Keys, EZ Drummer.
 
 "I will create music the way I want to whether a million people are listening or no one is listening."   Dan Fogelberg, Singer/Songwriter-1951-2007
#26
Jump to:
© 2025 APG vNext Commercial Version 5.1