Wow! Very interesting..

ORIGINAL: brundlefly

Incidentally, I discovered an interesting factoid during this testing:

When you change to a lower clock rate, Sonar has to "quantize" some of the tick values to the lower resolution. I think it's been suggested herer that this is a permanent loss of resolution. But it appears to me that Sonar is storing the original value in a different form with more resolution, because if you change back to the higher clock rate, the "odd" values come back. For example if an event happened on tick 19 at 960PPQ, it will round to tick 5 when you drop down to 240PPQ, but when you change back, you get the 19 back, not a 20 as you might expect. This opens up a whole other ball of wax about how Sonar stores event times. At lower clock resolutions, at least, it appears to be using more internal resolution than it displays.

ORIGINAL: dewdman42
Can you please tell us a few more specifics about the behavior you are experiencing?

Sorry dewdman, I was just writing off the top of my head.

I'm not in front of the DAW right now, but I'll try and verbally re-create:

+ TTS-1 (piano)
+ NI Battery 2
+ Broomstick Bass
+ VS3 w/guitar patch

I played a piano part in to the metronome. It sounded good on playback (to the metronome). I step-entered some drums from battery 2. Sounded not so good on playback. I played the drums in by hand in Battery 2, sounded better. I selectively quantized the kik and snare, it sounded awful.

I then went and hand-moved the drum hits in the PRV. Of note here was the PRV was showing notes OFF the beat lines, even ones I quantized 100% and even ones I step-entered. Moving them even further away started getting some timing back.

Thinking this was odd, I double-checked my piano part. Pretty tight to the grid.

I played everything else in by hand for 8 measures to test. It got a little loose after a while... meaning looser than what I know I played in.

I thought maybe it was too much MIDI, so I bounced everything down. Sounds kind of choppy.

Still don't have a solution. I was just going to re-sequence and try keeping everything as tight to the grid as possible and bouncing down to see if PDC might have something to do with it.

Of note here was the PRV was showing notes OFF the beat lines, even ones I quantized 100% and even ones I step-entered.

Huh, what??? I'd be interested in seeing *that* MIDI file. Care to share?

brundle-fly@hotmail.com

ORIGINAL: ...wicked
I played a piano part in to the metronome. It sounded good on playback (to the metronome). I step-entered some drums from battery 2. Sounded not so good on playback. I played the drums in by hand in Battery 2, sounded better. I selectively quantized the kik and snare, it sounded awful.

More detail needed. Describe "awful". Did you turn off the metronome for playback? There are some known bugs related to the metronome, so need to make sure that is not screwing it up now for you.

I then went and hand-moved the drum hits in the PRV. Of note here was the PRV was showing notes OFF the beat lines, even ones I quantized 100% and even ones I step-entered. Moving them even further away started getting some timing back.

Ok, that is weird and I'm confused. You're saying that when you quantized some music, the notes moved further away from the grid? Are you sure you were using simple normal quantization and not one of the groove or humanization variants? Just checking.

Thinking this was odd, I double-checked my piano part. Pretty tight to the grid.

I played everything else in by hand for 8 measures to test. It got a little loose after a while... meaning looser than what I know I played in.

I thought maybe it was too much MIDI, so I bounced everything down. Sounds kind of choppy.

Still don't have a solution. I was just going to re-sequence and try keeping everything as tight to the grid as possible and bouncing down to see if PDC might have something to do with it.

That does not sound alright and frankly to me sounds like bugs in Cakewalk, but in order to help them figure it out we need to figure out how to replicate the problem consistently and try to isolate exactly what the problem is. You have done a lot of different things in your description, I guess desperately just trying to get it to sound good. Can we try a couple tests, one by one and isolated?

First, I want to know if you enter some notes manually into the PRV and make sure they are quantized to the grid lines...when you play it back through a VSTi, does it sound tight?

And before you do that, make sure you address the known metronome bug(search this forum for info about that, fix that first)

Thanks Jim, for the well thought out post. I am certainly not a computer engineer nor did I ever have a much desire to be one. Just an old musican that does not know much of anything about the true whys or wherefores re this topic. Only my experience with it.

When I signed up with the Cakewalk forum coming up on 4 years ago, the issues that are now being discussed full tilt in this thread where painfully obvious to me. I have long sought after answers and continued upgrading in the hope the problem would be solved. But here we are now at version 7 (I'm still at 6 though) and on it goes.

Somewhere, there is a post by one of the bakers that mentioned some type of timing problem that had crept in and would take a long time to solve. I believe this was around version 3 or 4. I will have to look see if I can find it.


Here in a nutshell is why I just can't fathom how detectable midi errors can be a issue with the computing power that is available to skilled programers now. I'm talking midi only.

I also realize this is Cakewalks forum.

1: Several have already stated their satisfaction with OLD Roland hardware sequencers. No timing issues at all that I ever found. A google search will prove it too. We're talking 20 years ago now. And what does that thing have in it, maybe 48k memory, if that ?

2: I watch my ancient Timeline Microlynx take the MTC that Sonar spits out (and who knows if that stream is steady or not), convert that into smpte 29.97nd and absolutely without error resolve and LOCK the 16 track and the digtal DTRS machines to frame accuracy. It show's you that it's locked and it stays locked no matter what Sonar does. I can't even comprehend the math involved in keeping all that stuff resolved.

3: I'm running a dual core machine, disk drives that set basically stopped with almost nothing to do, enough memory to get a rocketship to mars and beyond, and with only a couple of audio tracks running. And yet midi timing all over the place.

I would hope that if midi events just can't be reliably handled on a pc running windows, someone would put a honest disclaimer on the package stating as much. It just should not be pot luck at this stage. Heaven knows we're talking about serious money being involved once folks have invested in high end sample libraries, interfaces, hardware instruments and everything else desired or required to set up a decent project studio and to then (afaic) be continually discouraged that the foundation of their investment is inhibiting creativity and find themselves spending most of their available time trying to figure out what the problem is.

Personally, I have wanted this topic to be brought out of the closet for a long time and I had just about given up on others caring about it. I hope the thread doesn't get all weird now.

Regards,

Danny

ORIGINAL: dewdman42
And before you do that, make sure you address the known metronome bug(search this forum for info about that, fix that first)

Will do. I'll report back with more info when I accomplish said tasks. thanks!

A couple of interesting points to note:

Two people (unless there's another post that I missed) alluded to the concept of sync'ing the audio and MIDI streams, and the fact that this requires a great deal of computation by the host computer, whether it's at a software or OS level. However, I'm not sure this is entirely relevant to the MIDI jitter discussion. Think about slaving a hardware sequencer to a 2" reel-to-reel 24-track tape machine using SMPTE, MTC, or whatever other sync box you want to use. Based on tape stretching and pulling, motor fluctuations, etc., there may be some subtle "drift" or tempo-skewing, but the hardware sequencer's MIDI events will still retain their proportional timing relationship to each other, even if the tempo drifts in and out. For that matter, even if the tape itself has some drift, it doesn't wreck the groove or timing of the audio recorded on it, because the proportional timing relationship of the events on that tape is preserved.

Now take, as a comparison, the MIDI jitter situation. The problem is that with MIDI jitter issues, the timing relationships between the MIDI events themselves is lost. This is what makes a performance sound less "tight" on playback than it did on recording. And it's not a MIDI / Audio sync issue. Perhaps there could be a MIDI / Audio sync solution to the problem, whereby the MIDI timestamping can be dependent on the audio clock instead of the system clock, but the MIDI jitter problem itself does not derive from an audio sync issue, as far as I can tell.

And I, too, am *extremely* pleased that significant numbers of other people are finally taking an interest in this.

My feeling is that playback through VSTi's should absolutely be intertwined with audio sync and should be 100% jitter free. If it is not, as some people seem to be claiming, then Cakewalk has a lot of work to do and I'm not impressed. I haven't noticed any problems here, but some people appear to have.

The problem that comes up related to midi drivers and recording from a midi keyboard is probably not solvable and we have spent a great deal of time discussing it in this thread and sharing some interesting facts, but its a limitation of windows. At any rate, my perception is that its not actually that bad and that most of the complaints people have are timing differences much much greater than 1-2ms, but rather large and blatantly obvious timing glitches at playback time.

There is no excuse for this when playing through soft instruments since everything is being rendered by the Sonar audio engine, which is supposed to know all about audio buffers, delay compensation, etc.. There is absolutely no reason for the midi timing in that case to not be 100% perfectly tight.

But short of just bringing this up all the time, I'm not sure Cakewalk will take action, it doesn't seem to be something that people complain about very often.

dewdman42, pianodano, Jim Wright, RTGraham, et al...., this is SO GREAT! I can't wait to have the time (couple of days) to go through every piece of information contained in this thread. I will also be forwarding it to Cakewalk. What an amazing resource. Mucho thanks. Let's keep going!

ORIGINAL: RTGraham

A couple of interesting points to note:

Two people (unless there's another post that I missed) alluded to the concept of sync'ing the audio and MIDI streams, and the fact that this requires a great deal of computation by the host computer, whether it's at a software or OS level. However, I'm not sure this is entirely relevant to the MIDI jitter discussion. Think about slaving a hardware sequencer to a 2" reel-to-reel 24-track tape machine using SMPTE, MTC, or whatever other sync box you want to use. Based on tape stretching and pulling, motor fluctuations, etc., there may be some subtle "drift" or tempo-skewing, but the hardware sequencer's MIDI events will still retain their proportional timing relationship to each other, even if the tempo drifts in and out. For that matter, even if the tape itself has some drift, it doesn't wreck the groove or timing of the audio recorded on it, because the proportional timing relationship of the events on that tape is preserved.

Absolutely correct. Issues related to syncing audio and MIDI streams are different from the issues that lead to MIDI Jitter. Audio/MIDI sync issues have to do with synchronizing two or more independent "clocks". MIDI jitter issues have to do with the fact that MIDI messages, as handled by Windows, have "flaky" timestamps.

Now take, as a comparison, the MIDI jitter situation. The problem is that with MIDI jitter issues, the timing relationships between the MIDI events themselves is lost. This is what makes a performance sound less "tight" on playback than it did on recording. And it's not a MIDI / Audio sync issue. Perhaps there could be a MIDI / Audio sync solution to the problem, whereby the MIDI timestamping can be dependent on the audio clock instead of the system clock, but the MIDI jitter problem itself does not derive from an audio sync issue, as far as I can tell.

Yes. I call this the "flaky Windows timestamp" problem. Why flaky? The timestamp is referenced to an internal Windows clock (system clock or something else) -- that may not keep time very well. Individual clock interrupts can be delayed due to higher-priority CPU work -- like streaming audio. The timer rate may be 1000 interrupts per second, but that does not mean a new interrupt occurs at the start of each millisecond. Clock interrupts can be skipped for, say, 3 milliseconds, and then come in a burst (3 interrupts in the same millisecond) to make up for it. You can see how this can introduce jitter (and also why lower PPQN values, which quantize the MIDI to something like 5 millisecond boundaries, can compensate for this somewhat).

Another issue - is that the low-level MIDI event scheduler doesn't fire MIDI messages out of the hardware port directly, it just sends MIDI events to the driver for the port. For something like USB, the event has to travel through the USB driver stack ... which can take a while .... and may then have to wait until the next USB frame is ready to go out the USB port. (With USB 1.1, frames are sent once per millisecond - which adds 0.5 milliseconds of jitter right there. With USB 2, microframes can be sent more often (8x more often?), so USB 2 can potentially support lower-jitter MIDI I/O). Of course, Firewire drivers also need to queue up data, assemble packets, fire them down the wire -- this all takes time as well. A direct PCI-based interface (say, on an EMU 1820M or 1616M) has a lot less overhead involved with getting events from the low-level MIDI event scheduler to the actual hardware MIDI DIN port. So -- less jitter, less lag, happier musicians (other things being equal, and unless the drummer is hitting on your girlfriend/boyfriend, of course).

All of the above affects incoming MIDI too, but in the reverse direction. A lot can happen between the point when an incoming MIDI event is processed by the MIDI port hardware, and the point at which the event is timestamped by the MIDI driver (or low-level sequencer engine, as may be).

BTW - I have some ideas about how to build a MIDI interface that uses robust timestamps synchronized with an audio sample clock, but some custom chip design would likely be required. If someone with chip-design chops would like to collaborate on an open-source hardware/software project to do just that -- let me know.

- Jim