ORIGINAL: Jim Wright
<snip>

I have some ideas about how to build a MIDI interface that uses robust timestamps synchronized with an audio sample clock, but some custom chip design would likely be required. If someone with chip-design chops would like to collaborate on an open-source hardware/software project to do just that -- let me know.

- Jim

I hardly believe what I have read here: that midi is not synchronized to the audio via the sample clock. How did that idea become the standard? What is the SOURCE of the MTC that is generated and output ??

Danny

Pianodano: MTC is the midi equivelant of SMPTE. Research those to find out more. Not related to audio at all.

Jim, I don't know anything about hardware or chips, but this is a great idea and sorely needed. If you can get anyone to design the hardware, let me know and I can help with the drivers. I personally think that if someone made an affordable midi interface with timestamping in the hardware and all the correct syncronization built in to work with audio, midi and video setups....and all DAW's universally....well...that would become the standard that everyone would follow. They should have done it like that to begin with.

I believe some of the MOTU stuff does some what you are talking about in terms of sample accurate sync....but I think it might only be their high end video related stuff like the digital timepiece that does it. I just checked the back of my Motu midi timepiece. Admittedly, this is the older one, perhaps the new USB one has more stuff, but all I see on the back is a video sync in and a word clock out. I will have to read the manual to find out how that is designed to work. Word clock is really more like a sample rate metronome...so I'm pretty sure the word clock out is just to make sure that my DAW is sample accurate...


Anyway, back to Sonar...... This weekend I am going to try to run some tests where I freeze tracks to audio and compare the midi notes to the waves I see on the audio track to see if there is any jitter happening during freeze or mixdown.

After that i will try to capture the audio that is playing when I am just hitting play and playing midi tracks through soft synths. then I compare that audio track to the midi track to look at the waves and try to determine if there is jitter happening.

Those two situations are what I am most concerned about right now. If Sonar is screwing that up, then Cakewalk needs to get busy....

From my technical knowledge one can't have sample-accurate synch bewteen audio and MIDI in a normal PC system, because the devices connected to the PCI-bus works asynchronus and so do the buffers associated for audio-playback do. So if one really wants sample-accurate synch, one has to ommit buffers completely, which is impossible in a bus-organized structure like PCs have.

Of course, if the MIDI-outs are on the same interface as the audio-ins/outs, but this would require special (additional) hardware, like Jim Wright already wrote.
Also, one has to develop a completely new driver structure, because MIDI has no "handshake" lines or other low-control mechanisms. So this would also mean to establish a new standard for that... Not to mention the hassle with the different OS'.

cheers,
Chris

I don't think sample accurate sync is really required at this level. Not even close. They are separate things, running at completely different rates of resolution. Audio is typically 44,000/sec for most of us, midi sequences require at the most a few thousand ticks per sec and more likely its in the hundreds; that you need.

If you look at Sonar, I don't think you will find any capability to have a midi event start at a certain sample count from the start of the track. You can only tell a midi event to happen at either BAR:BEAT:TICK, or you can tell it start at a SMPTE time, which is a measly 30 frames a sec and lower resolution than TICKS usually. And the BB:BB:TT resolution will take precedence, whatever it is.

There is absolutely no notion that the midi event will start at a particular sample number, that I am aware of.

There is only a need to

(A) make sure the audio sample rate is stable and

(B) make sure the midi tick rate is stable and

(C) make sure their timebase is synchronized pretty closely, which does not need to be sample accurate for that either. By timebase, I mean have a notion of where zero is on the timeline for both streams so that when you rewind and hit play, they both start playing with stable clocks from the same point in time. Or if you fast foward to a particular point, you can calculate how many samples forward that is and how many midi ticks forward and then start both clocks at the same time, again both with their own clocks running at completely different rates, but nonetheless, as accurately as possible, and located to the same place to start...and playing back accurately forward without drift.



(A) sample rate is kept stable with "word clock". By the way, audio sampling can experience its own form of jitter also, the thing is, its happening so fast you don't even realize it. Better quality A/D converters have more stable clocks in them which is one of many reasons they sound better. You can actually make a mediocre soundcard sound better by feeding it word clock from a more stable word clock master source such as the Apogee Big Ben. But even without word clock, most soundcards have their own internal clock that does a pretty reasonable job of sampling the audio at regular intervals, 44,000 times a second with low enough audio jitter to satisfy most of us.

(B) midi tick clock rate is more difficult today since with our DAW's most of us are letting WinXP be the clock, and its not very accurate. Even to do a measly 1000 times a second, WinXP can't do it very well. Most midi interfaces do not have an internal clock or ability to timestamp events according to their own internal clock. Only a few do.

(C) Once it starts playing, then no sample by sample synchronization can or should occur, they both just need to run with stable clocks. You could, I suppose, have the sample clock trigger the midi clock in some way. either by having them in the same device or by having a midi device that can listen to word clock...or perhaps ADAT sync? Dunno, but I don't think anything does it today. I'm not actually sure if the MOTU's with timestamping can listen to word clock, but its kind of overkill to try to force the midi to be so tightly locked to the sample clock, overkill which requires processing resources.

Nah, all that is really needed is for the midi interface to have its own clock that is very stable and to timestamp each event with some kind of timestamp. The problem is how to get a timebase into the midi interface so that the timestamp can then be interpretted by the midi driver later to mean something useful. I guess with ADAT sync you could get a sample accurate timestamp, but the driver would downgrade it to only a Bar:Beat:Tick resolution anyway in Sonar. Word clock could only be useful to establish a stable clock. So I suppose you could have a midi interface that basically timestamps everythign with the same clock rate as the sample rate...and then divide it back down in the midi driver. However, again, how do you get the timebase in there to establish the location in time? ADAT sync is the only way I know of.


Anyway, we're waxing theoretical again...not dicussing anything that Cakewalk has any control over. The stuff I highlighted in blue above is what I think we should be focusing on.

ORIGINAL: dewdman42

Pianodano: MTC is the midi equivelant of SMPTE. Research those to find out more. Not related to audio at all.

Yep, I know that. But what is the clock source of MTC in Sonar ? Or what is the timing reference ? Based on what has already been stated, I can't see a common source to the audio. Iow, if someone intends to sychcronize something to something else, it just makes sense that the something else would be the reference. Anyone care to clarify ?

Most likely the clock source for midi and the SMPTE counter(which is even lower resolution than midi most of the time) in sonar is the MM timer, part of windows XP.

The sample clock is not really usable for midi/smpte counters because from Sonar's perspective it does not get a smoothly running sample clock timer.

See what happens is that that soundcard has as smooth running sample clock inside its own hardware that is filling a buffer. When the buffer fills up, then Sonar is told to come fetch the data as a chunk from the buffer. You control the buffer size in your ASIO control panel. As you know, if you set the buffersize really low the CPU starts to puke. There is no CPU today that could keep up with Sonar trying to react to each sample individually and use that clock to also pay attention to midi tasks. See what I mean?

ORIGINAL: Tonmann

From my technical knowledge one can't have sample-accurate synch bewteen audio and MIDI in a normal PC system, because the devices connected to the PCI-bus works asynchronus and so do the buffers associated for audio-playback do. So if one really wants sample-accurate synch, one has to ommit buffers completely, which is impossible in a bus-organized structure like PCs have.

Of course, if the MIDI-outs are on the same interface as the audio-ins/outs, but this would require special (additional) hardware, like Jim Wright already wrote.
Also, one has to develop a completely new driver structure, because MIDI has no "handshake" lines or other low-control mechanisms. So this would also mean to establish a new standard for that... Not to mention the hassle with the different OS'.

cheers,
Chris

I wonder if I am the only one getting the feeling that the existing underlying engineering/ architecture is a piecemeal affair ? And is there possibly a correlation as to why midi has not been in the forefront of development for years around here . until now . . like maybe they where hoping it would go away ?

ORIGINAL: dewdman42

Pianodano: MTC is the midi equivelant of SMPTE. Research those to find out more. Not related to audio at all.

Jim, I don't know anything about hardware or chips, but this is a great idea and sorely needed. If you can get anyone to design the hardware, let me know and I can help with the drivers. I personally think that if someone made an affordable midi interface with timestamping in the hardware and all the correct syncronization built in to work with audio, midi and video setups....and all DAW's universally....well...that would become the standard that everyone would follow. They should have done it like that to begin with.

I believe some of the MOTU stuff does some what you are talking about in terms of sample accurate sync....but I think it might only be their high end video related stuff like the digital timepiece that does it. I just checked the back of my Motu midi timepiece. Admittedly, this is the older one, perhaps the new USB one has more stuff, but all I see on the back is a video sync in and a word clock out. I will have to read the manual to find out how that is designed to work. Word clock is really more like a sample rate metronome...so I'm pretty sure the word clock out is just to make sure that my DAW is sample accurate...


Anyway, back to Sonar...... This weekend I am going to try to run some tests where I freeze tracks to audio and compare the midi notes to the waves I see on the audio track to see if there is any jitter happening during freeze or mixdown.

After that i will try to capture the audio that is playing when I am just hitting play and playing midi tracks through soft synths. then I compare that audio track to the midi track to look at the waves and try to determine if there is jitter happening.

Those two situations are what I am most concerned about right now. If Sonar is screwing that up, then Cakewalk needs to get busy....

Dewdman42,

I know you said in a later post that the stuff in blue should be the focus,which is cool, but can we really expect the frozen audio to precisely correlate to the midi triger events if there is no exact synchronization to the audio? (Forgive me, but I am playing devils advocate). Furthermore, how could they ? What would would happen if you inserted another copy of the same synth and output the same midi trigger track to that synth, froze and compared the 2 frozen audio tracks. I think I already know the answer but I'd be interested in your best guess ?

I will need to double check this, or maybe someone else can:

I recorded some drums with the resolution set to 240. I typically notice, when at 960, my playing is a little behind the beat - by about 15 ticks or so - which usually seems like it's late, though my latest test seems to show that it IS recording what I'm playing, so maybe I'm just a little late - I don't know what would be considered "good" from a drumming perspective, but I am a pretty decent click player. At any rate, I recorded at 240 and my stuff was in the 5 tick off area, which is what I'd expect, and mathematically, it is proportionally off as it was before.

So for fun, I bumped the project up to 960. I went into the piano roll, expecting as mentioned above to find it to be about 20, and it showed as "about 5 ticks"...did I just get a tighter recording because I'm only 5 ticks off at 960 now as well?

another thought - how do things like pressure, controllers, etc factor into these resolutions? Do you need the higher res to get things like the continous action of things like hi-hat pedals and smooth filter sweeps? I just don't feel like doing the math!

Again...I will have to check to see that I REALLY went up to 960 and that it can then record at that rate, and that the 5 ticks REALLY is 5 ticks, not just showing that way or something.

(sigh) This stuff is not simple...

Well, without getting into another long-winded explanation....let's just say..during mixdown and freezing..its not realtime. Sonar should have all the time in the world to calculate every single midi note to make sure it goes through whatever plugin and produces WAV output that is exactly there it needs to be in the timeline. All of that can be calculated in non-realtime with certainty...and there is no reason there should be any jitter at all in the WAV output from Sonar when midi tracks are being rendered through soft instruments, totally inside Sonar.

Remember plugins are not guitar pedals. They don't receive an audio signal from sonar. They are pieces of software that churn numbers and produce more numbers. They are handed buffers of data in non-realtime and spit out more buffers of data in non-realtime, that Sonar then assembles into a final WAV buffer that eventually goes to your soundcard and sounds like realtime music. How all of those buffers are assembled into the final output buffer depends upon the calculated timestamps that are attached to the buffers as all this ahead-of-time calculating is happening.

There is no reason at all for midi events to be calculated into the wrong timestamp. If so, that's a bug. In the background, Sonar should be able to calculate every single audio and midi event as having a certain moment in time when it should occur in the final WAV output, down to the exact sample.

Now when you are talking about the actual hardware....hardware for audio and hardware for midi..they are seperate, with their own clocks that run in realtime and are not in sync. If Sonar has to generate a signal to the midi interface so that it will send a note to an external synth...well that can't really be calculated ahead of time very well. Some timer needs to kick off at the moment the note needs to be heard, so that the note can be sent to the midi interface in realtime. That's where the timing issues are probably not going to get any better anytime soon since it depends on better hardware integration related to timestamping.

Everything I just described above...for lack of a better word, is like virtual syncronization. The audio and midi streams are not really being syncronized perfectly...but rather in the background the midi and audio data is being mixed together into WAV output...and as such every single event can be calculated exactly its location and placed in the final WAV output buffer where its supposed to be. But that is not that same as real time sync.