ORIGINAL: Jim Wright
<snip>

I have some ideas about how to build a MIDI interface that uses robust timestamps synchronized with an audio sample clock, but some custom chip design would likely be required. If someone with chip-design chops would like to collaborate on an open-source hardware/software project to do just that -- let me know.

- Jim

I hardly believe what I have read here: that midi is not synchronized to the audio via the sample clock. How did that idea become the standard? What is the SOURCE of the MTC that is generated and output ??

Danny

From my technical knowledge one can't have sample-accurate synch bewteen audio and MIDI in a normal PC system, because the devices connected to the PCI-bus works asynchronus and so do the buffers associated for audio-playback do. So if one really wants sample-accurate synch, one has to ommit buffers completely, which is impossible in a bus-organized structure like PCs have.

Of course, if the MIDI-outs are on the same interface as the audio-ins/outs, but this would require special (additional) hardware, like Jim Wright already wrote.
Also, one has to develop a completely new driver structure, because MIDI has no "handshake" lines or other low-control mechanisms. So this would also mean to establish a new standard for that... Not to mention the hassle with the different OS'.

cheers,
Chris

ORIGINAL: dewdman42

Pianodano: MTC is the midi equivelant of SMPTE. Research those to find out more. Not related to audio at all.

Yep, I know that. But what is the clock source of MTC in Sonar ? Or what is the timing reference ? Based on what has already been stated, I can't see a common source to the audio. Iow, if someone intends to sychcronize something to something else, it just makes sense that the something else would be the reference. Anyone care to clarify ?

ORIGINAL: Tonmann

From my technical knowledge one can't have sample-accurate synch bewteen audio and MIDI in a normal PC system, because the devices connected to the PCI-bus works asynchronus and so do the buffers associated for audio-playback do. So if one really wants sample-accurate synch, one has to ommit buffers completely, which is impossible in a bus-organized structure like PCs have.

Of course, if the MIDI-outs are on the same interface as the audio-ins/outs, but this would require special (additional) hardware, like Jim Wright already wrote.
Also, one has to develop a completely new driver structure, because MIDI has no "handshake" lines or other low-control mechanisms. So this would also mean to establish a new standard for that... Not to mention the hassle with the different OS'.

cheers,
Chris

I wonder if I am the only one getting the feeling that the existing underlying engineering/ architecture is a piecemeal affair ? And is there possibly a correlation as to why midi has not been in the forefront of development for years around here . until now . . like maybe they where hoping it would go away ?

I will need to double check this, or maybe someone else can:

I recorded some drums with the resolution set to 240. I typically notice, when at 960, my playing is a little behind the beat - by about 15 ticks or so - which usually seems like it's late, though my latest test seems to show that it IS recording what I'm playing, so maybe I'm just a little late - I don't know what would be considered "good" from a drumming perspective, but I am a pretty decent click player. At any rate, I recorded at 240 and my stuff was in the 5 tick off area, which is what I'd expect, and mathematically, it is proportionally off as it was before.

So for fun, I bumped the project up to 960. I went into the piano roll, expecting as mentioned above to find it to be about 20, and it showed as "about 5 ticks"...did I just get a tighter recording because I'm only 5 ticks off at 960 now as well?

another thought - how do things like pressure, controllers, etc factor into these resolutions? Do you need the higher res to get things like the continous action of things like hi-hat pedals and smooth filter sweeps? I just don't feel like doing the math!

Again...I will have to check to see that I REALLY went up to 960 and that it can then record at that rate, and that the 5 ticks REALLY is 5 ticks, not just showing that way or something.

Here's another interesting article

http://www.soundonsound.com/sos/Sep02/articles/pcmusician0902.asp

http://www.soundonsound.com/sos/Oct02/articles/pcmusician1002.asp

dewdman42

Hi dewdman42,

My hats off to you for persueing this like you are, although I'm not sure where your going to end up or how significant your findings will be. I've beat my head against the wall for years with this and all I got was a lumb on the head. Hehe, maybe that's why I'm so brain dead.

If your going to do this I'd suggest you use a sine wav or a square wav around 1Khz. Edit them by cutting them off both front and back at a zero crossing and put them in your softsampler. They'll be a lot more accurate than a sidestick or hihat.

Since this thread is so long, I haven't read much of it so if this has already been covered I'm sorry, please forgive me.

T.S.

Dudes and Dudets,

My first instrument was drums. Then at 9 yrs I switched to guitar. When I was much older I got one of those Roland Midi drum sets, and after recording Midi drums I thought, good Lord I suck at this, why did I not hear it like that when I was recording!!?? So Quantize I did, much. Which sterilized a very good part I played.

Then I scored a cheap 5 piece acoustic kit for a "house kit" when I started doing others demos.

Guess what. I'm a good drummer on that kit.

- Midi jitters alright - alot.

Well I just did one simple test..just took a drum midi clip from EZ drummer. Listened to it, in sonar, turned it into a groove clip loop, stretched it out a while, listened to it for a while...seemed pretty solid to me.

I froze it. Still sounded solid.

I zoomed way in until I could see that 1ms of time was a few millilmeters of horizantal screen space. I could view both the midi drum map view and the wave data at the same time. I put PRV into the track view so I could see the detailed drum map there. Zoomed everything big. i can click on the PRV exactly on the beat lining up the midi event. the line shows up on the track view and exactly lines up with the wav file that was created when freezing the midi track through EZDrummer as a sound module

(shrug)

So far, I'm happy with that timing, no complaints... I have to go to gym now but later I will try routing the output from sonar into a wave file while playing to see if its off when not freezing or mixing...though its somewhat of a moot point.

I don't understand why others are having all-over-the-place results. We will need to dig deeper to try to understand what it is about their test that is causing this. So one of the people having problems, one of us that are not having problems should try to recreate exactly the same test. We should discuss various system aspects to see if that is the root cause also, but right now I'm inclined to say at least here, so far...I am not detecting any problem while mixing down.

I had metronome assigned to midi by the way instead of using one of the built in audio sounds.

Please give as many details as you can about the setup you used to create this problem and I will see if I can recreate the bad timing here.

Chris,

In current PC (and Mac) systems, the audio and MIDI devices do work asynchronously. However, the reasons why have to do more with the use of different timebases for audio and MIDI (and the completely asynchronous nature of MIDI message input/output) -- than with the use of buffers per se.

It's useful to consider how Firewire (1394) handles audio and MIDI. Both kinds of data are sent "over the wire' using CIPs (Common Isochronous Packets). Because audio and MIDI data are sent, literally, within the same data packets --- MIDI is forced to be sychronized exactly with the audio sample data.

Let me say that differently. At the moment when MIDI messages are packetized for transmission over Firewire -- the MIDI messages are no longer asynchronous -- they are now isochronous. Which is to say, the "implicit timestamps" of the MIDI messages are now fixed rigidly against the running audio sample count associated with the Firewire data packet that contains each particular MIDI message fragment. (I edited the MIDI-over-Firewire spec for the MMA; trust me on this).

Now, Firewire data transport - involves buffering. Both audio and MIDI data have to be "queued up" before they can be packetized and formatted into Firewire frames, and then finally sent over the wire. Note that the MIDI data is asynchronous on its way from the sequencer to the Firewire MIDI driver; it only becomes isochronous at the point, during internal driver procesing, where the asynchronous outgoing MIDI messages are converted to isochronous outgoing Firewire data bytes within particular packets.

Now -- if the sequencer could write the MIDI data directly into the outgoing Firewire data packets -- the sequencer could control, with very fine accuracy, exactly when particular MIDI messages would occur w.r.t. the audio data that's being sent over the same Firewire pipe. This is not how Firewire MIDI works currently - but it could.

Chris noted that a new driver structure would be needed for a hypothetical next-generation MIDI interface. New drivers, certainly. I don't think the driver APIs would necessarily have to change, because DirectMusic (on Windows) and CoreAudio (on Mac) both support high-res timestamps. We would just need a MIDI driver that integrates with the audio drivers to 'do the right things', in lockstep.

Ok. Enough theory and speculation for now.

---------------

As dewdman42 pointed out - we don't need sample accurate sync for MIDI. I think 1/3 millisecond would be fine for pretty much any purpose. Heck, a 1 millisecond time resolution that was both accurate and reliable should be enough.

As dewdman42 also pointed out -- the "MIDI jitter" that bothers most of us is probably much larger than 1-2 milliseconds, and is probably happening because of varous kinds of system configuration issues (or MIDI interfaces with lousy drivers, or various other reasons....) I think most of us would be content if current PC MIDI interfaces worked as well, in practice, as the ones on the 15-year-old hardware sequencers mentioned in many posts on this thread. Those products hardly use sample-accurate MIDI -but lots of people swear by them (and few swear at them, unlike software sequencers...)

As pianodano wrote:
>> I wonder if I am the only one getting the feeling that the existing underlying engineering/ architecture is a piecemeal affair ?

Nope. Frankly, it's totally piecemeal. MIDI was developed starting in 1979 (Dave Smith's 'Universal Synthesizer Interface' AES paper; I was in the audience), and first shipped in 1983. It was designed to be as cheap as possible. It was not designed with any thought of synchronizing with audio, or video, or doing anything but hooking two electronic music products together. Everything after that -- just grew.

Different companies proposed different ways of syncing up MIDI with audio, video, whatever. Some of 'em worked, some didn't. Some of the better ideas -- died, because the people with those ideas were lousy businessmen. Some of the sketchier ideas -- made it in the marketplace, because their inventors were much better businessmen than engineers. Kind of like the rest of the computer hardware/software revolution that happened over the last 30 years....

I've been involved with MIDI and AES standards work since 1981, off and on. There is no grand design. There's just a lot of people trying to work out the best compromises they can, considering both what we (music geeks) know how to build, or code, and what the marketplace seems to want.

Kind of shocking, isn't it ?

- Jim

Please give as many details as you can about the setup you used to create this problem and I will see if I can recreate the bad timing here.

1. Open a new project file.

2. Set to 960PPQ and 120BPM.

3. Insert a MIDI track and the TTS-1, and set the output of the MIDI track to the TTS-1.

4. Open the MIDI track's PRV.

5. Create a series of 64th note events with 15-tick duration on Eb6 (this is the Claves sound in GM Drums).

6. I copied them to 128 measures, but you can already see the problem in measure 1, so 8 or 16 should be plenty.

6.5 Oops forgot. Pan hard left, and set output level to max for best results.

7. Select the MIDI track and the TTS-1, and bounce MONO to Track 3.

8. Disabling Fast Bounce made no difference.

8.5 Go get a beer from the fridge.

9. Turn on PRV in the Track View.

10 Zoom till you can see two events and their corresponding wave pulses.

11. Scroll around, and you should immediately see that some transients start closer to the beginning of the MIDI event than others.

12. Zoom further until you can clearly see individual samples.

13. Set the Now indicator to show samples (click it to scroll through the formats).

14. Scroll to the first zero-crossing *after* the beginning of a wave pulse, and write down the sample values (MS Excel can be helpful here).

15. Scroll to the next drum hit, and find the same first zero-crossing, and record the sample value. Repeat several more times, recording the values, and subtracting them to find the interval between hits.

16. If the project is set up as described (and you run 44.1k sample rate like I do), the nominal distance between two drum hits should be 1378 samples. If you get the same result I got, you will find many intervals of 1407 to 1414, with every 4th or 5th one being 1274. If you run a different dample rate, just scale the numbers accordingly.

1274 is 104 samples (4.5ms) short of what it should be, but because all the other intervals are a little too long, things get close to being back on track every 4th or fifth pulse. To me, this looks like a typical case of some sort of cyclic mathematical MODding. Could be a programming error in the rendering algorithm of the TTS-1, or somewhere else in Sonar. I don't know. I haven't yet tried this with another soft synth.

Finally, I should mention that you may be tempted to use AudioSnap to find the transients. I did, and found that AudioSnap has problems of its own. I'll leave you to see these poblems for yourself. Suffice it to say that it is only partially helpful in this effort.

Phew... I need another beer. Hope I got everything right.

Dave

I will try the TTS test later tonight

Keep in mind that at 120-BPM, a quarter note is 500ms so working at 120-BPM is a good tempo for configureing your time.

I thought the same thing initially, but I've just decided the magic number for testing at 44.1kHz is 56.25BPM. This is the highest tempo you can have that will give a whole number of samples per tick at 960PPQ. The resulting values of importance are:

900 ticks/sec (1.111... ms/tick)

49 samples/tick

47040 samples/beat (at 4/4)

30 ticks/frame

I figure this should take all rounding errors and related artifacts out of the system if the timing master is samples. My initial test of TTS-1 rendering at this tempo still found timng errors of 100 samples or more, though at this tempo, that's only about 2 ticks.

Correction: Looks like 78.75BPM is the highest tempo that will give a whole number of samples/tick, 35, which is a nicer number in some ways, but ms/tick gets uglier:

1260 ticks/sec = .79365... ms/tick

35 samples/tick

33600 samples/beat

42ticks/frame

Correction 2: Doh! Brain not working so well today. 131.25 (21 samples/tick) and 183.75BPM (15 samples/tick) will also work, and stay under the 200BPM limit (?) in Sonar.

But I found that the lower tempo 56.25 still works best because AudioSnap finds all the transients in roughly the right place. Apparently the higher tempos have the tails of the pulses running into the next beat, which confuses AudioSnap, even though the transients still look clear.

Details, details, and more details...

ORIGINAL: Jim Wright

As pianodano wrote:
>> I wonder if I am the only one getting the feeling that the existing underlying engineering/ architecture is a piecemeal affair ?

Nope. Frankly, it's totally piecemeal. MIDI was developed starting in 1979 (Dave Smith's 'Universal Synthesizer Interface' AES paper; I was in the audience), and first shipped in 1983. It was designed to be as cheap as possible. It was not designed with any thought of synchronizing with audio, or video, or doing anything but hooking two electronic music products together. Everything after that -- just grew.

Different companies proposed different ways of syncing up MIDI with audio, video, whatever. Some of 'em worked, some didn't. Some of the better ideas -- died, because the people with those ideas were lousy businessmen. Some of the sketchier ideas -- made it in the marketplace, because their inventors were much better businessmen than engineers. Kind of like the rest of the computer hardware/software revolution that happened over the last 30 years....

I've been involved with MIDI and AES standards work since 1981, off and on. There is no grand design. There's just a lot of people trying to work out the best compromises they can, considering both what we (music geeks) know how to build, or code, and what the marketplace seems to want.

Kind of shocking, isn't it ?

- Jim

Great ! We have DAW's designed by commitee.

You are confusing when jitter can take place. You can't get MIDI jitter purely within SONAR while bouncing or freezing. (irrespective of whether its a fast or slow bounce)
MIDI events stored in a SONAR project are timestamped. When you bounce to a softsynth, the MIDI events are fed to the synth with the timestamps implicitly locked to the audio buffers. There is no chance of jitter occuring in this scenario since the synth knows exactly when to render the audio for the MIDI based on the current sample position and the timestamp on the MIDI event. Jitter doesn't apply at all here.

ORIGINAL: dewdman42

Anyway, back to Sonar...... This weekend I am going to try to run some tests where I freeze tracks to audio and compare the midi notes to the waves I see on the audio track to see if there is any jitter happening during freeze or mixdown.

After that i will try to capture the audio that is playing when I am just hitting play and playing midi tracks through soft synths. then I compare that audio track to the midi track to look at the waves and try to determine if there is jitter happening.

Those two situations are what I am most concerned about right now. If Sonar is screwing that up, then Cakewalk needs to get busy....