- DirectMusic will not ever have any more development, its true. I have spoken directly with some of their developers about it actually. But last I heard they were not going to drop the library, there will always be a way to download it, though they may not have ported it to 64bit, which for all intensive purposes, means...don't use it in the long run.

- When we talk about hardware timestamps, that has nothing to do with DirectMusic or not, it simply means whether or not the hardware puts the timestamp on the midi event before sending to Windows/OSX.

- If there is no hardware timestamp, which is most often the case, the next best place to apply a timestamp is inside the midi driver. This could still suffer from timing problems, but its still better than timestamping in the DAW. Again, some midi drivers might do it and some might not, I am not aware of a standard that says they all must. It is certainly possible and better if it does. The directmusic stuff does apply timestamps deep down inside its library somewhere I guess, but we don't really know for sure if its in the midi driver or the DirectMusic library, or whether its in some kernel layer vs user mode layer, etc.

- if the midi driver does not provide a timestamp either, then the host software(ie, Sonar) must do it, which is the worst place since its the last place, but also because it is running in user mode rather than kernel mode.

- Directmusic is a lot more than just a midi driver. Its a whole section of the DirectX api that does a bunch of really cool interesting stuff that has largely been ignored. It was used briefly in the gaming community but now that has all been replaced with audio-only stuff instead of midi. DirectMusic contains all kinds of fancy API's for embeddeding sound fonts (DLS) into your program and using midi to drive them, it even has auto-arranger capabilities in it...its actually quite complex. You can download the DirectMusic SDK and even get some software that makes use of the capabilities. Here is an interesting book about it if you are interested:

http://www.amazon.com/DirectX-Audio-Exposed-Interactive-Development/dp/1556222882

In any case, one of the things that DirectMusic has in it is some lower level routines for accessing the midi hardware. They are not well documented. It also has some even more poorly documented API's for using a different timer than the normal MM timer that has been talked about a lot here. It has some API's for queuing midi events in a manner similar to Apple's CoreMidi... trusting that something lower down is going to make sure the midi events play on schedule. People that have tried to take advantage of the directMusic API such as Steinberg, have done so thinking that somehow Microsoft will have ensured that the handling of midi events is handled more precisely. They did this because they were sick and tired of trying to wrestle with the sloppy MM timer. The important point is this, if the host software is not calling the DirectMusic API, then it makes no difference whether your midi card driver is directMusic compatible or not. See what I mean?

I'm not entirely sure what a directmusic compatible midi driver is. I think its some kind of driver, that if put together with a DAW like Steinberg that calls the DirectMusic API...MIGHT provide slightly more accurate timestamps...I'm not really sure.

What is sad is that Microsoft completely missed the boat in Vista to incorporate better midi handling into the new audio subsystem they rolled out with Vista. What they really need to have is built in, kernel level, handling of midi events, with timestamping provided by the OS. This is what Apple did on CoreMidi and what they should have done for Vista too, but they completely missed the boat.

A softsynth is free to have delay and if it exposes delay SONAR will compensate for it. My point was that it was not relevent to the problem being discussed since very few if any synths have delay (unless they use internal effects that require lookahead obviously)

Regarding timing being off during freeze some plugins have trouble with fast bounce esp with MP mode on. If the problem goes away during realtime bounce its a sure sign of that.

We shouldn't jump to conclusions about any particular plugin or SONAR being broken until the issue is properly investigated. The best test to remove the host from the equation is to use a basic test synth one to which the source code is available, like Twonar as I suggested earlier.

ORIGINAL: dewdman42

Noel,

So if I understand you correctly.. Sonar does not apply any delay compensation to softsynths?

Softsynths are not expected to have any delay unless intended by the softsynth on purpose or perhaps shoddy synth programming?

And there is no concept for DXi and VSTi's of delay compensation in general?

If that is the case, then anyone who is experiencing shoddy timing while freezing midi tracks can't do anything about it but blame the plugin, though I have to say, we will never have any way to know for sure it is the plugin's fault. You would have to try using the same plugin in some other DAW's to establish whether its Sonar having a timing problem or the plugin itself. At this point, Noel is very sure that its not Sonar. This leads me to want to know which plugins have timing problems. I would also like to know why Pianodano complains that he makes RealGuitar sound great in Sonar while playing back but then when he does a freeze the timing is off. ??? Why would the plugin do anything different during play vs during freeze?

As far as a soft synth intentionally rendering every note 10ms(300samples) late...sure a soft synth could intentionally render every note 10ms late...on purpose, like in a slow attack. That is why for this test everybody should be using something that has a sharp attack, not a blatantly long attack. I was not under the impression that anyone was using anything with an intentionally long attack. If a soft synth was adding another 10ms of latency between receiving midi and outputting the audio for no reason other than shoddy programming or complex programming...then I would probably never use that soft synth again. Even more so if the synth is doing it haphazardly or randomly.

What I heard in the past few days is that TTS has shoddy programming in it. I have not tried it yet here. It sounds like the implication here is that RealGuitar has shoddy programming too. What else?

me

Very easy - do a phase inversion test with realtime playback. Most plugins don't have a problem with fast bounce.

ORIGINAL: dewdman42

That is something I did not know. I wonder why? Does anyone know the technical reason why a soft synth plugin would require a real time bounce to work correctly? This of course instills new timing paranoia into my soul. How will I ever know for sure if its ok to use fast bounce on all my plugins? The only way to feel safe would be to always use real time bounce! YUCK!

Right, including multiprocessing synchronization issues. DX doesn't have anything to do with it though (and it supports running faster than realtime)

ORIGINAL: Jim Wright

dewdman42 -- ok, fixed my post. Sorry I misremembered what you said.

As for why some plugins screw up when Fast Bound is used -- I can imagine a number of ways that a plugin could be miscoded -- making assumptions about a fixed relationship between nominal sample rate and wall-clock time, when setting up callback timers for deferred processing is one. Hooking directly into the underlying DirectX Windows framework (which always runs realtime, I think) would be another. But I don't know, not having written audio plugins.

Noel?

ORIGINAL: Noel Borthwick [Cakewalk]
A softsynth is free to have delay and if it exposes delay SONAR will compensate for it.

Ok. So Sonar is doing the right thing there when it can. Thanks for the confirmation.

Presumably not all plugins actually expose a known delay? Which makes some sense since ideally they want them all to be the shortest delay possible for realtime interaction. The downside is the risk of jitter during bounce though I guess. And it sounds like there is not much way around it unless the plugin is smart enough to ensure otherwise at least a constant amount of delay, which is still delay, but better than jitter. but if a plugin pays no attention and just always tries as hard as it can to be as fast as possible, sometimes being faster than other times...the result will be jitter in the bounced audio. Until right now I never realized this could be a signficant issue.

Noel, another question, based on this so far. Let's say I'm dealing with a problematic plugin that seems to have variations in delay response. I guess using a larger buffer setting will not help much either huh? I mean the larger buffer gives Sonar more time to do everything it needs to do, but if that plugin is not reporting any kind of delay to Sonar, then Sonar can't do anything anyway other than assume the changing delay is intentional and leave it alone. yes?

My point was that it was not relevent to the problem being discussed since very few if any synths have delay (unless they use internal effects that require lookahead obviously)

Noted. And I know I have not really had any timing problems. I am only reverbiating reports from others which we are still trying to get enough details to pinpoint the cause.

Regarding timing being off during freeze some plugins have trouble with fast bounce esp with MP mode on. If the problem goes away during realtime bounce its a sure sign of that.

That's interesting. PianoDano, is your machine Multiprocessor or multi-core?

We shouldn't jump to conclusions

Certainly Not! We are only asking questions so far....digging deep. Please don't take these questions the wrong way.

about any particular plugin or SONAR being broken until the issue is properly investigated. The best test to remove the host from the equation is to use a basic test synth one to which the source code is available, like Twonar as I suggested earlier.

That is what we're trying to do here. I am gathering that the problem is not Sonar, but perhaps just a general aspect of the DXi and VSTi architectures. I was really not aware of the fact that a poorly coded soft synth could screw up the timing of bounced midi tracks so easily. Its rather discouraging, yet at the same time encouraging to hear that most of them don't.

ORIGINAL: brundlefly

Tested the RealGuitar. Took a while, because they definitely have some sort of randomizing algorithm that makes each attack a little different, which confused AudioSnap a little bit. It sounded funky on playback, too.

Quoting myself again, because I looked into this a little more. I Re-rendered RealGuitars using 16th-note intervals so that each note would decay to zero before the next, allowing me to see the initial transients with no overlap, and...

Well, well, well, those RealGuitar guys sure are sneaky. Turns out there are 6 or 7 distinct plucks sampled with differing attacks and amplitudes, which are played back pretty much in random order. I found this by rendering the audio twice, and comparing the results. Every once in a while, two transients would have the exact same delay, and when I zoomed in, I found that the waveforms were identical, sample for sample. But the 6 or 7 variations were never in the same order; sometime you'd get two or three of the same sample in a row, and sometimes you wouldn't see a particular sample for 9 or more notes.

Perhaps more relevant to this discussion, I also found that even the identical pluck samples didn't necessarily play back with the same delay. I observed differences in the timing of like plucks of up to 88 samples (2ms). It also seemed that some pluck samples were more likely to be very late than others. This might be because their attacks were longer.

The mean lateness, looking at the first deviation of the signal from dead zero was about 89 samples with a standard deviation of about 40 samples (< 1ms). Pretty reasonable, really, and probably just enough to humanize the sound a little without getting to the level of being sloppy.

Anyway, all this explains why inverting the rendered signal won't cancel out the real-time playback. No two "performances" of RealGuitar will be the same, due to this deliberate random variation.

Very intersting. Did you try strums or plucks or both ?? I guess you saw in the gui the slider to set chord recognition time ? I have had a feeling they were using multisamples and maybe a lot of them. That why I said earlier that I was disappointed they didn't have the stereo guitar up for demo. Keep up the good work guys.

Danny

At the risk of completely spacing-out most of the people reading this thread (and likely boring others) -- here's some more detail on the two kinds of MIDI APIs we're been talking about. The MM (or MME) API is what most applications probably still use. The DirectMusic API is more recent, has explicit support for timestamps (unlike MM/MME), but is less well supported.

The classic ("MM" or "MME") Windows MIDI API

A reasonable person, thinking about how MIDI is handled by Windows, would probably assume that Windows associates every MIDI event with a precise instant in time. Obviously, this is necessary in order for MIDI events to be received and then assembled into a coherent musical phrase. It's also necessary in order for a sequence of MIDI events to be transmitted with musically-correct timing relationships. So, a reasonable person might conclude that Windows takes care of this.

A reasonable person -- would be wrong.

The classic Windows API for sending and receiving MIDI events has no notion of time whatsoever. The classic Windows API call for sending MIDI notes is called midiOutShortMsg. It does not take a timestamp.

Instead - the application is responsible for calling midiOutShortMsg at "Time A", such that the MIDI data is transmitted from the MIDI DIN jack at "Time B", where 'Time B' is "exactly the right time" within the musical context (e.g. audio channels being played back, other MIDI events, possibly video playback....).

Note that Time A (when the application calls midiOutShortMsg) and Time B (when the MIDI event flies out the DIN jack) are different times. Lots of low-level processing can occur between Time A and Time B - and usually does. How long does that processing take? That depends, on many factors. Does the processing always take the same amount of wall-clock time? No, because many other processes can temporarily interrupt the low-level MIDI output processing. Things like incoming (or outgoing) audio transfers, network activity (a notorious source of jitter), antivirus scanning, many other things. Can you find out how long the interval between Time A and Time B -- will be? No.

The net result is that the time interval between calling midiOutShortMsg and the actual transmission of a MIDI message is variable, and not controlled by the sequencer application. This directly causes jitter.

Having said that - I'd bet that Sonar uses a kernel-level event scheduler that calls midiOutShortMsg. This means that critical event scheduling, for Sonar, most likely happens at kernel level, at a high priority, to minimize potential jitter caused by other Windows processes. However, if the MIDI device driver is poorly written, jitter will still occur (and Sonar can't do anything about that). If the MIDI device driver is for a USB or Firewire device - then the MIDI data will have to be queued up (again) to go through the USB or Firewire driver stack. Jitter can certainly happen in those driver stacks -- and again, Sonar can't do anything to minimize that kind of jitter.

The DirectMusic Windows MIDI API

DirectMusic MIDI API associates timestamps with each and every MIDI message. This is the key difference between DirectMusic MIDI and MME MIDI.

DirectMusic timestamps have a nominal resolution of 100 nanoseconds (yes, 0.1 microseconds). That does not mean the actual resolution is that good! If the DirectMusic port is a USB 1.1 MIDI port, all timestamps will be quantized to the 1 millisecond USB frame time. If the DirectMusic port is a Firewire MIDI port, the effective resolution will be something like 0.3 milliseconds (depends on driver implementation). If the DirectMusic port is "emulated' (wrapping a non-DirectMusic MIDI driver).

I like the DirectMusic API, in principle, because there is a clear path whereby accurate timestamps can be passed all the way from a user-mode sequencer application down to the final low-level MIDI hardware output driver. That doesn't mean the timestamps will always be honored (bad drivers can always be written), but at least it's possible for the timestamp to make it "all the way down".

With the MME API -- there is no way for a timestamp to be passed "all the way down", because the MME API does not handle timestamps at all. As a sequencer developer - I just had to hope that the MIDI events would actually get transmitted "pretty soon" after I called midiOutShortMsg -- and I also had to hope that the "pretty soon" would be the same length of time, , every time. If not -- jitter would occur. And it does.

One more thing. Dewdman42 has asked several times about the clock used by DirectMusic MIDI ports. This is from the DirectMusic (DirectX 9.0) SDK documentation:

To guarantee accurate timing with an acceptably low latency, DirectMusic incorporates a master clock in kernel mode. This clock is based on a hardware timer. DirectMusic automatically selects the system clock as the master clock, but an application can select a different one, such as the wave-out crystal on a sound card.
The master clock is a high-resolution timer that is shared by all processes, devices, and applications that are using DirectMusic. The clock is used to synchronize all audio playback in the system. It is a standard IReferenceClock interface. The IReferenceClock::GetTime method returns the current time as a 64-bit integer (defined as the REFERENCE_TIME type) in increments of 100 nanoseconds.

This hardware timer is not the same as the 1-millisecond resolution MME timer (although the MME timer is hopefully derived from it!). I believe this hardware timer probably has resolution in the microsecond range, but don't know for sure. Note that since the system hardware timer is not locked to any particular audio sample rate clock, the relationship between audio sample count and system time will drift over time. Therefore, a smart sequencer designer will take steps to periodically reconcile these two timebases (conforming MIDI time to audio time, not the other way around). Alternatively, it might be possible to tell DirectMusic to just directly use the audio sample clock from a particular audio interface .... but not every audio interface driver supports the IReferenceClock interface. If it doesn't - it's impossible to tell DirectMusic to use that audio interface as the master clock.

---

Here is a quote from 'earthvegaconnection'. This is a guy who has written both MME and DirectMusic replacement drivers for the 8PortSE parallel-port-based MIDI interface. I suspect he know what he's talking about.

Original: http://8portse.earthvegaconnection.com/DM.htm
Non-DirectMusic MIDI devices will appear as emulated MIDI ports to DirectMusic applications. True DirectMusic MIDI devices will appear as both a DirectMusic port as well as an emulated DirectMusic port. The emulated port in this case will be the one that MME applications can use. If you have the choice you should always choose the non-emulated ports. The emulation mode adds about 5 to 7 milliseconds of latency to the MIDI processing.

One of the biggest benefits of DirectMusic drivers is the timestamping of MIDI input. In the Windows MultiMedia system, time related MIDI input data is handled by the receiving application itself. Applications on Windows 2000 and XP always run in so-called user mode. Most of the hardware handling on the other hand, takes place in so-called kernel mode. Kernel mode processing takes priority over processes running in user mode. In practice this means that if another hardware device needs processing time at the time of delivery of the MIDI input, the MME application will just have to wait. Depending on the hardware device in question, the wait time can be considerable.

This results in MIDI input arriving late and, more importantly, variations in the delivery time of MIDI data from hardware to application. This effect is sometimes called MIDI timing jitter. DirectMusic solves this problem by timestamping the input data at the moment of arrival to a system wide reference clock. This timestamping takes place in kernel mode and thus has high priority. The DirectMusic application still runs in user mode but this time, when it receives it's input data, it already has an accurate timestamp.

The result of all this is that DirectMusic applications have more accurate MIDI recording capabilities. They will suffer much less from jitter in timing.

Yep. It all sounds good. now just convince Cakewalk to use the DirectMusic API. ;-)

Very intersting. Did you try strums or plucks or both ?? I guess you saw in the gui the slider to set chord recognition time ? I have had a feeling they were using multisamples and maybe a lot of them. That why I said earlier that I was disappointed they didn't have the stereo guitar up for demo.

I didn't mess with the configuration at all, except to figure out why I wasn't getting any sound initially (no sample set loaded by default, even though the demo has only one available - Doh!). I was really just interested in seeing how it did with the simplest possible MIDI stream, so it was all just consecutive notes.

As for the multi-sampling, this isn't unusual, of course. Most good synths, both hardware and software provide different samples for different velocity ranges if nothing else. But I wasn't ready for a synth that randomly played a different sample in response to identical inputs. Actually, I'm not sure I like it. Better to give the musician complete control over any variation by using velocity, modulation, aftertouch, etc. If they are going to include a randomizing function, great, but it really should be documented, and it should be optional.

Just for kicks I went to the KVR plugin developer forum and asked a few questions. This is a very interesting thread. PianoDano, pay special attention to the post from JCJR.

http://www.kvraudio.com/forum/viewtopic.php?t=194589