- DirectMusic will not ever have any more development, its true. I have spoken directly with some of their developers about it actually. But last I heard they were not going to drop the library, there will always be a way to download it, though they may not have ported it to 64bit, which for all intensive purposes, means...don't use it in the long run.

- When we talk about hardware timestamps, that has nothing to do with DirectMusic or not, it simply means whether or not the hardware puts the timestamp on the midi event before sending to Windows/OSX.

- If there is no hardware timestamp, which is most often the case, the next best place to apply a timestamp is inside the midi driver. This could still suffer from timing problems, but its still better than timestamping in the DAW. Again, some midi drivers might do it and some might not, I am not aware of a standard that says they all must. It is certainly possible and better if it does. The directmusic stuff does apply timestamps deep down inside its library somewhere I guess, but we don't really know for sure if its in the midi driver or the DirectMusic library, or whether its in some kernel layer vs user mode layer, etc.

- if the midi driver does not provide a timestamp either, then the host software(ie, Sonar) must do it, which is the worst place since its the last place, but also because it is running in user mode rather than kernel mode.

- Directmusic is a lot more than just a midi driver. Its a whole section of the DirectX api that does a bunch of really cool interesting stuff that has largely been ignored. It was used briefly in the gaming community but now that has all been replaced with audio-only stuff instead of midi. DirectMusic contains all kinds of fancy API's for embeddeding sound fonts (DLS) into your program and using midi to drive them, it even has auto-arranger capabilities in it...its actually quite complex. You can download the DirectMusic SDK and even get some software that makes use of the capabilities. Here is an interesting book about it if you are interested:

http://www.amazon.com/DirectX-Audio-Exposed-Interactive-Development/dp/1556222882

In any case, one of the things that DirectMusic has in it is some lower level routines for accessing the midi hardware. They are not well documented. It also has some even more poorly documented API's for using a different timer than the normal MM timer that has been talked about a lot here. It has some API's for queuing midi events in a manner similar to Apple's CoreMidi... trusting that something lower down is going to make sure the midi events play on schedule. People that have tried to take advantage of the directMusic API such as Steinberg, have done so thinking that somehow Microsoft will have ensured that the handling of midi events is handled more precisely. They did this because they were sick and tired of trying to wrestle with the sloppy MM timer. The important point is this, if the host software is not calling the DirectMusic API, then it makes no difference whether your midi card driver is directMusic compatible or not. See what I mean?

I'm not entirely sure what a directmusic compatible midi driver is. I think its some kind of driver, that if put together with a DAW like Steinberg that calls the DirectMusic API...MIGHT provide slightly more accurate timestamps...I'm not really sure.

What is sad is that Microsoft completely missed the boat in Vista to incorporate better midi handling into the new audio subsystem they rolled out with Vista. What they really need to have is built in, kernel level, handling of midi events, with timestamping provided by the OS. This is what Apple did on CoreMidi and what they should have done for Vista too, but they completely missed the boat.

Very easy - do a phase inversion test with realtime playback. Most plugins don't have a problem with fast bounce.

ORIGINAL: dewdman42

That is something I did not know. I wonder why? Does anyone know the technical reason why a soft synth plugin would require a real time bounce to work correctly? This of course instills new timing paranoia into my soul. How will I ever know for sure if its ok to use fast bounce on all my plugins? The only way to feel safe would be to always use real time bounce! YUCK!

ORIGINAL: Noel Borthwick [Cakewalk]
A softsynth is free to have delay and if it exposes delay SONAR will compensate for it.

Ok. So Sonar is doing the right thing there when it can. Thanks for the confirmation.

Presumably not all plugins actually expose a known delay? Which makes some sense since ideally they want them all to be the shortest delay possible for realtime interaction. The downside is the risk of jitter during bounce though I guess. And it sounds like there is not much way around it unless the plugin is smart enough to ensure otherwise at least a constant amount of delay, which is still delay, but better than jitter. but if a plugin pays no attention and just always tries as hard as it can to be as fast as possible, sometimes being faster than other times...the result will be jitter in the bounced audio. Until right now I never realized this could be a signficant issue.

Noel, another question, based on this so far. Let's say I'm dealing with a problematic plugin that seems to have variations in delay response. I guess using a larger buffer setting will not help much either huh? I mean the larger buffer gives Sonar more time to do everything it needs to do, but if that plugin is not reporting any kind of delay to Sonar, then Sonar can't do anything anyway other than assume the changing delay is intentional and leave it alone. yes?

My point was that it was not relevent to the problem being discussed since very few if any synths have delay (unless they use internal effects that require lookahead obviously)

Noted. And I know I have not really had any timing problems. I am only reverbiating reports from others which we are still trying to get enough details to pinpoint the cause.

Regarding timing being off during freeze some plugins have trouble with fast bounce esp with MP mode on. If the problem goes away during realtime bounce its a sure sign of that.

That's interesting. PianoDano, is your machine Multiprocessor or multi-core?

We shouldn't jump to conclusions

Certainly Not! We are only asking questions so far....digging deep. Please don't take these questions the wrong way.

about any particular plugin or SONAR being broken until the issue is properly investigated. The best test to remove the host from the equation is to use a basic test synth one to which the source code is available, like Twonar as I suggested earlier.

That is what we're trying to do here. I am gathering that the problem is not Sonar, but perhaps just a general aspect of the DXi and VSTi architectures. I was really not aware of the fact that a poorly coded soft synth could screw up the timing of bounced midi tracks so easily. Its rather discouraging, yet at the same time encouraging to hear that most of them don't.

At the risk of completely spacing-out most of the people reading this thread (and likely boring others) -- here's some more detail on the two kinds of MIDI APIs we're been talking about. The MM (or MME) API is what most applications probably still use. The DirectMusic API is more recent, has explicit support for timestamps (unlike MM/MME), but is less well supported.

The classic ("MM" or "MME") Windows MIDI API

A reasonable person, thinking about how MIDI is handled by Windows, would probably assume that Windows associates every MIDI event with a precise instant in time. Obviously, this is necessary in order for MIDI events to be received and then assembled into a coherent musical phrase. It's also necessary in order for a sequence of MIDI events to be transmitted with musically-correct timing relationships. So, a reasonable person might conclude that Windows takes care of this.

A reasonable person -- would be wrong.

The classic Windows API for sending and receiving MIDI events has no notion of time whatsoever. The classic Windows API call for sending MIDI notes is called midiOutShortMsg. It does not take a timestamp.

Instead - the application is responsible for calling midiOutShortMsg at "Time A", such that the MIDI data is transmitted from the MIDI DIN jack at "Time B", where 'Time B' is "exactly the right time" within the musical context (e.g. audio channels being played back, other MIDI events, possibly video playback....).

Note that Time A (when the application calls midiOutShortMsg) and Time B (when the MIDI event flies out the DIN jack) are different times. Lots of low-level processing can occur between Time A and Time B - and usually does. How long does that processing take? That depends, on many factors. Does the processing always take the same amount of wall-clock time? No, because many other processes can temporarily interrupt the low-level MIDI output processing. Things like incoming (or outgoing) audio transfers, network activity (a notorious source of jitter), antivirus scanning, many other things. Can you find out how long the interval between Time A and Time B -- will be? No.

The net result is that the time interval between calling midiOutShortMsg and the actual transmission of a MIDI message is variable, and not controlled by the sequencer application. This directly causes jitter.

Having said that - I'd bet that Sonar uses a kernel-level event scheduler that calls midiOutShortMsg. This means that critical event scheduling, for Sonar, most likely happens at kernel level, at a high priority, to minimize potential jitter caused by other Windows processes. However, if the MIDI device driver is poorly written, jitter will still occur (and Sonar can't do anything about that). If the MIDI device driver is for a USB or Firewire device - then the MIDI data will have to be queued up (again) to go through the USB or Firewire driver stack. Jitter can certainly happen in those driver stacks -- and again, Sonar can't do anything to minimize that kind of jitter.

The DirectMusic Windows MIDI API

DirectMusic MIDI API associates timestamps with each and every MIDI message. This is the key difference between DirectMusic MIDI and MME MIDI.

DirectMusic timestamps have a nominal resolution of 100 nanoseconds (yes, 0.1 microseconds). That does not mean the actual resolution is that good! If the DirectMusic port is a USB 1.1 MIDI port, all timestamps will be quantized to the 1 millisecond USB frame time. If the DirectMusic port is a Firewire MIDI port, the effective resolution will be something like 0.3 milliseconds (depends on driver implementation). If the DirectMusic port is "emulated' (wrapping a non-DirectMusic MIDI driver).

I like the DirectMusic API, in principle, because there is a clear path whereby accurate timestamps can be passed all the way from a user-mode sequencer application down to the final low-level MIDI hardware output driver. That doesn't mean the timestamps will always be honored (bad drivers can always be written), but at least it's possible for the timestamp to make it "all the way down".

With the MME API -- there is no way for a timestamp to be passed "all the way down", because the MME API does not handle timestamps at all. As a sequencer developer - I just had to hope that the MIDI events would actually get transmitted "pretty soon" after I called midiOutShortMsg -- and I also had to hope that the "pretty soon" would be the same length of time, , every time. If not -- jitter would occur. And it does.

One more thing. Dewdman42 has asked several times about the clock used by DirectMusic MIDI ports. This is from the DirectMusic (DirectX 9.0) SDK documentation:

To guarantee accurate timing with an acceptably low latency, DirectMusic incorporates a master clock in kernel mode. This clock is based on a hardware timer. DirectMusic automatically selects the system clock as the master clock, but an application can select a different one, such as the wave-out crystal on a sound card.
The master clock is a high-resolution timer that is shared by all processes, devices, and applications that are using DirectMusic. The clock is used to synchronize all audio playback in the system. It is a standard IReferenceClock interface. The IReferenceClock::GetTime method returns the current time as a 64-bit integer (defined as the REFERENCE_TIME type) in increments of 100 nanoseconds.

This hardware timer is not the same as the 1-millisecond resolution MME timer (although the MME timer is hopefully derived from it!). I believe this hardware timer probably has resolution in the microsecond range, but don't know for sure. Note that since the system hardware timer is not locked to any particular audio sample rate clock, the relationship between audio sample count and system time will drift over time. Therefore, a smart sequencer designer will take steps to periodically reconcile these two timebases (conforming MIDI time to audio time, not the other way around). Alternatively, it might be possible to tell DirectMusic to just directly use the audio sample clock from a particular audio interface .... but not every audio interface driver supports the IReferenceClock interface. If it doesn't - it's impossible to tell DirectMusic to use that audio interface as the master clock.

---

Here is a quote from 'earthvegaconnection'. This is a guy who has written both MME and DirectMusic replacement drivers for the 8PortSE parallel-port-based MIDI interface. I suspect he know what he's talking about.

Original: http://8portse.earthvegaconnection.com/DM.htm
Non-DirectMusic MIDI devices will appear as emulated MIDI ports to DirectMusic applications. True DirectMusic MIDI devices will appear as both a DirectMusic port as well as an emulated DirectMusic port. The emulated port in this case will be the one that MME applications can use. If you have the choice you should always choose the non-emulated ports. The emulation mode adds about 5 to 7 milliseconds of latency to the MIDI processing.

One of the biggest benefits of DirectMusic drivers is the timestamping of MIDI input. In the Windows MultiMedia system, time related MIDI input data is handled by the receiving application itself. Applications on Windows 2000 and XP always run in so-called user mode. Most of the hardware handling on the other hand, takes place in so-called kernel mode. Kernel mode processing takes priority over processes running in user mode. In practice this means that if another hardware device needs processing time at the time of delivery of the MIDI input, the MME application will just have to wait. Depending on the hardware device in question, the wait time can be considerable.

This results in MIDI input arriving late and, more importantly, variations in the delivery time of MIDI data from hardware to application. This effect is sometimes called MIDI timing jitter. DirectMusic solves this problem by timestamping the input data at the moment of arrival to a system wide reference clock. This timestamping takes place in kernel mode and thus has high priority. The DirectMusic application still runs in user mode but this time, when it receives it's input data, it already has an accurate timestamp.

The result of all this is that DirectMusic applications have more accurate MIDI recording capabilities. They will suffer much less from jitter in timing.

Very intersting. Did you try strums or plucks or both ?? I guess you saw in the gui the slider to set chord recognition time ? I have had a feeling they were using multisamples and maybe a lot of them. That why I said earlier that I was disappointed they didn't have the stereo guitar up for demo.

I didn't mess with the configuration at all, except to figure out why I wasn't getting any sound initially (no sample set loaded by default, even though the demo has only one available - Doh!). I was really just interested in seeing how it did with the simplest possible MIDI stream, so it was all just consecutive notes.

As for the multi-sampling, this isn't unusual, of course. Most good synths, both hardware and software provide different samples for different velocity ranges if nothing else. But I wasn't ready for a synth that randomly played a different sample in response to identical inputs. Actually, I'm not sure I like it. Better to give the musician complete control over any variation by using velocity, modulation, aftertouch, etc. If they are going to include a randomizing function, great, but it really should be documented, and it should be optional.

This is a very interesting thread. I haven't really considered this before. I've had instances where anomalies occured, but didn't really know what the cause was, and it doesn't always happen. I will pay closer attention from now on.

Thanks for helping to stretch the old gray matter.

ORIGINAL: dewdman42

Just for kicks I went to the KVR plugin developer forum and asked a few questions. This is a very interesting thread. PianoDano, pay special attention to the post from JCJR.

http://www.kvraudio.com/forum/viewtopic.php?t=194589

Thanks. I spend time there but over at the Receptor forum. Just found this post this over a musiclab. http://www.musiclab.com/community/forum/viewtopic.php?t=2130

ORIGINAL: Jim Wright

OK, I did some (very informal, non-scientific) testing of 3 different MIDI interfaces I happen to own.

My DAW: AMD Athlon 939 X2 4800+, MSI Neo2 Platinum socket 939 motherboard, 2x1G of OCZ RAM, tons of disk storage, EMU 1820M, Matrox 650P video card with 2 LCDs.

The three MIDI interaces used are: Edirol UM550 (USB), EMU 1820M (PCI) and M-Audio Axiom 61 (USB, using MIDI DIN Input on back of keyboard).

The UM550 can act as a MIDI Patchbay as well as a MIDI interface. This is how my gear was connected:
1) Korg keyboard was connected to UM550 MIDI IN #1.
2) UM550 MIDI OUT #1 was patched to EMU MIDI IN #1
3) UM550 MIDI OUT #2 was patched to Axiom USB Input

Sonar (6.2.1) was set up to record each interface to a different MIDI track.
* Track 1 - was Axiom USB MIDI
* Track 2 - was UM550 MIDI IN #1
* Track 3 - was EMU PCI MIDI Input.

Sonar was set for 120 BPM, 960 ppqn.

Test procedure:
A) Enable Record on all three MIDI tracks
B) Disable echo for all tracks
C) Start recording (all three tracks simultaneously)
D) Play notes on the Korg keyboard, (a series of 18 staccato notes, of roughly quarter-note duration - I was not listening to a metronome, just playing free).

Results were as follows:

         EMU PCI         Both USB       Delta
 
         1:02:501        1:02:503        2
         1:03:458        1:03:460        2
         1:04:478        1:04:478        0
         2:01:462        2:01:464        2
         2:02:493        2:02:493        0
         2:03:501        2:03:503        2
         2:04:602        2:04:602        0
         3:01:560        3:01:564        4
         3:02:670        3:02:670        0
         3:03:658        3:03:658        0
         3:04:735        3:04:735        0
         4:01:739        4:01:739        0
         4:02:727        4:02:729        2
         4:03:735        4:03:735        0
         4:04:800        4:04:804        4
         5:01:792        5:01:792        0
         5:02:796        5:02:798        2
         5:03:752        5:03:752        0

The two USB interfaces (UM550 and Axiom 61) had exactly the same timing for all notes. This surprised me a bit; I thought the Edirol (with 'FPT') might perform slightly better, but this was not true in my very limited test.

As you can see, 10 notes had the same timing with both PCI and USB interfaces. 6 notes were 2 ticks later with the USB interfaces. 2 notes were 4 ticks later with the USB interfaces.

Since 1 tick is about 0.5 milliseconds (@120BPM, using 960 PPQN) - the maximum jitter was about 2 milliseconds (4 ticks). Note that the deltas are even numbers (2 ticks == 1 millisecond, 4 ticks == 2 milliseconds). This is not surprising: the USB frame rate is 1 millisecond, which introduces a subtle quantizing to 1 millisecond boundaries.

To recap this very unscientific test:

• The PCI interface seemed to have the tightest timing (but wasn't compared against any kind of absolute reference, so it's a somewhat dubious conclusion).
  
• The two USB interface performed the same
  
• There's no way to know how much jitter was present in the PCI interface stream. There was no reference for comparison, and the source MIDI notes were played freely, not sequenced.
  
• There's no way to know the latency of any of these interfaces, from the measurements I took.
  
• Both USB interfaces had a maximum jitter of about 2 milliseconds (4 ticks) - on top of whatever jitter was also present in the PCI interface MIDI.
  
• USB Jitter was quantized to 1 millisecond boundaries (delta was 1 millisecond or 2 milliseconds, but never 0.5 or 1.5 milliseconds).
  
• Iwas recording 3 tracks in parallel. It's possible that this could introduce some extra processing overhead (3 drivers were being tickled at the same time) that would not be present if only 1 driver was in use. In other words -- it's possible that USB jitter would be reduced a bit if only one MIDI interface was being used.

Jitter of 2 milliseconds should be fine for most music and most musicians (although 1 millisecond max jitter is what I and others recommend). If you're experiencing significantly higher MIDI jitter levels -- there's probably something wrong with your system configuration.

- Jim

Jim,

I know it would be a lot to ask, but could you maybe do the same test while playing to the metronome at 960ppqn. And repeated again with about 4 or 5 audio tracks running and every midi port you've got enabled? I would love to see if your results are similar to what I see occur. I have a reasonably complex midi setup

My outboard midi setup uses a Maudio 8x8 via USB patched as follows:

Roland R-5 drum machine 1in - 1 out
Microlynx Sychronizer 2 out Syncs tape machine and DTRS machines via MTC
Tascam DA-78hr master 3in - 3 out Only used for storing menu settings and offets.
Roland MC500MKII seq 4in - 4 out
Roland Juno 106 5in - 5 out
Yamaha TX81z 6in - 6 out
Yamaha P120 piano 7in - 7 out
Yamaha DTExtreme drums 8in- 8out

Maudio 1x1 USB - Korg PA80 arranger 1-16
Korg padkontrol USB
Yamaha Tyros arranger USB1 1-16 USB2 1-16 (or 32 channesl available via both ports enabled)

Yamaha MotifXS USB 1 1-16 only used. Also used as main control surface. The mdi out of this instrument fires the 16 channels on the Receptor. So that setup is: Motif USB>Sonar track>Motif USB>Motif midi out>Receptor.

Tascam IF/FW firewire - Midi 1-16. Controls mixer TC and mixer automation - sometimes used as the control surface
FrontierTranzport via USB

Audio via the Tascam 24 i/o IF/FW via firewire on separate bus.

Sonar 6.2.1

Computer - All services disabled that I can. Dedicated music only.

ASUS P4 3.2ghz. 2gig ram, hyperthreading. Regardless what is said, mine runs Sonar much better with HT enabled.

I just read somewhere that it is recommended (by someone) that ports not used should be disabled. I hope that's not for real. That's a new one on me. I have be totally frustrated for years about the way sonar reorders ports when one on the list is not found. You can imagine how time comsuming it is getting all the ports listed in the correct order if Sonar reoders them.

I had planned to get another Maudio 8x8 for a few more of the keyboards at hand, but I have held off because of the timing problems (be it latency, jitter, me, whatever, I have run into that's bogging me down.

Thanks

Danny

Using the inverted sum trick will not really tell you if the timing is off, it will only tell you if the audio is at all different.

If you know that the waveforms in the two tracks are identical (or should be with a non-randomizing synth given a stream if identical MIDI events), then an cancellation errors the can't be cured by balancing amplitudes are due to timing.

Jim, Thanks for the quote from 'earthvegaconnection'. So that's what (emulated) means in DXDIAG. All of my midi ports are emulated, so none actually have DirectMusic drivers. I wonder if any sequencer actually uses DirectMusic midi timestamps anyhow...

dewdman, thanks, and I like the new avatar. The old one looked kindof like a shady character. I'm still an old geezer.

Jim, your tests show just about the same results as mine for the 1820m midi ports, so I guess I did nothing wrong. Next test for me will be to repeat the test on a medium size project that contains midi, soft synts, and audio to see if the results are the similar.

ORIGINAL: pianodano
So what you're saying is there could be minute timing inaccuracies in the samples themslves ??

Well sure there could be, but that would equally be a bug in RealGuitar as anything else. But I actually doubt that is the problem. Presumably they would have made sure that the samples would all have their attack transients lined up so that they sound correct. If that were not the case, then when you play back your sequence before freezing it would not sound right either...the random sample selection would be causing time problems then too, if that were the case.

I am much more suspicious that the plugin is not handling midi timestamps correctly during freeze. Did you try both fast and slow freezing by the way? Do they also ship a VST version or only DX version? If so, did you try the VST version? You're the one that said that if you make the audio buffer smaller before doing a freeze it sounds better, right?

what if you do a real time bounce, will the result sound correct that way?

The original version I have (1.5) was DXI only. IT required around 190 meg to laod up the stereo sample( I think). 2L runs as a VSTi and somehow only partially load the sample (again I think). Worked all weekend so I haven't had time to startup the stuff to even see which freeze method is being used. It would be the default I am sure, since I only installed 6.2 about a month ago and never had a reason to change anything in 3,4,5 or 6.

However how many synths do you konw that would garantee that each time the produced audio output would ge exactly the same? Probably none.

Actually, I would expect most sample-based "synths" to play back the same thing every time. Likewise, most DSPs will run the same numbers the same way under the same conditions, so it's probbly more often the case than not that you get identical output form identical input. If a synth uses a heavy dose of modeling technology, maybe not, but I think the randomizing ones are the exception. That may be changing, though. I have to confess that my perspective is based more on older equipment.

Anyway, I just wanted to clarify that timing issues will usually be revealed by a cancellation test.

ORIGINAL: dewdman42

ORIGINAL: brundlefly
Actually, I would expect most sample-based "synths" to play back the same thing every time.

Unless its Real Guitar, which uses randomized sample alternation. ;) but yea, the cancelation test is an interesting one. I intend to run it on every single soft instrument I own. But if I don't get a perfect cancelation I can't jump to the conclusion immediately that the timing is off. MAYBE that's what it is...or maybe its just a really good synth that attempts to make every note sound a little different, which is not such a bad thing.

Yep, that is what's amazing to me and have I have understood this for several years. As far as I can see there is no way to really EVER nail down just what the heck the cause is or to even get 2 people to definitively agree what the problem is.

But if I don't get a perfect cancelation I can't jump to the conclusion immediately that the timing is off.

No, but it's a good starting point for eliminating synths that don't have an issue. If you do get a cancellation failure, you just zoom in and look at the waveform. It didn't take long to see what RealGuitar was doing.