• SONAR
  • 64bit floating point vs 64bit fixed point (p.3)
2006/01/11 23:50:09
attalus

ORIGINAL: bogdan


ORIGINAL: Ron Kuper [Cakewalk]

Floating point implementations will generally waste memory and result in slower execution (on most CPU's).
I am pretty sure floating point is faster than fixed point on Pentium and x64 CPUs, and uses the same amount of memory.


Regarding memory you're wrong. Fixed point implementation (when done correctly), will not use the same number of bits for all variables. It has more flexibility. Also, people often use block floating point for arrays, etc. That saves a lot based on array sizes. If you have a simple algorithm it doesn't really matter what you do, but for more memory intensive algorithms, fixed point will always win.

Regarding execution speed it depends. It depends on the instruction set, ALU structure, existence or not of a barrel shifter and type and number of the accumulators. If you have a single cycle integer MAC operations with a barrel shifter at the output, fixed point would be either faster or the same speed as floating point. (for most algorithms) Sometimes fixed point would require more operations to implement what floating point would do in less. However, based on the number of cycles used and number of memory accesses necessary it is not always easy to guess which one would be faster.

I'm no expert on Pentium instruction set, so I do not know if what I mentioned above is available or not. However, I worked on most existing DSP's since the 1980's including many RISC CPU's. On all of them fixed point was always faster and used considerably less memory. There was no exception.

As for the types of instructions, you should definitely use "integer" instructions for fixed point math. They will be faster. As for the saturation, you shouldn't worry about it much. People who do fixed point for living, know how to deal with it efficiently. Also, most good CPU's know how to handle integer saturation properly. (all DSP CPU's do it for sure)

Finally, 1-bit shift following multiply is only needed for some types of fixed point algorithms. If the CPU does not support implied shift (happens in the same cycle as the multiply) you do not really have to use it. That is, you do not need to use always Q15, Q31 or Q47 number formats, you can use arbitrary Q-formats and keep track of where the format goes, so you adjust only the final results instead of all intermediate products or sums.......

You would be amazed to find out how much faster you can get things to execute with fixed point math. (especially if the CPU architecture enables you to do it with efficient integer operations as mentioned above)

However, I must point out (again), it would come at a price. Highly skilled fixed point DSP programmers (paid considerbly more than a high school floating point "wizz") are not easy to find. Development is harder and takes more time and thinking since the algorithms have to be converted with a lot of care. Debugging is harder and maintenance will cost more.

Regards,
Bogdan



I had to re-read your posts several times to get some understanding but i think i got the gist of it now, essentially your saying fixed point uses less memory and is faster than 64bit floating, In what ways would the benifits of 64bit fixed point show in a Audio host application over floating point? Or in vst plugins if all where programmed well?
2006/01/12 01:33:00
sinc
ORIGINAL: attalus

I have no doubt converters play there role but i'm sure aswell that the 192khz extra bandwidth plays it's role aswell.I'm sure the same is with the neve 88D 40bit processing and the SSL beyond 192khz bandwith aswell, Im sure they convert better, but i'll bet my right arm that they just plain sound better overall.In the end you have a good point but i'm pretty convinced soundcards can always be improved, and at this point i believe 32bit would be a hearable difference if the converters and the rest of the card is built well!



Yes, soundcards can always be improved, but at this point, 32-bit soundcards just plain don't make sense.

The A/D converter in a soundcard essentially converts voltage in the wire into a number. The range of numbers depends on the bit depth. 24-bit audio has 2^24 or 16,777,216 different possible values. The so-called "professional" audio level operates at a nominal level of -4dBu, which equates to only a bit over one volt. This is just the "nominal voltage", though - something of an "average". The actual peaks can be much higher. Say the A/D converter can handle 5 or even 10 volts maximum.

Divide this into 16,777,216 pieces, though, and each piece ends up being extremely tiny - easily less than a microvolt. In order to really sample sound at a true 24-bit resolution, the A/D converter would need to be able to accurately detect changes of less than a microvolt.

It turns out that it takes some pretty impressive electronics to achieve this feat. Changes of a microvolt can easily be caused by a variety of sources, including magnetic interference, fluctuations in the power source, and even moment-to-moment temperature changes inside the electrical components. This means that it would be very difficult at best to get any soundcard to work at a true 24-bit resolution inside of something like a home computer.

Essentially, what happens is that noise inside of the system covers up the really fine details. Imagine feeding a steady signal into a 24-bit A/D converter such that the A/D converter should theoretically read a constant 5000. At the voltage levels of audio systems, the actual result would not be a constant 5000. The value would randomly jump around 5000, sometimes reading above, sometimes below. Oversampling can help remove some of this variation and correct for some of the noise, but not all of it. If we convert the readings to binary, we may notice that the first 20 bits of every value are always the same. However, those last 4 bits just jump around, basically randomly. This means that our soundcard is really giving us audio with an effective bit depth of 20 bits, because those last 4 bits of resolution are covered up by random noise.

As it turns out, the absolute best of the soundcards most of us on this forum use get an effective bit depth of not much more than 20 bits. Well, actually, the sound may sound a bit better than something dithered to 20 bits, because of an interesting phenomenon where the human brain can identify sounds lower than a noise threshold. (This is the ability that sometimes causes people to think they hear voices in white noise - the human brain tries to discern the sound beneath the noise floor, even if there's no sound there.) Essentially, the brain can make out some of the gist of the sounds hidden in those lower 4 bits, even though they are mostly masked by noise. But for all practical purposes, the effective dynamic range ends up being roughly 20 bits (or approx. 120dB).

The Neve console you mention is a different beast. It is completely custom-designed with high-end audio in mind. So, it can have all kinds of custom features designed to minimize noise in the A/D converters, such as special power supply, special housing, and of course, a high-end converter. But even that Neve console is only 96/24. It just may have an effective bit depth of something closer to 24 than the soundcards most of us work with. It may even have an effective bit depth of pretty close to 24. But obviously, you need to pay for that.

But getting back to computer soundcards... Since the best of them are still rather far from actually using all 24-bits, it would be an utter waste to build one that was 32-bit, at least right now. Especially since A/D converters tend to get geometrically more difficult to make as resolution goes up. In other words, it is not 50% harder/more expensive to build a theoretical 32-bit soundcard, but many, many times more expensive. And if the effective bit depth stays at 20-bits because of electronic noise, it is even more of a waste. The perceived performance of the 32-bit soundcard would be identical to that of a 24-bit soundcard, and both would be getting almost the same results as a 20-bit soundcard.

Since one of the main problems with increasing bit depth is that the voltage changes are too small to be measured over the electronic noise, it seems one option might be to raise the voltage level in the analog lines. But this means that a new analog audio standard would have to be developed, like a +20dBV standard or something. But this would have a ripple effect throughout all audio electronics - every piece of analog equipment would have to be built to handle higher voltages, which means better (i.e. more expensive) power supplies and other components, which means EVERYTHING will cost more...

And then there's the fact that sound quality depends on more than just bit depth. The quality of the low-pass filters, the oversampling algorithm, and the stability of the timing crystal are also of critical importance. It would do no good to create a 32-bit soundcard if jitter in timing kept the effective bit depth at 20. And then there's the D/A converters, which have basically all the same problems in reverse. Getting a D/A converter to run at higher than 20-bits is just as hard as getting the A/D converter to do it.

Then there's also the question as to whether or not 32-bits is really worth it at all. 16-bit audio can only encode a dynamic range of 96dB. The human ear can discern a range of 120dB. This means that the range of human hearing is roughly 16 TIMES greater than the dynamic range of a 16-bit CD (24dB greater dynamic range = 16 times the dynamic range). It's no wonder that the sound of 16-bit audio is noticeably lacking, compared to real life. The dynamic range of 24-bit audio, however, is 144dB. If you recorded both the absolute quietest sound you could and the absolute loudest sound you could using 24-bit audio, then played it back on a system with true 24-bit D/A converters and speakers to match so that the absolute quietest sound registered 0dBspl (which, by definition, is the absolute quietest sound that the "average" human ear can hear), then the loudest sound would be way above the average human's pain threshold, a good twice as loud as a jet engine. A more practical choice would be to play the absolute loudest stuff at a more reasonable volume, which means any noise or distortion in our recorded tracks would be WAY, WAY, WAY below the threshold of human hearing. And this is just with true 24-bit audio. If the error with 24-bit recording is so far below human hearing that there's no possible way it could be discerned, what's the point of recording at 32-bit?

(Keep in mind that the record bit depth should not be confused with the bit depth for processing. We always want our intermediate results to be at a higher bit-depth than either our starting point or our desired ending results. That way the accumulated error stays at a minimum. If we round or truncate to 16-bit or even 24-bit after every mathematical operation, the distortion becomes apparent rather quickly during processing. So processing our audio with 32, 40, 48, or even 64 bits of precision is desirable, even if we only record at 24-bit. Recording at 32-bit would imply that we would probably want at least 48-bit or 64-bit processing.)

At some point the electronics might improve to the point that common soundcards can actually operate at true 24-bit, even if the nominal voltage levels of the analog signals don't change. Or, the audio industry might even adopt a new standard (undoubtedly not anytime soon), and increase the analog voltages. But until then, recording at 32-bit doesn't make any sense at all, if it ever will make sense.
2006/01/12 04:23:18
AndyW
And just to add some pain and consternation to all this.. ...there are very few places on the planet where the audio noise floor would allow true 144dB dynamic range....so even if everything was PERFECT, you had a perfectly recorded, noise free recording of an instrument on a noiseless A/D converter whose quietest sound and loudest sound had 144dB of difference, and played it back on a perfect 24 D/A converter in an anechoic room, your actual ambient noise from just *breathing* would raise the noise floor and limit the dynamic range.

So much of this belongs in the "these go to 11" or Emperor's New clothes category...just make music, people! The Beatles recorded to a 4 or 8 track with a noise floor probably around -60dB. Besides, we squash the crap out of all our mixes anyway nowadays! It is so ironic to listen to so many people crow about 64 bit this and 32 bit that and then squash their 24bit mix to have about 10 bits or less of true dynamic range(myself included)....which some 13 year old will play as a 128bps mp3 on her ipod.

What we need to do is have the meter on our audio apps have an option to switch the scale to "bit depth" from dB and people could really undestand what it means and see how much dynamic range they really use.
2006/01/12 06:28:34
Muziekschuur
Compression reduces the dynamic range and can make stuff audible what wasn't audible before. Above mentioned stories are known and found to be true regarding "RECORDING". But are they true when mixing and mastering is concerned.

A mastering engineer does not wanna alter what is considdered 'GOOD'. So an application wich does not damage original material is important to the mastering engineer.

In the mixing proces reverbs are often placed before LIMITING. So the resolution of the reverb is important. As the resolution when limiting is reduced (again... perhaps) details may show up of artefacts of damaging by compression, effects or clipping at the recordingstage.

TC electronics M6000 and the Lexicon L48 systems have resolutions of 48bit (double precision, don't know if this is floating or fixed (as I am not a programmer I am unaware of such technical details). It would be great to get such quality within a DAW. With the coming of 4 cores in systems (on one socket) systems will get powerfull enough to do 24bit/96khz. And when used on Vista an internal resolution of 64bit/96khz. This will have impact on VST instruments such as convolution reverbs and their level of detail.

If I am wrong here I wouldn't mind to be set straight.

Thanks for such a great insight on this subject.

There is another thing why I am in favor of such stuff. I was told that people can hear up to 22khz. But are sensitive to phasechanges up to 200khz.

Ambisonics and surround in general will be more powerfull if such high resolutions are possible I think. 96khz/64bit will not damage sound up to 40Khz. So alot more phasechanges can be heard in the high end (if tweeters are RIBBON (wich can give up to 40Khz (measured, but probably higher) Some dometweeters (look at Tannoy ellipse for example) can go that high, but are probably less...

I am interested if above can be shot down...... Amaze me.


Muziekschuur
2006/01/12 07:52:33
jlgrimes
The difference between 64 bit floating point and 64 bit fixed point is fixed point means that for a 64 bit word your decimal point will always be in the same place.

Floating point allows the decimal to move and is much better for representing very small real numbers between 0 and 1 and also allow some representation of large numbers higher than the 64 bit integer limit.

Both has the same number of possibilities but which one is better depends on the application.

I think for audio processing most coders prefer floating point math because of the high decimal resolution but there would probably be some apps where fixed point would be better.
2006/01/12 10:50:58
Master Chief [Cakewalk]
Fixed point implementation (when done correctly), will not use the same number of bits for all variables.
I can see how that makes sense for coefficients or internal state for DSP code. But how much memory are we talking about here? It seems to me that the only memory overhead of any significance in real DSP for things like delay lines and other long buffers. For these you need the buffer to be the same word length as the sample format.

I'm no expert on Pentium instruction set, so I do not know if what I mentioned above is available or not.
Pentium does not have a single cycle MAC instruction. However if your code is fully pipelined you can get 1 cycle throughput on floating point multiplies and adds. It's generally accepted that for audio work, floating point is more efficient on Pentium.

You would be amazed to find out how much faster you can get things to execute with fixed point math. (especially if the CPU architecture enables you to do it with efficient integer operations as mentioned above)
That's a big "if". Since SONAR only runs on Pentium class processors, for our application floating point wins.
2006/01/12 11:18:46
kp
What about if using SSE/SSE2 to work on multiple streams of ints in parallel (yes, I know SSE can also work on multiple fp data too - just musing)?
2006/01/12 16:58:20
Scott Reams
The only real (but very important) advantage of floating point is ease of use.


A very real advantage of floating point that hasn't been mentioned is that the end user doesn't really have to concern him/herself with clipping within the mix engine. Headroom beyond 0db is enormous. With a fixed-point mix engine (Protools) clipping can occur at every stage. Much more care must be taken by the end user in this regard.

-S
2006/01/12 17:50:49
Guest
Headroom beyond 0db is enormous.


even with 48bit fixed point, you've got ~385db of dynamic range. that's insane ;-) you
can get clip on the PT engine .. but you really have to try ..

also, it's important to note that you lose the 64bit precision/carry-forward when going
in and out of a plug-in which doesn't deal with that precision. so, it's not a license to
go nuts even in Sonar.

jeff
2006/01/13 03:52:48
Muziekschuur
Yes, it would be nice if there was a list with suppported 64bit VST plugins (so we know the internal signalflow isn't 'damaged' if we like that signalpath.......

May be a lil off topic here....

Muziekschuur
© 2024 APG vNext Commercial Version 5.1

Use My Existing Forum Account

Use My Social Media Account