64bit floating point vs 64bit fixed point

your title says 64 fixed versus 64 floating... do you mean 48 fixed versus 64 floating?


IIRC, fixed point is better in general than floating because it utilizes a "double sampling technology" wheras floating does not..

because it utilizes a "double sampling technology" wheras floating does not..

what does double sampling have to do with the way the number is represented?

IMHO, it doesn't really matter. The human ear can't hear distinctions in levels
so fine. Granted, performing many mixing and effect calculations can introduce
errors into the soup. But I doubt anyone could hear the difference.

We would need a massive ear and cognitive upgrades for our hardware to be able
to discern the difference.

I don't think any system uses 64-bit fixed today.

In theory 64-bit has the ability to represent more dynamic range with greater precision than 48-bit fixed point. In practice maybe there is some esoteric DSP that is more stable/accurate using 64-bit float vs. 48-bit fixed.

how does 64bit float compare to 64bit fixed and is the difference significant?

depends on where they fix the exponent ;-) .. i'm pretty sure ron's right .. never seen a
64bit fixed point audio system.

jeff

ORIGINAL: mewsicknerd

IMHO, it doesn't really matter. The human ear can't hear distinctions in levels
so fine. Granted, performing many mixing and effect calculations can introduce
errors into the soup. But I doubt anyone could hear the difference.

We would need a massive ear and cognitive upgrades for our hardware to be able
to discern the difference.

I do believe soundcards will improve in near future and a know for a fact there has been serious talk in the music industry about raising the quality of cd from 16bit 44.1 khz to a higher bit and samplerate.Are you going to tell me you don't hear a difference between cd and dvd's? Even now if my memory serves me correctly, there are virtual media players that accept higher quality songs then what cd's accept.

64-bit floating point (IEEE Format) uses 52 bits for mantissa, 1 bit for sign and 11 bits for exponent.

48-bit fixed point (in signed format) would be equivalent to 47-bits of mantissa and 1 sign bit.

So, it is true that 64-bit floats would give you more precision. Real question is, why would you care for 5 bits more precision, once you already have 48 bits?

Floating point is a convenience. It is really useful only if you have to deal with a variety of huge dynamic ranges so you don't have to deal with exponent manually. It's already there for every number you use.

Floating point implementations will generally waste memory and result in slower execution (on most CPU's).

The only real (but very important) advantage of floating point is ease of use. Hence, it is faster to make software that works. It is easier to debug floating point software. Companies will make money faster if they use floating point....

Fixed point is primarily used for real-time hardware implementations. Also, fixed point is used if you want to do something extremely fast (hence, it is used for most real time hardware implementations that have to fit in small memory, or that have to serve many channels in parallel at relatively low cost)

Floating point hardware would cost more and it would end up using considerably more memory and potentially multiple CPU's. Fixed point solutions will always win in terms of speed/memory/power consumption/footprint for real-time hardware solutions. However, the cost of development and later maintenance is much larger. Most of the cost in all these cases would be absorbed by the consumer. That's you!

Generally, it is much harder to master fixed point math and fixed point algorithms than the floating point.

So much about floating point vs. fixed point...

As for what is better, it depends on the skill of the designers and programmers. You have to listen and judge for yourself.

Bottom line: there is no general answer to your question. It depends. (when comparing take into account audio quality, CPU usage, memory usage)

Regards,
Bogdan

Floating point implementations will generally waste memory and result in slower execution (on most CPU's).

I am pretty sure floating point is faster than fixed point on Pentium and x64 CPUs, and uses the same amount of memory.

Why faster? To do a fixed-point multiply you must multiply and shift (2 instructions instead of 1 -- though maybe it's all a wash in the CPUs microarchitecture). Then we you add 2 fixed-point numbers you have to manage overflow more carefully, which means you either need extra code to do saturation addition, or you *must* use integer SSE instructions since they provide saturation addition.

Definitely the same memory, because a 32-bit float is the same word size as a 32-bit fixed.

ORIGINAL: attalus
but i assure you a mark of the unicorn 24bit 192khz sound better than my m-audio delta 66 24bit 96khz wich suggest sound can always be improved khz wise,

yes, but how much of that differnce in sound is attributable to better AD/DA converters?

ORIGINAL: Ron Kuper [Cakewalk]

Floating point implementations will generally waste memory and result in slower execution (on most CPU's).

I am pretty sure floating point is faster than fixed point on Pentium and x64 CPUs, and uses the same amount of memory.

Regarding memory you're wrong. Fixed point implementation (when done correctly), will not use the same number of bits for all variables. It has more flexibility. Also, people often use block floating point for arrays, etc. That saves a lot based on array sizes. If you have a simple algorithm it doesn't really matter what you do, but for more memory intensive algorithms, fixed point will always win.

Regarding execution speed it depends. It depends on the instruction set, ALU structure, existence or not of a barrel shifter and type and number of the accumulators. If you have a single cycle integer MAC operations with a barrel shifter at the output, fixed point would be either faster or the same speed as floating point. (for most algorithms) Sometimes fixed point would require more operations to implement what floating point would do in less. However, based on the number of cycles used and number of memory accesses necessary it is not always easy to guess which one would be faster.

I'm no expert on Pentium instruction set, so I do not know if what I mentioned above is available or not. However, I worked on most existing DSP's since the 1980's including many RISC CPU's. On all of them fixed point was always faster and used considerably less memory. There was no exception.

As for the types of instructions, you should definitely use "integer" instructions for fixed point math. They will be faster. As for the saturation, you shouldn't worry about it much. People who do fixed point for living, know how to deal with it efficiently. Also, most good CPU's know how to handle integer saturation properly. (all DSP CPU's do it for sure)

Finally, 1-bit shift following multiply is only needed for some types of fixed point algorithms. If the CPU does not support implied shift (happens in the same cycle as the multiply) you do not really have to use it. That is, you do not need to use always Q15, Q31 or Q47 number formats, you can use arbitrary Q-formats and keep track of where the format goes, so you adjust only the final results instead of all intermediate products or sums.......

You would be amazed to find out how much faster you can get things to execute with fixed point math. (especially if the CPU architecture enables you to do it with efficient integer operations as mentioned above)

However, I must point out (again), it would come at a price. Highly skilled fixed point DSP programmers (paid considerbly more than a high school floating point "wizz") are not easy to find. Development is harder and takes more time and thinking since the algorithms have to be converted with a lot of care. Debugging is harder and maintenance will cost more.

Regards,
Bogdan

ORIGINAL: attalus

I have no doubt converters play there role but i'm sure aswell that the 192khz extra bandwidth plays it's role aswell.I'm sure the same is with the neve 88D 40bit processing and the SSL beyond 192khz bandwith aswell, Im sure they convert better, but i'll bet my right arm that they just plain sound better overall.In the end you have a good point but i'm pretty convinced soundcards can always be improved, and at this point i believe 32bit would be a hearable difference if the converters and the rest of the card is built well!

Yes, soundcards can always be improved, but at this point, 32-bit soundcards just plain don't make sense.

The A/D converter in a soundcard essentially converts voltage in the wire into a number. The range of numbers depends on the bit depth. 24-bit audio has 2^24 or 16,777,216 different possible values. The so-called "professional" audio level operates at a nominal level of -4dBu, which equates to only a bit over one volt. This is just the "nominal voltage", though - something of an "average". The actual peaks can be much higher. Say the A/D converter can handle 5 or even 10 volts maximum.

Divide this into 16,777,216 pieces, though, and each piece ends up being extremely tiny - easily less than a microvolt. In order to really sample sound at a true 24-bit resolution, the A/D converter would need to be able to accurately detect changes of less than a microvolt.

It turns out that it takes some pretty impressive electronics to achieve this feat. Changes of a microvolt can easily be caused by a variety of sources, including magnetic interference, fluctuations in the power source, and even moment-to-moment temperature changes inside the electrical components. This means that it would be very difficult at best to get any soundcard to work at a true 24-bit resolution inside of something like a home computer.

Essentially, what happens is that noise inside of the system covers up the really fine details. Imagine feeding a steady signal into a 24-bit A/D converter such that the A/D converter should theoretically read a constant 5000. At the voltage levels of audio systems, the actual result would not be a constant 5000. The value would randomly jump around 5000, sometimes reading above, sometimes below. Oversampling can help remove some of this variation and correct for some of the noise, but not all of it. If we convert the readings to binary, we may notice that the first 20 bits of every value are always the same. However, those last 4 bits just jump around, basically randomly. This means that our soundcard is really giving us audio with an effective bit depth of 20 bits, because those last 4 bits of resolution are covered up by random noise.

As it turns out, the absolute best of the soundcards most of us on this forum use get an effective bit depth of not much more than 20 bits. Well, actually, the sound may sound a bit better than something dithered to 20 bits, because of an interesting phenomenon where the human brain can identify sounds lower than a noise threshold. (This is the ability that sometimes causes people to think they hear voices in white noise - the human brain tries to discern the sound beneath the noise floor, even if there's no sound there.) Essentially, the brain can make out some of the gist of the sounds hidden in those lower 4 bits, even though they are mostly masked by noise. But for all practical purposes, the effective dynamic range ends up being roughly 20 bits (or approx. 120dB).

The Neve console you mention is a different beast. It is completely custom-designed with high-end audio in mind. So, it can have all kinds of custom features designed to minimize noise in the A/D converters, such as special power supply, special housing, and of course, a high-end converter. But even that Neve console is only 96/24. It just may have an effective bit depth of something closer to 24 than the soundcards most of us work with. It may even have an effective bit depth of pretty close to 24. But obviously, you need to pay for that.

But getting back to computer soundcards... Since the best of them are still rather far from actually using all 24-bits, it would be an utter waste to build one that was 32-bit, at least right now. Especially since A/D converters tend to get geometrically more difficult to make as resolution goes up. In other words, it is not 50% harder/more expensive to build a theoretical 32-bit soundcard, but many, many times more expensive. And if the effective bit depth stays at 20-bits because of electronic noise, it is even more of a waste. The perceived performance of the 32-bit soundcard would be identical to that of a 24-bit soundcard, and both would be getting almost the same results as a 20-bit soundcard.

Since one of the main problems with increasing bit depth is that the voltage changes are too small to be measured over the electronic noise, it seems one option might be to raise the voltage level in the analog lines. But this means that a new analog audio standard would have to be developed, like a +20dBV standard or something. But this would have a ripple effect throughout all audio electronics - every piece of analog equipment would have to be built to handle higher voltages, which means better (i.e. more expensive) power supplies and other components, which means EVERYTHING will cost more...

And then there's the fact that sound quality depends on more than just bit depth. The quality of the low-pass filters, the oversampling algorithm, and the stability of the timing crystal are also of critical importance. It would do no good to create a 32-bit soundcard if jitter in timing kept the effective bit depth at 20. And then there's the D/A converters, which have basically all the same problems in reverse. Getting a D/A converter to run at higher than 20-bits is just as hard as getting the A/D converter to do it.

Then there's also the question as to whether or not 32-bits is really worth it at all. 16-bit audio can only encode a dynamic range of 96dB. The human ear can discern a range of 120dB. This means that the range of human hearing is roughly 16 TIMES greater than the dynamic range of a 16-bit CD (24dB greater dynamic range = 16 times the dynamic range). It's no wonder that the sound of 16-bit audio is noticeably lacking, compared to real life. The dynamic range of 24-bit audio, however, is 144dB. If you recorded both the absolute quietest sound you could and the absolute loudest sound you could using 24-bit audio, then played it back on a system with true 24-bit D/A converters and speakers to match so that the absolute quietest sound registered 0dBspl (which, by definition, is the absolute quietest sound that the "average" human ear can hear), then the loudest sound would be way above the average human's pain threshold, a good twice as loud as a jet engine. A more practical choice would be to play the absolute loudest stuff at a more reasonable volume, which means any noise or distortion in our recorded tracks would be WAY, WAY, WAY below the threshold of human hearing. And this is just with true 24-bit audio. If the error with 24-bit recording is so far below human hearing that there's no possible way it could be discerned, what's the point of recording at 32-bit?

(Keep in mind that the record bit depth should not be confused with the bit depth for processing. We always want our intermediate results to be at a higher bit-depth than either our starting point or our desired ending results. That way the accumulated error stays at a minimum. If we round or truncate to 16-bit or even 24-bit after every mathematical operation, the distortion becomes apparent rather quickly during processing. So processing our audio with 32, 40, 48, or even 64 bits of precision is desirable, even if we only record at 24-bit. Recording at 32-bit would imply that we would probably want at least 48-bit or 64-bit processing.)

At some point the electronics might improve to the point that common soundcards can actually operate at true 24-bit, even if the nominal voltage levels of the analog signals don't change. Or, the audio industry might even adopt a new standard (undoubtedly not anytime soon), and increase the analog voltages. But until then, recording at 32-bit doesn't make any sense at all, if it ever will make sense.

Compression reduces the dynamic range and can make stuff audible what wasn't audible before. Above mentioned stories are known and found to be true regarding "RECORDING". But are they true when mixing and mastering is concerned.

A mastering engineer does not wanna alter what is considdered 'GOOD'. So an application wich does not damage original material is important to the mastering engineer.

In the mixing proces reverbs are often placed before LIMITING. So the resolution of the reverb is important. As the resolution when limiting is reduced (again... perhaps) details may show up of artefacts of damaging by compression, effects or clipping at the recordingstage.

TC electronics M6000 and the Lexicon L48 systems have resolutions of 48bit (double precision, don't know if this is floating or fixed (as I am not a programmer I am unaware of such technical details). It would be great to get such quality within a DAW. With the coming of 4 cores in systems (on one socket) systems will get powerfull enough to do 24bit/96khz. And when used on Vista an internal resolution of 64bit/96khz. This will have impact on VST instruments such as convolution reverbs and their level of detail.

If I am wrong here I wouldn't mind to be set straight.

Thanks for such a great insight on this subject.

There is another thing why I am in favor of such stuff. I was told that people can hear up to 22khz. But are sensitive to phasechanges up to 200khz.

Ambisonics and surround in general will be more powerfull if such high resolutions are possible I think. 96khz/64bit will not damage sound up to 40Khz. So alot more phasechanges can be heard in the high end (if tweeters are RIBBON (wich can give up to 40Khz (measured, but probably higher) Some dometweeters (look at Tannoy ellipse for example) can go that high, but are probably less...

I am interested if above can be shot down...... Amaze me.


Muziekschuur

Fixed point implementation (when done correctly), will not use the same number of bits for all variables.

I can see how that makes sense for coefficients or internal state for DSP code. But how much memory are we talking about here? It seems to me that the only memory overhead of any significance in real DSP for things like delay lines and other long buffers. For these you need the buffer to be the same word length as the sample format.

I'm no expert on Pentium instruction set, so I do not know if what I mentioned above is available or not.

Pentium does not have a single cycle MAC instruction. However if your code is fully pipelined you can get 1 cycle throughput on floating point multiplies and adds. It's generally accepted that for audio work, floating point is more efficient on Pentium.

You would be amazed to find out how much faster you can get things to execute with fixed point math. (especially if the CPU architecture enables you to do it with efficient integer operations as mentioned above)

That's a big "if". Since SONAR only runs on Pentium class processors, for our application floating point wins.

The only real (but very important) advantage of floating point is ease of use.

A very real advantage of floating point that hasn't been mentioned is that the end user doesn't really have to concern him/herself with clipping within the mix engine. Headroom beyond 0db is enormous. With a fixed-point mix engine (Protools) clipping can occur at every stage. Much more care must be taken by the end user in this regard.

-S

Yes, it would be nice if there was a list with suppported 64bit VST plugins (so we know the internal signalflow isn't 'damaged' if we like that signalpath.......

May be a lil off topic here....

Muziekschuur