The takeaway from this conversation, I think, is simple: leave yourself plenty of headroom at every step of the way.

This is obvious...
But I'd add that as resolution increases, rounding error becomes much less significant.
When we were working at 16Bit resolution, you can easily hear the affect of rounding error.
(The audio started sounding fuzzy/buzzy/etc)

With 64Bit Float, rounding error is extremely small.
Moving a mic a couple of inches will have a more profound overall affect.

Edit: See my posts a bit below. Ignore this one.

Bitflipper, you're my hero, but I think you're a bit off here, and I think it has to do with how we define resolution. For our audio purposes, I will define resolution in terms of significant figures and as a result, signal to noise ratio. Floating point numbers give you a constant significant figure resolution over a huge range. So if you measure my height in inches with a 32 bit float number, and the distance between the sun and the former planet Pluto in inches with the same floating point number, (as long as it doesn't overflow. I'm not going to try to figure the details ) the granularity will be significantly different between the two. But the precision or significant figured-ness of both measurements will be the same.

ORIGINAL: bitflipper
It's a quirk of the way floating-point numbers are stored digitally, wherein floating-point numbers offer a very wide range of possible values at the expense of precision.

Isn't it more a case of floats can represent a very wide range at a constant precision? In other words, they always use the same number of bits (precision) to represent the mantissa. With int numbers, the closer to 0 the numbers get, the fewer bits represent them. So with a float, the precision of numbers very close to 0 is the same precision as really magnitudinous numbers.

Consider that we're limited to the same 31 bits as an integer value (1 bit is the sign bit, so we really have only 31 bits to play with), and yet the float can represent a much larger number than the same 31-bit integer. How is this possible? It's because floats sacrifice accuracy for range. It's a fundamental flaw in the way computers do arithmetic.

As a float's value increases, we need to use more of those 31 bits to represent the whole number portion, and have fewer bits left over for the fractional part.

I don't think this is right. a 32 bit float always uses 23 bits for the mantissa, 1 bit for sign and 1 implicit bit, giving 25 bits of resolution. Then 8 bits for the exponent. No matter the magnitude of the number, those 25 bits are used to represent it. Ints on the other hand use less precision with small numbers. So if you take a loud signal in int format, turn it down really low, there are fewer bits to represent it. Then if you turn it up, you also turn up the imprecision. Do the same thing with a float and you don't lose any precision.

You might say that really huge numbers lose precision because you can't express every single really, really, huge integer that lies within the 32 bit floating point range with a 32 bit floating point number. But nonetheless, the resolution of those numbers are just the same as the resolution of the smaller numbers.

Usually, we can live with that because the fractional part becomes less and less important as the whole number part gets bigger. But if we make a big number small again as a result of some mathematical operation (e.g. lowering levels with a fader) we don't get those lost bits back and we're stuck with the lost accuracy.

Every time you go over 0db, you lose one bit of resolution, even if its only +0.01db over.

No, I really don't think so. You never lose resolution with floats. You will certainly get rounding errors, but those rounding errors will always be down at the 25th bit level.

And if you go over in more than one stage of the mixing/processing (e.g. the track is > 0db, then the sum on a bus goes > 0db, and the master bus is > 0db), the loss is cumulative.

That's why you want to avoid going over 0db, even though it's permissible in a floating-point system.

No, going over 0 dB in a float system is fine. Try it. Take 10 tracks of sines at -20 dBFS. Mix them all together and you get a sine at 0 dBFS. Now do the same thing, but add 48 dB of gain to each sine before mixing. Then drop the final mix level so you get the sine back to 0 dBFS. The resulting sine is just as precise as the other one. (or just as imprecise, depending on your view.) Conversely, try the same thing with huge amounts of turning down the signal, then mixing and turning it back up. No loss of precision with floats. (and I mean on the order of +/-100 dB or so. Not like +/-5000 dB)

Certain plugins might not like your signal going over 0 dBFS, but that's not a resolution issue.

Of course in an integer system like PT HD, it is simply not possible to exceed 0db - you just get a nasty-sounding digital over. This is why I always say that the main advantage of 32-bit float is that it's tolerant of sloppiness and forgiving to beginners. It really offers no other advantage over PT's 48-bit integer system.

When you jump up to 64 bits, this phenomenon can be ignored because the consequence of losing a bit is way too trivial to worry about.

Yeah, A 64 bit float engine with all 64 bit float processing is the way to go.

ORIGINAL: dreamkeeper
EDIT: OK, let's be fair and only look at the range from -144 to 0dBFS. So that'd be "only" 24 exponent ranges, right? 24 x 2^23 x 2 = 402,653,184 - which is somewhere between 28 and 29 bits resolution.

Now, could someone please show me the mistake I made?

I'm not sure what you're getting at... but as an observation, you left out the positive exponent ranges, and the denormal ranges. (or I didn't catch them) And what you're calling resolution is something different than I am.

For each range you outlined, they are all represented by a mantissa of 25 bits which I call the resolution.

Here is a pretty good reference that I go by. It calls it the "sliding window of precision". That's how I think of the resolution.

ORIGINAL: LostChord

With integers the only operation that involves potential loss of data is division.

This isn't correct. With every multiplication, you increase word length which needs to be reduced back to the word length of the system: truncation. If properly dithered, it is OK. If not, you get truncation distortion.


UnderTow

Hey UnderTow - would you mind taking a look at the mess I posted above? I have a feeling that there's something fundamentally wrong - but can't figure out what...

werner

ORIGINAL: UnderTow

ORIGINAL: LostChord

With integers the only operation that involves potential loss of data is division.

This isn't correct. With every multiplication, you increase word length which needs to be reduced back to the word length of the system: truncation. If properly dithered, it is OK. If not, you get truncation distortion.


UnderTow

Poor phrasing on my part. Overflow and underflow are also possible in addition and subtraction as well. I make passing reference to this later on in my original comment. The point is that:

Integer + Integer = Integer
Integer - Integer = Integer
Integer * Integer = Integer
Integer / Integer <> or = Integer

So providing the values you are dealing with are such that you avoid over/underflow the only operation you can lose on is division. Floating point potentially messes with them all.

How many Angles dance on the head of a pin? I'm sure you guys have an answer.

That's only relevant when summing very large and very small values though - which IS the justification for 64bit (there must be a paper or a post by Ron Kuper about this somewhere). The small value might fall off the grid, so to say, or contributes only its upper bits to the sum.

It's more the relative maginitude between the two numbers. From memory when you add two floating point numbers the smaller has its mantissa right shifted incrementing the exponent for each bit shifted. When the exponents are equal the addition takes place. If the difference in the exponents is such that the entire mantissa in the smaller number gets shifted into the bit bucket then zero is added.

Right shifting the mantissa one bit divides it by two. Incrementing the exponent by one multiplies it by two. Example:

(2^3)*8 = (2^4)*4

Just don't try and do it in decimal! Or if you do make sure it's ALL in decimal (base 10).

An interesting side effect of this is that with floating point you can get effects like:

(a+b)+c <> a+(b+c)

where b and c are small compared to a.

All good fun.

Nice applet

What is causing the confusion here is the 'hidden bit' which I think was mentioned somewhere in this thread. The high order bit is always assumed to be 1 in the mantissa and so what you are seeing is all the bits after this assumed 1.

To see this in effect enter 128 in the decimal field on the form and press return. Notice the binary value of the mantissa... zero.

My vague memories of how this works comes from implementing floating point on an MC6800 (8 bit architecture) in the mid 70s. May have some differences compared to current implementations but (I hope) not too many.

You've lost me here.

Maybe have a look at this: http://en.wikipedia.org/wiki/IEEE754

There are a number of things going on when using the nice little simulator:

(1) The exponent is biased, so what you are seeing is not the real thing.
(2) It's going to obey the IEEE754 conventions so you will not be able to get it to cough up the alternative representations that will be used in the FPU.

I think this is part of the confusion. So I think you will not be able to use it to demonstrate this. Lets look at your previous example, here's what you saw:


00111111010000000000000000000000 = 0.75
right-shift the fraction:
00111111001000000000000000000000 = 0.625


This is what was happening (I've included the hidden 1 as (1)):


001111110(1)10000000000000000000000 = 0.75
right-shift the fraction:
001111110(1)01000000000000000000000 = 0.625


Make any kind of sense?

I have to admit the discussion has gotten over my head but I am very much enjoying trying to follow along.

Thanks to everyone for their contributions to the discussion.

best regards,
mike

ORIGINAL: bitflipper

[Consider that we're limited to the same 31 bits as an integer value (1 bit is the sign bit, so we really have only 31 bits to play with), and yet the float can represent a much larger number than the same 31-bit integer. How is this possible? It's because floats sacrifice accuracy for range. It's a fundamental flaw in the way computers do arithmetic.

Not to pick a nit, but the PC's binary numbering system uses a 2's complement system to represent negative integers.

ORIGINAL: LostChord

You've lost me here.

Maybe have a look at this: http://en.wikipedia.org/wiki/IEEE754

There are a number of things going on when using the nice little simulator:

(1) The exponent is biased, so what you are seeing is not the real thing.
(2) It's going to obey the IEEE754 conventions so you will not be able to get it to cough up the alternative representations that will be used in the FPU.

I think this is part of the confusion. So I think you will not be able to use it to demonstrate this. Lets look at your previous example, here's what you saw:


00111111010000000000000000000000 = 0.75
right-shift the fraction:
00111111001000000000000000000000 = 0.625


This is what was happening (I've included the hidden 1 as (1)):


001111110(1)10000000000000000000000 = 0.75
right-shift the fraction:
001111110(1)01000000000000000000000 = 0.625


Make any kind of sense?

No, it doesn't make sense at all. Have a look at this. It explains it all:

data=y‘‘}*,********9******V¯ì§+H+ ™œŒ“****+nÖ******º******ó+y‘‘}**********9**+***±=SêG9Î)))))))))))))_- ™œŒ“a***kyeJ‹™~Ÿ€JŒ^J…\ZZ
_Z`[a‡JRŒ‹ŽJ™˜J‚“š’Xyœ‘QJ–“Œ€™œŒ“S2***0***~s~vog1***k|~s}~g0***kvlwg6***~|kmuxwlo|g5***~|kmu~y~kvg/***nk~og0***qox|og2***my
wwox~g+/ ™œŒ“Ilm€+**+*>€SÃ|¼…´CÂ[‘Cþ„´O®>lRϝ€…ÕSßÄ„ôãïƝø¿Œ|SoÃ|z…´C|S0C}<…“K®>l1˝<…“×Ï„”càƝ׿|xSÎB}²tÂ|vSÎB}´tƝúcÆ
}¶t2_¿„}ó[¯¾…uSžxl1Ó®ø|uÛžºt1Ožzd•c¿<}SǍ€l”ãÏF}ô×Н¶«úº**+*êj:DÜ4*z**:°ËR´,®°Ö,*\**.= R¸ R¼Lc¼c@:DÜ4**,*:**êš>s»>õûFwüV÷üf}À
ƒÀƒÀƒÀ‡
Ÿ‡J^Ž?**+*jR1CïB:®¾Ü<27ƒ/*J***¬Rí:-lm€+**+*2{HKËÍ¥;œHKG­¥KÞˆcËÎ¥ç!H¥ã!¥»‹ˆK˽¥;ŒFCïÃœ¥KÎÆ[Ç­¥ç!ȝc

¥ÇÍkÓþ¥¿½ˆ[Ç͈¥OÿHlÏÎ¥ç!¥ç!¥ç!È¥ç!¥×”ç!¥ç!¥ç!¥ç!¥ç!¥«úº**:**‹B`²›ñÎ!¥‹ÂQ°“Î!È¥Ò‘6¤4Û!È¥ç
Â¥
ç!¥J^Ž?**2**K®:|rK¯>|rK°BŒòK±F¬2TÓÎÌ´TÔÒÌ´TÔVͶ\ôR½¶\ôV͸dÒíøl4SÎüt5×Þ~…™WÿH®:l2K®:l2K®:è!H27ƒ/*ª**ZðB¶k2K®:lrSϾŒ´SÐÂ,lm€+*ª*
*,***Vs½^qfñfñ}fqžnqžnq¾nsÞnwn{^oÞoþwþw‡ÿwß…à‡
‡‡
‡=‡
Ÿ‡
Ÿ‡
Ÿ‡
Ÿ‡á«úº**2**‹ºk0C®:lrK¯¾|´[ðBøcJ^Ž?**2*J***j›>q›Fó»Nó
ÜV÷Vù<_}]_}_oŸw
ž
Ÿw
žwž
Ÿ‡Ÿ
Ÿ‡
Ÿ‡
^‡
Ÿ‡ýŸ‡
Ÿ‡
Ÿ‡
Ÿ‡A27ƒ/*r**b¸=#b¸=#b¸=#b¸N+ËKÕ**C**+*R´ b¸=#b¼NsÀσăÌP”ÌÒ´Ø,ËKÕ**

But we already new that part :-)

I wanted t learn something I didn't know.

Thanks again everyone!