Ruby float precision

Question

As I understand it, Ruby (1.9.2) floats have a precision of 15 decimal digits. Therefore, I would expect rounding float x to 15 decimal places would equal x. For this calculation this isn't the case.

x = (0.33 * 10)
x == x.round(15) # => false

Incidentally, rounding to 16 places returns true.

Can you please explain this to me?

Solution 1

Part of the problem is that 0.33 does not have an exact representation in the underlying format, because it cannot be expressed by a series of 1 / 2ⁿ terms. So, when it is multiplied by 10 a number slightly different than 0.33 is being multiplied.

For that matter, 3.3 does not have an exact representation either.

Part One

When numbers don't have an exact base-10 representation, there will be a remainder when converting the least significant digit for which there was information in the mantissa. This remainder will propagate to the right, possibly forever, but it's largely meaningless. The apparent randomness of this error is due to the same reason that explains the apparently-inconsistent rounding you and Matchu noticed. That's in part two.

Part Two

And this information (the right-most bits) is not aligned neatly with the information conveyed by a single decimal digit, so the decimal digit will typically be somewhat smaller than its value would have been if the original precision had been greater.

This is why a conversion might round to 1 at 15 digits and 0.x at 16 digits: because a longer conversion has no value for the bits to the right of the end of the mantissa.

Solution 2

Well, though I'm not certain on the details of how Ruby handles floats internally, I do know why this particular bit of code is failing on my box:

 > x = (0.33 * 10)
=> 3.3000000000000003
 > x.round(15)
=> 3.300000000000001

The first float keeps 16 decimal places for a total of 17 digits, for whatever reason. So, rounding to 15 discards those digits.

Comments

[Pascal Cuoq]

In all likelihood, your language uses IEEE 754 double-precision floating-point numbers internally. The "15 decimal digits" is only an approximation, since floating-point numbers are actually represented in base 2, not 10.

[oligan]

A question about floating accuracy that I didn't want to close-hammer! +1.000000000001!

[Reactormonk]

I wonder why you can == floating numbers, it's a trap.

[Admin]

Why does 3.3000000000000003.round(15) == 3.300000000000001 and not 3.3? Is this just down to binary representation and floating point rounding errors?

[Rudy Velthuis]

@georgehemmings: Yes indeed, it is down to the binary representation of 0.1 (or multiples of it) not being a finite series of 1 / 2^n terms.

[Admin]

Thanks for your help so far. I get the general idea, but I'm still trying to prove it to myself. If Ruby only keeps approx 15 significant digits why does sprintf("%.50f",1.1) print non-zeros after the 15th decimal place? Where are these values coming from?

[Matchu]

+1 Yay, someone knows the details! The fact that Ruby is willing to show other decimal places when coerced into it seems to indicate that it rounds some in to_s and, in fact, keeps other info beyond what it shows by default.

[DigitalRoss]

George ... the way conversion works is that the value is divided and then the remainder is converted for digits further to the right. So unless this coincidentally goes to zero, the conversion can produce non-zero digits for quite some time after running out of mantissa bits. It can go on forever for repeating quotients. I suppose someone might come up with an algorithm to taper off to zero deliberately, but note that so-converted numbers will be further from the "real" number than the ones with the trailing remainder bits. It's best just not to convert extra digits in the first place.

[DigitalRoss]

I should also add that this problem also occurs for irrational numbers and for numbers that really do have more precision than the mantissa can specify.

[Admin]

Thanks, I get it now. I decided to see what C# does; it refuses to show more than 17 decimal places, even when asked for 50. For 1.1 it only shows 16 decimal places. (1.1D).ToString("G50").Length == 18