floating accuracy - ruby: converting from float to integer in ruby produces strange results

Question

ree-1.8.7-2010.02 :003 > (10015.8*100.0).to_i
 => 1001579 
ree-1.8.7-2010.02 :004 > 10015.8*100.0
 => 1001580.0 
ree-1.8.7-2010.02 :005 > 1001580.0.to_i
 => 1001580 

ruby 1.8.7 produces the same. Does anybody knows how to eradicate this heresy? =)

Answer 1

Actually, all of this make sense.

Because 0.8 cannot be represented exactly by any series of 1 / 2 ** x for various x, it must be represented approximately, and it happens that this is slightly less than 10015.8.

So, when you just print it, it is rounded reasonably.

When you convert it to an integer without adding 0.5, it truncates .79999999... to .7

When you type in 10001580.0, well, that has an exact representation in all formats, including float and double. So you don't see the truncation of a value ever so slightly less than the next integral step.

Floating point is not inaccurate, it just has limitations on what can be represented. Yes, FP is perfectly accurate but cannot necessarily represent every number we can easily type in using base 10. (Update/clarification: well, ironically, it can represent exactly every integer, because every integer has a 2 ** x composition, but "every fraction" is another story. Only certain decimal fractions can be exactly composed using a 1/2**x series.)

In fact, JavaScript implementations use floating point storage and arithmetic for all numeric values. This is because FP hardware produces exact results for integers, so this got the JS guys 52-bit math using existing hardware on (at the time) almost-entirely 32-bit machines.