Surprising Imprecision in Simple Math

Shocked was I to find javascript’s conclusion that

52.48 – 40 = 12.479999999999997

So, actually it makes sense for any floating-point (base 10) type math system. Javascript gets the most complaints, but it must affect other languages as well.

Think of it like this: If all you had was a tool for cutting apple sections in half, you could never give someone exactly one tenth of an apple!

You could give them exactly half or exactly a quarter and so on, but not a third or a fifth, etc. Well most computers think in ones and zeroes, so they’re limited to halfsies.

So, for those of us who need to get work done rather than ponder such things, here are two solutions:

The Quick (brown) Fix (jumped over the fence):

Simply round everything to proper decimal places before outputting. So, for money amounts:

outputtableValue = exactValue.toFixed(2);

Note that this changes the number to a string, so you would need to use parseFloat to convert back to a number. (But don’t do that, silly! Just keep using exactValue until you need to output again.)

The Slow Way (that will make you feel all warm inside):

Explanation

The Code

 

Leave a Reply