Double minus int giving unexpected results

15,683

Solution 1

Subtracting an int from a double does not change only the integer part of the double. It can also change the scale. In your particular case, the integer part before the subtraction (6) requires more bits to represent than the integer part after subtraction (1). This causes the entire representation of the number to allow for more fractional bits. Results can be (as you found out) a bit unexpected. (Pun intended)

Solution 2

If you want a hardcore explanation, read the classic What Every Computer Scientist Should Know About Floating-Point Arithmetic. It explains why there are inevitably tiny rounding errors in floating point arithmetic like the one you're seeing.

If you just want a summary: Computers represent everything in binary. Binary representations while doing floating point arithmetic results in tiny inaccuracies in some situations.

Share:
15,683
user1086516
Author by

user1086516

Updated on August 12, 2022

Comments

  • user1086516
    user1086516 over 1 year

    Possible Duplicate:
    Floating point arithmetic not producing exact results in Java
    Floating point inaccuracy examples

    In Java, given the following code:

        double amount = scan.nextDouble();
    
        tenBills = (int)amount / (int)10;
        amount = amount - (tenBills * 10);
    
        fiveBills = (int)amount / (int)5;
        amount = amount - (fiveBills * 5);
    

    After the first calculation, given an input of say 16 amount will equal 6.66 . But after the second calculation amount will be 1.6600000000000001 . I don't understand why subtracting an int from a double would cause such a result.