java integer rounding (division related)

29,447

I'm expecting integer math to round all fractions down.

First of all, you're not doing integer division: You're doing floating point division. (float)-8 is a floating point expression. Therefore, the s in (float)-8/s gets promoted to (float) before the division happens.

Then the floating point result is converted to an int. Reuseman said that integer division rounds toward zero. Well, float->int conversion also rounds toward zero.

Share:
29,447
Keybounce
Author by

Keybounce

Updated on July 09, 2022

Comments

  • Keybounce
    Keybounce almost 2 years

    I have run into a surprise with integer division not rounding down as expected.

    Simple code:

    public class HelloMath {
    
        public static void main(String[] args) {
            for (int s=1; s< 20; s++)
            {
                int div = 1<<s;
                int res = (int) ((float)-8/ s);
                System.out.printf("Bit %d, result %d\n", s, res);
            }
        }
    
    }
    

    Even with the explicit (float) casts, the output is:

    Bit 1, result -8
    Bit 2, result -4
    Bit 3, result -2
    Bit 4, result -2
    Bit 5, result -1
    Bit 6, result -1
    Bit 7, result -1
    Bit 8, result -1
    Bit 9, result 0
    Bit 10, result 0
    Bit 11, result 0
    Bit 12, result 0
    Bit 13, result 0
    Bit 14, result 0
    Bit 15, result 0
    Bit 16, result 0
    Bit 17, result 0
    Bit 18, result 0
    Bit 19, result 0
    

    I was expecting -1 all the way down.

    The real code where this is happening does this:

    public static int fluidTo8th(int fluid)
    {
        if (0 == fluid)
            return 0;   // Technically, this isn't needed :-).
        int wholePart = (fluid-1) * 8 / RealisticFluids.MAX_FLUID; // -1 gets rounding correct;
        // consider fluid of exactly 1/8th.
        return 1+wholePart;
    }
    

    RealisticFluids.MAX_FLUID has the value (1<<20). The code is supposed to take an input that is 1 to MAX_FLUID (0 should only happen if the input block is air), and return a number from 0 to 7 -- 1 to 1/8th max is 0, 1/8th +1 to 2/8th is 2, up to 7/8th + 1 to 8/8th is 7.

    I'm expecting integer math to round all fractions down. But that's not happening -- I wind up with off-by-one errors all over the place as 5.999 winds up becoming 6 instead of 5.

    1. Where is this behavior documented?
    2. What's the easiest work-around to get the rounding down that I expected?

    (full code context: https://github.com/keybounce/Finite-Fluids/blob/master/src/main/java/com/mcfht/realisticfluids/Util.java)

  • Andreas
    Andreas almost 8 years
    Java specification backing your statement: JLS §5.1.3 ... the floating-point value is rounded to an integer value V, rounding toward zero using IEEE 754 round-toward-zero mode (§4.2.3).