Why does integer division code give the wrong answer?

60,955

Solution 1

You're dividing integers, which means that you're using integer division.

In integer division the fractional part of the result is thrown away.

Try the following:

float res = (float) quantity / standard;
            ^^^^^^^

The above forces the numerator to be treated as a float which in turn promotes the denominator to float as well, and a float-division is performed instead of an int-division.

Note that if you're dealing with literals, you can change

float f = 6800 / 500;

to include the f suffix to make the denominator a float:

float f = 6800f / 500;
              ^

Solution 2

If you concerned about precision I would suggest using double which has more than double the number of digits of precision. However floating point only accurately represents fractions which are a sum or powers of 0.5. This means 0.6 is only approximately represented. This doesn't have to be a problem with appropriate rounding.

double d = (double) 6800 / 500;

or

double d = 6800.0 / 500;

Solution 3

In my case I was doing this:

double a = (double) (MAX_BANDWIDTH_SHARED_MB/(qCount+1));

Instead of the "correct" :

double a = (double)MAX_BANDWIDTH_SHARED_MB/(qCount+1);
Share:
60,955
Julian C.
Author by

Julian C.

Updated on January 13, 2020

Comments

  • Julian C.
    Julian C. over 4 years

    I have a very simple division in Java (it's a product quantity / production per hour), however whenever I make this division I get strange errors:

    float res = quantity / standard;
    

    I have tried the above division with several values and I always get errors, however the one that I've tried everywhere else and gotten right was this:

    Everywhere in the world:

    13.6 = 6800 / 500;
    

    Java:

    13.0 = 6800 / 500;
    

    I've researched BigDecimal and BigInteger, however I haven't found a way to create this division with them, is there any other way to do this division in Java without having precision errors??

    Any help will be greatly appreciated.