Why does integer division code give the wrong answer?
Solution 1
You're dividing integers, which means that you're using integer division.
In integer division the fractional part of the result is thrown away.
Try the following:
float res = (float) quantity / standard;
^^^^^^^
The above forces the numerator to be treated as a float
which in turn promotes the denominator to float as well, and a float-division is performed instead of an int-division.
Note that if you're dealing with literals, you can change
float f = 6800 / 500;
to include the f
suffix to make the denominator a float:
float f = 6800f / 500;
^
Solution 2
If you concerned about precision I would suggest using double
which has more than double the number of digits of precision. However floating point only accurately represents fractions which are a sum or powers of 0.5. This means 0.6 is only approximately represented. This doesn't have to be a problem with appropriate rounding.
double d = (double) 6800 / 500;
or
double d = 6800.0 / 500;
Solution 3
In my case I was doing this:
double a = (double) (MAX_BANDWIDTH_SHARED_MB/(qCount+1));
Instead of the "correct" :
double a = (double)MAX_BANDWIDTH_SHARED_MB/(qCount+1);
Julian C.
Updated on January 13, 2020Comments
-
Julian C. over 4 years
I have a very simple division in Java (it's a product quantity / production per hour), however whenever I make this division I get strange errors:
float res = quantity / standard;
I have tried the above division with several values and I always get errors, however the one that I've tried everywhere else and gotten right was this:
Everywhere in the world:
13.6 = 6800 / 500;
Java:
13.0 = 6800 / 500;
I've researched BigDecimal and BigInteger, however I haven't found a way to create this division with them, is there any other way to do this division in Java without having precision errors??
Any help will be greatly appreciated.