Error in astype float32 vs float64 for integer
Solution 1
A 32-bit float can exactly represent about 7 decimal digits of mantissa. Your number requires more, and therefore cannot be represented exactly.
The mechanics of what happens are as follows:
A 32-bit float has a 24-bit mantissa. Your number requires 27 bits to be represented exactly, so the last three bits are getting truncated (set to zero). The three lowest bits of your number are 011
2; these are getting set to 000
2. Observe that 011
2 is 3
10.
Solution 2
A float32
only has 24 bits of significand precision, which is roughly seven digits (log10(2**24) = 7.22). You're expecting it to store an 8-digit number exactly, which in general is impossible.
jsexauer
Updated on June 23, 2022Comments
-
jsexauer almost 2 years
I'm sure this is due to a lapse in my understanding in how casting between different precision of float works, but can someone explain why the value is getting cast as 3 less than its true value in 32 vs 64 bit representation?
>>> a = np.array([83734315]) >>> a.astype('f') array([ 83734312.], dtype=float32) >>> a.astype('float64') array([ 83734315.])