Floating point precision in Python array

16,900

Because they are different numbers and different numbers have different rounding effects.

(Practically any of the Related questions down the right-hand side will explain the cause of the rounding effects themselves.)


Okay, more serious answer. It appears that numpy performs some transformation or calculation on the numbers in an array:

>>> t = numpy.array([0.22])
>>> t[0]
0.22


>>> t = numpy.array([0.24])
>>> t[0]
0.23999999999999999

whereas Python doesn't automatically do this:

>>> t = 0.22
>>> t
0.22

>>> t = 0.24
>>> t
0.24

The rounding error is less than numpy's "eps" value for float, which implies that it should be treated as equal (and in fact, it is):

>>> abs(numpy.array([0.24])[0] - 0.24) < numpy.finfo(float).eps
True

>>> numpy.array([0.24])[0] == 0.24
True

But the reason that Python displays it as '0.24' and numpy doesn't is because Python's default float.__repr__ method uses lower precision (which, IIRC, was a pretty recent change):

>>> str(numpy.array([0.24])[0])
0.24

>>> '%0.17f' % 0.24
'0.23999999999999999'
Share:
16,900
Admin
Author by

Admin

Updated on June 04, 2022

Comments

  • Admin
    Admin almost 2 years

    I apologize for the really simple and dumb question; however, why is there a difference in precision displayed for these two cases?

    1)

    >> test = numpy.array([0.22])
    >> test2 = test[0] * 2
    >> test2
    0.44
    

    2)

    >> test = numpy.array([0.24])
    >> test2 = test[0] * 2
    >> test2
    0.47999999999999998
    

    I'm using python2.6.6 on 64-bit linux. Thank you in advance for your help.

    This also hold seems to hold for a list in python

    >>> t = [0.22]
    >>> t
    [0.22]
    
    >>> t = [0.24]
    >>> t
    [0.23999999999999999]