compare two floats for equality in Python

Question

When comparing whether two float in Python, I see code always like this to compare for a small value epsilon, wondering what is the best practices to select the right epsilon value? And what is the scene behind it? Thanks.

epsilon = 0.000001
abs(a - b)<epsilon

Solution 1

The answer is quite complex since you need to know how single or double precision floats are saved (Wikipedia), as a rule of thumb you can use this Table on Wikipedia as reference for choosing epsilon. But there might be some exceptions specially if you don't exactly know if it is float32 or float64 (or for Linux/Mac there are also float96 and float128 around).

But I guess best practise would be to use some predefined function like numpy_assert_array_almost_equal (numpy required).

I guess everyone is handling it somehow different and as long as you can trust your results every method has its pros and cons. And always keep in mind that floats can go totally haywire with the wrong kind of arithmetic operations. i.e. where small differences of big values are being calculated. And in the end the value of epsilon depends on which precision you need and that should be tested there.

Solution 2

There is an assert function in numpy for this purpose, which uses seven decimal precision by default.

from numpy.testing import assert_almost_equal

a = 0.000000001
b = 0.0000000001

>>> assert_almost_equal(a, b)
# Nothing returned.

b = 1
>>> assert_almost_equal(a, b)
AssertionError: 
Arrays are not almost equal to 7 decimals
 ACTUAL: 1e-09
 DESIRED: 1

Solution 3

if you are looking for the best epsilon ever, to get best comparison you could use python's sys epsilon using:

>>> import sys
>>> sys.float_info.epsilon
2.220446049250313e-16

but if you are more looking to have this epsilon dynamically based on your a and b I would suggest go for:

abs(f1-f2) < tol*max(abs(f1),abs(f2))

or

abs(a-b) <= max( rel_tol * max(abs(a), abs(b)), abs_tol )

Solution 4

what is the best practices to select the right epsilon value?

It depends on the requirements of the application.

If it is planning a Earth-bound trajectory for reentry of a spacecraft I am in, I would choose a very small value, like epsilon = (a+b) * 1e-15.

If it is projecting the U.S. federal deficit (which inherently has great uncertainties), a much larger epsilon is likely suitable: epsilon = (a+b) * 0.002.

View more solutions

Comments

[Lin Ma]

Thanks MSeifert, I read through this document (en.wikipedia.org/wiki/…) and really very informative. I am confused by one thing, in the table, there are two columns called "Machine epsilon", wondering what are their differences?

[Lin Ma]

Hi Alexander, very neat, but how to choose the value of decimal in your practices?

[Lin Ma]

Thanks user 12321, a bit lost, what is the benefit of dynamically using epsilon? An example is appreciated.

[Lin Ma]

Thanks wallyk for the samples, and I agree with you different precision is for different cases. My question is more about from Python internal representation, what is the precision boundary it could be. As your example showed, in some cases, you can use 1e-15, but how do I know if Python internally could have such precision for floats or doubles? Your insights are appreciated. Thanks.

[Alexander]

It is very subjective and entirely depends on what you're working on. A question like that is a programming question, per se, and isn't really suitable for SO in my opinion.

[Alexander]

** NOT a programming question

[Lin Ma]

Thanks Alexander, I think Python should have its own precision boundary for float and why not just use it as epsilon? Thanks.

[MSeifert]

@LinMa The difference is how you want to define it: The first one is just the second one divided by two. So the first one is propably the one you want because you compare the absolute of the difference of two floats. I guess the first one is like a +/- error and the second is more like the absolute error on any float. But I'm not exactly sure about that.

[Lin Ma]

Thansk MSeifert, do you think we could leverage some Python built-in stuff, like sys.float_info.epsilon?

[MSeifert]

@LinMa Unfortunatly there is not easy way to determine such a epsilon. Because comparing the quality of a result has another epsilon as the initial values. Just suposse you have a variable A with floating point error epsilon1. If you look at the error of B which is A+A it's floating point error is now 2*epsilon. For adding that might be reconstrubtable but suppose you use A+B*C/D**E % math.exp(F) what will be the error of this result?

[Lin Ma]

Thanks MSeifert, what exactly do you mean floating point error? An example is appreciated. And confused for your comments, "A+A it's floating point error is now 2*epsilon", since I think as long as we are using float, the floating point error is fixed as Python is using unified schema for floating points.

[MSeifert]

Maybe I am wrong but try a=0 then try for i in range(1000): a+=0.1 and print(a-100) then again the for loop and print(a-200) you can try this again and again and at least on my computer the difference is getting bigger.

[Lin Ma]

Thanks MSeifert, tested code and you are correct. Wondering what does it prove?

[MSeifert]

It proves that you cannot define an universial epsilon for arbitary float comparisons because you may or may not know how the float was processed.

[Lin Ma]

Thanks MSeifert, I think float is using 4-byte always, and why universal epsilon does not work? A bit more details are appreciated. :)