python - invalid value encountered in log

Question

I have the following expression: log = np.sum(np.nan_to_num(-y*np.log(a+ 1e-7)-(1-y)*np.log(1-a+ 1e-7)))

it is giving me the following warning:

RuntimeWarning: invalid value encountered in log
  log = np.sum(np.nan_to_num(-y*np.log(a+ 1e-7)-(1-y)*np.log(1-a+ 1e-7)))

I don't understand what might be the invalid value or why am I getting it. Any and every help is appreciated.

NOTE: This is a cross-entropy cost function where I added 1e-7 to avoid having zeros inside log. y & a are numpy arrays and numpy is imported as np.

Solution 1

You probably still have negative values inside the log, which gives nan with real numbers.

a and y should represent probability between 0 to 1, So you need to check why do you have smaller/larger values there. Adding 1e-7 shows there is something fishy, because np.log(0) gives -inf, which I think is the value you want.

Solution 2

You can use math.log() replacing numpy.log(), which could raise error

>>> import numpy
>>> numpy.log(0)
-inf
>>> numpy.__version__
'1.3.0'
>>> import math
>>> math.log(0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: math domain error

Comments

[RandomWalker]

A minor comment: math.log() take one real number as an input while numpy.log() can take a list of real as input. Of course, this difference does not matter in OP's case.

[WoodyDRN]

I think adding the 1e-7 will prevent division by 0 error?