Keras: how to get tensor dimensions inside custom loss?
22,666
Two things here:
- If you want to get a tensor shape you should use
int_shape
function fromkeras.backend
. - The first dimension is set to be a batch dimension so
int_shape(y_true)[0]
will return you a batch size. You should useint_shape(y_true)[1]
.
Related videos on Youtube
Author by
mrgloom
Updated on July 09, 2022Comments
-
mrgloom almost 2 years
I'm trying to write my custom loss function: I want to apply
categorical_crossentropy
to the parts of input vector and then sum.Assume y_true, y_pred are 1D vectors.
Code:
def custom_loss(y_true, y_pred): loss_sum= 0.0 for i in range(0,y_true.shape[0],dictionary_dims): loss_sum+= keras.backend.categorical_crossentropy(y_true[i*dictionary_dims:(i+1)*dictionary_dims], y_pred[i*dictionary_dims:(i+1)*dictionary_dims]) return loss_sum
But I get an error:
for i in range(0,y_true.shape[0],dictionary_dims): TypeError: __index__ returned non-int (type NoneType)
So how to access shape of input tensors to get subset of tensor?
Update: Also tried to write loss via tensorflow directly:
def custom_loss_tf(y_true, y_pred): print('tf.shape(y_true)',tf.shape(y_true)) # print('type(tf.shape(y_true))',type(tf.shape(y_true))) # sys.exit() loss_sum= 0.0 for i in range(0,y_true.shape[0],dictionary_dims): loss_sum+= keras.backend.categorical_crossentropy(y_true[i*dictionary_dims:(i+1)*dictionary_dims], y_pred[i*dictionary_dims:(i+1)*dictionary_dims]) return loss_sum
Output:
tf.shape(y_true) Tensor("Shape:0", shape=(2,), dtype=int32) type(tf.shape(y_true)) <class 'tensorflow.python.framework.ops.Tensor'>
Not sure what is
shape=(2,)
mean, but this is not what I'm expecting, becausemodel.summary()
shows that last layer is(None, 26)
:_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) (None, 80, 120, 3) 0 _________________________________________________________________ conv2d_1 (Conv2D) (None, 80, 120, 32) 896 _________________________________________________________________ max_pooling2d_1 (MaxPooling2 (None, 40, 60, 32) 0 _________________________________________________________________ activation_1 (Activation) (None, 40, 60, 32) 0 _________________________________________________________________ conv2d_2 (Conv2D) (None, 40, 60, 32) 9248 _________________________________________________________________ max_pooling2d_2 (MaxPooling2 (None, 20, 30, 32) 0 _________________________________________________________________ activation_2 (Activation) (None, 20, 30, 32) 0 _________________________________________________________________ conv2d_3 (Conv2D) (None, 20, 30, 64) 18496 _________________________________________________________________ max_pooling2d_3 (MaxPooling2 (None, 10, 15, 64) 0 _________________________________________________________________ activation_3 (Activation) (None, 10, 15, 64) 0 _________________________________________________________________ conv2d_4 (Conv2D) (None, 10, 15, 64) 36928 _________________________________________________________________ max_pooling2d_4 (MaxPooling2 (None, 5, 7, 64) 0 _________________________________________________________________ activation_4 (Activation) (None, 5, 7, 64) 0 _________________________________________________________________ flatten_1 (Flatten) (None, 2240) 0 _________________________________________________________________ head (Dense) (None, 26) 58266 =================================================================
-
mrgloom almost 7 yearsFor some reason
K.int_shape(y_true)
gives me(None, None)
and forK.int_shape(y_pred)
it's(None, 26)
so it looks valid. -
Gery Vessere over 4 yearsI think this is because y_true is only known during training, whereas when you are compiling your model, y_pred is known from the model.