logits and labels must be broadcastable error in Tensorflow RNN

Question

I am new to Tensorflow and deep leaning. I am trying to see how the loss decreases over 10 epochs in my RNN model that I created to read a dataset from kaggle which contains credit card fraud data. I am trying to classify the transactions as fraud(1) and not fraud(0). When I try to run the below code I keep getting the below error:

> 2018-07-30 14:59:33.237749: W
> tensorflow/core/kernels/queue_base.cc:277]
> _1_shuffle_batch/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed Traceback (most recent call last):  
> File
> "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1322, in _do_call
>     return fn(*args)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1307, in _run_fn
>     options, feed_dict, fetch_list, target_list, run_metadata)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1409, in _call_tf_sessionrun
>     run_metadata) tensorflow.python.framework.errors_impl.InvalidArgumentError: logits
> and labels must be broadcastable: logits_size=[1,2] labels_size=[1,24]
>    [[Node: softmax_cross_entropy_with_logits_sg =
> SoftmaxCrossEntropyWithLogits[T=DT_FLOAT,
> _device="/job:localhost/replica:0/task:0/device:CPU:0"](add, softmax_cross_entropy_with_logits_sg/Reshape_1)]]
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):   File
> "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 96, in
> <module>
>     train_neural_network(x)   File "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 79, in
> train_neural_network
>     _, c = sess.run([optimizer, cost], feed_dict={x: feature_batch, y: label_batch})   File
> "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 900, in run
>     run_metadata_ptr)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1135, in _run
>     feed_dict_tensor, options, run_metadata)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1316, in _do_run
>     run_metadata)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py",
> line 1335, in _do_call
>     raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: logits
> and labels must be broadcastable: logits_size=[1,2] labels_size=[1,24]
>    [[Node: softmax_cross_entropy_with_logits_sg =
> SoftmaxCrossEntropyWithLogits[T=DT_FLOAT,
> _device="/job:localhost/replica:0/task:0/device:CPU:0"](add, softmax_cross_entropy_with_logits_sg/Reshape_1)]]
> 
> Caused by op 'softmax_cross_entropy_with_logits_sg', defined at:  
> File "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 96, in
> <module>
>     train_neural_network(x)   File "/home/suleka/Documents/untitled1/RNN_CrediCard.py", line 63, in
> train_neural_network
>     cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction,
> labels=y))   File
> "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py",
> line 250, in new_func
>     return func(*args, **kwargs)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py",
> line 1968, in softmax_cross_entropy_with_logits
>     labels=labels, logits=logits, dim=dim, name=name)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py",
> line 1879, in softmax_cross_entropy_with_logits_v2
>     precise_logits, labels, name=name)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py",
> line 7205, in softmax_cross_entropy_with_logits
>     name=name)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py",
> line 787, in _apply_op_helper
>     op_def=op_def)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py",
> line 3414, in create_op
>     op_def=op_def)   File "/home/suleka/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py",
> line 1740, in __init__
>     self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access
> 
> InvalidArgumentError (see above for traceback): logits and labels must
> be broadcastable: logits_size=[1,2] labels_size=[1,24]     [[Node:
> softmax_cross_entropy_with_logits_sg =
> SoftmaxCrossEntropyWithLogits[T=DT_FLOAT,
> _device="/job:localhost/replica:0/task:0/device:CPU:0"](add, softmax_cross_entropy_with_logits_sg/Reshape_1)]]

Can anyone point out what I am doing wrong in my code and also any problem in my code if possible. Thank you in advance.

Shown below is my code:

import tensorflow as tf
from tensorflow.contrib import rnn



# cycles of feed forward and backprop
hm_epochs = 10
n_classes = 2
rnn_size = 128
col_size = 30
batch_size = 24
try_epochs = 1
fileName = "creditcard.csv"

def create_file_reader_ops(filename_queue):
    reader = tf.TextLineReader(skip_header_lines=1)
    _, csv_row = reader.read(filename_queue)
    record_defaults = [[1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1.], [1]]
    col1, col2, col3, col4, col5, col6, col7, col8, col9, col10, col11, col12, col13, col14, col15, col16, col17, col18, col19, col20, col21, col22, col23, col24, col25, col26, col27, col28, col29, col30, col31 = tf.decode_csv(csv_row, record_defaults=record_defaults)
    features = tf.stack([col1, col2, col3, col4, col5, col6, col7, col8, col9, col10, col11, col12, col13, col14, col15, col16, col17, col18, col19, col20, col21, col22, col23, col24, col25, col26, col27, col28, col29, col30])
    return features, col31


def input_pipeline(fName, batch_size, num_epochs=None):
    # this refers to multiple files, not line items within files
    filename_queue = tf.train.string_input_producer([fName], shuffle=True, num_epochs=num_epochs)
    features, label = create_file_reader_ops(filename_queue)
    min_after_dequeue = 10000 # min of where to start loading into memory
    capacity = min_after_dequeue + 3 * batch_size # max of how much to load into memory
    # this packs the above lines into a batch of size you specify:
    feature_batch, label_batch = tf.train.shuffle_batch(
        [features, label],
        batch_size=batch_size,
        capacity=capacity,
        min_after_dequeue=min_after_dequeue)
    return feature_batch, label_batch


creditCard_data, creditCard_label = input_pipeline(fileName, batch_size, try_epochs)


x = tf.placeholder('float',[None,col_size])
y = tf.placeholder('float')


def recurrent_neural_network_model(x):
    #giving the weights and biases random values
    layer ={ 'weights': tf.Variable(tf.random_normal([rnn_size, n_classes])),
            'bias': tf.Variable(tf.random_normal([n_classes]))}

    x = tf.split(x, 24, 0)
    print(x)

    lstm_cell = rnn.BasicLSTMCell(rnn_size)
    outputs, states = rnn.static_rnn(lstm_cell, x, dtype=tf.float32 )
    output = tf.matmul(outputs[-1], layer['weights']) + layer['bias']

    return output

def train_neural_network(x):
    prediction = recurrent_neural_network_model(x)
    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction, labels=y))
    optimizer = tf.train.AdamOptimizer().minimize(cost)


    with tf.Session() as sess:

        gInit = tf.global_variables_initializer().run()
        lInit = tf.local_variables_initializer().run()
        coord = tf.train.Coordinator()
        threads = tf.train.start_queue_runners(coord=coord)
        for epoch in range(hm_epochs):
            epoch_loss = 0

            for counter in range(101):
                    feature_batch, label_batch = sess.run([creditCard_data, creditCard_label])
                    print(label_batch.shape)
                    _, c = sess.run([optimizer, cost], feed_dict={x: feature_batch, y: label_batch})
                    epoch_loss += c
            print('Epoch', epoch, 'compleated out of', hm_epochs, 'loss:', epoch_loss)



train_neural_network(x)

Solution 1

Make sure that the number of labels in the final classification layer is equal to the number of classes you have in your dataset. InvalidArgumentError (see above for traceback): logits and labels must be broadcastable: logits_size=[1,2] labels_size=[1,24] as shown in your question might suggest that you are have just two classes in your final classification layer while you actually need 24.

In my case, I had 7 classes in my dataset, but I mistakenly used 4 labels in the final classification layer. Therefore, I had to change from

tf.keras.layers.Dense(4, activation="softmax")

to

tf.keras.layers.Dense(7, activation="softmax")

Solution 2

When you say

    cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=prediction, labels=y))

the prediction and labels have incompatible shapes. You need to change how the predictions are computed to get one per example in your minibatch.

Solution 3

A little late to the party but I had the same error with a CNN, I messed around with different types of cross entropy and the error was resolved by using sparce_softmax_cross_entropy_with_logits().

cost = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=prediction, labels=y))

Solution 4

This error occurs because there is a mismatch in the count of Predicted class and the input. I copied a code and encountered this error

This was the original code where output is 5 classes

model.add(Dense(5, activation = "softmax"))

In my case, I had 30 classes and correcting the classes count fixed it

model.add(Dense(30, activation = "softmax"))

View more solutions

Comments

[Suleka_28]

Yes, that was also one of the problems in my code and also I had made several more mistakes. I have been trying to use softmax_cross_entropy_with_logits when it is a binary classification where I should have used a sigmoid_cross_entropy_with_logits with n_classes variable set to 1.

[prb_cm]

Hi, could you please let me know how did you manage to compute predictions per example for a given minibatch. I am facing the same error as you. logits and labels must be broadcastable: logits_size=[32,31] labels_size=[32,199].

[Admin]

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.