Error when checking target: expected dense_3 to have shape (3,) but got array with shape (1,)

Question

I am working on training a VGG16-like model in Keras, on a 3 classes subset from Places205, and encountered the following error:

ValueError: Error when checking target: expected dense_3 to have shape (3,) but got array with shape (1,)

I read multiple similar issues but none helped me so far. The error is on the last layer, where I've put 3 because this is the number of classes I'm trying right now.

The code is the following:

import keras from keras.datasets
import cifar10 from keras.preprocessing.image 
import ImageDataGenerator from keras.models 
import Sequential 
from keras.layers import Dense, Dropout, Activation, Flatten from keras.layers import Conv2D, MaxPooling2D 
from keras import backend as K import os


# Constants used  
img_width, img_height = 224, 224  
train_data_dir='places\\train'  
validation_data_dir='places\\validation'  
save_filename = 'vgg_trained_model.h5'  
training_samples = 15  
validation_samples = 5  
batch_size = 5  
epochs = 5


if K.image_data_format() == 'channels_first':
    input_shape = (3, img_width, img_height) else:
    input_shape = (img_width, img_height, 3)

model = Sequential([
    # Block 1
    Conv2D(64, (3, 3), activation='relu', input_shape=input_shape, padding='same'),
    Conv2D(64, (3, 3), activation='relu', padding='same'),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    # Block 2
    Conv2D(128, (3, 3), activation='relu', padding='same'),
    Conv2D(128, (3, 3), activation='relu', padding='same'),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    # Block 3
    Conv2D(256, (3, 3), activation='relu', padding='same'),
    Conv2D(256, (3, 3), activation='relu', padding='same'),
    Conv2D(256, (3, 3), activation='relu', padding='same'),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    # Block 4
    Conv2D(512, (3, 3), activation='relu', padding='same'),
    Conv2D(512, (3, 3), activation='relu', padding='same'),
    Conv2D(512, (3, 3), activation='relu', padding='same'),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    # Block 5
    Conv2D(512, (3, 3), activation='relu', padding='same',),
    Conv2D(512, (3, 3), activation='relu', padding='same',),
    Conv2D(512, (3, 3), activation='relu', padding='same',),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    # Top
    Flatten(),
    Dense(4096, activation='relu'),
    Dense(4096, activation='relu'),
    Dense(3, activation='softmax') ])

model.summary()

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

# no augmentation config train_datagen = ImageDataGenerator() validation_datagen = ImageDataGenerator()
     train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = validation_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary')

model.fit_generator(
    train_generator,
    steps_per_epoch=training_samples // batch_size,
    epochs=epochs,
    validation_data=validation_generator,
    validation_steps=validation_samples // batch_size)

model.save_weights(save_filename)

Solution 1

The problem is with your label-data shape. In a multiclass problem you are predicting the probabibility of every possible class, so must provide label data in (N, m) shape, where N is the number of training examples, and m is the number of possible classes (3 in your case).

Keras expects y-data in (N, 3) shape, not (N,) as you've problably provided, that's why it raises an error.

Use e.g. OneHotEncoder to convert your label data to one-hot encoded form.

Solution 2

As mentioned by others, Keras expects "one hot" encoding in multiclass problems.

Keras comes with a handy function to recode labels:

print(train_labels)
[1. 2. 2. ... 1. 0. 2.]

print(train_labels.shape)
(2000,)

Recode labels using to_categorical to get the correct shape of inputs:

from keras.utils import to_categorical
train_labels = to_categorical(train_labels)

print(train_labels)
[[0. 1. 0.]
 [0. 0. 1.]
 [0. 0. 1.]
 ...
 [0. 1. 0.]
 [1. 0. 0.]
 [0. 0. 1.]]

print(train_labels.shape)
(2000, 3)  # viz. 2000 observations, 3 labels as 'one hot'

Other importent things to change/check in multiclass (compared to binary classification):

Set class_mode='categorical' in the generator() function(s).

Don't forget that the last dense layer must specify the number of labels (or classes):

model.add(layers.Dense(3, activation='softmax'))

Make sure that activation= and loss= is chosen so to suit multiclass problems, usually this means activation='softmax' and loss='categorical_crossentropy'.

Solution 3

Had the same issue. To solve the problem you can simply change in validation_generator and train_generator the class mode from 'binary' to 'categorical' - that's because you have 3 classes-which is not binary.

Solution 4

Problem : expected dense_3 to have shape (3,) but got array with shape (1,)

If you are using it for classification then the number of variables should be correct in the parameter for adding a dense layer.

variables_for_classification=5 #change it as per your number of categories
model.add(Dense(variables_for_classification, activation='softmax'))

model.fit(X_train, Y_train, epochs=epochs, batch_size=batch_size,validation_split=0.1,callbacks=[EarlyStopping(monitor='val_loss', patience=3, min_delta=0.0001)])

To make it more clear. As I was using the LSTM to predict the category of the news and the categories were 5- business, tech, politics, sports, entertainment

In that dense function when I put 5 it worked correctly.

Solution 5

The reason for this is you would have used 'binary' class_mode in the fit_generator() method for a multi class problem. Change that to 'categorical' and the error goes.

View more solutions

Comments

[Sumax]

My problem was solved as I had wrongly provided output shape for the last Dense layer. Thank you so much! (3 instead of 2 for Binary Classification)

[user5305519]

fit_generator() does not have keyword argument 'class_mode', I think you meant flow_from_directory() instead

[Andrea Moro]

And what's the difference in passing a value directly, or by a variable? There's none, so clearly not the issue.