ValueError: Input 0 is incompatible with layer conv_1: expected ndim=3, found ndim=4

Question

I am trying to make a variational auto encoder to learn to encode DNA sequences, but am getting an unexpected error.

My data is an array of one-hot arrays.

The issue I'm getting is a Value Error. It's telling me that I have a four dimensional input, when my input is clearly three-dimensional (100, 4008, 4).

In fact, when I print out the seq layer, it says that it's shape is (?, 100, 4008, 4).

When I take out a dimension, it then gives me an error for being two dimensional.

Any help will be highly appreciated!

Code is:

from keras.layers import Input 
from keras.layers.convolutional import Conv1D
from keras.layers.core import Dense, Activation, Flatten, RepeatVector, Lambda
from keras import backend as K
from keras.layers.wrappers import TimeDistributed
from keras.layers.recurrent import GRU
from keras.models import Model
from keras import objectives

from one_hot import dna_sequence_to_one_hot

from random import shuffle
import numpy as np

# take FASTA file and convert into array of vectors
seqs = [line.rstrip() for line in open("/home/ubuntu/sequences.fa", "r").readlines() if line[0] != ">"]
seqs = [dna_sequence_to_one_hot(s) for s in seqs]
seqs = np.array(seqs)

# first random thousand are training, next thousand are validation
test_data = seqs[:1000]
validation_data = seqs[1000:2000]

latent_rep_size = 292
batch_size = 100
epsilon_std = 0.01
max_length = len(seqs[0])
charset_length = 4
epochs = 100

def sampling(args):
    z_mean_, z_log_var_ = args
    # batch_size = K.shape(z_mean_)[0]
    epsilon = K.random_normal_variable((batch_size, latent_rep_size), 0., epsilon_std)
    return z_mean_ + K.exp(z_log_var_ / 2) * epsilon

# loss function
def vae_loss(x, x_decoded_mean):
    x = K.flatten(x)
    x_decoded_mean = K.flatten(x_decoded_mean)
    xent_loss = max_length * objectives.categorical_crossentropy(x, x_decoded_mean)
    kl_loss = - 0.5 * K.mean(1 + z_log_var - K.square(z_mean) - K.exp(z_log_var), axis = -1)
    return xent_loss + kl_loss

# Encoder
seq = Input(shape=(100, 4008, 4), name='one_hot_sequence')
e = Conv1D(9, 9, activation = 'relu', name='conv_1')(seq)
e = Conv1D(9, 9, activation = 'relu', name='conv_2')(e)
e = Conv1D(9, 9, activation = 'relu', name='conv_3')(e)
e = Conv1D(10, 11, activation = 'relu', name='conv_4')(e)
e = Flatten(name='flatten_1')(e)
e = Dense(435, activation = 'relu', name='dense_1')(e)
z_mean = Dense(latent_rep_size, name='z_mean', activation = 'linear')(e)
z_log_var = Dense(latent_rep_size, name='z_log_var', activation = 'linear')(e)
z = Lambda(sampling, output_shape=(latent_rep_size,), name='lambda')([z_mean, z_log_var])

encoder = Model(seq, z)

# Decoder
d = Dense(latent_rep_size, name='latent_input', activation = 'relu')(z)
d = RepeatVector(max_length, name='repeat_vector')(d)
d = GRU(501, return_sequences = True, name='gru_1')(d)
d = GRU(501, return_sequences = True, name='gru_2')(d)
d = GRU(501, return_sequences = True, name='gru_3')(d)
d = TimeDistributed(Dense(charset_length, activation='softmax'), name='decoded_mean')(d)



# create the model, compile it, and fit it
vae = Model(seq, d)
vae.compile(optimizer='Adam', loss=vae_loss, metrics=['accuracy'])
vae.fit(x=test_data, y=test_data, epochs=epochs, batch_size=batch_size, validation_data=validation_data)

Solution 1

In the documentation it is mentioned that we need to mention the input in a specific format which is (None,NumberOfFeatureVectors). In your case it will be (None,4)

https://keras.io/layers/convolutional/

When using this layer as the first layer in a model, provide an input_shape argument (tuple of integers or None, e.g. (10, 128) for sequences of 10 vectors of 128-dimensional vectors, or (None, 128) for variable-length sequences of 128-dimensional vectors.

Solution 2

Try to put the input to the network like this: Input(shape=(None, 4)

Generally, it's for the case you don't know the length of your sequence, but I had the same problem and for some reason, it was solved when I did it

Hope it works!

Solution 3

Specify the kernel_size in your convolutional layers as a tuple, not integer, even if it requires only a dimension:

e = Conv1D(9, (9), activation = 'relu', name='conv_1')(seq)

Although in Keras documentation is stated that both an integer and a tuple are valid, I found the second to be more handy with dimensionality.