Keras Multi-inputs AttributeError: 'NoneType' object has no attribute 'inbound_nodes'

Question

I'm trying to build a model as ilustrated in below diagram. The idea is to take more than one categorical features (one-hot vectors) and embed them separately, then combine those embedded vectors with a 3D tensor for a LSTM.

With following code in Keras2.0.2, when creating the Model() object with multiple inputs, it raises an AttributeError: 'NoneType' object has no attribute 'inbound_nodes' similar to this question. Can anyone help me to figure out what's the problem?

Model:

Code:

from keras.layers import Dense, LSTM, Input
from keras.layers.merge import concatenate
from keras import backend as K
from keras.models import Model

cat_feats_dims = [315, 14] # Dimensions of the cat_feats
emd_inputs = [Input(shape=(in_size,)) for in_size in cat_feats_dims]
emd_out = concatenate([Dense(20, use_bias=False)(inp) for inp in emd_inputs])
emd_out_3d = K.repeat(emd_out, 10)

lstm_input = Input(shape=(10,5))

merged = concatenate([emd_out_3d,lstm_input])

lstm_output = LSTM(32)(merged)
dense_output = Dense(1, activation='linear')(lstm_output)

model = Model(inputs=emd_inputs+[lstm_input], outputs=[dense_output])

#ERROR MESSAGE
Traceback (most recent call last):
  File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\IPython\core\interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-a9da7f276aa7>", line 14, in <module>
    model = Model(inputs=emd_inputs+[lstm_input], outputs=[dense_output])
  File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\legacy\interfaces.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1648, in __init__
    build_map_of_graph(x, seen_nodes, depth=0)
  File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1644, in build_map_of_graph
    layer, node_index, tensor_index)
  File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1644, in build_map_of_graph
    layer, node_index, tensor_index)
  File "C:\Program Files\Anaconda2\envs\mle-env\lib\site-packages\keras\engine\topology.py", line 1639, in build_map_of_graph
    next_node = layer.inbound_nodes[node_index]
AttributeError: 'NoneType' object has no attribute 'inbound_nodes'

Solution 1

keras.backend.repeat is a function, not a layer. Try using keras.layers.core.RepeatVector instead. It has the same functionality as the function.

emd_out_3d = RepeatVector(10)(emd_out)

Solution 2

Not just for that case, but in general case, if you like to add some function into your model whose has no equivalent layer implementation, you can make that function as Lambda layer.

for instance, I needed to add mean operator on axis=1 into my model. Here is the code as supposed my current tensor named xinput and output tensor is output the code should be as follows.

# suppose my tensor named xinput
meaner=Lambda(lambda x: K.mean(x, axis=1) )
agglayer = meaner(xinput)    
output = Dense(1, activation="linear", name="output_layer")(agglayer)

Instead of using Lambda function, but adding K.mean function directly, you will get the same error.