Keras Multiply() layer in functional API
17,021
Solution 1
With keras
> 2.0:
from keras.layers import multiply
output = multiply([dense_all, dense_att])
Solution 2
Under the functional API, you just use the multiply
function, note the lowercase "m". The Multiply class is a layer as you see, intended to be used with the sequential API.
More information in https://keras.io/layers/merge/#multiply_1
Solution 3
You need to add one more open close parenthesis at front.
from keras.layers import Multiply
att_mull = Multiply()([dense_all, dense_att])
Author by
StatsSorceress
Updated on June 11, 2022Comments
-
StatsSorceress almost 2 years
Under the new API changes, how do you do element-wise multiplication of layers in Keras? Under the old API, I would try something like this:
merge([dense_all, dense_att], output_shape=10, mode='mul')
I've tried this (MWE):
from keras.models import Model from keras.layers import Input, Dense, Multiply def sample_model(): model_in = Input(shape=(10,)) dense_all = Dense(10,)(model_in) dense_att = Dense(10, activation='softmax')(model_in) att_mull = Multiply([dense_all, dense_att]) #merge([dense_all, dense_att], output_shape=10, mode='mul') model_out = Dense(10, activation="sigmoid")(att_mull) return 0 if __name__ == '__main__': sample_model()
Full trace:
Using TensorFlow backend. Traceback (most recent call last): File "testJan17.py", line 13, in <module> sample_model() File "testJan17.py", line 8, in sample_model att_mull = Multiply([dense_all, dense_att]) #merge([dense_all, dense_att], output_shape=10, mode='mul') TypeError: __init__() takes exactly 1 argument (2 given)
EDIT:
I tried implementing tensorflow's elementwise multiply function. Of course, the result is not a
Layer()
instance, so it doesn't work. Here's the attempt, for posterity:def new_multiply(inputs): #assume two only - bad practice, but for illustration... return tf.multiply(inputs[0], inputs[1]) def sample_model(): model_in = Input(shape=(10,)) dense_all = Dense(10,)(model_in) dense_att = Dense(10, activation='softmax')(model_in) #which interactions are important? new_mult = new_multiply([dense_all, dense_att]) model_out = Dense(10, activation="sigmoid")(new_mult) model = Model(inputs=model_in, outputs=model_out) model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy']) return model