Does K.function method of Keras with Tensorflow backend work with network layers?

Question

I recently have started using Keras to build neural networks. I built a simple CNN to classify MNIST dataset. Before learning the model I used K.set_image_dim_ordering('th') in order to plot a convolutional layer weights. Right now I am trying to visualize convolutional layer output with K.function method, but I keep getting error.

Here is what I want to do for now:

input_image = X_train[2:3,:,:,:]

output_layer = model.layers[1].output
input_layer = model.layers[0].input

output_fn = K.function(input_layer, output_layer)

output_image = output_fn.predict(input_image)
print(output_image.shape)

output_image = np.rollaxis(np.rollaxis(output_image, 3, 1), 3, 1)
print(output_image.shape)

fig = plt.figure()
for i in range(32):
    ax = fig.add_subplot(4,8,i+1)
    im = ax.imshow(output_image[0,:,:,i], cmap="Greys")
    plt.xticks(np.array([]))
    plt.yticks(np.array([]))
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([1, 0.1, 0.05 ,0.8])
fig.colorbar(im, cax = cbar_ax)
plt.tight_layout()

plt.show()

And this is what I get:

  File "/home/kinshiryuu/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1621, in function
return Function(inputs, outputs, updates=updates)

  File "/home/kinshiryuu/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1569, in __init__
raise TypeError('`inputs` to a TensorFlow backend function '

TypeError: `inputs` to a TensorFlow backend function should be a list or tuple.

Solution 1

You should do the following changes:

output_fn = K.function([input_layer], [output_layer])
output_image = output_fn([input_image])

K.function takes the input and output tensors as list so that you can create a function from many input to many output. In your case one input to one output.. but you need to pass them as a list none the less.

Next K.function returns a tensor function and not a model object where you can use predict(). The correct way of using is just to call as a function

Solution 2

I think you can also use K.function to get gradients.

self.action_gradients = K.gradients(Q_values, actions)
self.get_action_gradients=K.function[*self.model.input, K.learning_phase()], outputs=action_gradients) 

which basically runs the graph to obtain the Q-value to calculate the gradient of the Q-value w.r.t. action vector in DDPG. Source code here (lines 64 to 70): https://github.com/nyck33/autonomous_quadcopter/blob/master/criticSolution.py#L65

In light of the accepted answer and this usage here (originally from project 5 autonomous quadcopter in the Udacity Deep Learning nanodegree), a question remains in my mind, ie. is K.function() something that can be used fairly flexibly to run the graph and to designate as outputs of K.function() for example outputs of a particular layer, gradients or even weights themselves?

Lines 64 to 67 here: https://github.com/nyck33/autonomous_quadcopter/blob/master/actorSolution.py

It is being used as a custom training function for the actor network in DDPG:

#caller
self.actor_local.train_fn([states, action_gradients, 1])
#called
self.train_fn = K.function(inputs=[self.model.input, action_gradients, K.learning_phase()], \
            outputs=[], updates=updates_op)

outputs is given a value of an empty list because we merely want to train the actor network with the action_gradients from the critic network.