Keras - is it possible to view the weights and biases of models in Tensorboard

Question

I just got started with Keras and built a Q-learning example program. I created a tensorboard callback and I include it in the call to model.fit, but the only things that appear in TensorBoard are the scalar summary for the loss and the network graph. Interestingly, if I open up the dense layer in the graph, I see a little summary icon labeled "bias_0" and one labeled "kernel_0", but I don't see these appearing in the distributions or histograms tabs in TensorBoard like I did when I built a model in pure tensorflow.

Do I need to do something else to enable these in Tensorboard? Do I need to look into the details of the model that Keras produces and add my own tensor_summary() calls?

Solution 1

You can get the weights and biases per layer and for the entire model with .get_weights().

For example if the first layer of your model is the dense layer for which you would like to have your weights and biases, you can get them with:

weights, biases = model.layers[0].get_weights()

Solution 2

I debugged this and found that the problem was I was not providing any validation data when I called fit(). The TensorBoard callback will only report on the weights when validation data is provided. That seems a bit restrictive, but I at least have something that works.

Comments

[Autonomous]

Possible duplicate of this.

[Ocie Mitchell]

I am interested in outputting and analyzing the weights using Tensorboard, not just printing them out. Maybe .get_weights() will give me something I can feed into Tensorboard.

[Autonomous]

I don't think this should be the case. In my case, I have to process validation data differently than the train data, so I wrote my own callback to compute validation accuracy. So similarly you may be able to write your own callback.

[Ocie Mitchell]

In the versions of Keras I have been using (including 2.04), in TensorBoard.on_epoch_end, the second line is: if self.validation_data and self.histogram_freq: so if self.validation_data is not provided, the tensor summaries will be skipped. It looks like the validation_data is used to generate the model.inputs and model.targets for the summary, but the weights are also lumped in with this, even though the weights are part of the model and shouldn't need validation data to exist.

[KeithWM]

This might be somewhat useful, but the question is about tensorboard, and I don't think calls to model.layers[i].get_weights() are likely to be as useful as getting tensortboard working properly.

[Elbek]

Even with validation data. Tensorboard is not showing weights and biases. Which version of Tensorflow are you using?