Keras : history not accessible for loss or accuracy

Question

I am training a CNN in Keras with Tensorflow backend,

mod1=gmodel.fit(images, train_labels,
      batch_size=100,
      epochs=2,
      verbose=1,
      validation_data=(test_images, test_labels))

and at every epoch I can see printed in the output the accuracy and loss (until here everything seems ok).

Epoch 1/10
1203/1203 [==============================] - 190s - loss: 0.7600 - acc: 0.5628 
- val_loss: 0.5592 - val_acc: 0.6933
Epoch 2/10
1203/1203 [==============================] - 187s - loss: 0.5490 - acc: 0.6933 
- val_loss: 0.4589 - val_acc: 0.7930
Epoch 3/10
....

At the end, I want to plot the validation loss so in previous projects I have accessed the validation loss via

mod1.history['val_loss']

but I am getting an error as if .history() was empty.

TypeError                                 Traceback (most recent call last)
<ipython-input-23-ecdd306e9232> in <module>()
----> 1 modl.history()
TypeError: 'History' object is not callable

EDIT (after answer below): When I try to access the loss, for example:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-34-06fcc6efb374> in <module>()
----> 1 mod1.history['val_loss']

TypeError: 'History' object is not subscriptable

I haven't found anything like this problem before, so I am lost as to what could be happening or how to debug.

Any pointers or ideas are greatly appreciated.

Solution 1

model.fit(x_train, y_train,batch_size=128,validation_data=(x_test, y_test))

vy = model.history.history['val_loss']
ty = model.history.history['loss']

Please use the validation_data in model.fit statement for test data then the only "model.history.history" will come

Reference: https://keras.io/callbacks/

Solution 2

When model is fitted it will return a history object, you cannot call() or subscript it directly like history['loss'].

If you fitted it using model.fit() then you have to query as follows

model.history.history.keys() -> will give you ['acc','loss','val_acc','val_loss']
If you monitored loss and mentioned metrics as accuracy during compile process.

You can access all metrics using same format ,for ex :model.history.history['acc']

But if you fitted a model and assigned history object to a local variable like this history = model.fit(X, Y) then the mode of access would be

history.history['acc']
history.history['val_acc']

Here we don't need to mention model object as history object is now saved in a local variable.

And also don't forget to add validation data or use validation split parameter of fit to access the validation metrics.

Solution 3

Although you say you have called mod1.history['val_loss'], your error message tells a different story - most probably, as Daniel Moller has already commented, you have in fact used something like mod1.history() (i.e. with parentheses). Here is what I get (Python 3.5):

mod1.history()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-20-67bafe3187cc> in <module>()
----> 1 mod1.history()

TypeError: 'dict' object is not callable

mod1.history is not a function to be called with (), but a Python dictionary:

mod1.history
# result:
{'acc': [0.82374999999999998,
  0.94294999999999995,
  0.95861666666666667,
  ...],
 'loss': [0.62551526172161098,
  0.18810810926556587,
  0.13734668906728426,
  ...],
 'val_loss': [12.05395287322998,
  11.584557554626464,
  10.949809835815429,
  ...]}

mod1.history['val_loss']
# result:
[12.05395287322998,
 11.584557554626464,
 10.949809835815429,
 ...]

Comments

[Daniel Möller]

"History" object is not callable. It's clear you shouldn't be using (). Parentheses "call" a function.

[desertnaut]

1) when editing your question after an answer has been posted, it's good practice to indicate so ("EDIT"), so that the respondent does not look like an idiot 2) when someone has answered the question as you had it initially described, upvoting is a nice courtesy 3) I have actually shown the case for mod1.history['val_loss'] below; try simply mod1.history and type(mod1.history) - if you do not get a proper dictionary, there is something else wrong in parts of your code that you don't show here (even now, you show snippets with both modl ('l') & mod1 ('1') - one must be a typo)...

[user3177938]

Thanks for pointing out that mod1.history() would not work anyway. What I was really aiming at, though, which is mod1.history['val_loss'] gives me a "not subscriptable" error. I am not a Keras expert, but I have accessed the loss and accuracy before, and I am very confused as to why this time I cannot do it as well.

[desertnaut]

@user3177938 this was not the error you reported originally!

[user3177938]

I thought, perhaps mistakenly, that since I was getting that error on mod1.history['val_loss'] it is because there was no dictionary in history. So, that is why I tried doing mod1.history()

[desertnaut]

@patti_jane Solve what exactly? The question is a mess. If you face an issue that looks similar, I suggest you open a new question with a reproducible example

[patti_jane]

@desertnaut I was having " History is not subscriptable" error, but I realized that I was calling history for "gmodel" instead of "mod1" --> mod1=gmodel.fit(). It seemed weird to me even though using mod1.history["loss] user3177938 still got the same error.

[surajs1n]

Please add some explanation along with the code you wrote.

[Yatin Arora]

Until and unless you don't have validation data history will not be accessible.