keras: how to save the training history attribute of the history object

Question

In Keras, we can return the output of model.fit to a history as follows:

 history = model.fit(X_train, y_train, 
                     batch_size=batch_size, 
                     nb_epoch=nb_epoch,
                     validation_data=(X_test, y_test))

Now, how to save the history attribute of the history object to a file for further uses (e.g. draw plots of acc or loss against epochs)?

Solution 1

What I use is the following:

    with open('/trainHistoryDict', 'wb') as file_pi:
        pickle.dump(history.history, file_pi)

In this way I save the history as a dictionary in case I want to plot the loss or accuracy later on.

Solution 2

Another way to do this:

As history.history is a dict, you can convert it as well to a pandas DataFrame object, which can then be saved to suit your needs.

Step by step:

import pandas as pd

# assuming you stored your model.fit results in a 'history' variable:
history = model.fit(x_train, y_train, epochs=10)

# convert the history.history dict to a pandas DataFrame:     
hist_df = pd.DataFrame(history.history) 

# save to json:  
hist_json_file = 'history.json' 
with open(hist_json_file, mode='w') as f:
    hist_df.to_json(f)

# or save to csv: 
hist_csv_file = 'history.csv'
with open(hist_csv_file, mode='w') as f:
    hist_df.to_csv(f)

Solution 3

The easiest way:

Saving:

np.save('my_history.npy',history.history)

Loading:

history=np.load('my_history.npy',allow_pickle='TRUE').item()

Then history is a dictionary and you can retrieve all desirable values using the keys.

Solution 4

The model history can be saved into a file as follows

import json
hist = model.fit(X_train, y_train, epochs=5, batch_size=batch_size,validation_split=0.1)
with open('file.json', 'w') as f:
    json.dump(hist.history, f)

Solution 5

A history objects has a history field is a dictionary which helds different training metrics spanned across every training epoch. So e.g. history.history['loss'][99] will return a loss of your model in a 100th epoch of training. In order to save that you could pickle this dictionary or simple save different lists from this dictionary to appropriate file.

View more solutions