Notebook

Part 4a - Saving a Model¶

In this notebook we will cover the following topics:

Saving a model to disk

In [1]:

import numpy as np
np.warnings.filterwarnings('ignore')  # Hide np.floating warning

import keras

from keras.datasets import cifar10
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D

# Prevent TensorFlow from grabbing all the GPU memory
import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.allow_growth=True
sess = tf.Session(config=config)

import holoviews as hv
hv.extension('bokeh')

Using TensorFlow backend.

Load the Data¶

In [2]:

from keras.datasets import cifar10
import keras.utils

(x_train, y_train), (x_test, y_test) = cifar10.load_data()

# Save an unmodified copy of y_test for later, flattened to one column
y_test_true = y_test[:,0].copy()

x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

num_classes = 10
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

# The data only has numeric categories so we also have the string labels below 
cifar10_labels = np.array(['airplane', 'automobile', 'bird', 'cat', 'deer', 
                           'dog', 'frog', 'horse', 'ship', 'truck'])

Train the Model¶

Let's quickly train our simple model so we can save the results

In [3]:

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
                 activation='relu',
                 input_shape=x_train.shape[1:]))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),
              metrics=['accuracy'])

In [4]:

history = model.fit(x_train, y_train,
          batch_size=128,
          epochs=10,
          verbose=1,
          validation_data=(x_test, y_test))

Train on 50000 samples, validate on 10000 samples
Epoch 1/10
50000/50000 [==============================] - 5s 103us/step - loss: 1.7929 - acc: 0.3541 - val_loss: 1.3845 - val_acc: 0.5069
Epoch 2/10
50000/50000 [==============================] - 4s 77us/step - loss: 1.3650 - acc: 0.5134 - val_loss: 1.2388 - val_acc: 0.5520
Epoch 3/10
50000/50000 [==============================] - 4s 78us/step - loss: 1.1949 - acc: 0.5765 - val_loss: 1.1182 - val_acc: 0.6049
Epoch 4/10
50000/50000 [==============================] - 4s 78us/step - loss: 1.0851 - acc: 0.6182 - val_loss: 1.0442 - val_acc: 0.6356
Epoch 5/10
50000/50000 [==============================] - 4s 79us/step - loss: 0.9922 - acc: 0.6536 - val_loss: 0.9387 - val_acc: 0.6704
Epoch 6/10
50000/50000 [==============================] - 4s 79us/step - loss: 0.9205 - acc: 0.6778 - val_loss: 0.9655 - val_acc: 0.6630
Epoch 7/10
50000/50000 [==============================] - 4s 79us/step - loss: 0.8633 - acc: 0.6976 - val_loss: 0.9120 - val_acc: 0.6852
Epoch 8/10
50000/50000 [==============================] - 4s 78us/step - loss: 0.8114 - acc: 0.7168 - val_loss: 0.8800 - val_acc: 0.6973
Epoch 9/10
50000/50000 [==============================] - 4s 78us/step - loss: 0.7555 - acc: 0.7342 - val_loss: 0.8765 - val_acc: 0.7014
Epoch 10/10
50000/50000 [==============================] - 4s 78us/step - loss: 0.7164 - acc: 0.7497 - val_loss: 0.8671 - val_acc: 0.7043

Saving the model is as easy as calling the save() method. This records the weights and the structure of the model so that it can be recreated by another program entirely from this file. (Assuming that program is using the same version of Keras.)

In [5]:

model.save('cifar10_model.hdf5')

Note that the file format is HDF5, which is a common data format for numerical data. Keras requires the h5py Python package be present in order to read and write HDF5 files.

Depending on the number of weights in the model, this file can get very big:

In [6]:

! ls -lh cifar10_model.hdf5

-rw-rw-r-- 1 seibert seibert 19M Apr  3 11:16 cifar10_model.hdf5

We can poke around to see the structure of it using the HDF5 command line tools:

In [7]:

! h5ls -r cifar10_model.hdf5

/                        Group
/model_weights           Group
/model_weights/conv2d_1  Group
/model_weights/conv2d_1/conv2d_1 Group
/model_weights/conv2d_1/conv2d_1/bias:0 Dataset {32}
/model_weights/conv2d_1/conv2d_1/kernel:0 Dataset {3, 3, 3, 32}
/model_weights/conv2d_2  Group
/model_weights/conv2d_2/conv2d_2 Group
/model_weights/conv2d_2/conv2d_2/bias:0 Dataset {64}
/model_weights/conv2d_2/conv2d_2/kernel:0 Dataset {3, 3, 32, 64}
/model_weights/dense_1   Group
/model_weights/dense_1/dense_1 Group
/model_weights/dense_1/dense_1/bias:0 Dataset {128}
/model_weights/dense_1/dense_1/kernel:0 Dataset {12544, 128}
/model_weights/dense_2   Group
/model_weights/dense_2/dense_2 Group
/model_weights/dense_2/dense_2/bias:0 Dataset {10}
/model_weights/dense_2/dense_2/kernel:0 Dataset {128, 10}
/model_weights/dropout_1 Group
/model_weights/dropout_2 Group
/model_weights/flatten_1 Group
/model_weights/max_pooling2d_1 Group
/optimizer_weights       Group
/optimizer_weights/training Group
/optimizer_weights/training/Adadelta Group
/optimizer_weights/training/Adadelta/Variable:0 Dataset {3, 3, 3, 32}
/optimizer_weights/training/Adadelta/Variable_10:0 Dataset {3, 3, 32, 64}
/optimizer_weights/training/Adadelta/Variable_11:0 Dataset {64}
/optimizer_weights/training/Adadelta/Variable_12:0 Dataset {12544, 128}
/optimizer_weights/training/Adadelta/Variable_13:0 Dataset {128}
/optimizer_weights/training/Adadelta/Variable_14:0 Dataset {128, 10}
/optimizer_weights/training/Adadelta/Variable_15:0 Dataset {10}
/optimizer_weights/training/Adadelta/Variable_1:0 Dataset {32}
/optimizer_weights/training/Adadelta/Variable_2:0 Dataset {3, 3, 32, 64}
/optimizer_weights/training/Adadelta/Variable_3:0 Dataset {64}
/optimizer_weights/training/Adadelta/Variable_4:0 Dataset {12544, 128}
/optimizer_weights/training/Adadelta/Variable_5:0 Dataset {128}
/optimizer_weights/training/Adadelta/Variable_6:0 Dataset {128, 10}
/optimizer_weights/training/Adadelta/Variable_7:0 Dataset {10}
/optimizer_weights/training/Adadelta/Variable_8:0 Dataset {3, 3, 3, 32}
/optimizer_weights/training/Adadelta/Variable_9:0 Dataset {32}

Interestingly, we can see that the HDF5 file also records the state of the optimizer, so that we can resume training from a saved model. This is a useful way to checkpoint your work. In fact, Keras has a ModelCheckpoint callback that does this automatically after every epoch.

Experiments to Try¶

Try using the ModelCheckpoint callback to save the model in every epoch.

If you screw everything up, you can use File / Revert to Checkpoint to go back to the first version of the notebook and restart the Jupyter kernel with Kernel / Restart.

In [ ]: